1.선형회귀.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "1. 선형회귀.ipynb",
      "provenance": [],
      "authorship_tag": "ABX9TyN1XCxenMLW6BogIJ2W3kyE",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/skyil7/SejongUniv_AI/blob/master/1_%EC%84%A0%ED%98%95%ED%9A%8C%EA%B7%80.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMZmjJqp482w",
        "colab_type": "text"
      },
      "source": [
        "# Linear Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XqXMlTwE5Gkp",
        "colab_type": "text"
      },
      "source": [
        "## Theoretical Overview\n",
        "$$ H(x) = Wx +b$$\n",
        "\n",
        "$$ cost(W, b) = \\frac{1}{m} \\sum^m_{i=1} \\left( H(x^{(i)}) - y^{(i)} \\right)^2$$\n",
        "\n",
        "- H(x): x값에 대한 예측\n",
        "- cost(W,b): W,b에 대해 함수 H(x)의 손실값"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F3TmX7Lg40VR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch\n",
        "import torch.optim as optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lw29q_sl7GgK",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "80312a96-1b54-4519-95cb-750957433ccd"
      },
      "source": [
        "torch.manual_seed(1) #시드를 고정"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<torch._C.Generator at 0x7fb6551fe490>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FLL00ihV7pPE",
        "colab_type": "text"
      },
      "source": [
        "## Data\n",
        "아래의 조건을 충족하는 Fake Data를 만들어 사용\n",
        "$$ y(x) = 2x+1$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vSCMadY17ywZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_train = torch.FloatTensor([[1], [2], [3]])\n",
        "y_train = torch.FloatTensor([[3], [5], [7]])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bBZ4Pz6P8ROy",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 92
        },
        "outputId": "4914edd7-4faf-48f0-81cf-d0cea815fd61"
      },
      "source": [
        "print(x_train)\n",
        "print(x_train.shape)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[1.],\n",
            "        [2.],\n",
            "        [3.]])\n",
            "torch.Size([3, 1])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8St3rJ6o8dJ2",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 92
        },
        "outputId": "1b022b57-f721-40ba-eda2-5dd05e953e27"
      },
      "source": [
        "print(y_train)\n",
        "print(y_train.shape)"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[3.],\n",
            "        [5.],\n",
            "        [7.]])\n",
            "torch.Size([3, 1])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y_oMSoFi9BhQ",
        "colab_type": "text"
      },
      "source": [
        "## Weight Initialization\n",
        "\n",
        "requires_grad는 파라메터가 학습되어야 하는지를 지정한다."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "izJbLNFw9IML",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "897a4317-3776-4355-956c-7879eaf71c5f"
      },
      "source": [
        "W = torch.zeros(1, requires_grad=True)\n",
        "print(W)"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([0.], requires_grad=True)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EZNjEfyP9QOQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "c5134de0-1f6b-433c-fec6-9f0517fe67e3"
      },
      "source": [
        "b = torch.zeros(1, requires_grad=True)\n",
        "print(b)"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([0.], requires_grad=True)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "j1Pszwxw9kFf",
        "colab_type": "text"
      },
      "source": [
        "## Hypothesis 설정\n",
        "$$ H(x) = Wx + b $$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EOfMTkR09sgO",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "9cffeca1-a327-4982-c50c-b6bc8d0df436"
      },
      "source": [
        "hypothesis = x_train * W + b\n",
        "print(hypothesis)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[0.],\n",
            "        [0.],\n",
            "        [0.]], grad_fn=<AddBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SEP4eRAo9067",
        "colab_type": "text"
      },
      "source": [
        "## Cost\n",
        "$$ cost(W, b) = \\frac{1}{m} \\sum^m_{i=1} \\left( H(x^{(i)}) - y^{(i)} \\right)^2$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jKAl-khJ96eU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "e57e48f5-bd95-4978-e817-b628128d51fd"
      },
      "source": [
        "print(hypothesis - y_train)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[-3.],\n",
            "        [-5.],\n",
            "        [-7.]], grad_fn=<SubBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N_LFkAXH-AOQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "80bbd87a-b381-48a8-83b0-297782214d60"
      },
      "source": [
        "cost = torch.mean((hypothesis - y_train)**2)    #MSE\n",
        "print(cost)"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(27.6667, grad_fn=<MeanBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OXKeWtnF-OP-",
        "colab_type": "text"
      },
      "source": [
        "## Gradient Descent"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tTe-plz6-Sre",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "optimizer = optim.SGD([W,b], lr=0.01)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6StgW4-J-YFZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "optimizer.zero_grad()  # Optimizer 초기화\n",
        "cost.backward()        # 구한 loss로부터 Back Prop을 통해 각 변수마다 loss에 대한 gradient 산출\n",
        "optimizer.step()       # 파라메터 업데이트"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uTqzaKBX-iIi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 55
        },
        "outputId": "a44c7a78-672f-4a2f-975a-8bde658c4f7b"
      },
      "source": [
        "print(W)\n",
        "print(b)"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([0.2267], requires_grad=True)\n",
            "tensor([0.1000], requires_grad=True)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wdhLdnqV_fxT",
        "colab_type": "text"
      },
      "source": [
        "## Validate Training Result"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0WBSlNen_lq9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 73
        },
        "outputId": "e3edb664-6b01-482d-b5d4-7c33a67487d0"
      },
      "source": [
        "hypothesis = x_train * W + b\n",
        "print(hypothesis)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[0.3267],\n",
            "        [0.5533],\n",
            "        [0.7800]], grad_fn=<AddBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s1EsjgGq_n3h",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "outputId": "e9255812-1543-43a1-8f33-70dd6af41d76"
      },
      "source": [
        "cost = torch.mean((hypothesis - y_train)**2)    #MSE\n",
        "print(cost)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(21.8693, grad_fn=<MeanBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gAcxhYLU_wef",
        "colab_type": "text"
      },
      "source": [
        "## Training\n",
        "Loss가 올바르게 감소하는 것을 확인했으니, 충분히 학습시키기"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WrC1sw-g_0X7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "3304d230-8e82-48df-d6cd-ba5a565f5a7a"
      },
      "source": [
        "epochs = 1000\n",
        "for epoch in range(1, epochs+1):\n",
        "    # Train one step\n",
        "    optimizer.zero_grad()\n",
        "    cost.backward()\n",
        "    optimizer.step()\n",
        "\n",
        "    # Calculate new cost\n",
        "    hypothesis = x_train * W + b\n",
        "    cost = torch.mean((hypothesis - y_train)**2)\n",
        "\n",
        "    if epoch % 100 == 0:\n",
        "        print('Epoch {:4d}/{} W: {:.3f} b: {:.3f} Cost: {:.6f}'.format(epoch, epochs, W.item(), b.item(), cost.item()))"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch  100/1000 W: 2.035 b: 0.921 Cost: 0.000895\n",
            "Epoch  200/1000 W: 2.027 b: 0.938 Cost: 0.000553\n",
            "Epoch  300/1000 W: 2.021 b: 0.951 Cost: 0.000342\n",
            "Epoch  400/1000 W: 2.017 b: 0.962 Cost: 0.000211\n",
            "Epoch  500/1000 W: 2.013 b: 0.970 Cost: 0.000130\n",
            "Epoch  600/1000 W: 2.010 b: 0.976 Cost: 0.000081\n",
            "Epoch  700/1000 W: 2.008 b: 0.981 Cost: 0.000050\n",
            "Epoch  800/1000 W: 2.006 b: 0.985 Cost: 0.000031\n",
            "Epoch  900/1000 W: 2.005 b: 0.988 Cost: 0.000019\n",
            "Epoch 1000/1000 W: 2.004 b: 0.991 Cost: 0.000012\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yB5aDwSTA6lO",
        "colab_type": "text"
      },
      "source": [
        "## Test"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e2zrXzIZA8r6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "x_test = torch.FloatTensor([[5], [7], [10]])\n",
        "y_test = torch.FloatTensor([[11], [15], [21]])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fvP6TnTPBK2S",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 92
        },
        "outputId": "3dce38f4-0c27-4742-940a-1b8bb625236a"
      },
      "source": [
        "hypothesis = W * x_test + b\n",
        "cost = torch.mean((hypothesis - y_test)**2)\n",
        "\n",
        "print(hypothesis)\n",
        "print('Cost:',cost)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[11.0109],\n",
            "        [15.0188],\n",
            "        [21.0308]], grad_fn=<AddBackward0>)\n",
            "Cost: tensor(0.0005, grad_fn=<MeanBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}