03.Introduction to NumPy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to NumPy\n",
    "by Maxwell Margenot\n",
    "\n",
    "Part of the Quantopian Lecture Series:\n",
    "\n",
    "* [www.quantopian.com/lectures](https://www.quantopian.com/lectures)\n",
    "* [github.com/quantopian/research_public](https://github.com/quantopian/research_public)\n",
    "\n",
    "Notebook released under the Creative Commons Attribution 4.0 License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy is an incredibly powerful package in Python that is ubiquitous throughout the Quantopian platform. It has strong integration with Pandas, another tool we will be covering in the lecture series. NumPy adds support for multi-dimensional arrays and mathematical functions that allow you to easily perform linear algebra calculations. This lecture will be a collection of linear algebra examples computed using NumPy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "NumPy是一个非常强大的Python包，它几乎无处不在(不单单在Quantopian平台中)。它和Pandas是好朋友，我们将在另一门课中专门介绍Pandas. NumPy增加了对多维数组的支持，自带很多用于数学计算的函数，能让你轻松地进行线性代数的计算。\n",
=======
    "NumPy是一个非常强大的Python包，它几乎无处不在。它和Pandas是好朋友，我们将在另一门课中专门介绍Pandas. NumPy增加了对多维数组的支持，自带很多用于数学计算的函数，能让你轻松地进行线性代数的计算。\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "本课将用Numpy为你展示一些它的应用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Basic NumPy arrays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The most basic way that we could make use of NumPy in finance is calculation the mean return of a portfolio. Say that we have a list containing the historical return of several stocks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Numpy数组基础"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "Numpy在金融中应用，最基础的应该是计算一个投资组合平均回报。\n",
    "我们用若干只股票的历史回报构成一个列表。"
=======
    "Numpy在金融中应用，最基础的应该是计算一个投资组合平均收益。\n",
    "假设这里有列表,他们由几只股票的历史数据构成。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "stock_list = [3.5, 5, 2, 8, 4.2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make an array by calling a function on the list:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "通过`numpy.array()`使得上述的list转换成`NumPy`中的数据结构:`numpy.ndarray`"
=======
    "将这个列表传入到numpy.array()函数中，将它变成numpy中的数组:numpy.ndarray"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "returns = np.array(stock_list)\n",
    "print(returns, type(returns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll notice that the type of our array is 'ndarray', not just 'array'. This is because NumPy arrays can be created with multiple dimensions. If we pass np.array() a list of lists, it will create a 2-dimensional array. If pass a list of lists of lists, it will create a 3-dimensional array, and so on and so forth."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "你可能注意到我们的数组类型是`ndarray`,而不是`array`.这是因为`NumPy`数组可以是多维的.如果我们给`np.array()`传递一个列表的列表，他将生成一个二维数组.如果传递的是一个列表的列表的列表，它将会生成一个三维数组，以此类推。"
=======
    "你可能注意到我们的数组类型是'ndarray',而不是'array'.这是因为Numpy数组可以是多维的.如果我们给np.array()传递一个列表的列表，他将生成一个二维数组.如果你传递的是一个列表的列表的列表，它将会生成一个三维数组，以此类推。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "A = np.array([[1, 2], [3, 4]])\n",
    "print(A, type(A))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can access the dimensions of an array by looking at its `shape` member variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "我们可以通过`.shape`查看数组的维度。"
=======
    "我们可以通过`.shape`查看数组的维度的详细情况。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Arrays are indexed in much the same way as lists in Python. Elements of a list begin indexing from $0$ and end at $n - 1$, where $n$ is the length of the array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数组的索引和Python中的列表一样. 一个长度为$n$的列表的索引起始于 $0$ 结束于 $n - 1$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(returns[0], returns[len(returns) - 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can take a slice of an array using a colon, just like in a list."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和list一样，我们用分号进行切片操作."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(returns[1:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A slice of an array, like in a list, will select a group of elements in the array starting from the first element indicated and going up to (but not including) the last element indicated.\n",
    "\n",
    "In the case of multidimensional arrays, many of the same conventions with slicing and indexing hold. We can access the first column of a 2-dimensional array like so:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数组的切片，也和列表一样，从指定的第一个元素开始，按照一定的步长(默认是1)选择元素，直到你指定的最后一个元素，但是不包含最后一个元素.\n",
    "在多维数组的情形中，情况也是类似的。接下来的例子中，我们可以这样去访问一个二维数组的第一列："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A[:, 0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the first row of a 2-dimensional array like so:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "访问数组首行："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A[0, :])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that each slice of the array returns yet another array!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意每次通过切片返回的是另一个numpy数组！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(type(A[0,:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Passing only one index to a 2-dimensional array will result in returning the row with the given index as well, providing us with another way to access individual rows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果二维数组只用一个索引，得到的将是某行数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Accessing the index of an individual element will return only the element."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
<<<<<<< HEAD
    "访问单个元素,返回的只是你索引的元素(而不是一个numpy数组)."
=======
    "访问单个元素,最后返回是你索引的数据(而不是一个numpy数组)"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A[1, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Array functions\n",
    "\n",
    "Functions built into NumPy can be easily called on arrays. Most functions are applied to an array element-wise (as scalar multiplication is). For example, if we call `log()` on an array, the logarithm will be taken of each element."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数组函数\n",
    "numpy内置函数使用起来特别简单，大多数函数都是直接作用于元素层面的，我们来看下np.log()例子，自己感受下吧."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.log(returns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some functions return a single value. This is because they treat the array as a collection (similar to a list), performing the designated function. For example, the `mean()` function will do exactly what you expect, calculating the mean of an array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "有些函数会返回一个单独的值，他们把array当做一个整体来看待，比如说`np.mean()`函数，它会计算一个数组的平均值。"
=======
    "有些函数会返回一个单独的值，他们把array当做一个整体来看待，比如说np.mean()函数，它会计算一个数组的平均值。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.mean(returns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or the `max()` function will return the maximum element of an array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "np.max()会计算数组最大值"
=======
    "np.max()会计算数组最大的值"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.max(returns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For further reading on the universal functions in NumPy, check out the [documentation](https://docs.scipy.org/doc/numpy/user/quickstart.html#universal-functions)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更多的介绍请参考[文档](https://docs.scipy.org/doc/numpy/user/quickstart.html#universal-functions)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Return to the returns\n",
    "\n",
    "Now let's modify our returns array with scalar values. If we add a scalar value to an array it will be added to every element of the array. If we multiply an array by a scalar value it will be multiplied against every element of the array. If we do both, both will happen!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Return to the returns\n",
<<<<<<< HEAD
    "现在我们用标量和returns做计算，比如做加法，那么这个标量将会和return的每个元素进行相加。乘法也是一样，我们来试试同时进行乘法和加法会发生什么!"
=======
    "现在我们用标量和returns做计算，比如做加法，那么这个标量将会和return的每个元素进行相加。乘法也是一样，我们来试试看。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
<<<<<<< HEAD
    "collapsed": true,
=======
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "returns*2 + 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy also has functions specifically built to operate on arrays. Let's take the mean and standard deviation of this group of returns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "numpy有专门作用于数组的函数，让我们来计算下这组回报的均值和标准差。"
=======
    "numpy有专门作用于数组的函数，比如np.std(),计算数组的标准差。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(\"Mean: \", np.mean(returns), \"Std Dev: \", np.std(returns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's simulate a universe of stocks using NumPy's functions. First we need to create the arrays to hold the assets and returns that we will use to build a portfolio. This is because arrays are created with a fixed size. Their dimensions can't be changed without creating a new array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "我们用numpy的函数来虚拟一个股票池。首先我们要创建一个数组，用来存储资产和回报的数据，然后用这个去创建一个投资组合。NumPy数组的尺寸是不能改变的(可以改变形状)。"
=======
    "我们用numpy的函数来模拟一个股票池。首先我们要创建一个数组，用来存储资产和收益的数据，然后用这个去创建一个投资组合。数组的尺寸是不能改变的(可以改变形状)。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "N = 10\n",
    "assets = np.zeros((N, 100))\n",
    "returns = np.zeros((N, 100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This function, `zeroes()`, creates a NumPy array with the given dimensions that is entirely filled in with $0$. We can pass a single value or a tuple of as many dimensions as we like. Passing in the tuple `(N, 100)`, will return a two-dimensional array with $N$ rows and $100$ columns. Our result is a $N \\times 100$ array.\n",
    "\n",
    "Now we will simulate a base asset. We want the universe of stocks to be correlated with each other so we will use this initial value to generate the others."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "np.zeros顾名思义，给定维度，创建一个全是0的矩阵，函数的参数是一个标量或者一个tuple(元组),比如说(N,100),那么生成的就是10(N=10)行100列的二维数组。\n",
    "\n",
    "我们首先随机生成一个基准资产值，股票池中的股票都以这个基准值设定他们的初始净值，这样做能让股票之间保持**相关性**。"
=======
    "np.zeros顾名思义，就是创建一个全是0的矩阵，函数的参数是一个标量或者一个tuple(元组),比如说(N,100),那么生成的就是10行100列的矩阵。\n",
    "现在我们给每只股票设定一个初始的净值，以供后面使用。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.random.seed(1)\n",
    "R_1 = np.random.normal(1.01, 0.03, 100)\n",
    "returns[0] = R_1\n",
    "assets[0] = np.cumprod(R_1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "returns[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "np.cumprod?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "a = np.array([[1, 2, 3], [4, 5, 6]])\n",
    "a.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "np.cumprod(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `random` module in NumPy is exceedingly useful. It contains methods for sampling from many different probability distributions, some of which are covered in the [random variables lecture](https://www.quantopian.com/lectures/random-variables) in the Quantopian lecture series. In this case we draw $N = 100$ random samples from a normal distribution with mean $1.01$ and standard deviation $0.03$. We treat these as the daily percentage returns of our asset and take the cumulative product of these samples to get the current price.\n",
    "\n",
    "The way we have generated our universe, the individual $R_i$ vectors are each 1-dimensional arrays and the `returns` and `assets` variables contain 2-dimensional arrays. Above, we set the initial row of both `returns` and `assets` to be the first $R_i$ vector and the cumulative asset price based on those returns, respectively.\n",
    "\n",
    "We will now use this base asset to create a few other random assets that are correlated with it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "`random`是非常有用的模块，能对不同概率分布的进行随机取样，更详细的内容会在后面的课程：[随机变量](https://www.quantopian.com/lectures/random-variables)中讲到.上面的例子，我们用期望为$1.01$，标准差为$0.03$的正态分布随机生成了100个样本，把这些值当做每只股票的日回报率(即100天每天的回报率)，通过np.cumprod做累乘，算出每天的资产价格。\n",
    "\n",
    "至此，我们已经生成了我们的universe(股票池),$R_i$向量是一个一维数组，`returns` 和 `assets`是二维的\n",
    "数组。我们用$R_i$向量去初始化`returns` 和 `assets`每行的数据，最后得到初始的`returns` 和 `assets`。"
=======
    "`random`是非常有用的模块，能对不同概率分布的进行随机取样，更详细的知识可以在后面的课程：[随机变量](https://www.quantopian.com/lectures/random-variables)会讲到.上面的例子，我们用期望为$1.01$，标准差为$0.03$的正态分布随机生成了100个样本，把这些值当做每只股票的日收益率，通过np.cumprod做累乘，算出当前资产价格。\n",
    "\n",
    "至此，我们已经生成了我们的universe(证券池),$R_i$向量是一个一维的数组，`returns` 和 `assets`变量是二维的数组。我们用$R_i$向量去初始化`returns` 和 `assets`每行的数据，最后得到初始的`returns` 和 `assets`。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Generate assets that are correlated with R_1\n",
    "np.random.seed(2)\n",
    "for i in range(1, N):\n",
    "    R_i = R_1 + np.random.normal(0.001, 0.02, 100)\n",
    "    returns[i] = R_i # Set each row of returns equal to the new R_i array\n",
    "    assets[i] = np.cumprod(R_i)\n",
    "    \n",
    "mean_returns = [(np.mean(R) - 1)*100 for R in returns]\n",
    "return_volatilities = [np.std(R) for R in returns]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "len(mean_returns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we generate the remaining $N - 1$ securities that we want in our universe by adding random noise to $R_1$. This ensures that our $N - 1$ other assets will be correlated with the base asset because they have some underlying information that is shared.\n",
    "\n",
    "Let's plot what the mean return of each asset looks like:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "这里，我们基于$R_1$，加上用正态分布随机生成的噪声，得到股票池中另外$N - 1$只股票的回报，这样能确保他们拥有相关性。\n",
    "我们用画图工具来看下每只股票回报的均值。"
=======
    "这里，我们基于$R_1$，加上用正态分布随机生成的噪声，得到股票池总另外$N - 1$只证券的收益，这样能确保他们拥有相关性。\n",
    "我们用画图工具来看下每只证券收益均值。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
<<<<<<< HEAD
    "collapsed": true,
=======
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "plt.bar(np.arange(len(mean_returns)), mean_returns)\n",
    "plt.xlabel('Stock')\n",
    "plt.ylabel('Returns')\n",
    "plt.title('Returns for {0} Random Assets'.format(N));"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculating Expected Return\n",
    "\n",
    "So we have a universe of stocks. Great! Now let's put them together in a portfolio and calculate its expected return and risk.\n",
    "\n",
    "We will start off by generating $N$ random weights for each asset in our portfolio."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "### 计算回报期望\n",
    "\n",
    "我们现在已经有了一个股票池(universe).非常好！我们用他们生成一个投资组合，然后计算他的预期回报率和风险值。\n",
=======
    "### 计算收益期望\n",
    "\n",
    "我们现在已经有了一个股票池(universe).非常好！我们用他们生成一个投资组合，然后计算他的预期收益率和风险值。\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "我们首先给我们组合中每只股票设定一个初始的随机权重值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "weights = np.random.uniform(0, 1, N)\n",
    "weights = weights/np.sum(weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "weights.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to rescale the weights so that they all add up to $1$. We do this by scaling the weights vector by the sum total of all the weights. This step ensures that we will be using $100\\%$ of the portfolio's cash.\n",
    "\n",
    "To calculate the mean return of the portfolio, we have to scale each asset's return by its designated weight. We can pull each element of each array and multiply them individually, but it's quicker to use NumPy's linear algebra methods. The function that we want is `dot()`. This will calculate the dot product between two arrays for us. So if $v = \\left[ 1, 2, 3 \\right]$ and $w = \\left[4, 5, 6 \\right]$, then:\n",
    "\n",
    "$$ v \\cdot w = 1 \\times 4 + 2 \\times 5 + 3 \\times 6 $$\n",
    "\n",
    "For a one-dimensional vector, the dot product will multiply each element pointwise and add all the products together! In our case, we have a vector of weights, $\\omega = \\left[ \\omega_1, \\omega_2, \\dots \\omega_N\\right]$ and a vector of returns, $\\mu = \\left[ \\mu_1, \\mu_2, \\dots, \\mu_N\\right]$. If we take the dot product of these two we will get:\n",
    "\n",
    "$$ \\omega \\cdot \\mu = \\omega_1\\mu_1 + \\omega_2\\mu_2 + \\dots + \\omega_N\\mu_N = \\mu_P $$\n",
    "\n",
    "This yields the sum of all the asset returns scaled by their respective weights. This the the portfolio's overall expected return!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "p_returns = np.dot(weights, mean_returns)\n",
    "print(\"Expected return of the portfolio: \", p_returns)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "这里p_returns就是资产组合平均回报。简单来说，假设组合里都是股票，每个股票都有一个权重值，当然这个不是随机的，为了方便计算，我们取得随机值，并做缩放处理处理(每项权重值/权重值之和)，使得各项权重之和等于1.我们用股票的平均回报乘以股票对应的权重值，然后求和，就能得到这个资产组合的平均回报。\n",
=======
    "这里p_returns就是我们常说的资产组合净值。简单来说，假设组合里都是股票，每个股票都有一个权重值，当然这个不是随机的，为了方便计算，我们取得随机值，并做归一化处理(权重值/权重值之和)，使得权重之和等于1.我们用权重去乘以每只股票的平均收益率，然后求和，就能得到这个资产组合的净值。\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "我们这里使用的是`np.dot`,对于一维数组(向量)，将数组中对应位置的元素进行相乘，并将结果相加。\n",
    "\n",
    "$$v = \\left[ 1, 2, 3 \\right] , \\left[4, 5, 6 \\right]:$$\n",
    "\n",
    "$$ v \\cdot w = 1 \\times 4 + 2 \\times 5 + 3 \\times 6 $$\n",
    "\n",
<<<<<<< HEAD
    "在我们的例子中,权重: $\\omega = \\left[ \\omega_1, \\omega_2, \\dots \\omega_N\\right]$ 和回报: $\\mu = \\left[ \\mu_1, \\mu_2, \\dots, \\mu_N\\right]$.\n",
=======
    "在我们的例子中,权重: $\\omega = \\left[ \\omega_1, \\omega_2, \\dots \\omega_N\\right]$ 和收益: $\\mu = \\left[ \\mu_1, \\mu_2, \\dots, \\mu_N\\right]$.\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "$$ \\omega \\cdot \\mu = \\omega_1\\mu_1 + \\omega_2\\mu_2 + \\dots + \\omega_N\\mu_N = \\mu_P $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "p_variance = np.power((np.multiply(weights, mean_returns) - p_returns), 2)\n",
    "p_std = np.sum(np.sqrt(p_variance))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "np.sum(p_variance)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "p_std"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calculating the mean return is fairly intuitive and does not require too much explanation of linear algebra. However, calculating the variance of our portfolio requires a bit more background."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "计算平均回报很直观，然而，计算投资组合的回报方差，这个时候就需要线性代数来帮忙了。"
=======
    "计算收益的均值很直观，并不需要线性代数的相关知识，然而，计算我们投资组合的方差就需要一些背景知识了。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Beware of NaN values\n",
    "\n",
    "Most of the time, all of these calculations will work without an issue. However, when working with real data we run the risk of having `nan` values in our arrays. This is NumPy's way of saying that the data there is missing or doesn't exist. These `nan` values can lead to errors in mathematical calculations so it is important to be aware of whether your array contains `nan` values and to know how to drop them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 对NaN值要小心\n",
    "\n",
<<<<<<< HEAD
    "多数候进行数值计算并不会有问题。但当NumPy遇到了缺省值时，就会用`nan`来表示，当做数值计算的时候，这个会带来麻烦，所以在进行计算的时候，需要消除他们带来的影响。"
=======
    "大多数时候计算并不会有问题。但当我们的numpy遇到了缺省值时，就会用`nan`来表示，当我们做数值计算的时候，这个会带来麻烦，所以在进行计算的时候，消除他们带来的影响。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "v = np.array([1, 2, np.nan, 4, 5])\n",
    "print(v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see what happens when we try to take the mean of this array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如有`nan`时，我们计算均值时会发生什么?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.mean(v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clearly, `nan` values can have a large impact on our calculations. Fortunately, we can check for `nan` values with the `isnan()` function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这下，你知道`nan`会对我们计算有多大影响了吧！\n",
    "还好我们有检测函数，np.isnan()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "np.isnan(v)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Calling `isnan()` on an array will call the function on each value of the array, returning a value of `True` if the element is `nan` and `False` if the element is valid. Now, knowing whether your array contains `nan` values is all well and good, but how do we remove `nan`s? Handily enough, NumPy arrays can be indexed by boolean values (`True` or `False`). If we use a boolean array to index an array, we will remove all values of the array that register as `False` under the condition. We use the `isnan()` function in create a boolean array, assigning a `True` value to everything that is *not* `nan` and a `False` to the `nan`s and we use that to index the same array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "np.isnan()吃掉数组，吐出的是`True`和`False`，如果元素是`nan`,吐出`True`,反之就是`False`\n",
    "，然后利用索引就可以把不是`nan`的值给筛选出来了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "ix = ~np.isnan(v) # the ~ indicates a logical not, inverting the bools\n",
    "print(v[ix]) # We can also just write v = v[~np.isnan(v)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.mean(v[ix]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few shortcuts to this process in the form of NumPy functions specifically built to handle them, such as `nanmean()`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也可以直接使用内置函数`nanmean()`实现上述的功能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.nanmean(v))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `nanmean()` function simply calculates the mean of the array as if there were no `nan` values at all! There are a few more of these functions, so feel free to read more about them in the [documentation](https://docs.scipy.org/doc/numpy/user/index.html). These indeterminate values are more an issue with data than linear algebra itself so it is helpful that there are ways to handle them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "`nanmean()` 轻松实现了不含`nan`的均值的计算， [点我查看Numpy的使用指南](https://docs.scipy.org/doc/numpy/user/index.html). 我们在进行线性代数计算之前，首先要处理好诸如缺省值这些数据问题。"
=======
    "`nanmean()` 轻松实现了不含`nan`的均值的计算， [点我查看Numpy的使用指南](https://docs.scipy.org/doc/numpy/user/index.html). 我们在进行线性代数计算之前，首先处理好诸如缺省值这些数据问题。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conclusion\n",
    "\n",
    "Linear algebra is pervasive in finance and in general. For example, the calculation of *optimal* weights according to modern portfolio theory is done using linear algebra techniques. The arrays and functions in NumPy allow us to handle these calculations in an intuitive way. For a quick intro to linear algebra and how to use NumPy to do more significant matrix calculations, proceed to the next section."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 小结\n",
    "\n",
<<<<<<< HEAD
    "线代在金融里的应用是无处不在的。`modern portfolio`理论的权重值的*最优化(optimal)*问题就需要用到线性代数。\n",
=======
    "线代在金融里是无处不在的。线代资产组合理论的权重值的*optimal*问题就需要用到线性代数。\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "下面就让我们开启线代之旅吧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A brief foray into linear algebra\n",
    "\n",
    "Let's start with a basic overview of some linear algebra. Linear algebra comes down to the mutiplication and composition of scalar and matrix values. A scalar value is just a real number that we multiply against an array. When we scale a matrix or array using a scalar, we multiply each individual element of that matrix or array by the scalar.\n",
    "\n",
    "A matrix is a collection of values, typically represented by an $m \\times n$ grid, where $m$ is the number of rows and $n$ is the number of columns. The edge lengths $m$ and $n$ do not necessarily have to be different. If we have $m = n$, we call this a square matrix. A particularly interesting case of a matrix is when $m = 1$ or $n = 1$. In this case we have a special case of a matrix that we call a vector. While there is a matrix object in NumPy we will be doing everything using NumPy arrays because they can have dimensions greater than $2$. For the purpose of this section, we will be using matrix and array interchangeably.\n",
    "\n",
    "We can express the matrix equation as:\n",
    "\n",
    "$$ y = A\\cdot x $$\n",
    "\n",
    "Where $A$ is an $m \\times n$ matrix, $y$ is a $m \\times 1$ vector, and $x$ is a $n \\times 1$ vector. On the right-hand side of the equation we are multiplying a matrix by a vector. This requires a little bit more clarification, lest we think that we can go about multiplying any matrices by any other matrices.\n",
    "\n",
    "#### Matrix multiplication\n",
    "\n",
    "With matrix multiplication, the order in which the matrices are multiplied matters. Multiplying a matrix on the left side by another matrix may be just fine, but multiplying on the right may be undefined."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线性代数简介\n",
    "\n",
    "让我们来回顾下线性代数的一些基本概念。我们先来看看线性代数中标量和矩阵的乘法。一个标量就是一个实数，当我们用一个标量乘以一个矩阵的时候，就是将矩阵中每个元素都乘以这个标量，从而实现了矩阵的缩放。\n",
    "\n",
<<<<<<< HEAD
    "矩阵就是数值的集合，通常我们会将其展示成$m \\times n$表格的形式，$m$代表矩阵的行数，$n$代表矩阵的列数。$m$和$n$并不要求相等，当他们相等时，我们称这个矩阵为`方阵`。对于$m = 1$ 或者 $n = 1$的矩阵，这种特殊的矩阵，有时也称作向量。在`Numpy`中我们用多维(维度大于等于2)数组表示一个矩阵对象，在本节课中，我们视情况使用矩阵或者数组这两种称谓。\n",
=======
    "矩阵就是数值的集合，通常我们会将其展示成$m \\times n$表格的形式，$m$代表矩阵的行数，$n$代表矩阵的列数。$m$和$n$并不要求相等，当他们相等时，我们称这个矩阵为`方阵`。对于$m = 1$ 或者 $n = 1$的矩阵，这种特殊的矩阵，有时也称作向量。在`Numpy`中我们用多维(维度大于等于2)数组表示一个矩阵对象，在本节课中，我们视情况使用矩阵或者数组。\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "一个矩阵等式如下:\n",
    "\n",
    "$$ y = A\\cdot x $$\n",
    "\n",
    "A是一个$m \\times n$ 的矩阵, $y$ 是一个 $m \\times 1$ 向量, and $x$ 是一个 $n \\times 1$ 向量.所以等式的右边是一个矩阵乘以一个向量，需要注意的是，矩阵乘法，对于矩阵的形状有严格要求。\n",
    "\n",
    "#### 矩阵乘法\n",
    "\n",
    "故在矩阵乘法中，两个矩阵的顺序很重要，$$ A\\cdot x $$可以成立，但是$$ x\\cdot A $$就不一定。(矩阵乘法不满足交换律)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "A = np.array([\n",
    "        [1, 2, 3, 12, 6],\n",
    "        [4, 5, 6, 15, 20],\n",
    "        [7, 8, 9, 10, 10]        \n",
    "    ])\n",
    "B = np.array([\n",
    "        [4, 4, 2],\n",
    "        [2, 3, 1],\n",
    "        [6, 5, 8],\n",
    "        [9, 9, 9]\n",
    "    ])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A.shape)\n",
    "print(B.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the above-defined matrices, $A$ and $B$, have different dimensions. $A$ is $3 \\times 5$ and $B$ is $4 \\times 3$. The general rule of what can and cannot be multiplied in which order is based on the dimensions of the matrices. Specifically, the number of columns in the matrix on the left must be equal to the number of rows in the matrix on the right. In super informal terms, let's say that we have an $m \\times n$ matrix and a $p \\times q$ matrix. If we multiply the first by the second on the right, we get the following:\n",
    "\n",
    "$$ (m \\times n) \\cdot (p \\times q) = (m \\times q) $$\n",
    "\n",
    "So the resultant product has the same number of rows as the left matrix and the same number of columns as the right matrix. This limitation of matrix multiplication with regards to dimensions is important to keep track of when writing code. To demonstrate this, we use the `dot()` function to multiply our matrices below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "注意矩阵 $A$ 和 $B$的维度。 $A$ 是 $3 \\times 5$, 而 $B$ 是 $4 \\times 3$，这种情况是无法进行矩阵相乘的，矩阵乘法规则是乘号左边的矩阵列数要和乘号右边的矩阵的行数相等才可以进行相乘。用数学的语言可以表示为:形如$m \\times n$ 和 $p \\times q$ 两个矩阵：\n",
=======
    "注意矩阵 $A$ 和 $B$的维度。 $A$ 是 $3 \\times 5$ 而 $B$ 是 $4 \\times 3$，这种情况是无法进行矩阵相乘的，矩阵乘法规则是乘号左边的矩阵列数要和乘号右边的矩阵的行数相等才可以进行相乘。用数学的语言可以表示为:形如$m \\times n$ 和 $p \\times q$ 两个矩阵：\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "$$ (m \\times n) \\cdot (p \\times q) = (m \\times q) $$\n",
    "\n",
    "可以看出两个矩阵相乘后的新矩阵，他的行数等于乘号左边矩阵行数，列数等于乘号右边的矩阵列数。在我们进行编程时，要特别留意矩阵的维度。我们通常使用`np.dot()`来进行矩阵乘法的计算。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.dot(B, A))\n",
    "print(np.dot(B, A).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These results make sense in accordance with our rule. Multiplying a $3 \\times 5$ matrix on the right by a $4 \\times 3$ matrix results in an error while multiplying a $4 \\times 3$ matrix on the right by a $3 \\times 5$ matrix results in a $4 \\times 5$ matrix."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算的结果验证了我们的法则:\n",
    "    \n",
    "$$ (4 \\times 5) \\cdot (5 \\times 3) = (4 \\times 3) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.dot(A.T, B.T))\n",
    "print(np.dot(A.T, B.T).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Portfolio Variance\n",
    "\n",
    "Let's return to our portfolio example from before. We calculated the expected return of the portfolio, but how do we calculate the variance We start by trying to evaluate the portfolio as a sum of each individual asset, scaled by it's weight.\n",
    "\n",
    "$$ VAR[P] = VAR[\\omega_1 S_1 + \\omega_2 S_2 + \\cdots + \\omega_N S_N] $$\n",
    "\n",
    "Where $S_0, \\cdots, S_N$ are the assets contained within our universe. If all of our assets were independent of each other, we could simply evaluate this as\n",
    "\n",
    "$$ VAR[P] = VAR[\\omega_1 S_1] + VAR[\\omega_2 S_2] + \\cdots + VAR[\\omega_N S_N] = \\omega_1^2\\sigma_1^2 + \\omega_2^2\\sigma_2^2 + \\cdots + \\omega_N^2\\sigma_N^2 $$\n",
    "\n",
    "However, all of our assets depend on each other by their construction. They are all in some way related to our base asset and therefore each other. We thus have to calculate the variance of the portfolio by including the individual pairwise covariances of each asset. Our formula for the variance of the portfolio:\n",
    "\n",
    "$$ VAR[P] = \\sigma_P^2 = \\sum_i \\omega_i^2\\sigma_i^2 + \\sum_i\\sum_{i\\neq j} \\omega_i\\omega_j\\sigma_i\\sigma_j\\rho_{i, j}, \\ i, j \\in \\lbrace 1, 2, \\cdots, N \\rbrace $$\n",
    "\n",
    "Where $\\rho_{i,j}$ is the correlation between $S_i$ and $S_j$, $\\rho_{i, j} = \\frac{COV[S_i, S_j]}{\\sigma_i\\sigma_j}$. This seems exceedingly complicated, but we can easily handle all of this using NumPy arrays. First, we calculate the covariance matrix that relates all the individual stocks in our universe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "### 组合回报的方差\n",
    "\n",
    "Let's return to our portfolio example from before. We calculated the expected return of the portfolio, but how do we calculate the variance We start by trying to evaluate the portfolio as a sum of each individual asset, scaled by it's weight.\n",
    "\n",
    "让我们回到刚才那个投资组合的例子。我们计算了组合投资回报的期望值(均值)，我们应该如何计算这个组合的回报的方差呢?\n",
=======
    "### 组合收益的方差\n",
    "\n",
    "Let's return to our portfolio example from before. We calculated the expected return of the portfolio, but how do we calculate the variance We start by trying to evaluate the portfolio as a sum of each individual asset, scaled by it's weight.\n",
    "\n",
    "让我们回到刚才那个投资组合的例子。我们计算了组合投资收益的期望值(均值)，我们应该如何计算这个组合的收益的方差呢?\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "$$ VAR[P] = VAR[\\omega_1 S_1 + \\omega_2 S_2 + \\cdots + \\omega_N S_N] $$\n",
    "\n",
    "$S_0, \\cdots, S_N$ 代表了我们组合中每个投资标的物的资产. 如果所有的投资标的都是相互独立的，那么我们可以使用如下的公式进行计算:\n",
    "\n",
    "$$ VAR[P] = VAR[\\omega_1 S_1] + VAR[\\omega_2 S_2] + \\cdots + VAR[\\omega_N S_N] = \\omega_1^2\\sigma_1^2 + \\omega_2^2\\sigma_2^2 + \\cdots + \\omega_N^2\\sigma_N^2 $$\n",
    "\n",
    "然而，我们的标的物都是彼此相关的，因为他们由$R_1$加上一个随机值生成(这些随机值都服从同一个正态分布)。因此，我们就要计算各个标的物之间的协方差，我们的计算公式就变成了:\n",
    "\n",
    "$$ VAR[P] = \\sigma_P^2 = \\sum_i \\omega_i^2\\sigma_i^2 + \\sum_i\\sum_{i\\neq j} \\omega_i\\omega_j\\sigma_i\\sigma_j\\rho_{i, j}, \\ i, j \\in \\lbrace 1, 2, \\cdots, N \\rbrace $$\n",
    "\n",
    "$\\rho_{i,j}$ 为$S_i$ 和 $S_j$的相关系数: $\\rho_{i, j} = \\frac{COV[S_i, S_j]}{\\sigma_i\\sigma_j}$.\n",
<<<<<<< HEAD
    "这个公式看起来极其复杂，但是我们通过使用NumPy数组可以轻松实现。首先，我们计算各个标的物的之间的协方差矩阵。"
=======
    "这个公式看起来极其复杂，但是我们通过使用Numpy数组可以轻松实现。首先，我们计算各个标的物的之间的协方差矩阵。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "cov_mat = np.cov(returns)\n",
    "print(cov_mat.shape)\n",
    "print(cov_mat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "np.sum(cov_mat.diagonal())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This array is not formatted particularly nicely, but a covariance matrix is a very important concept. The covariance matrix is of the form:\n",
    "\n",
    "$$ \\left[\\begin{matrix}\n",
    "VAR[S_1] & COV[S_1, S_2] & \\cdots & COV[S_1, S_N] \\\\\n",
    "COV[S_2, S_1] & VAR[S_2] & \\cdots & COV[S_2, S_N] \\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "COV[S_N, S_1] & COV[S_N, S_2] & \\cdots & VAR[S_N]\n",
    "\\end{matrix}\\right] $$\n",
    "\n",
    "So each diagonal entry is the variance of that asset at that index and each off-diagonal holds the covariance of two assets indexed by the column and row number. What is important is that once we have the covariance matrix we are able to do some very quick linear algebra to calculate the variance of the overall portfolio. We can represent the variance of the portfolio in array form as:\n",
    "\n",
    "$$ \\sigma_p^2 = \\omega \\ C \\ \\omega^\\intercal$$\n",
    "\n",
    "Where $C$ is the covariance matrix of all the assets and $\\omega$ is the array containing the weights of each individual asset. The superscript $\\intercal$ on the second $\\omega$ listed above denotes the **transpose** of $\\omega$. For a reference on the evaluation of the variance of a portfolio as a matrix equation, please see the Wikipedia article on [modern portfolio theory](https://en.wikipedia.org/wiki/Modern_portfolio_theory).\n",
    "\n",
    "The transpose of an array is what you get when you switch the rows and columns of an array. This has the effect of reflecting an array across what you might imagine as a diagonal. For example, take our array $A$ from before:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "协方差矩阵是一个非常重要的概念. 它的形式如下:\n",
    "\n",
    "$$ \\left[\\begin{matrix}\n",
    "VAR[S_1] & COV[S_1, S_2] & \\cdots & COV[S_1, S_N] \\\\\n",
    "COV[S_2, S_1] & VAR[S_2] & \\cdots & COV[S_2, S_N] \\\\\n",
    "\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
    "COV[S_N, S_1] & COV[S_N, S_2] & \\cdots & VAR[S_N]\n",
    "\\end{matrix}\\right] $$\n",
    "\n",
<<<<<<< HEAD
    "其主对角线的元素对应是每个标的物(股票)回报的方差，非主对角线的元素用行和列的序号来表示对应的两个标的物回报之间的协方差。值得注意的是：当我们拿到协方差矩阵后，可以通过线性代数的方法快速计算出投资组合回报的方差，计算公式为:\n",
    "\n",
    "$$ \\sigma_p^2 = \\omega \\ C \\ \\omega^\\intercal$$\n",
    "\n",
    "其中$C$是协方差矩阵，$\\omega$是股票权重值的矩阵，$\\omega^\\intercal$是$\\omega$的转置矩阵，关于投资组合回报的方差计算公式，可以参考wiki百科[modern portfolio theory](https://en.wikipedia.org/wiki/Modern_portfolio_theory).\n",
=======
    "其主对角线的元素对应是每个标的物(证券)收益的方差，非主对角线的元素用行和列的序号来表示对应的两个标的物收益之间的协方差。至关重要的一点就是：当我们拿到协方差矩阵后，可以通过线性代数的方法快速计算出投资组合收益的方差，计算公式为:\n",
    "\n",
    "$$ \\sigma_p^2 = \\omega \\ C \\ \\omega^\\intercal$$\n",
    "\n",
    "其中$C$是协方差矩阵，$\\omega$是证券的权重值，$\\omega^\\intercal$是$\\omega$的转置矩阵，关于投资组合收益的方差计算所用的矩阵表达式，可以参考wiki百科[modern portfolio theory](https://en.wikipedia.org/wiki/Modern_portfolio_theory).\n",
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "\n",
    "转置矩阵就是将行列互换得到的新矩阵，好比是沿着对角线做了镜像反转。我们来举个栗子好了：\n",
    "对于矩阵$A$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The transpose looks like a mirror image of the same array."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "转置后矩阵如同镜像翻转了一般。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(np.transpose(A))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But $\\omega$ here is a 1-dimensional array, a vector! It makes perfect to take the transpose of $A$, a $3 \\times 5$ array, as the output will be a $5 \\times 3$ array, but a 1-dimensional array is not quite as intuitive. A typical 1-dimensional array can be thought of as a $1 \\times n$ horizontal vector. Thus, taking the tranpose of this array essentially means changing it into a $n \\times 1$ vertical vector. This makes sense because 1-dimensional arrays are still arrays and any multiplication done between 1-dimensional and higher dimensional arrays must keep in line with our dimensionality issue of matrix multiplication.\n",
    "\n",
    "To make a long story short, we think of $\\omega$ as $1 \\times N$ since we have $N$ securities. This makes it so that $\\omega^\\intercal$ is $N \\times 1$. Again, our covariance matrix is $N \\times N$. So the overall multiplication works out like so, in informal terms:\n",
    "\n",
    "$$ \\text{Dimensions}(\\sigma_p^2) = \\text{Dimensions}(\\omega C \\omega^\\intercal) = (1 \\times N)\\cdot (N \\times N)\\cdot (N \\times 1) = (1 \\times 1)$$\n",
    "\n",
    "Multiplying the covariance matrix on the left by the plain horizontal vector and on the right by that vector's transpose results in the calculation of a single scalar ($1 \\times 1$) value, our portfolio's variance.\n",
    "\n",
    "So knowing this, let's proceed and calculate the portfolio variance! We can easily calculate the product of these arrays by using `dot()` for matrix multiplication, though this time we have to do it twice."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可 $\\omega$ 是一个一维数组，一个向量! 当我们对$A$进行转置操作时，它从 $3 \\times 5$ 的数组就变成了 $5 \\times 3$ 的数组，但是一维数组就没这么直观。通常我们把一维数组想象成一个 $1 \\times n$ 水平向量，当对它进行转置操作后，就变成一个 $n \\times 1$ 的垂直向量。这样就能保证当一维数组和更高纬度数组做矩阵乘法的时候，能遵守矩阵乘法的规则。\n",
    "\n",
<<<<<<< HEAD
    "简单来说，当我们有 $N$ 只股票时，我们把$\\omega$ 看做一个 $1 \\times N$的矩阵， $\\omega^\\intercal$ 的`shape`就是 $N \\times 1$.我们协方差矩阵的`shape`是 $N \\times N$。所以当进行完所有的矩阵乘法运算后：\n",
    "\n",
    "$$ \\text{Dimensions}(\\sigma_p^2) = \\text{Dimensions}(\\omega C \\omega^\\intercal) = (1 \\times N)\\cdot (N \\times N)\\cdot (N \\times 1) = (1 \\times 1)$$\n",
    "\n",
    "水平向量乘以协方差矩阵再乘以垂直向量后，结果就是一个标量($1 \\times 1$) -- 组合回报的方差。\n",
    "\n",
    "到这里，就是计算组合回报方差全过程!\n",
    "\n",
    "接下来我们只用使用两次`np.dot()`进行矩阵乘法运算，我们就大功告成了。"
=======
    "为了长话短说，当我们有 $N$ 只证券时，我们把$\\omega$ 看做一个 $1 \\times N$的矩阵， $\\omega^\\intercal$ 的`shape`就是 $N \\times 1$.我们协方差矩阵的`shape`是 $N \\times N$。所以当进行完所有的矩阵乘法运算后：\n",
    "\n",
    "$$ \\text{Dimensions}(\\sigma_p^2) = \\text{Dimensions}(\\omega C \\omega^\\intercal) = (1 \\times N)\\cdot (N \\times N)\\cdot (N \\times 1) = (1 \\times 1)$$\n",
    "\n",
    "水平向量乘以协方差矩阵再乘以垂直向量后，结果就是一个标量($1 \\times 1$) -- 组合收益的方差。\n",
    "\n",
    "到这里，就是计算组合收益方差全过程!接下来我们只用使用两次`np.dot()`进行矩阵乘法运算，我们就大功告成了。"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "print(\"shape of weights:\", weights.shape)\n",
    "print(\"shape of covariance:\", cov_mat.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "p_var = np.dot(np.dot(weights.T, cov_mat), weights)\n",
    "p_std_cov = np.sqrt(p_var)\n",
    "print(\"the portfolio std is:\", p_std_cov)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
<<<<<<< HEAD
    "collapsed": true,
=======
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Calculating the portfolio volatility(投资组合的波动率)\n",
    "var_p = np.dot(np.dot(weights, cov_mat), weights.T)\n",
    "vol_p = np.sqrt(var_p)\n",
    "print(\"Portfolio volatility: \", vol_p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To confirm this calculation, let's simply evaluate the volatility of the portfolio using only NumPy functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了验证计算结果，我们使用NumPy内置函数来计算投资组合的波动率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
<<<<<<< HEAD
   "metadata": {
    "collapsed": true
   },
=======
   "metadata": {},
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   "outputs": [],
   "source": [
    "# Confirming calculation\n",
    "vol_p_alt = np.sqrt(np.var(np.dot(weights, returns), ddof=1))\n",
    "print(\"Portfolio volatility: \", vol_p_alt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `ddof` parameter is a simple integer input that tells the function the number of degrees of freedom to take into account. This is a more statistical concept, but what this tells us that our matrix calculation is correct!\n",
    "\n",
    "A lot of this might not make sense at first glance. It helps to go back and forth between the theory and the code representations until you have a better grasp of the mathematics involved. It is definitely not necessary to be an expert on linear algebra and on matrix operations, but linear algebra can help to streamline the process of working with large amounts of data. For further reading on NumPy, check out the [documentation](https://docs.scipy.org/doc/numpy/user/index.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
<<<<<<< HEAD
    "`ddof`参数表示的是自由度。`np.var`默认的自由度为0，这里我设为1，告诉函数在自由度为1的情况下计算方差，这是一个统计学的概念，通过这个函数的计算，可以验证我们用矩阵的方法计算出的方差是没有问题的。\n",
    "\n",
    "今天介绍的很多东西初看似乎没有什么用，但是当你更好地掌握了这些数学概念之后，就能很好通过代码的形式去表现这些数学理论知识。我们并不要求你精通线性代数或者矩阵操作，但是线性代数能帮助优化数据的处理流程。更多的请查看NumPy使用指南：[点我查看](https://docs.scipy.org/doc/numpy/user/index.html)."
=======
    "The `ddof` parameter is a simple integer input that tells the function the number of degrees of freedom to take into account. This is a more statistical concept, but what this tells us that our matrix calculation is correct!\n",
    "\n",
    "A lot of this might not make sense at first glance. It helps to go back and forth between the theory and the code representations until you have a better grasp of the mathematics involved. It is definitely not necessary to be an expert on linear algebra and on matrix operations, but linear algebra can help to streamline the process of working with large amounts of data. For further reading on NumPy, check out the [documentation](https://docs.scipy.org/doc/numpy/user/index.html).\n",
    "\n",
    "`ddof`参数表示的是自由度`np.var`默认的自由度为0，这里我设为1，便可计算出带有相关性的方差，这是一个统计学的概念，虽然不好理解，但是能告诉我们，通过矩阵计算出的方差是没有问题的。\n",
    "\n",
    "今天介绍的很多东西看似没有什么用，但是当你更好地掌握了这些数学概念之后，你能很好通过代码的形式去表现这些数学理论知识。我们并不要求你精通线性代数或者矩阵操作，但是线性代数能帮助优化数据的处理流程。更多的请查看NumPy使用指南：[点我查看](https://docs.scipy.org/doc/numpy/user/index.html)."
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
<<<<<<< HEAD
   "version": "3.5.4"
=======
   "version": "3.6.2"
>>>>>>> 27de8dec4bde8c8ba17d90df1376c379bee91aa9
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}