python_foundations.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Foundations of python\n",
    "\n",
    "This notebook introduces students to some of the most common parts of the python programming language. It is intended as a *cheat sheet* for students to refer to when working on the batch processing script. Look through the aspects of python below and have a go at running each cell. Modify the code (eg. add new variables or function) and run again to help you understand how the computer is interpreting and acting on the instructions.\n",
    "\n",
    "\n",
    "## Introduction\n",
    "\n",
    "\n",
    "This is a ***jupyter notebook***. It is neat a way to have runable python code alongside notes and embeded graphs and outputs from the code. Notebooks are commonly used to develop bits of research code. The notebook is made up of code cells, which you can edit and run, and markdown cells, where text can be written.\n",
    "\n",
    "The cell below includes a bit of python code. Click on the box and then press the ***Run*** button at the top of the screen to see it run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Hello world')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The basics\n",
    "\n",
    "The cell below gives a brief overview of some of the most common bits of syntax in python. It consists of a set of comments (which the computer will not try to run as commands), and a set of commands, where you tell the computer what you want it to do.\n",
    "\n",
    "The code below sets some ***variables***, prints those out and does some maths on it. Note that there are two broad types of ***variables*** below; numbers (known as ***floats*** or ***integers*** and words (known as ***strings*** or ***characters***). Python treats these different data types differently, allowing you to build up complex programs.\n",
    "\n",
    "It is **very important** that you make sure you keep track what type each piece of data is saved as and only use the right type of data in operations. For example, $\\sqrt{x}$ would not mean anything if ***x=\"cabbage\"***.\n",
    "\n",
    "\n",
    "**Click and run each of the code blocks below to see what it does. Have a go at modifying values and rerunning. Make sure you understand how the code is making the computer perform tasks**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Examples of the basic aspects of python3\n",
    "'''\n",
    "\n",
    "# Comments can be behind \"#\"\n",
    "'''Or behind three quotation marks'''\n",
    "# use # for comments in text, and ''' for descriptions of functions\n",
    "\n",
    "\n",
    "# print something to the screen\n",
    "print(\"Welcome to python3\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# set a variable. The variable will hold this value until you over-write it.\n",
    "x=2.718281828459045\n",
    "\n",
    "\n",
    "# write variable to screen\n",
    "print(x)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# set a \"string\", ie a word. Note we have overwritten \"x\" set earlier and \n",
    "# changed it from a number to a word\n",
    "x=\"This is a word\"\n",
    "\n",
    "\n",
    "# print a string variable to the screen\n",
    "print(x)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Multiple variables can be printed out\n",
    "x=2.718281828459045\n",
    "y=6.52\n",
    "z=\"a note\"\n",
    "print(z,x,y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# basic maths can be done in python\n",
    "x=2.718281828459045\n",
    "y=3.2\n",
    "z=x*y\n",
    "print(x,\"times\",y,\"equals\",z)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Arrays and lists\n",
    "\n",
    "We can also store lists of things in a single variable, which is a useful way of passing sets of data around (it would get very tedious to have to define thousands of separate variables to hold every brightness value from a 1 Mpixel photograph)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''\n",
    "Basics of lists and \n",
    "arrays in python3\n",
    "'''\n",
    "\n",
    "\n",
    "# We can also store lists of things in a single variable\n",
    "x=[1,2,3,4,5]\n",
    "\n",
    "# This can be used to pass around large amounts of data without needing thousands of variables\n",
    "print(x)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# it can be accessed by telling pythong with \"element\" you want\n",
    "print(x[0])\n",
    "\n",
    "\n",
    "# we can also count from the back\n",
    "print(x[-1])\n",
    "print(x[-2])\n",
    "\n",
    "\n",
    "# Note that the elements start from \"0\" and go, in this case, to 4.\n",
    "print(x[0],x[4])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# lists can be of any type\n",
    "x=[\"This\",\"is\",\"now\",\"a\",\"string\"]\n",
    "\n",
    "\n",
    "# or even a mix\n",
    "x=[1,2,\"sausage\",4,5]\n",
    "print(x)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many other ways to hold large amounts of data in python. eg. dictionaries and tuples, but lists are perhaps the most common.\n",
    "\n",
    "In GIS and EO, we often want to hold large arrays of numbers this can be efficiently done using a \"numpy\" array, which have been specifically designed for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "# set some data (most often we will read from file)\n",
    "x=np.array([4,3,2,1])\n",
    "\n",
    "\n",
    "# It can be accessed in a similar way to lists\n",
    "print(x[0],x[2])\n",
    "\n",
    "\n",
    "# but now we can perform maths on the whole array\n",
    "y=x/2\n",
    "print(\"original\",x,\"modified\",y)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# arrays can also be multi-dimensional. Let us make one full of \"-1\"\n",
    "x=np.full((100,100),-1)\n",
    "\n",
    "# the \"np.full()\" command tells it to make an array full of a given value\n",
    "# the \"(100,100)\" tells it to make it 100*100 elements\n",
    "# the \"-1\" tells it what value to fill the array with\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can store large amounts of data in a single variable. Let us see how we can analyse that."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plotting data\n",
    "\n",
    "Often we need to visualise data. Python has a very useful plotting package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# import a useful plotting package.\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "# let us read some data in.\n",
    "import numpy as np\n",
    "filename='data/practice/practice_data.csv'  # this is the name of a file on the disk\n",
    "x,y=np.loadtxt(filename,delimiter=',',unpack=True,dtype=float,comments=\"#\")\n",
    "\n",
    "\n",
    "# set the labels\n",
    "plt.xlabel('x')\n",
    "plt.ylabel('y')\n",
    "\n",
    "# tell it which data to plot, and what format. '.' means point\n",
    "plt.plot(x,y,'.')\n",
    "\n",
    "# print to screen\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# or we plot with a line\n",
    "plt.plot(x,y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Program flow: Loops, functions and if's\n",
    "\n",
    "## Functions\n",
    "So far we have written \"monolithic\" code. We can create reusable chunks of code called \"functions\". We have already been using these from some packages (eg ***plt.plot()***), but can create our own.\n",
    "\n",
    "To create a function, define it, the populate it with commands with a set number of indentations, as in the cell below. We can then reuse this function throughout our code. This allows us to build libraries of tools which we can reuse throughout our work for decades to come.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def myFunc(x):   # function name and input variables\n",
    "  '''Describe the function here'''\n",
    "  y=x*x       # do some analysis\n",
    "  return(y)   # pass the results back\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What happens when you run the cell above?\n",
    "\n",
    "The answer is hopefully nothing. Here you have defined a set of instructions for the computer, but you have not asked it to carry out those instructions. You can think of defining a function a bit like planning a hillwalking route. Everything needed to complete the walk is there in the instructions, but the walking (the time consuming part) does not start until you set off.\n",
    "\n",
    "We set off a function by ***calling*** it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# set some variable ready to pass to our function\n",
    "x=2\n",
    "\n",
    "# call our function\n",
    "res=myFunc(x)\n",
    "\n",
    "# write the results to screen\n",
    "print(\"The result of\",x,'times',x,\"is\",res)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The important thing to understand with functions is what data goes in (*the bit inside the brackets after the function name*) and what data comes out (*the bit in the return*). If you pass the wrong data to or from a function, it will not give you an accurate result.\n",
    "\n",
    "For the function above, it takes a single variable which it will then try to times by itself. This must be a number (as a word times a word has no meaning). **Importantly** whatever data you pass to the function will be renamed within it. So the variable names inside the code can be different from those outside the code. This lets you iteravely call a function and pass a whole list of variables to it (eg. a filename list).\n",
    "\n",
    "It is good practice to group all commands in to functions to make them easier to rerun and your code more reusable. We will make extensive use functions in the biomass mapping exercise."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loops\n",
    "\n",
    "\n",
    "The real power of a computer comes from its ability to perform repetative tasks quickly we can \"loop\" over a set of tasks, repeating them one by one.\n",
    "\n",
    "Below is an example of simple loop:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# make a list of numbers\n",
    "numList=[1,2,3,4,5,6,7,8,9]\n",
    "\n",
    "# the loop below will set the variable, i, equal to each element\n",
    "# of the list, numList, then carry out whatever instructions are\n",
    "# indented within the loop\n",
    "for i in numList:\n",
    "  print(i)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are also inbut functions, such as ***range()***, which can make lists of numbers between given bounds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# loop over a list made by the range() function\n",
    "for i in range(0,10):\n",
    "  print(i)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is an example of reading data through a function and then looping over it to write the elements out one by one.\n",
    "\n",
    "Here ***len(x)*** returns the length of the array. This way we can read different sized data files and the code will automatically scale the loop to fit the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def readData(filename):\n",
    "  '''A function to read some data'''\n",
    "  x,y=np.loadtxt(filename,delimiter=',',unpack=True,dtype=float,comments=\"#\")\n",
    "  return(x,y)\n",
    "\n",
    "x,y=readData('data/practice/practice_data.csv')\n",
    "\n",
    "# loop over the length of the array\n",
    "for i in range(0,len(x)):\n",
    "  print(x[i],y[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## If's\n",
    "\n",
    "We won't use these in the course today, but they are included for completeness. Feel free to skip to the second notebook now. If you want to learn more python, read on.\n",
    "\n",
    "An if statement lets us control the flow of a program, allowing it to do different things in different conditions. This allows our program to make \"decisions\", making it more flexible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "z=0.34\n",
    "threshold=0.5\n",
    "\n",
    "if(z>threshold):\n",
    "  print(\"z is bigger than\",threshold,z)\n",
    "else:\n",
    "  print(\"z is smaller than\",threshold,z)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can be used to change the behaviour of the program based on the data that is passed to it. For example, the code below will only print out elements of a dataset if they are more than a certain number of standard deviations from the mean.\n",
    "\n",
    "Here we haves made use of the ***mean()*** and ***std()*** functions within the ***numpy*** package to calculate the mean and standard deviation of our data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read the data\n",
    "x,y=np.loadtxt(filename,delimiter=',',unpack=True,dtype=float,comments=\"#\")\n",
    "\n",
    "\n",
    "# eg, only print values more than 2 stdandard deviations from the mean\n",
    "meanY=np.mean(y)\n",
    "stdevY=np.std(y)\n",
    "# set thresholds\n",
    "minThresh=meanY-2*stdevY\n",
    "maxThresh=meanY+2*stdevY\n",
    "\n",
    "# only write those values\n",
    "for i in range(0,len(x)):\n",
    "  if( (y[i]<minThresh) | (y[i]>maxThresh)):\n",
    "    print(x[i],y[i])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}