Python for Social Scientists

title: Python for Social Scientists author: name: Renee Chu twitter: "@reneighbor" output: slides.html controls: true

--

Python for Social Scientists

Tutorial, PyCon 2014 Montreal

--

Intended audience:

People learning to code who have completed a Python workshop or an online class

--

Prereqs:

python
matplotlib
numpy
text editor

--

Survey of programming levels in the audience

--

My background

Econ major, liberal arts college
No coding at school
First job in sales and support
Learned coding through workshops (Railsbridge and PyLadies) and online (Stanford Engineering Everywhere, Learn Python the Hard Way)

--

Advice I received:

Pick a project you're passionate about.

--

Hacking on data is:

Interesting
Widely available
Easier setup than web

--

Answer some interesting questions:

Should development resources be spent on family planning or fighting disease?

http://www.tubechop.com/watch/2507136

--

What would we need to be able to verify this on an empirical level for at least one country?

Childhood mortality across years
Births per woman across years

--

World Bank Data Indicators page

Thousands of indicators
214 Countries
1960-2012
http://data.worldbank.org/indicator

--

Answer some other interesting questions:

--

Answer some other interesting questions:

Who has more mobile phone subscriptions per 100 people:

Finland
United States

--

Mobile phone subscriptions per 100 people, 1960-2011

Source: World Bank, http://data.worldbank.org/indicator/IT.CEL.SETS.P2

--

Answer some interesting questions:

Who has more mobile phone subscriptions per 100 people:

Finland
United States
El Salvador

--

Mobile phone subscriptions per 100 people, 1960-2011

--

Plans are cheaper in lower-income countries...

US: $59.00/mo
Finland: $40.10/mo
India: $12.90/mo

Unbundled total package (voice+sms+data) plans available to individual consumers. (No El Salvador data available.) Open Technology Initiative (http://newamerica.net/publications/policy/an_international_comparison_of_cell_phone_plans_and_prices)

--

Prices are much cheaper within-network (incentive for multiple accounts)

Tigo El Salvador "Basic" mobile postpaid, per min:

Between Tigo: $0.08
Other networks: $0.13
Landlines: $0.13
To USA/Canda: $0.09

http://www.tigo.com.sv/planes-pospago

--

Today we're going to:

Import CSV data into Python
Find a MatPlotLib example
Pipe our CSV data into MatPlotLib

--

1. Import CSV data into Python

--

Get started:

In your folder for personal projects:

git clone https://github.com/reneighbor/python-for-social-scientists.git

--

What's here?

python_for_social_scientists/

data/
- fertility.csv
- childhood_deaths.csv
read_data.py
chart_csv.py

--

Running a dead-simple program

Open up the folder
Open up "read_data.py"
Type:

print "Hello world"

--

Find your new file in your terminal

cd Projects/personal-projects/programming-for-social-scientists
python read_data.py

You should see "Hello World" spit back at you

--

Follow these steps:

[Erase the print statement]

import csv

libraries are a bunch of functions and helpers written by other people

--

Follow these steps:

import csv

csvfile = open('data/childhood_deaths.csv', 'rU')

--

Follow these steps:

import csv

csvfile = open('data/childhood_deaths.csv', 'rU')
reader = csv.DictReader(csvfile)

DictReader lets you traverse the contents the csv file like a dictionary

--

This is a Python dictionary

{
	'Country Name': 'Finland', 
	'Country Code': 'FIN', 
	'2007': '114.924474', 
	'2008': '128.4719884',
	'2009': '144.1530224'
}

--

* This is a Python dictionary

{
	'2007': '114.924474',
	'2009': '144.1530224' 
	'Country Name': 'Finland', 
	'2008': '128.4719884',
	'Country Code': 'FIN',
}

--

Follow these steps:

import csv

csvfile = open('data/childhood_deaths', 'rU')
reader = csv.DictReader(csvfile)

for row in reader:
	print row

--

Follow these steps:

Run it! In your terminal:

cd Projects/personal-projects/programming-for-social-scientists
python read_data.py

--

Follow these steps:

import csv

csvfile = open('data/childhood_deaths', 'rU')
reader = csv.DictReader(csvfile)

for row in reader:
	if row['Country Name'] == "Finland":
		print row

--

Any questions?

--

What are some things we observe about the output?

--

What are some things we observe about the output?

Not sorted
Data in strings
Every row

--

2. Find a MatPlotLib example

--

Find something to copy

http://matplotlib.org/gallery.html

--

Examine the code-- What do we know?

First half is drawing the rects, second is labels
Values the list

--

Run the sample code

Create a new file "mens_womens.py" inside python_for_social_scientists
In your terminal run "python mens_womens.py"
Do you get the chart?
Edit the values for "menMeans" and "womensMeans". Do you see a change?

--

MatPlotLib Bar chart simplified

Look at basic_chart.py

--

In your terminal:

python basic_chart.py

--

In your text editor:

How do we get the other countries data?

In basic_chart.py add 2 series, el_salvador_data and usa_data
Uncomment the commented-out lines
Run it again

--

Piping in the CSV data

open chart-csv.py
What needs to be done in order to extract data?

--

Piping in the CSV data

What needs to be done in order to extract data?

Turn data from strings into ints
Stip out county names
Sort by year

--

Give it a try

-- Break

--

Share what you've written

--

Comparing 2 countries

How would you edit chart_csv to graph multiple countries?

--

Comparing 2 countries

run extractData() on 2 other countries
comment out the rectangle-drawing for those countries

--

Comparing 2 indicators

What do we need to do to compare mortality against fertility?

--

Comparing 2 indicators

What do we need to do to compare mortality against fertility?

Take in 2 CSV readers
Take in only one country
rename "rects" to be indicators instead of countries

--

From Github:

programming_for_social_scientists compare_indicators_starter.py

Re-arranged to accomediate two comparison-drawing functions and a main()

--

Give it a try

-- El Salvador 1960-2013

--

Public service announcement:

correlation != causation

--

What we've done:

Imported a CSV and turned it into a dict
Went to MatPlotLib and found a bar chart to borrow
Drew a series comparing countries from one CSV
Drew a series comparing indicators from two CSVs

--

Your Turn

Pick a data set that interests you.
Write code to visualize it.
Teach us something!

Files

README.md

Latest commit

History

README.md

File metadata and controls