Merge pull request #20 from gboeing/sp24

spring updates

.github/workflows/tests.yml

@@ -7,9 +7,7 @@ on:
     branches: [main]
 
 jobs:
-
   build:
-
     name: ${{ matrix.os }}
     runs-on: ${{ matrix.os }}
     strategy:
@@ -19,29 +17,28 @@ jobs:
 
     defaults:
       run:
-        shell: bash -l {0}
+        shell: bash -elo pipefail {0}
 
     steps:
-
       - name: Checkout repo
         uses: actions/checkout@v3
         with:
           fetch-depth: 2
 
-      - name: Setup Conda environment with Micromamba
-        uses: mamba-org/provision-with-micromamba@v14
+      - name: Create environment with Micromamba
+        uses: mamba-org/setup-micromamba@v1
         with:
           cache-downloads: true
-          cache-env: true
-          channels: conda-forge
-          channel-priority: strict
+          cache-environment: true
           environment-file: environment.yml
-          environment-name: ppd599
+          post-cleanup: none
 
       - name: Test environment
         run: |
-          python -m ipykernel install --sys-prefix --name ppd599 --display-name "Python (ppd599)"
-          conda list
-          conda info --all
+          python -m ipykernel install --sys-prefix --name ppde642 --display-name "Python (ppde642)"
           jupyter kernelspec list
           ipython -c "import osmnx; print('OSMnx version', osmnx.__version__)"
+
+      - name: Lint
+        run: |
+          SKIP=no-commit-to-branch pre-commit run --all-files

.gitignore

@@ -2,6 +2,7 @@
 data/*
 modules/*/*.gal
 modules/*/*.png
+modules/*/cache/*
 modules/*/keys.py
 syllabus/pdf/*.pdf
 

.pre-commit-config.yaml

@@ -0,0 +1,50 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: "v4.5.0"
+    hooks:
+      - id: check-added-large-files
+        args: [--maxkb=50]
+      - id: check-ast
+      - id: check-builtin-literals
+      - id: check-case-conflict
+      - id: check-docstring-first
+      - id: check-json
+      - id: check-merge-conflict
+        args: [--assume-in-merge]
+      - id: check-yaml
+      - id: debug-statements
+      - id: detect-private-key
+      - id: end-of-file-fixer
+      - id: fix-byte-order-marker
+      - id: mixed-line-ending
+      - id: no-commit-to-branch
+        args: [--branch, main]
+      - id: trailing-whitespace
+
+  - repo: https://github.com/pre-commit/mirrors-prettier
+    rev: "v3.0.3"
+    hooks:
+      - id: prettier
+        types_or: [markdown, yaml]
+
+  - repo: https://github.com/nbQA-dev/nbQA
+    rev: "1.7.1"
+    hooks:
+      - id: nbqa-isort
+        additional_dependencies: [isort]
+        args: [--line-length=100, --sl]
+      - id: nbqa-black
+        additional_dependencies: [black]
+        args: [--line-length=100]
+      - id: nbqa-flake8
+        additional_dependencies: [flake8]
+        args: [--max-line-length=100]
+
+  - repo: local
+    hooks:
+      - id: nbconvert
+        name: clear notebook output
+        entry: jupyter nbconvert
+        language: system
+        types: [jupyter]
+        args: ["--clear-output", "--inplace"]

LICENSE.txt

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2013-2023 Geoff Boeing, https://geoffboeing.com/
+Copyright (c) 2013-2024 Geoff Boeing, https://geoffboeing.com/
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,13 +1,11 @@
-[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/gboeing/ppd599/main?urlpath=lab)
-[![Build Status](https://github.com/gboeing/ppd599/workflows/tests/badge.svg?branch=main)](https://github.com/gboeing/ppd599/actions?query=workflow%3A%22tests%22)
+[![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/gboeing/ppde642/main?urlpath=lab)
+[![Build Status](https://github.com/gboeing/ppde642/workflows/tests/badge.svg?branch=main)](https://github.com/gboeing/ppde642/actions?query=workflow%3A%22tests%22)
 
-
-# PPD599: Advanced Urban Analytics
+# PPDE642: Advanced Urban Analytics
 
 This is the second part of a two-course series on **urban data science** that I teach at the **University of Southern California**'s Department of Urban Planning and Spatial Analysis.
 
-This course series takes a computational social science approach to working with urban data. It uses Python and Jupyter notebooks to introduce coding and statistical methods that students can reproduce and experiment with in the cloud. The series as a whole presumes no prior knowledge as it introduces coding, stats, spatial analysis, and applied machine learning from the ground up, but PPD599 assumes you have completed [PPD534](https://github.com/gboeing/ppd534) or its equivalent.
-
+This course series takes a computational social science approach to working with urban data. It uses Python and Jupyter notebooks to introduce coding and statistical methods that students can reproduce and experiment with in the cloud. The series as a whole presumes no prior knowledge as it introduces coding, stats, spatial analysis, and applied machine learning from the ground up, but PPDE642 assumes you have completed [PPD534](https://github.com/gboeing/ppd534) or its equivalent.
 
 ## Urban Data Science course series
 
@@ -17,13 +15,11 @@ The first course in the series, **PPD534**, starts with the basics of coding wit
 
 **PPD534**'s lecture materials are available on [GitHub](https://github.com/gboeing/ppd534) and interactively on [Binder](https://mybinder.org/v2/gh/gboeing/ppd534/main).
 
+### PPDE642: Advanced Urban Analytics
 
-### PPD599: Advanced Urban Analytics
-
-The second course, **PPD599**, assumes you have completed PPD534 (or its equivalent) and builds on its topics. It introduces spatial analysis, network analysis, spatial models, and applied machine learning. It also digs deeper into the tools and workflows of urban data science in both research and practice.
-
-**PPD599**'s lecture materials are available in this repo and interactively on [Binder](https://mybinder.org/v2/gh/gboeing/ppd599/main).
+The second course, **PPDE642**, assumes you have completed PPD534 (or its equivalent) and builds on its topics. It introduces spatial analysis, network analysis, spatial models, and applied machine learning. It also digs deeper into the tools and workflows of urban data science in both research and practice.
 
+**PPDE642**'s lecture materials are available in this repo and interactively on [Binder](https://mybinder.org/v2/gh/gboeing/ppde642/main).
 
 ## Not a USC student?
 

assignments/assignment2.md

@@ -6,9 +6,9 @@ You will clean, organize, describe, and visualize the data you downloaded in Ass
 
 Create a new Jupyter notebook. The first cell of your notebook should be markdown explaining what your research question and hypotheses are, where you found your data set, and what it contains. Given your proposed project:
 
-  1. Load your data set and clean/process it as needed.
-  1. Identify at least two variables of interest and calculate relevant descriptive statistics.
-  1. Using the techniques we learned in class, visualize interesting aspects of your data set. Create at least 4 visualizations using at least 3 different visualization types (e.g., scatterplots, barplots, maps, etc).
+1. Load your data set and clean/process it as needed.
+1. Identify at least two variables of interest and calculate relevant descriptive statistics.
+1. Using the techniques we learned in class, visualize interesting aspects of your data set. Create at least 4 visualizations using at least 3 different visualization types (e.g., scatterplots, barplots, maps, etc).
 
 Make sure your code is well-commented throughout for explanatory clarity. Your notebook should be well-organized into high-level sections using markdown headers representing the steps above, plus subheaders as needed. Each visualization should be followed by a markdown cell that explains what you are visualizing, why it is interesting, and why you made your specific graphical design decisions. What story does each visual tell? How does it enrich, confirm, or contradict the descriptive statistics you calculated earlier?
 

assignments/assignment4.md

@@ -6,9 +6,9 @@ You will conduct a spatial analysis using a spatial dataset (ideally the same on
 
 Create a new Jupyter notebook. The first cell of your notebook should be markdown explaining what your research question and hypotheses are, where you found your data set, and what it contains. Use geopandas to load your data set and clean/process it as needed. Make sure your code is well-commented throughout for explanatory clarity. Using the techniques we learned in class, do the following:
 
-  1. conduct a spatial analysis to look for hot/cold spots and assess spatial autocorrelation
-  1. compute spatial diagnostics to pick an appropriate spatial regression model
-  1. estimate and interpret a spatial regression model
+1. conduct a spatial analysis to look for hot/cold spots and assess spatial autocorrelation
+1. compute spatial diagnostics to pick an appropriate spatial regression model
+1. estimate and interpret a spatial regression model
 
 Your notebook should be separated into high-level sections using markdown headers representing the steps above. Each section should conclude with a markdown cell that succinctly explains your analysis/visuals, why you set it up the way you did, and how you interpret its results. Your notebook should conclude with a markdown cell that explains 1) what evidence does this analysis provide for your research question and hypothesis, 2) what is the "big picture" story, and 3) how can planners or policymakers use this finding.
 

assignments/final-project.md

@@ -8,27 +8,27 @@ The final project is a cumulative assignment that requires you to use the skills
 
 Identify a conference of interest and familiarize yourself with their paper submission requirements. You might consider the following conferences, among others:
 
-  - Transportation Research Board (TRB)
-  - Association of Collegiate Schools of Planning (ACSP)
-  - American Planning Association's National Planning Conference (APA)
-  - American Association of Geographers (AAG)
-  - Urban Affairs Association (UAA)
+- Transportation Research Board (TRB)
+- Association of Collegiate Schools of Planning (ACSP)
+- American Planning Association's National Planning Conference (APA)
+- American Association of Geographers (AAG)
+- Urban Affairs Association (UAA)
 
 Develop an urban research question that fits with the themes of your chosen conference. Develop a research design to answer this question, then collect data, clean and organize it, visualize it, and analyze it.
 
 Write a conference paper organized into five sections:
 
-  1. introduction: provide a short (3 paragraph) summary of the study's importance, methods, and findings/implications (1 paragraph each)
-  2. background: explain the context of your study and provide a short lit review of relevant related work to establish what is known and what urgent open questions remain
-  3. methods: present your data and your analysis methods with sufficient detail that a reader could reproduce your study
-  4. results: present your findings and include supporting visuals
-  5. discussion: circle back to your research question, interpret your findings, and discuss their importance and how planners or policymakers could use them to improve some aspect of urban living
+1. introduction: provide a short (3 paragraph) summary of the study's importance, methods, and findings/implications (1 paragraph each)
+2. background: explain the context of your study and provide a short lit review of relevant related work to establish what is known and what urgent open questions remain
+3. methods: present your data and your analysis methods with sufficient detail that a reader could reproduce your study
+4. results: present your findings and include supporting visuals
+5. discussion: circle back to your research question, interpret your findings, and discuss their importance and how planners or policymakers could use them to improve some aspect of urban living
 
 Format your paper according to the conference's guidelines. For the purposes of this course, your paper must be at least 3000 words in length (not including tables, figures, captions, or references). It must include the following, at a minimum:
 
-  - a table of descriptive statistics
-  - a table of spatial regression or machine learning model results
-  - 4 aesthetically-pleasing figures containing data visualizations including at least 1 map
+- a table of descriptive statistics
+- a table of spatial regression or machine learning model results
+- 4 aesthetically-pleasing figures containing data visualizations including at least 1 map
 
 You are strongly encouraged, but not required, to actually submit this paper to the conference.
 

assignments/mini-lecture.md

@@ -8,21 +8,21 @@ This exercise is intended to be informal and an opportunity for self-discovery.
 
 Instructions:
 
-  - Pick a method listed in the syllabus for those weeks or covered in the reading material.
-  - Learn how the method works by reading the week's reading material.
-  - Practice the method in your own notebook on your own data.
-  - Google for additional usage examples and further information.
-  - Prepare a mini-lecture notebook that would take 8-10 minutes to present that 1) briefly introduces why someone would use the method and how it works (~2 minutes), 2) demonstrates in code how to use the method for a simple data analysis (~5 minutes), 3) summarizes what the analysis revealed (~2 minutes).
+- Pick a method listed in the syllabus for those weeks or covered in the reading material.
+- Learn how the method works by reading the week's reading material.
+- Practice the method in your own notebook on your own data.
+- Google for additional usage examples and further information.
+- Prepare a mini-lecture notebook that would take 8-10 minutes to present that 1) briefly introduces why someone would use the method and how it works (~2 minutes), 2) demonstrates in code how to use the method for a simple data analysis (~5 minutes), 3) summarizes what the analysis revealed (~2 minutes).
 
-8 minutes is not a lot of time, so keep your lecture notebook simple and brief. Have a clean dataset ready to go at the beginning of your lecture. Do not show us a lot of preparatory steps setting things up in your  notebook. Jump right into the analysis that demonstrates your method.
+8 minutes is not a lot of time, so keep your lecture notebook simple and brief. Have a clean dataset ready to go at the beginning of your lecture. Do not show us a lot of preparatory steps setting things up in your notebook. Jump right into the analysis that demonstrates your method.
 
 You will be graded according to the following. In your notebook, did you:
 
-  - summarize why someone would use this method and how it works, at a high-level
-  - demonstrate the method with a simple data analysis
-  - summarize what your analysis revealed
-  - keep it all succinct
+- summarize why someone would use this method and how it works, at a high-level
+- demonstrate the method with a simple data analysis
+- summarize what your analysis revealed
+- keep it all succinct
 
 Make sure your notebook runs from the top without any errors (i.e., restart the kernel and run all cells) and that all the output can be seen inline without me having to re-run your notebook. Via Blackboard, submit your notebook and data files, all zipped as a single file, named `LastName_FirstName_Lecture.zip`. If your submission file exceeds Blackboard's maximum upload size limit, you may provide a Google Drive link to your zipped data in the comment field when you submit.
 
-Note that if you pick a supervised learning method, your assignment is due prior to class in module 11. If you pick an unsupervised learning method, your assignment is due prior to class in module 12. The "presentation" is pretend: you are just creating the lecture notebook you would have presented, and submitting it via Blackboard *before that module's class session* begins.
+Note that if you pick a supervised learning method, your assignment is due prior to class in module 11. If you pick an unsupervised learning method, your assignment is due prior to class in module 12. The "presentation" is pretend: you are just creating the lecture notebook you would have presented, and submitting it via Blackboard _before that module's class session_ begins.

environment.yml

@@ -1,36 +1,38 @@
-name: ppd599
+name: ppde642
 
 channels:
   - conda-forge
+  - pytorch
 
 dependencies:
   - beautifulsoup4
-  - black
   - cartopy
   - cenpy
-  - conda
   - contextily
-  - dill
-  - flake8
   - folium
-  - gensim
   - geopandas
-  - isort
   - jupyterlab
   - mapclassify
-  - osmnx=1.3.0
-  - nbqa
-  - nltk
+  - osmnx=1.8.1
   - pandana
   - pandas
+  - pre-commit
   - pysal
-  - python=3.10.*
+  - python=3.11.*
   - rasterio
-  - rtree
   - seaborn
   - scikit-learn
   - scipy
   - statsmodels
+
+  # computer vision and NLP
+  - gensim
+  - nltk
+  - pillow
+  - pytorch
+  - torchvision
+
+  # others (unused)
   # bokeh
   # datashader
   # holoviews