Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SB125 Fund Split Analysis and website #1255

Merged
merged 77 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
503890d
start of notebook
csuyat-dot Aug 8, 2024
944e084
testing reading in some fund request sheets from GCS, renaming cols a…
csuyat-dot Aug 8, 2024
5596d89
added list of file names, made function to read in/lightly clean data…
csuyat-dot Aug 9, 2024
721a575
testing loop to extract basename from excel files
csuyat-dot Aug 9, 2024
7270bc8
starting to look at individual fund request templates, will flag whic…
csuyat-dot Aug 13, 2024
923a294
created function that loops through list of files to determine if any…
csuyat-dot Aug 14, 2024
ef8c0db
update function that returns tuple of list of good and bad files. muc…
csuyat-dot Aug 14, 2024
ac5507a
created a function that creates a dictionary of file names and cleane…
csuyat-dot Aug 14, 2024
6cb2d28
used loop to see each dataframe in one go, see what needs to be edite…
csuyat-dot Aug 14, 2024
9344277
added more steps to cleaning script to forward fill NaN values with p…
csuyat-dot Aug 14, 2024
ee27264
cleaned DFs from good list that needed minor tweaking (removed some r…
csuyat-dot Aug 14, 2024
82a8206
tested concat-ing all the dictionary dataframe values into a single DF
csuyat-dot Aug 14, 2024
4f99f9f
setting up cells to sum all the capital and operations columns
csuyat-dot Aug 14, 2024
6949a06
more cleaning to remove unnecessary rows from indiviaul DFs
csuyat-dot Aug 15, 2024
2b8515d
more cleaning and removing rows. testing melting to DF to be more tid…
csuyat-dot Aug 15, 2024
7d12618
more cleaning. testing .groupby().agg()
csuyat-dot Aug 15, 2024
a8f4eb1
testing altair charts
csuyat-dot Aug 15, 2024
a8ad11c
started to explore the fund request files that need extra work. mappe…
csuyat-dot Aug 16, 2024
d670aa0
still cleaing the bad files. using a lot of .read_excel() arguments
csuyat-dot Aug 16, 2024
2c580ca
cleaned all the bad DFs except kern and ventura. was able to concat n…
csuyat-dot Aug 16, 2024
1c1b2b4
minor fixes. about to start cleanign ventura and kern
csuyat-dot Aug 16, 2024
e588ee9
splitting ventura data into multiple tables
csuyat-dot Aug 19, 2024
d2badc3
sucessfully merged the ventura TIRCP capital and operating DFs
csuyat-dot Aug 19, 2024
a59e3ce
cleaning the ventura zetcp capital DF, about to merge with rest of ve…
csuyat-dot Aug 19, 2024
b6c8143
finished ventura DF, working on Kern, started function that reads in …
csuyat-dot Aug 19, 2024
a35f731
completed cleaning kern data. concat all the little DFs, added to oth…
csuyat-dot Aug 19, 2024
22927e1
trying to save as parquet, but fund souce column has some int from Me…
csuyat-dot Aug 19, 2024
75216c3
trying out different altair chart configs
csuyat-dot Aug 20, 2024
3f13cd1
adjusted values in merced and humboldt so parquet can save out, also …
csuyat-dot Aug 20, 2024
59af39c
cleaned up some cleaning steps to not depend on numeric row index, re…
csuyat-dot Aug 20, 2024
9b5194c
started to update the cleaning steps into individual functions. also …
csuyat-dot Aug 20, 2024
9c17580
completed converting cleaning functions for good DFs, now starting to…
csuyat-dot Aug 20, 2024
030490e
ran everything in the NB, everything still works even with the new fu…
csuyat-dot Aug 20, 2024
b915986
minor changes
csuyat-dot Aug 21, 2024
891ff5d
continued making functions out of the cleaning steps for problem DFs.…
csuyat-dot Sep 3, 2024
94179e6
completed adding all cleaning functions to script. also made function…
csuyat-dot Sep 3, 2024
3c101a3
minor changes
csuyat-dot Sep 3, 2024
307364c
minor fixes to script. did full run, no errors. good to go
csuyat-dot Sep 3, 2024
76cd649
removed all the old work and test functions. now reading in final cle…
csuyat-dot Sep 3, 2024
5cb23c1
testing chart sizes and adding data labels
csuyat-dot Sep 3, 2024
324e504
still experimenting with altair charts
csuyat-dot Sep 3, 2024
14ecc14
reorganizing some chart
csuyat-dot Sep 4, 2024
1bfd49e
more cleaning in the NB
csuyat-dot Sep 4, 2024
69ece76
added chart showing funding amount by funding source
csuyat-dot Sep 5, 2024
59de1d8
created charts showing the top RTPA that requetsed the most and shows…
csuyat-dot Sep 5, 2024
4051de1
testing some charts for individual agencies for top RTPAs
csuyat-dot Sep 5, 2024
e75bfea
still trying to chart agenciesby RTPAs
csuyat-dot Sep 5, 2024
1ee3ace
got chart of all RTPAs and their agencies by project type and FY
csuyat-dot Sep 6, 2024
aa6f63a
minor changes
csuyat-dot Sep 6, 2024
5ca3ae8
tryign to get data labels on big chart. but looks like stacked bar ch…
csuyat-dot Sep 10, 2024
5b26b36
got independent x labels for line charts, but seems like have to to k…
csuyat-dot Sep 10, 2024
41c1587
added titles to charts
csuyat-dot Sep 10, 2024
c94bb09
fixing titles on the big charts
csuyat-dot Sep 10, 2024
4afd853
made data labels black
csuyat-dot Sep 11, 2024
23913ce
fixed operating charts so its actually charting op data and not cap
csuyat-dot Sep 11, 2024
89fc5fd
added data labels to other charts
csuyat-dot Sep 12, 2024
b8ed4c7
brining in rtpa/district/county crosswalk data. also testing to add d…
csuyat-dot Sep 16, 2024
1157d45
started to test having a drop down filter on the faceted charts
csuyat-dot Sep 18, 2024
c882085
tryign to deal with white space and filter position
csuyat-dot Sep 18, 2024
0d73aaf
minor cleaning to plot visuals. added points to line charts
csuyat-dot Sep 19, 2024
58c015d
trying to add filter selector to other charts
csuyat-dot Sep 20, 2024
e718bcd
adjusted plot size
csuyat-dot Sep 24, 2024
4a503d4
set default filter to VCTC, RTPA with the most agencies
csuyat-dot Sep 24, 2024
c3d1d5b
moved some charts around, started work on making a gradient line char…
csuyat-dot Sep 25, 2024
ff18f6f
made a side-by-side chart of opX vs capX by agencies in RTPAs. per am…
csuyat-dot Sep 25, 2024
62b985a
swapped axis on alot of charts to be more horizontal
csuyat-dot Sep 25, 2024
0c301f5
get HTML code to move filter to the side of plots. change side-by-sid…
csuyat-dot Sep 26, 2024
49f2ad6
updated axis names and titles on side-by-side chart
csuyat-dot Sep 26, 2024
3f03fde
renamed initial NB to explore. started new NB to be the final publish…
csuyat-dot Oct 7, 2024
ad970cd
started to organize the NB by sorting and commenting the variables. a…
csuyat-dot Oct 8, 2024
e94dd1b
started README.md in preparation to make jupyter book
csuyat-dot Oct 11, 2024
47e2f2f
initial draft of read me compelted
csuyat-dot Oct 11, 2024
9f6a63c
added site .yml to portfolio/sites folder
csuyat-dot Oct 11, 2024
9a14399
added captions to each chart and added currency format to .mark_text()
csuyat-dot Oct 14, 2024
eee1478
minor adjustment, swapped capx and opx chart postiion
csuyat-dot Oct 14, 2024
b10efc1
first build/deploy of NB to portfolio site
csuyat-dot Oct 14, 2024
1afac11
changed title of NB to be different from the README title, also respo…
csuyat-dot Oct 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Git LFS file not shown
33 changes: 33 additions & 0 deletions portfolio/sb125_fund_split_analysis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# SB125 Fund Split Analysis
As a condition of receiving SB125 (TIRCP and ZETCP programs) funds, RTPAs are required to complete an Initial Allocation Package describing their full four-year plan for use of these formulaic funds.

Agencies or RTPAs are requried to submit a summary Excel table (`SB 125 Fund Request.xlsx`) that explains the proposed uses of TIRCP and ZETCP funds by fiscal year of availability. This table is subdivided between funding for capital projects and funding for operations expenses, with each project and its implementing agency specifically identified, and each operator that will receive operations funding also specifically identified.

This analysis aims to answer the question of "of all the fund request tables received, how much money is allocated to operating expenses and capital projects?"

Answering these questions will help inform CalSTA, Caltrans and other stakeholders of how agencies plan to use these funds to fulfill the objectives of SB125 and transform transit in California.


## Definitions
- SB125: Senate Bill-125 Transportation budget trailer bill
- TIRCP: Transit and Intercity Rail Capital Program
- ZETCP: Zero-Emission Transit Capital Program
- RTPA: Regional transportation planning agencies
- CalSTA: California State Transportation Agency
- State Fiscal Year: July 1 to June 30


## Methodology
Excel tables were cleaned and prepared to a uniform format. Fund data was then aggregated by RTPA, implementing agency and state fiscal year.


## Data Sources
All fund request excel tables were extracted from all available allocation packages.


## Who We Are
This website was created by the [California Department of Transportation](https://dot.ca.gov/)'s Division of Data and Digital Services. We are a group of data analysts and scientists who analyze transportation data, such as General Transit Feed Specification (GTFS) data, or data from funding programs such as the Active Transportation Program. Our goal is to transform messy and indecipherable original datasets into usable, customer-friendly products to better the transportation landscape. For more of our work, visit our [portfolio](https://analysis.calitp.org/).

<img src="https://raw.githubusercontent.com/cal-itp/data-analyses/main/portfolio/Calitp_logo_MAIN.png" alt="Alt text" width="200" height="100"> <img src="https://raw.githubusercontent.com/cal-itp/data-analyses/main/portfolio/CT_logo_Wht_outline.gif" alt="Alt text" width="129" height="100">

<br>Caltrans®, the California Department of Transportation® and the Caltrans logo are registered service marks of the California Department of Transportation and may not be copied, distributed, displayed, reproduced or transmitted in any form without prior written permission from the California Department of Transportation.
43 changes: 43 additions & 0 deletions portfolio/sb125_fund_split_analysis/_config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Book settings
# Learn more at https://jupyterbook.org/customize/config.html

title: SB125 Fund Split Analysis
author: Cal-ITP
copyright: "2024"
#logo: calitp_logo_MAIN.png

# Force re-execution of notebooks on each build.
# See https://jupyterbook.org/content/execute.html
execute:
execute_notebooks: 'off'
allow_errors: false
timeout: -1

# Define the name of the latex output file for PDF builds
latex:
latex_documents:
targetname: book.tex

launch_buttons:
binderhub_url: "https://mybinder.org"
jupyterhub_url: "https://hubtest.k8s.calitp.jarv.us"
thebe: true

repository:
url: https://github.com/cal-itp/data-analyses/ # Online location of your book
# path_to_book: docs # Optional path to your book, relative to the repository root
path_to_book: sb125_analyses/sb125_fund_split_analysis
branch: main # Which branch of the repository should be used when creating links (optional)

# Add GitHub buttons to your book
# See https://jupyterbook.org/customize/config.html#add-a-link-to-your-repository
html:
use_issues_button: true
use_repository_button: true
use_edit_page_button: true
google_analytics_id: 'G-JCX3Z8JZJC'

sphinx:
config:
html_js_files:
- https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js
6 changes: 6 additions & 0 deletions portfolio/sb125_fund_split_analysis/_toc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
format: jb-book
parts:
- caption: null
chapters:
- file: 00__sb125_fund_split_analysis__.ipynb
root: README
7 changes: 7 additions & 0 deletions portfolio/sites/sb125_fund_split_analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
title: SB125 Fund Split Analysis
directory: ./sb125_analyses/sb125_fund_split_analysis/
readme: ./sb125_analyses/sb125_fund_split_analysis/README.md
parts:
- caption: SB125 Fund Split Analysis
- chapters:
- notebook: ./sb125_analyses/sb125_fund_split_analysis/sb125_fund_split_analysis.ipynb
33 changes: 33 additions & 0 deletions sb125_analyses/sb125_fund_split_analysis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# SB125 Fund Split Analysis
As a condition of receiving SB125 (TIRCP and ZETCP programs) funds, RTPAs are required to complete an Initial Allocation Package describing their full four-year plan for use of these formulaic funds.

Agencies or RTPAs are requried to submit a summary Excel table (`SB 125 Fund Request.xlsx`) that explains the proposed uses of TIRCP and ZETCP funds by fiscal year of availability. This table is subdivided between funding for capital projects and funding for operations expenses, with each project and its implementing agency specifically identified, and each operator that will receive operations funding also specifically identified.

This analysis aims to answer the question of "of all the fund request tables received, how much money is allocated to operating expenses and capital projects?"

Answering these questions will help inform CalSTA, Caltrans and other stakeholders of how agencies plan to use these funds to fulfill the objectives of SB125 and transform transit in California.


## Definitions
- SB125: Senate Bill-125 Transportation budget trailer bill
- TIRCP: Transit and Intercity Rail Capital Program
- ZETCP: Zero-Emission Transit Capital Program
- RTPA: Regional transportation planning agencies
- CalSTA: California State Transportation Agency
- State Fiscal Year: July 1 to June 30


## Methodology
Excel tables were cleaned and prepared to a uniform format. Fund data was then aggregated by RTPA, implementing agency and state fiscal year.


## Data Sources
All fund request excel tables were extracted from all available allocation packages.


## Who We Are
This website was created by the [California Department of Transportation](https://dot.ca.gov/)'s Division of Data and Digital Services. We are a group of data analysts and scientists who analyze transportation data, such as General Transit Feed Specification (GTFS) data, or data from funding programs such as the Active Transportation Program. Our goal is to transform messy and indecipherable original datasets into usable, customer-friendly products to better the transportation landscape. For more of our work, visit our [portfolio](https://analysis.calitp.org/).

<img src="https://raw.githubusercontent.com/cal-itp/data-analyses/main/portfolio/Calitp_logo_MAIN.png" alt="Alt text" width="200" height="100"> <img src="https://raw.githubusercontent.com/cal-itp/data-analyses/main/portfolio/CT_logo_Wht_outline.gif" alt="Alt text" width="129" height="100">

<br>Caltrans®, the California Department of Transportation® and the Caltrans logo are registered service marks of the California Department of Transportation and may not be copied, distributed, displayed, reproduced or transmitted in any form without prior written permission from the California Department of Transportation.
Loading
Loading