Docs enhancements

.gitignore

@@ -1,6 +1,5 @@
 *.pyc
 private.py
-*.ipynb*
 private/
 # Setuptools distribution folder.
 /dist/

README.md

@@ -1,262 +1,27 @@
 # swmmio
 *v0.7.3 (2024/12/02)*
 
-![example workflow](https://github.com/aerispaha/swmmio/actions/workflows/python-app.yml/badge.svg)
-[![Documentation Status](https://readthedocs.org/projects/swmmio/badge/?version=latest)](https://swmmio.readthedocs.io/en/latest/?badge=latest)
-
-![Kool Picture](docs/img/impact_of_option.png?raw=true "Impact of Option")
-`swmmio` is a set of python tools aiming to provide a means for version control and visualizing results from the EPA Stormwater Management Model (SWMM). Command line tools are also provided for running models individually and in parallel via Python's `multiprocessing` module. These tools are being developed specifically for the application of flood risk management, though most functionality is applicable to SWMM modeling in general.
-
-
-### Prerequisites
-`swmmio` functions primarily by interfacing with .inp and .rpt (input and report) files produced by SWMM. Functions within the `run_models` module rely on a SWMM5 engine which can be downloaded [here](https://www.epa.gov/water-research/storm-water-management-model-swmm).
-
-### Installation:
-Before installation, it's recommended to first activate a [virtualenv](https://github.com/pypa/virtualenv) to not crowd your system's package library. If you don't use any of the dependencies listed above, this step is less important. SWMMIO can be installed via pip in your command line:
-
-```bash
-pip install swmmio
-```
-
-### Basic Usage
-The `swmmio.Model()` class provides the basic endpoint for interfacing with SWMM models. To get started, save a SWMM5 model (.inp) in a directory with its report file (.rpt). A few examples:   
-```python
-import swmmio
-
-#instantiate a swmmio model object
-mymodel = swmmio.Model('/path/to/directory with swmm files')
-
-# Pandas dataframe with most useful data related to model nodes, conduits, and subcatchments
-nodes = mymodel.nodes.dataframe
-links = mymodel.links.dataframe
-subs = mymodel.subcatchments.dataframe
-
-#enjoy all the Pandas functions
-nodes.head()
-```
-<table border=1 class=dataframe><thead><tr style=text-align:right><th><th>InvertElev<th>MaxDepth<th>SurchargeDepth<th>PondedArea<th>Type<th>AvgDepth<th>MaxNodeDepth<th>MaxHGL<th>MaxDay_depth<th>MaxHr_depth<th>HoursFlooded<th>MaxQ<th>MaxDay_flood<th>MaxHr_flood<th>TotalFloodVol<th>MaximumPondDepth<th>X<th>Y<th>coords<tr><th>Name<th><th><th><th><th><th><th><th><th><th><th><th><th><th><th><th><th><th><th><tbody><tr><th>S42A_10.N_4<td>13.506673<td>6.326977<td>5.0<td>110.0<td>JUNCTION<td>0.69<td>6.33<td>19.83<td>0<td>12:01<td>0.01<td>0.20<td>0.0<td>11:52<td>0.000<td>6.33<td>2689107.0<td>227816.000<td>[(2689107.0, 227816.0)]<tr><th>D70_ShunkStreet_Trunk_43<td>8.508413<td>2.493647<td>5.0<td>744.0<td>JUNCTION<td>0.04<td>0.23<td>8.74<td>0<td>12:14<td>NaN<td>NaN<td>NaN<td>NaN<td>NaN<td>NaN<td>2691329.5<td>223675.813<td>[(2691329.5, 223675.813)]<tr><th>TD61_1_2_90<td>5.150000<td>15.398008<td>0.0<td>0.0<td>JUNCTION<td>0.68<td>15.40<td>20.55<td>0<td>11:55<td>0.01<td>19.17<td>0.0<td>11:56<td>0.000<td>15.40<td>2698463.5<td>230905.720<td>[(2698463.5, 230905.72)]<tr><th>D66_36.D.7.C.1_19<td>19.320000<td>3.335760<td>5.0<td>6028.0<td>JUNCTION<td>0.57<td>3.38<td>22.70<td>0<td>12:00<td>0.49<td>6.45<td>0.0<td>11:51<td>0.008<td>3.38<td>2691999.0<td>230309.563<td>[(2691999.0, 230309.563)]</table>
-
-```python
-#write to a csv
-nodes.to_csv('/path/mynodes.csv')
-
-#calculate average and weighted average impervious
-avg_imperviousness = subs.PercImperv.mean()
-weighted_avg_imp = (subs.Area * subs.PercImperv).sum() / len(subs)
-```
-
-### Nodes and Links Objects 
-Specific sections of data from the inp and rpt can be extracted with `Nodes` and `Links` objects. 
-Although these are the same object-type of the `swmmio.Model.nodes` and `swmmio.Model.links`, 
-accessing them directly allows for custom control over what sections of data are retrieved. 
-
-```python
-from swmmio import Model, Nodes
-m = Model("coolest-model.inp")
-
-# pass custom init arguments into the Nodes object instead of using default settings referenced by m.nodes() 
-nodes = Nodes(
-    model=m, 
-    inp_sections=['junctions', 'storage', 'outfalls'],
-    rpt_sections=['Node Depth Summary', 'Node Inflow Summary'],
-    columns=[ 'InvertElev', 'MaxDepth', 'InitDepth', 'SurchargeDepth', 'MaxTotalInflow', 'coords']
-)
-
-# access data 
-nodes.dataframe 
-```
-
-### Generating Graphics
-Create an image (.png) visualization of the model. By default, pipe stress and node flood duration is visualized if your model includes output data (a .rpt file should accompany the .inp).
-
-```python
-swmmio.draw_model(mymodel)
-```
-
-![Default Draw Output](docs/img/default_draw.png?raw=true "Sewer Stress, Node Flooding")
-
-Use pandas to calculate some interesting stats, and generate a image to highlight
-what's interesting or important for your project:
-
-```python
-#isolate nodes that have flooded for more than 30 minutes
-flooded_series = nodes.loc[nodes.HoursFlooded>0.5, 'TotalFloodVol']
-flood_vol = sum(flooded_series) #total flood volume (million gallons)
-flooded_count = len(flooded_series) #count of flooded nodes
-
-#highlight these nodes in a graphic
-nodes['draw_color'] = '#787882' #grey, default node color
-nodes.loc[nodes.HoursFlooded>0.5, 'draw_color'] = '#751167' #purple, flooded nodes
-
-#set the radius of flooded nodes as a function of HoursFlooded
-nodes.loc[nodes.HoursFlooded>1, 'draw_size'] = nodes.loc[nodes.HoursFlooded>1, 'HoursFlooded'] * 12
+_Programmatic pre and post processing for EPA Stormwater Management Model (SWMM)_
 
-#make the conduits grey, sized as function of their geometry
-conds['draw_color'] = '#787882'
-conds['draw_size'] = conds.Geom1
 
-#add an informative annotation, and draw:
-annotation = 'Flooded Volume: {}MG\nFlooded Nodes:{}'.format(round(flood_vol), flooded_count)
-swmmio.draw_model(mymodel, annotation=annotation, file_path='flooded_anno_example.png')
-```
-![Flooded highlight](docs/img/flooded_anno_example.png?raw=true "Node Flooding with annotation")
-
-### Building Variations of Models
-Starting with a base SWMM model, other models can be created by inserting altered data into a new inp file. Useful for sensitivity analysis or varying boundary conditions, models can be created using a fairly simple loop, leveraging the `modify_model` package.
-
-For example, climate change impacts can be investigated by creating a set of models with varying outfall Fixed Stage elevations:
-
-```python
-import os
-import swmmio
-
-#initialize a baseline model object
-baseline = swmmio.Model(r'path\to\baseline.inp')
-rise = 0.0 #set the starting sea level rise condition
-
-#create models up to 5ft of sea level rise.
-while rise <= 5:
-
-    #create a dataframe of the model's outfalls
-    outfalls = baseline.inp.outfalls
-
-    #create the Pandas logic to access the StageOrTimeseries column of  FIXED outfalls
-    slice_condition = outfalls.OutfallType == 'FIXED', 'StageOrTimeseries'
-
-    #add the current rise to the outfalls' stage elevation
-    outfalls.loc[slice_condition] = pd.to_numeric(outfalls.loc[slice_condition]) + rise
-    baseline.inp.outfalls = outfalls
-
-    #copy the base model into a new directory    
-    newdir = os.path.join(baseline.inp.dir, str(rise))
-    os.mkdir(newdir)
-    newfilepath = os.path.join(newdir, baseline.inp.name + "_" + str(rise) + '_SLR.inp')
-
-    #Overwrite the OUTFALLS section of the new model with the adjusted data
-    baseline.inp.save(newfilepath)
-
-    #increase sea level rise for the next loop
-    rise += 0.25
-
-```
-
-### Access Model Network
-The `swmmio.Model` class returns a Networkx MultiDiGraph representation of the model via that `network` parameter:
-```python
-
-#access the model as a Networkx MutliDiGraph
-G = model.network
-
-#iterate through links
-for u, v, key, data in model.network.edges(data=True, keys=True):
-
-        print (key, data['Geom1'])
-        # do stuff with the network
-```  
-
-### Running Models
-Using the command line tool, individual SWMM5 models can be run by invoking the swmmio module in your shell as such:
-```
-$ python -m swmmio --run path/to/mymodel.inp
-```
-If you have many models to run and would like to take advantage of your machine's cores, you can start a pool of simulations with the `--start_pool` (or `-sp`) command. After pointing `-sp` to one or more directories, swmmio will search for SWMM .inp files and add all them to a multiprocessing pool. By default, `-sp` leaves 4 of your machine's cores unused. This can be changed via the `-cores_left` argument.
-```
-$ #run all models in models in directories Model_Dir1 Model_Dir2
-$ python -m swmmio -sp Model_Dir1 Model_Dir2  
-
-$ #leave 1 core unused
-$ python -m swmmio -sp Model_Dir1 Model_Dir2  -cores_left=1
-```
-<div class="warning">
-    <p class="first admonition-title">Warning</p>
-    <p class="last">Using all cores for simultaneous model runs can put your machine's CPU usage at 100% for extended periods of time. This probably puts stress on your hardware. Use at your own risk.</p>
-</div>
-
-
-
-### Flood Model Options Generation
-swmmio can take a set of independent storm flood relief (SFR) alternatives and combine them into every combination of potential infrastructure changes. This lays the ground work for identifying the most-efficient implementation sequence and investment level.
-
-Consider the simplified situaiton where a city is interested in solving a flooding issue by installing new relief sewers along Street A and/or Street B. Further, the city wants to decide whether they should be 1 or 2 blocks long. Engineers then decide to build SWMM models for 4 potential relief sewer options:
-*  A1 -> One block of relief sewer on Street A
-*  A2 -> Two blocks of relief sewer on Street A
-*  B1 -> One block of relief sewer on Street B
-*  B2 -> Two blocks of relief sewer on Street B
-
-To be comprehensive, implementation scenarios should be modeled for combinations of these options; it may be more cost-effective, for example, to build relief sewers on one block of Street A and Street B in combination, rather than two blocks on either street independently.
-
-swmmio achieves this within the version_control module. The `create_combinations()` function builds models for every logical combinations of the segmented flood mitigation models. In the example above, models for the following scenarios will be created:
-*  A1 with B1
-*  A1 with B2
-*  A2 with B1
-*  A2 with B2
+![workflow status](https://github.com/aerispaha/swmmio/actions/workflows/python-app.yml/badge.svg)
+[![Documentation Status](https://readthedocs.org/projects/swmmio/badge/?version=latest)](https://swmmio.readthedocs.io/en/latest/?badge=latest)
 
-For the `create_combinations()` function to work, the model directory needs to be set up as follows:
-```
-├───Baseline
-        baseline.inp
-├───Combinations
-└───Segments
-    ├───A
-    │   ├───A1
-    │   │   A1.inp
-    │   └───A2
-    │       A2.inp
-    └───B
-        ├───B1
-        │   B1.inp
-        └───B2
-            B2.inp
-```
-The new models will be built and saved within the Combinations directory. `create_combinations()` needs to know where these directories are and optionally takes version_id and comments data:
 
-```python
-#load the version_control module
-from swmmio.version_control import version_control as vc
+![image](_static/img/flooded_anno_example.png)
 
-#organize the folder structure
-baseline_dir = r'path/to/Baseline/'
-segments_dir = r'path/to/Segments/'
-target_dir = r'path/to/Combinations/'
 
-#generate flood mitigation options
-vc.create_combinations(
-    baseline_dir,
-    segments_dir,
-    target_dir,
-    version_id='initial',
-    comments='example flood model generation comments')
-```
+## Introduction
+`swmmio` is a Python tool for engineers and hydrologists who need to supercharge their ability to modify and analyze EPA SWMM models and results. Using a familiar Pandas interface, users can replace manual procesess that used to live in spreadsheets with scripts and automation.
 
-The new models will be saved in subdirectories within the `target_dir`. New models (and their containing directory) will be named based on a concatenation of their parent models' names. It is recommended to keep parent model names as concise as possible such that child model names are manageable. After running `create_combinations()`, your project directory will look like this:
-```
-├───Baseline
-├───Combinations
-│   ├───A1_B1
-│   ├───A1_B2
-│   ├───A2_B1
-│   └───A2_B2
-└───Segments
-    ├───A
-    │   ├───A1
-    │   └───A2
-    └───B
-        ├───B1
-        └───B2
+The core `swmmio.Model` object provides accessors to related elements in the INP and RPT. For example, `swmmio.Model.subcatchments` provides a DataFrame (or GeoDataFrame) joining data from the `[SUBCATCHMENTS]` and `[SUBAREAS]` tables in the model.inp file and, if available, the `Subcatchment Runoff Summary` from the model.rpt file. 
 
-```
+Additionally, `swmmio` provides a lower-level API for reading and writing (almost) all of the sections of the model.inp file which is useful for programmatically modifying EPA SWMM models.
 
-### SWMM Model Version Control
-To add more segments to the model space, create a new segment directory and rerun the `create_combinations()` function. Optionally include a comment summarizing how the model space is changing:
-```python
-vc.create_combinations(
-    baseline_dir,
-    alternatives_dir,
-    target_dir,
-    version_id='addA3',
-    comments='added model A3 to the scope')
-```
-The `create_combinations()` function can also be used to in the same way to propogate a change in an existing segment (parent) model to all of the children. Version information for each model is stored within a subdirectory called `vc` within each model directory. Each time a model is modified from the `create_combinations()` function, a new "BuildInstructions" file is generated summarizing the changes. BuildInstructions files outline how to recreate the model with respect to the baseline model.
 
+## Installation
+```bash
+pip install swmmio
+``` 
 
-TO BE CONTINUED...
+For documentation and tutorials, see our [documentation](https://swmmio.readthedocs.io/). 

docs/changelog.md

@@ -0,0 +1,5 @@
+# Changelog
+
+```{include} ../CHANGELOG.md
+
+```

docs/conf.py

@@ -19,6 +19,7 @@
 import sys
 import swmmio
 
+
 sys.path.insert(0, os.path.abspath('../../swmmio'))
 
 # -- Project information -----------------------------------------------------
@@ -43,14 +44,27 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'sphinx.ext.autodoc',
-    'sphinx.ext.coverage',
-    'sphinx.ext.mathjax',
-    'sphinx.ext.viewcode',
-    'sphinx.ext.autosummary',
-    'myst_parser',
+    "sphinx.ext.autosummary",
+    "sphinx.ext.autodoc",
+    "IPython.sphinxext.ipython_console_highlighting",
+    "IPython.sphinxext.ipython_directive",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.mathjax",
+    "sphinx.ext.todo",
+    "sphinx.ext.coverage",
+    "sphinx.ext.viewcode",
+    "sphinx_copybutton",
+    "myst_nb",
+    "numpydoc",
 ]
 
+myst_enable_extensions = ["colon_fence"]
+myst_heading_anchors = 3
+
+# sphinx-copybutton configurations
+copybutton_prompt_text = r">>> |\.\.\. |\$ |In \[\d*\]: | {2,5}\.\.\.: | {5,8}: "
+copybutton_prompt_is_regexp = True
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 
@@ -59,7 +73,9 @@
 #
 source_suffix = {
     '.rst': 'restructuredtext',
-    '.md': 'markdown',
+    '.md': 'myst-nb',
+    ".ipynb": "myst-nb",
+    ".myst": "myst-nb",
 }
 
 # The master toctree document.
@@ -75,7 +91,12 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = []
+exclude_patterns = [
+    # 'usage/visualizing_models.ipynb', 
+    # 'usage/making_art_with_swmm.ipynb'
+]
+
+# nb_execution_mode = "cache"
 
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
@@ -93,7 +114,12 @@
 # further.  For a list of options available for each theme, see the
 # documentation.
 #
-# html_theme_options = {}
+html_theme_options = {
+    "github_url": "https://github.com/pyswmm/swmmio",
+    "navbar_align": "content",
+    "navigation_depth": 4,
+    "collapse_navigation": True,
+}
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -185,3 +211,11 @@
 # A list of files that should not be packed into the epub file.
 epub_exclude_files = ['search.html']
 
+
+numpydoc_class_members_toctree = False
+
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/3/", None),
+    "pandas": ("https://pandas.pydata.org/pandas-docs/stable", None),
+    "pyswmm": ("https://pyswmm.github.io/pyswmm/", None),
+}