Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Soiling algorithm updates #435

Open
wants to merge 53 commits into
base: development
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
c8aae89
Matt’s updates to SRR algorithm (detect negative shifts in soiling ra…
Jun 24, 2024
e1401a6
committing updates to merge with aggregated_filters_for_trials
Jul 22, 2024
f23a497
Making sure there will be no merge conflicts
Jul 22, 2024
30f5548
Merge remote-tracking branch 'remotes/origin/aggregated_filters_for_t…
Jul 22, 2024
9e3a411
Improvements in order to pass checks and pytesting
Jul 31, 2024
9e3d89d
Merge pull request #417 from noromo01/development
martin-springer Aug 2, 2024
c08a0e9
Merge remote-tracking branch 'origin/aggregated_filters_for_trials' i…
martin-springer Aug 2, 2024
35a3ec9
formatting conftest.py and soiling_test.py
Aug 2, 2024
3fdf0b0
fixing formatting
Aug 5, 2024
669ec75
lint soiling.py
martin-springer Aug 6, 2024
23710a0
lint line length
martin-springer Aug 6, 2024
fa1d79b
revert TrendAnalysis notebook changes
martin-springer Aug 6, 2024
d642210
revert conftest.py
martin-springer Aug 6, 2024
9122b56
revert notebook requirements
martin-springer Aug 6, 2024
cd4fbb6
added piecewise and neg_shift PI data back to conftest.py
Aug 7, 2024
e9a2552
formatting fixes
Aug 8, 2024
612c9f1
minor formatting issue in soiling.py
Aug 10, 2024
0f020b5
testing some changes to pass notebook checks
Aug 13, 2024
6d5ce23
trying another minor change for notebook checks
Aug 13, 2024
b99c2de
soiling.py change to pass notebook checks
Aug 13, 2024
ab28608
Trying some changes in the notebooks to pass tests
Aug 16, 2024
2dbbeae
Fixing pytests and reverting notebooks
Aug 17, 2024
febe693
undoing some black formatting
Aug 19, 2024
ca7627b
cleaning up formatting redundancies in soiling_test.py
Aug 19, 2024
8b3fa4a
reformatting soiling.py and minor reformatting to soiling_test.py
Aug 19, 2024
efa5042
run black on soiling.py
martin-springer Aug 21, 2024
21da67d
fixing flake8 formatting
Aug 21, 2024
5ef6c81
fixing flake8 formatting
Aug 21, 2024
e66c295
removing _collapse_cleaning_events so half_norm_clean results are not…
Aug 21, 2024
628cfe8
fixing notebook failures
Aug 22, 2024
3860ada
Switching back code towards final review version
Aug 27, 2024
f64077a
fixing minor formatting
Aug 27, 2024
7517ddc
added fix for bare except and pytests for segmented_soiling_period fu…
Oct 24, 2024
cee6105
updated changelogs
Oct 24, 2024
9734118
Merge remote-tracking branch 'origin/aggregated_filters_for_trials' i…
Oct 24, 2024
cfd5858
deleted updates in wrong changelogs
Oct 24, 2024
6172ea0
fixed lint error
Oct 24, 2024
693c570
remove soiling experimental warning label
martin-springer Nov 6, 2024
b96ebcc
Merge branch 'development' into qnguyen345-bare_except_error
martin-springer Nov 6, 2024
e5159d7
update changelog
martin-springer Nov 6, 2024
fcac604
rerun notebooks
martin-springer Nov 6, 2024
6346480
Merge remote-tracking branch 'github-desktop-qnguyen345/bare_except_e…
martin-springer Nov 8, 2024
fd4dc47
fix double quotations left over from merge
martin-springer Nov 8, 2024
dc46cf6
remove double import
martin-springer Nov 8, 2024
8492873
remove experimental soiling warnings from plotting
martin-springer Nov 8, 2024
7df691c
remove warnings import
martin-springer Nov 8, 2024
b0c4510
quick fix mean runtimewarning
martin-springer Nov 8, 2024
48f02dc
re-run notebooks
martin-springer Nov 8, 2024
0299804
fix linting in soiling
martin-springer Nov 8, 2024
08e0090
update nbval and remove semicolons from nb's
martin-springer Nov 11, 2024
ec105a3
re-run notebooks
martin-springer Nov 11, 2024
5ded716
nbval update sanitize-with argument
martin-springer Nov 11, 2024
9b1fd4c
Merge branch 'development' into qnguyen345-bare_except_error
martin-springer Dec 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/nbval.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Run notebook and check output
run: |
# --sanitize-with: pre-process text to remove irrelevant differences (e.g. warning filepaths)
pytest --nbval --sanitize-with docs/nbval_sanitization_rules.cfg docs/${{ matrix.notebook-file }}
pytest --nbval --nbval-sanitize-with docs/nbval_sanitization_rules.cfg docs/${{ matrix.notebook-file }}
- name: Run notebooks again, save files
run: |
pip install nbconvert[webpdf]
Expand Down
275 changes: 117 additions & 158 deletions docs/TrendAnalysis_example.ipynb

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions docs/TrendAnalysis_example_NSRDB.ipynb

Large diffs are not rendered by default.

328 changes: 75 additions & 253 deletions docs/degradation_and_soiling_example.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/sphinx/source/changelog/v2.2.0-beta.2.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ Requirements
Contributors
------------
* Martin Springer (:ghuser:`martin-springer`)
* Michael Deceglie (:ghuser:`mdeceglie`)
* Michael Deceglie (:ghuser:`mdeceglie`)
12 changes: 11 additions & 1 deletion docs/sphinx/source/changelog/v3.0.0-beta.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,17 @@ when compared with older versions of RdTools

* Upgrade pvlib 0.9.0 to 0.11.0 (:pull:`428`)

* Upgrade soiling algorithms SRR and CODS. Remove experimental warning label. (:pull:`426`)

Enhancements
------------
* Added a new wrapper function for clearsky filters (:pull:`412`)
* Improve test coverage, especially for the newly added filter capabilities (:pull:`413`)
* Added codecov.yml configuration file (:pull:`420`)
* Added new methods perfect_clean_complex and inferred_clean_complex which detects negative shifts and piecewise changes in the slope for soiling detection in :py:func:`~rdtools.soiling.soiling_srr`(:pull:`426`)
* Availability module no longer considered experimental (:pull:`429`)
* Add capability to seed the CircularBlockBootstrap (:pull:`429`)
* Allow sub-daily aggregation in :py:func:`~rdtools.degradation.degradation_year_on_year` (:pull:`390`)
* Allow sub-daily aggregation in :py:func:`~rdtools.degradation.degradation_year_on_year` (:pull:`390`)

Bug fixes
---------
Expand All @@ -31,11 +34,15 @@ Bug fixes
* Deploy workflow was replaced with trusted publisher workflow for pypi (:pull:`427`)
* Fix pandas 2.0.0 deprications and update syntax changes (:pull:`428`)
* Fix numpy 2.0.0 deprications and update syntax changes (:pull:`428`)
* Fixed pylint bare except error for :py:func:`~rdtools.soiling.segmented_soiling_period`
in ``soiling.py`` (:pull:`432`)

Tests
-----
* Testing matrix was updated to include python = [3.9, 3.10, 3.11, 3.12] (:pull:`428`)
* nbval sanitization rules were added for date and time stamp (:pull:`428`)
* Added pytests to cover invalid segementations for
:py:func:`~rdtools.soiling.segmented_soiling_period` in ``soiling_cods_test.py`` (:pull:`432`)

Documentation
-------------
Expand Down Expand Up @@ -184,5 +191,8 @@ Contributors
* Martin Springer (:ghuser:`martin-springer`)
* Michael Deceglie (:ghuser:`mdeceglie`)
* Kirsten Perry (:ghuser:`kperrynrel`)
* Matthew Muller (:ghuser:`mmuller`)
* Noah Moyer (:ghuser:`noromo01`)
* Quyen Nguyen (:ghuser:`qnguyen345`)
* Dirk Jordan (:ghuser:`dirkjordan`)
* Chris Deline (:ghuser:`cdeline`)
12 changes: 6 additions & 6 deletions docs/system_availability_example.ipynb

Large diffs are not rendered by default.

45 changes: 6 additions & 39 deletions rdtools/plotting.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import pandas as pd
import plotly.express as px
import numpy as np
import warnings


def degradation_summary_plots(yoy_rd, yoy_ci, yoy_info, normalized_yield,
Expand Down Expand Up @@ -133,15 +132,10 @@ def soiling_monte_carlo_plot(soiling_info, normalized_yield, point_alpha=0.5,
profile_alpha=0.05, ymin=None, ymax=None,
profiles=None, point_color=None,
profile_color='C1'):
'''
"""
Create figure to visualize Monte Carlo of soiling profiles used in the SRR
analysis.

.. warning::
The soiling module is currently experimental. The API, results,
and default behaviors may change in future releases (including MINOR
and PATCH releases) as the code matures.

Parameters
----------
soiling_info : dict
Expand All @@ -168,13 +162,7 @@ def soiling_monte_carlo_plot(soiling_info, normalized_yield, point_alpha=0.5,
Returns
-------
fig : matplotlib.figure.Figure
'''
warnings.warn(
'The soiling module is currently experimental. The API, results, '
'and default behaviors may change in future releases (including MINOR '
'and PATCH releases) as the code matures.'
)

"""
fig, ax = plt.subplots()
renormalized = normalized_yield / soiling_info['renormalizing_factor']
ax.plot(renormalized.index, renormalized, 'o', alpha=point_alpha,
Expand All @@ -197,14 +185,9 @@ def soiling_monte_carlo_plot(soiling_info, normalized_yield, point_alpha=0.5,
def soiling_interval_plot(soiling_info, normalized_yield, point_alpha=0.5,
profile_alpha=1, ymin=None, ymax=None,
point_color=None, profile_color=None):
'''
"""
Create figure to visualize valid soiling profiles used in the SRR analysis.

.. warning::
The soiling module is currently experimental. The API, results,
and default behaviors may change in future releases (including MINOR
and PATCH releases) as the code matures.

Parameters
----------
soiling_info : dict
Expand All @@ -228,13 +211,7 @@ def soiling_interval_plot(soiling_info, normalized_yield, point_alpha=0.5,
Returns
-------
fig : matplotlib.figure.Figure
'''
warnings.warn(
'The soiling module is currently experimental. The API, results, '
'and default behaviors may change in future releases (including MINOR '
'and PATCH releases) as the code matures.'
)

"""
sratio = soiling_info['soiling_ratio_perfect_clean']
fig, ax = plt.subplots()
renormalized = normalized_yield / soiling_info['renormalizing_factor']
Expand All @@ -249,14 +226,9 @@ def soiling_interval_plot(soiling_info, normalized_yield, point_alpha=0.5,


def soiling_rate_histogram(soiling_info, bins=None):
'''
"""
Create histogram of soiling rates found in the SRR analysis.

.. warning::
The soiling module is currently experimental. The API, results,
and default behaviors may change in future releases (including MINOR
and PATCH releases) as the code matures.

Parameters
----------
soiling_info : dict
Expand All @@ -268,12 +240,7 @@ def soiling_rate_histogram(soiling_info, bins=None):
Returns
-------
fig : matplotlib.figure.Figure
'''
warnings.warn(
'The soiling module is currently experimental. The API, results, '
'and default behaviors may change in future releases (including MINOR '
'and PATCH releases) as the code matures.'
)
"""

soiling_summary = soiling_info['soiling_interval_summary']
fig, ax = plt.subplots()
Expand Down
Loading
Loading