Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate derrf distribution parameters on startup #9599

Merged
merged 1 commit into from
Jan 6, 2025

Conversation

larsevj
Copy link
Contributor

@larsevj larsevj commented Dec 19, 2024

Issue
Resolves #9523

  • PR title captures the intent of the changes, and is fitting for release notes.
  • Added appropriate release note label
  • Commit history is consistent and clean, in line with the contribution guidelines.
  • Make sure unit tests pass locally after every commit (git rebase -i main --exec 'pytest tests/ert/unit_tests -n logical -m "not integration_test"')

When applicable

  • When there are user facing changes: Updated documentation
  • New behavior or changes to existing untested code: Ensured that unit tests are added (See Ground Rules).
  • Large PR: Prepare changes in small commits for more convenient review
  • Bug fix: Add regression test for the bug
  • Bug fix: Create Backport PR to latest release

Copy link

codspeed-hq bot commented Dec 19, 2024

CodSpeed Performance Report

Merging #9599 will not alter performance

Comparing larsevj:validate_derrf_function (b51d234) with main (ab373dd)

Summary

✅ 24 untouched benchmarks

@codecov-commenter
Copy link

codecov-commenter commented Dec 19, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.71%. Comparing base (839610b) to head (b51d234).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9599      +/-   ##
==========================================
- Coverage   91.74%   91.71%   -0.03%     
==========================================
  Files         430      430              
  Lines       26639    26661      +22     
==========================================
+ Hits        24439    24452      +13     
- Misses       2200     2209       +9     
Flag Coverage Δ
cli-tests 39.69% <75.00%> (+<0.01%) ⬆️
everest-models-test 34.16% <0.00%> (-0.03%) ⬇️
gui-tests 71.82% <16.66%> (-0.04%) ⬇️
integration-test 38.26% <0.00%> (-0.04%) ⬇️
performance-tests 51.75% <16.66%> (+0.06%) ⬆️
test 39.58% <0.00%> (-0.03%) ⬇️
unit-tests 73.97% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -224,6 +224,32 @@ def _check_valid_triangular_parameters(prior: PriorDict) -> None:
).set_context(self.name)
)

def _check_valid_derrf_parameters(prior: PriorDict) -> None:
Copy link
Collaborator

@oyvindeide oyvindeide Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be outside the scope for this PR, but did you consider converting this from a function to a dataclass? Something like:

@pydantic.dataclass
class DERRF:
    name: str
    steps: PositiveInt
    min: float
    max: float
    skewness: float
    width: confloat(gt=0)

    @model_validator(mode="after")
    def validate_min_max(self) -> Self:
        if not self.max > self.min
            raise ValueError(f"Max ({self.max}) must be larger than max ({self.max})
        return self
 
    def transform(self, x: float) -> float:
        """
        Bin the result of `trans_errf` with `min=0` and `max=1` to closest of `nbins`
        linearly spaced values on [0,1]. Finally map [0,1] to [min, max].
        """
        q_values = np.linspace(start=0, stop=1, num=self.steps)
        q_checks = np.linspace(start=0, stop=1, num=self.steps + 1)[1:]
        y = TransformFunction.trans_errf(x, [0, 1, self.skewness, self.width])
        bin_index = np.digitize(y, q_checks, right=True)
        y_binned = q_values[bin_index]
        result = self.min + y_binned * (self.max - self.min)
        if result > self.max or result < self.min:
            warnings.warn(
                "trans_derff suffered from catastrophic loss of precision, clamping to min,max",
                stacklevel=1,
            )
            return np.clip(result, self.min, self.max)
        if np.isnan(result):
            raise ValueError(
                "trans_derrf returns nan, check that input arguments are reasonable"
            )
        return result

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not, but will look into it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this will require a storage migration?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I guess it will, though it would be possible to work around it for now, and only migrate once we do something like this for all parameter types. If that is the direction you are heading with rewriting parameter configs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest we merge this fix for now and then you can do this conversion later.

@larsevj larsevj force-pushed the validate_derrf_function branch from 5655414 to 711f8e7 Compare January 6, 2025 11:45
@larsevj larsevj added the release-notes:bug-fix Automatically categorise as bug fix in release notes label Jan 6, 2025
@larsevj larsevj force-pushed the validate_derrf_function branch from 711f8e7 to c3de39f Compare January 6, 2025 11:55
@larsevj larsevj force-pushed the validate_derrf_function branch from c3de39f to b51d234 Compare January 6, 2025 12:06
@larsevj larsevj merged commit 3cee24d into equinor:main Jan 6, 2025
41 checks passed
@larsevj larsevj deleted the validate_derrf_function branch January 6, 2025 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes:bug-fix Automatically categorise as bug fix in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid values for DERRF distribution leads to "float division by zero" in update
4 participants