Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace ecl2df with res2df #244

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/fmu-ensemble.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:

- name: 📦 Install test dependencies
run: |
pip install ecl2df
pip install res2df
pip install .[tests,docs]

- name: 🧾 List all installed packages
Expand Down
22 changes: 11 additions & 11 deletions docs/advancedusage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -253,34 +253,34 @@ in the realization object to use, potentially only the directory).
:header-rows: 1

where PORV and VOLUME are sums over each zone, Z is the minimum (thus apex pr.
zone) and PERMX is an arithmetic mean. In the language of `ecl2df
<https://equinor.github.io/ecl2df/>`_ this could be done with a code like this:
zone) and PERMX is an arithmetic mean. In the language of `res2df
<https://equinor.github.io/res2df/>`_ this could be done with a code like this:

.. code-block:: python

from ecl2df import grid, EclFiles
from res2df import grid, ResdataFiles

eclfiles = EclFiles('MYDATADECK.DATA') # There is a file zones.lyr alongside this.
grid_df = grid.df(eclfiles) # Produce a dataframe with one row pr. cell
resdatafiles = ResdataFiles('MYDATADECK.DATA') # There is a file zones.lyr alongside this.
grid_df = grid.df(resdatafiles) # Produce a dataframe with one row pr. cell
my_aggregators = {'PORV': 'sum', 'VOLUME': 'sum', 'Z': 'min', 'PERMX': 'mean'}
stats_df = grid_df.groupby("ZONE").agg(my_aggregators)
print(stats_df)


``ScratchRealization`` objects contain the methods ``runpath()`` which will give
the full path to the directory the realization resides in, this can be used
freely by your function. For easier coupling with ecl2df, the function
``get_eclfiles()`` is provided.
freely by your function. For easier coupling with res2df, the function
``get_resdatafiles()`` is provided.

To be able to inject the ecl2df lines above into the API of fmu.ensemble and the
To be able to inject the res2df lines above into the API of fmu.ensemble and the
:py:meth:`apply() <fmu.ensemble.ensemble.ScratchEnsemble.apply>` function, we
need to to put it into a wrapper function. This wrapper function will always
receive a Realization object as a named argument, and it must return a
dataframe. The wrapper function can look like this:

.. code-block:: python

from ecl2df import grid, EclFiles
from res2df import grid, ResdataFiles

def my_realization_stats(args):
"""A custom function for performing a particular calculation
Expand All @@ -290,8 +290,8 @@ dataframe. The wrapper function can look like this:
args (dict): A dictionary with parameters to my custom function.
The keys 'realization' and 'localpath' are reserved for fmu.ensemble."""
realization = args["realization"] # Provided by fmu.ensemble apply()
eclfiles = realization.get_eclfiles()
grid_df = grid.df(eclfiles)
resdatafiles = realization.get_resdatafiles()
grid_df = grid.df(resdatafiles)
my_aggregators = {'PORV': 'sum', 'VOLUME': 'sum', 'Z': 'min', 'PERMX': 'mean'}
stats_df = grid_df.groupby("ZONE").agg(my_aggregators)
return stats_df.reset_index() # Zone names are in the index, lost if not reset.
Expand Down
20 changes: 10 additions & 10 deletions src/fmu/ensemble/realization.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@
from .util.rates import compute_volumetric_rates
from .util.dates import unionize_smry_dates

HAVE_ECL2DF = False
HAVE_RES2DF = False
try:
import ecl2df
import res2df

HAVE_ECL2DF = True
HAVE_RES2DF = True
except ImportError:
HAVE_ECL2DF = False
HAVE_RES2DF = False

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -852,9 +852,9 @@ def parameters(self):
"""
return self.data["parameters.txt"]

def get_eclfiles(self):
def get_resdatafiles(self):
"""
Return an ecl2df.EclFiles object to connect to the ecl2df package
Return an res2df.ResdataFiles object to connect to the res2df package

If autodiscovery, it will search for a DATA file in
the standard location eclipse/model/...DATA.
Expand All @@ -866,10 +866,10 @@ def get_eclfiles(self):
>>> real.find_files("eclipse/model/MYMODELPREDICTION.DATA")

Returns:
ecl2df.EclFiles. None if nothing found
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe copy this function for now, and expose both but with a deprecation warning added to get_eclfiles. You can then have two import checks: HAVE_ECL2DF and HAVE_RES2DF.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this resolve the problem of breaking changes @asnyv ?

res2df.ResdataFiles. None if nothing found
"""
if not HAVE_ECL2DF:
logger.warning("ecl2df not installed. Skipping")
if not HAVE_RES2DF:
logger.warning("res2df not installed. Skipping")
return None
data_file_row = self.files[self.files["FILETYPE"] == "DATA"]
data_filename = None
Expand All @@ -895,7 +895,7 @@ def get_eclfiles(self):
return None
if not os.path.exists(data_filename):
return None
return ecl2df.EclFiles(data_filename)
return res2df.ResdataFiles(data_filename)

def get_eclsum(self, cache=True, include_restart=True):
"""
Expand Down
1 change: 0 additions & 1 deletion src/fmu/ensemble/util/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Common utility functions used in fmu.ensemble"""


import os
from collections.abc import MutableMapping

Expand Down
1 change: 0 additions & 1 deletion src/fmu/ensemble/util/dates.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Common utility functions used in fmu.ensemble"""


import datetime
import dateutil
import pandas as pd
Expand Down
1 change: 0 additions & 1 deletion src/fmu/ensemble/util/rates.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Common utility functions for rates used in fmu.ensemble"""


import calendar
import dateutil
import logging
Expand Down
1 change: 0 additions & 1 deletion src/fmu/ensemble/virtualensemble.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
"""Module containing a VirtualEnsemble class"""


import os
import re
import shutil
Expand Down
1 change: 1 addition & 0 deletions src/fmu/ensemble/virtualrealization.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Contains the VirtualRealization class"""

import os
import fnmatch
import shutil
Expand Down
1 change: 1 addition & 0 deletions tests/test_ensemble.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Testing fmu-ensemble."""

# pylint: disable=protected-access

import os
Expand Down
1 change: 1 addition & 0 deletions tests/test_realization.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Testing fmu-ensemble."""

# pylint: disable=protected-access

import datetime
Expand Down
52 changes: 27 additions & 25 deletions tests/test_ecl2df.py → tests/test_res2df.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Testing incorporation of ecl2df in fmu-ensemble."""
"""Testing incorporation of res2df in fmu-ensemble."""

import os
import logging
Expand All @@ -7,19 +7,19 @@

from fmu.ensemble import ScratchEnsemble, ScratchRealization

HAVE_ECL2DF = True
HAVE_RES2DF = True
try:
import ecl2df
import res2df
except ImportError:
HAVE_ECL2DF = False
HAVE_RES2DF = False

logger = logging.getLogger(__name__)


def test_ecl2df_real():
"""Check that we can utilize ecl2df on single realizations"""
def test_res2df_real():
"""Check that we can utilize res2df on single realizations"""

if not HAVE_ECL2DF:
if not HAVE_RES2DF:
pytest.skip()

if "__file__" in globals():
Expand All @@ -30,15 +30,15 @@ def test_ecl2df_real():
realdir = os.path.join(testdir, "data/testensemble-reek001", "realization-0/iter-0")
real = ScratchRealization(realdir)

eclfiles = real.get_eclfiles()
assert isinstance(eclfiles, ecl2df.EclFiles)
compdat_df = ecl2df.compdat.df(eclfiles)
resdatafiles = real.get_resdatafiles()
assert isinstance(resdatafiles, res2df.ResdataFiles)
compdat_df = res2df.compdat.df(resdatafiles)
assert not compdat_df.empty
assert "KH" in compdat_df


def test_reek():
"""Import the reek ensemble and apply ecl2df functions on
"""Import the reek ensemble and apply res2df functions on
the realizations"""

if "__file__" in globals():
Expand All @@ -49,19 +49,19 @@ def test_reek():
reekens = ScratchEnsemble(
"reektest", testdir + "/data/testensemble-reek001/" + "realization-*/iter-0"
)
if not HAVE_ECL2DF:
if not HAVE_RES2DF:
pytest.skip()

def extract_compdat(kwargs):
"""Callback fnction to extract compdata data using ecl2df
"""Callback fnction to extract compdata data using res2df
on a ScratchRealization"""
eclfiles = kwargs["realization"].get_eclfiles()
if not eclfiles:
resdatafiles = kwargs["realization"].get_resdatafiles()
if not resdatafiles:
print(
"Could not obtain EclFiles object for realization "
"Could not obtain ResdataFiles object for realization "
+ str(kwargs["realization"].index)
)
return ecl2df.compdat.deck2dfs(eclfiles.get_ecldeck())["COMPDAT"]
return res2df.compdat.deck2dfs(resdatafiles.get_deck())["COMPDAT"]

allcompdats = reekens.apply(extract_compdat)
assert not allcompdats.empty
Expand All @@ -70,16 +70,18 @@ def extract_compdat(kwargs):
# Pr. now, only realization-0 has eclipse/include in git


def test_smry_via_ecl2df():
"""Test that we could use ecl2df for smry extraction instead
def test_smry_via_res2df():
"""Test that we could use res2df for smry extraction instead
of the native code inside fmu-ensemble"""

def get_smry(kwargs):
"""Callback function to extract smry data using ecl2df on a
"""Callback function to extract smry data using res2df on a
ScratchRealization"""
eclfiles = kwargs["realization"].get_eclfiles()
return ecl2df.summary.df(
eclfiles, time_index=kwargs["time_index"], column_keys=kwargs["column_keys"]
resdatafiles = kwargs["realization"].get_resdatafiles()
return res2df.summary.df(
resdatafiles,
time_index=kwargs["time_index"],
column_keys=kwargs["column_keys"],
)

if "__file__" in globals():
Expand All @@ -90,12 +92,12 @@ def get_smry(kwargs):
reekens = ScratchEnsemble(
"reektest", testdir + "/data/testensemble-reek001/" + "realization-*/iter-0"
)
if not HAVE_ECL2DF:
if not HAVE_RES2DF:
pytest.skip()

callback_smry = reekens.apply(get_smry, column_keys="FOPT", time_index="yearly")
direct_smry = reekens.get_smry(column_keys="FOPT", time_index="yearly")

assert callback_smry["FOPT"].sum() == direct_smry["FOPT"].sum()
assert callback_smry["REAL"].sum() == direct_smry["REAL"].sum()
# BUG in ecl2df, dates are missing!!
# BUG in res2df, dates are missing!!
1 change: 1 addition & 0 deletions tests/test_util.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Test general utility functions in use by fmu.ensemble"""

import datetime
import logging

Expand Down
1 change: 1 addition & 0 deletions tests/test_virtualrealization.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Testing fmu-ensemble, virtual realizations"""

# pylint: disable=protected-access,duplicate-code

import os
Expand Down
Loading