Does lca_algebraic support parameters defined in excel import?

[xiaoshir]

Based on the exmaple_notebook, I noticed that the lca_algebraic has treated all defined parameters as Project parameters. I have a foreground database in excel, parameterized, importable to bw, in the form of this.

In order to use lca_algebraic, I have made all parameters in my foreground as project parameters (they used to be database and activity parameters), but after that, I couldn't update my existing activity using sympolic formula, and an error of "NameError: name 'parameter name' is not defined" was raised. I checked my project parameters, using the following, and they are indeed defined there.

from bw2data.parameters import *
print("Project parameters:")
for p in ProjectParameter.select():
print(p.amount, p)

I noticed in the exmaple_notebook,

list_parameters()

is used to check the defined parameters, printing this table turns out to be blank in my case. I have tried the same approach as shown in the exmaple_notebook by defining the parameter using script/command, and found such defined parameter will show up under list_parameters().

So my questions are:

• Does lca_algebraic support only project parameters? and why?
• Does lca_algebraic support project parameters imported from excel? or it only supports script/command created parameters?

Thanks.

[raphaeljolivet]

Hi, thanks for this feedback.

1. Yes lca_algebraic only supports project parameters at the moment. This project is a generalisation of several use cases, and I did not find yet the need for other type of parameters. Is it an issue to you ? Could you maybe explain your usage of activity parameters ?
2. The parameters should be defined via script. the library store them in the project, but keeps some extra meta data (like distribution) in a separate / non persistent dictionary.

The philosophy behind this is to have all the definition of the model (activities and parameters) done in the code, and to always start fresh by removing user activities and parameters :
I found that having persistent data between two sessions is very prone to errors, since you never know in which state you left it. This is especially true when you play with Notebook, since you may execute cells in different order.
So I try to organize my code starting fresh / empty at the beginning.

Anyway, I think that it might be a good idea to :

• Try to store all meta data in Brightway project and get rid of this extra dictionnary
• Improve the possibility to import / export a dataset from excel without having to define parameters again.

I am not familiar with Excel import in BW, I will look at it.
This is a feedback I also received by email.

[xiaoshir]

Thanks for your anwser Raphael:) I think you can close it now.

[ntropy-esa]

Hello Raphael & Xiaoshir,

Me and a student are starting a new project where we want to use LCA_algebraic.

We have existing bw2 projects, developed in the activity-brower, with only Project parameters (as you, I don't understand the value of Activity and Database parameters, except for organizing things within the project - but it's very limiting as well, since activity parameters cannot be accessed outside of the activity).

Two things:

• We would be glad if we could convert existing ab projects to LCA_algebraic, via some nice script.
• Reacting on the issue above: indeed, it sounds also good to me to store all the metadata in the bw2 dictionaries. I had in mind that the data structure in bw2 is very flexible and that we can save any extra keys in the parameters.

If you have time in February, do you think we could set up a meeting & try to fast-code these scripts/improvements together?

[raphaeljolivet]

@ntropy-esa I am currently checking how AB is storing parameters and uncertainty.
I am developing two functions to store / load lca_algebraic parameters from / to Brightway2 project the way AB does it.
I will push it in a separate branch.

I would be glad if you can test it and provide feedbacks.

[ntropy-esa]

Great ! Have also found few minor things that could be improved / that I am currently working on:

• handling of multiple outputs of an activity via substitution (exchanges of type 'production') lead to wrong impact results
• function searchAct by name does not handle the case where two activities in ecoinvent have same name but different reference products, e.g.

findTechAct('softwood forestry, pine, sustainable forest management', loc='SE')
bundle, energy wood, measured as dry mass softwood forestry, pine, sustainable forest management
cleft timber, measured as dry mass softwood forestry, pine, sustainable forest management

• a function similar to actToExpression, but that implements a matrix inversion rather than a graph traversal, to support the case where we have internal loops in the foreground system

[raphaeljolivet]

@ntropy-esa I have pushed a new branch :
https://github.com/oie-mines-paristech/lca_algebraic/tree/feature/parameters-load-and-persistance

You have two new functions loadParams() persistParams() (new***Param also persists by default unless save=False)

This is compatible with the way Brightway2 and AB thread uncertainties, as described here :
https://stats-arrays.readthedocs.io/en/latest/

Boolean and Enum parameters are stored as "discrete uniform" params.
Enum parameters produce one boolean parameter for each possible enum value.

Float params support uniform, normal, triangle and beta distribution (not sure about the mapping of the A and B attributes though)

Please give it a try a provide some feedbacks about your use cases : Can you import existing AB project and parameters without redefining the parameters in the code ?

[ntropy-esa]

Started to test tonight.

Test 1 - Importing existing AB project and parameters without redefining the parameters in the code

• Created a dummy project in AB with 4 activities, defined 4 parameters in AB and 1 formula, and defined their uncertainties.
• Opening notebook, loading lca_algebraic, setting project
• running function loadParam()

Result:

• all parameters get skipped
• the key in the param dictionary from AB is 'uncertainty type' rather than 'type' (at least in my version of AB, latest version)
• note: bwParam.dict and bwParam.data are not the same (dict has the keys name & amount and all keys from data, data just has the uncertainty data)

# bwParam.dict
{'name':'some name',
'amount':4,
'uncertainty type': 4, # = uniform, according to https://stats-arrays.readthedocs.io/en/latest/
'loc': nan,
'scale': nan,
'shape': nan,
'minimum': 2.0,
'maximum': 5.0,
'negative': False}

Options to fix this:

1. either we make a translation from how dict are saved in lca_algebraic and in bw2, and use it every time we load params or persist params
2. or we re-write lca_algebraic param dict to match nomenclature in bw2 (or at least the stats-array nomenclature), but it's limited because EnumParams/BoolParams are not defined in bw2

I think the main use case is that people have created a model in the AB, with essentially only float parameters, and want to analyse it with lca_algebraic.

For further development, I think the following mappins may be useful:

        mapping_types_bw2alg = {
            7: (ParamType.ENUM, None),
           #7: ParamType.BOOL,
            3: (ParamType.FLOAT, DistributionType.NORMAL), 
            4: (ParamType.FLOAT, DistributionType.LINEAR),
            5: (ParamType.FLOAT, DistributionType.TRIANGLE),
            10: (ParamType.FLOAT, DistributionType.BETA),
        }


        mapping_keys_bw2alg = {
            'uncertainty type': ('type', 'distrib'),
            'loc': None, # not sure
            'scale':'std',
            'shape':'b', # not sure
            'minimum':'min',
            'maximum':'max',
            'negative':None,
            'amount':'default',
            'name':'name',
        }

        mapping_keys_alg2bw = {
            'type': 'uncertainty type',
            'default': 'amount', 
            'description': 'description',
            'min': 'minimum',
            'max': 'maximum',
            'a': 'loc',
            'b': 'shape',
            'std': 'scale',
            'unit': None,
            'label': None,
            'label_fr': None,
            'group': None,
            'distrib': None
        }

With these mappings, I edited loadParams() and it now works to import AB-defined parameters with a FLOAT uncertainty type (linear, normal, beta) (but mapping of params to be checked as you said). You can have a look at it here:
ntropy-esa@d2dd2e4

[ntropy-esa]

One more thing:

• using persistParams() will overwrite the existing parameters in AB, with the dictionary keys from lca_agl, meaning they won't be functional in the activity browser anymore

[raphaeljolivet]

Hi, I am surprised.
I tried it myself and did not have those issues.
Are you sure you were up to ate with remote branch (last commit is a7a0bcb)

[raphaeljolivet]

@ntropy-esa : Did you see my comment above ?

[ntropy-esa]

Yes, I just didn't have time yet to process it. Indeed, when I forked the branch, it did not have the last comits. I re-fetched it, and your mappings between bw2 param dicts and lca_algebraic works like a charm. Well done. I will have more time to test and adapt for our case in the coming weeks. Elias<mailto:[email protected]>

…

________________________________ De : Raphael Jolivet <[email protected]> Envoyé : vendredi 5 février 2021 10:40 À : oie-mines-paristech/lca_algebraic <[email protected]> Cc : ntropy-esa <[email protected]>; Mention <[email protected]> Objet : Re: [oie-mines-paristech/lca_algebraic] Does lca_algebraic support parameters defined in excel import? (#8) @ntropy-esa<https://github.com/ntropy-esa> : Did you see my comment above ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#8 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AJF5QB5EDY3ALO3OIU6OHI3S5O4HBANCNFSM4VY3CA5Q>.

[ljlazar]

Hi,
I also would like to use a bw2 project developed in the Activity Browser (AB) with lca-algebraic. Unfortunately loadParams() stops at the certain parameter where the uncertainty was modified in AB. Removing the uncertainty in AB did not work, is there another way to reset it? The error message occurring is:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[22], line 2
      1 # Load parameters previously  persisted in the dabatase.
----> 2 loadParams()

File ~\Miniconda3\envs\bw_testing\lib\site-packages\lca_algebraic\params.py:597, in loadParams(global_variable, dbname)
    594     type = _UncertaintyType.UNIFORM
    596 # Uncertainty type to distribution type
--> 597 args["distrib"] = _DistributionTypeMapReverse[type]
    599 if type == _UncertaintyType.TRIANGLE :
    600     args["default"] = data["loc"]

KeyError: 0

Thank you very much for any hints!