Cannot import name 'unicode' from 'openpyxl.compat' #241
Comments
|
I found the same issue with the following versions: koala2=0.0.36 openpyxl.compat no longer supports the unicode type (aka Python 2 code). The good thing is that in Python 3, the native str method is already unicode. Much of the code in koala that references unicode is mainly to switch between the two Python versions (is that correct?). Now that Python 2 is no longer supported, it should be easier. Until the bugfix is put into the repository, the fix I made locally is: replace the "from openpyxl.compat import unicode" line with "unicode = str" in all the koala files that have it. There is also an issue with networkx no longer (>2.4) having a 'node' attribute. Replaced with "nodes" in koala/astnodes.py. |
jia-chenhua
added a commit
to rangl-labs/netzerotc
that referenced
this issue
Sep 23, 2021
…e env.py under 'reference_environment_direct_deployment' folder, by using the pycel package, which unfortunately has not implemented Excel's RRI() function yet. An alternative solution is to use https://www.excelpython.org/koala/, which has a bug that needs to be fixed manually vallettea/koala#241 by changing all its files.
jia-chenhua
added a commit
to rangl-labs/netzerotc
that referenced
this issue
Oct 14, 2021
* Prototype implementation of direct action on deployment numbers in the env.py under 'reference_environment_direct_deployment' folder, by using the pycel package, which unfortunately has not implemented Excel's RRI() function yet. An alternative solution is to use https://www.excelpython.org/koala/, which has a bug that needs to be fixed manually vallettea/koala#241 by changing all its files. * Corrected env.py and added a new test script test_reference_environment_direct_deployment.py for the new env.py for direct deployment. Compiling of the 'Pathways to Net Zero.xlsx' only happens during environment creation by env = gym.make("reference_environment_direct_deployment:reference-environment-direct-deployment-v0"), which should take 1 minute or less. Before running the test_reference_environment_direct_deployment.py lines-by-lines, make sure the current directory is netzerotc/environment/, and place the 'Pathways to Net Zero - Simplified.xlsx' (can be downloaded from MS Teams/EXT-ATI/OREC-RANGL/General/Files) inside netzerotc/environment/compiled_workbook_objects/. Note: All RRI() functions/formulae in the original 'Pathways to Net Zero.xlsx' have been replaced by the original values, and 3 controllable technologies' all future deployment numbers from 2031 to 2050 have been removed because future is unknown in RL, and also, in the 'Outputs' spreadsheet of the 'Pathways to Net Zero.xlsx', all values depending on weighted sum of future values are changed to depending only on current year's values. The simplified/modified 'Pathways to Net Zero - Simplified.xlsx' contains changelog/history of changes so it's easy to see which cells' formulae have been replaced by raw values. * Fixed the compiled object's cell's map/address initialization issue: before resetting or setting any values in the compiled object, all reward components cells have to be evaluated first in order to get initialized, and after setting the action of deployment increments, the reward component cells have to be re-evaluated again to take the updated/reset value into consideration when doing the formula calculation. Now the env.py's output rewards components are (almost) exactly the same as Excel's workbook's total capex opex revenue emissions: In 'Pathways to Net Zero - Simplified.xlsx' - 'Gale' spreadsheet, simply apply an increments of [1.0 1.0 2.0] to offshore wind, blue hydrogen, and green hydrogen on top of 2030 values and set the new values as 2031 values, then apply constant increments [2.0 2.0 5.0] (as in the test_reference_environment_direct_deployment.py) for all following years, then in the 'Outputs' and 'CCUS', the total capex opex revenue and emissions from 2031 to 2050 are exactly the same as env.py's outputs. Moreover, even in the original 'Pathways to Net Zero.xlsx', when doing the same operation, the total opex revenue and emissions from 2031 to 2050 are still exactly the same as env.py's outputs, whereas the total capex is different because in the original 'Pathways to Net Zero.xlsx', offshore wind's capex depends on some numbers that are weighted sum of future values, which is not removed and replaced by current year's values only in the 'Pathways to Net Zero - Simplified.xlsx'. * Update README.md Update readme for actions to control deployment directly (rather than indirectly via scenarios) * Added requirement * Update README.md Update for direct deployment * Fix typos * Updated the jobs number calculation using a simple formula: 0.25 * (capex + opex + decomm) / 0.05. * The training works successfully now (but needs to switch between different folders before and after creating the environment by running env = gym.make("reference_environment_direct_deployment:reference-environment-direct-deployment-v0")). The training takes about 1 sec per episode, and the 10 uploaded modes are trained by 1000 episodes per model. Please feel free to do some evaluations using the codes in evaluate.py. * Fixed inconsistence in def to_observation(self): and def observation_space(self): such that the training and evaluation now work properly. Updated some trained models with 100 episodes per model and the plot. * Implemented the randomization on prices and costs in spreadsheet 'CCUS' row 23,24,26 and spreadsheet 'Outputs' row 148, 149, 150, 153, 154, 155, 158, 159, 163, 164, 165, 166; Trained 10 models with 100 episodes per model. * Fixed the evaluation of trained models. * Clipped the multiplicative noise by max(noise, 0.1) such that the price/cost will not become negative or zero. * Clipped noise by max(noise, 0.001) in env.py, re-trained the models and evalute them by plotting. * fix: apply black formatter to all python files * fix: use relative paths when loading resources * fix: typo in comment * test: add demonstration of serialisation/de-serialisation * Update README.md * Revert "fix: apply black formatter to all python files" This reverts commit 87f9d59. # Conflicts: # environment/reference_environment_direct_deployment/env.py # local_agent_training_and_evaluation/train.py Successfully implemented pycel serialization (for cells involved in env.py, but not necessarily for other cells) by using the evaluate_all_and_serialize.py in 'compiled_workbook_objects', and changed env.py to de-serialize the workbook, and also env.reset will also reset the env.param by self.param = Parameters(). * Implemented #3 of #36 and #1 of #5. Training and evaluation tests passed, and there is no longer the data type or array dimension issues experienced before with the def to_observation(self): and the def observation_space(self):. * step: test containerisation * step: use nztc:5000 in place of environment:5000 * fix: use evalai_rangl network * step: add base evaluation content * Updated README.md for #34 (comment); Fixed indexing of 'obs_high' in def observation_space(self):. * Implemented #35 (comment) to reset param when `env.reset()` is invoked, as an alternative to the current approach of creating a completely new `self.param = Parameters()` which will take 0.2~0.25 seconds to load the serialized .pkl/.yml; Correctly initialize the `state.randomized_costs` by setting them to fixed (non-randomized) 2030's values; Increased the upper bounds of costs/prices in the observation space by 10 times; Included non-independently randomized costs/prices in the `state.randomized_costs` and the observation space for #5 (comment). * step: add README for random agent submission * Update README.md Edit title from 'Random-agent...' to 'Agent...' because this workflow is the same for all agents. Move testing up before submission to reflect first-time use case. Summarise sample terminal output. * Update README.md Replace the challenge overview content with a link to the content on EvalAI. Add a note about the submission folder. * Add control of noise in Parameters() in env.py; Record deployment numbers (after applying actions of increments) to state.deployments, and add plotting of deployment numbers from 2031 to 2050. Add compiling and serialization of IEV model workbook in evaluate_all_and_serialize.py. Co-authored-by: Jia-Chen Hua <[email protected]> Co-authored-by: moriartyjm <[email protected]> Co-authored-by: jia-chenhua <[email protected]> Co-authored-by: Tomasz Kosmala <[email protected]>
masonlr
added a commit
to rangl-labs/netzerotc
that referenced
this issue
Oct 15, 2021
* Prototype implementation of direct action on deployment numbers in the env.py under 'reference_environment_direct_deployment' folder, by using the pycel package, which unfortunately has not implemented Excel's RRI() function yet. An alternative solution is to use https://www.excelpython.org/koala/, which has a bug that needs to be fixed manually vallettea/koala#241 by changing all its files. * Corrected env.py and added a new test script test_reference_environment_direct_deployment.py for the new env.py for direct deployment. Compiling of the 'Pathways to Net Zero.xlsx' only happens during environment creation by env = gym.make("reference_environment_direct_deployment:reference-environment-direct-deployment-v0"), which should take 1 minute or less. Before running the test_reference_environment_direct_deployment.py lines-by-lines, make sure the current directory is netzerotc/environment/, and place the 'Pathways to Net Zero - Simplified.xlsx' (can be downloaded from MS Teams/EXT-ATI/OREC-RANGL/General/Files) inside netzerotc/environment/compiled_workbook_objects/. Note: All RRI() functions/formulae in the original 'Pathways to Net Zero.xlsx' have been replaced by the original values, and 3 controllable technologies' all future deployment numbers from 2031 to 2050 have been removed because future is unknown in RL, and also, in the 'Outputs' spreadsheet of the 'Pathways to Net Zero.xlsx', all values depending on weighted sum of future values are changed to depending only on current year's values. The simplified/modified 'Pathways to Net Zero - Simplified.xlsx' contains changelog/history of changes so it's easy to see which cells' formulae have been replaced by raw values. * Fixed the compiled object's cell's map/address initialization issue: before resetting or setting any values in the compiled object, all reward components cells have to be evaluated first in order to get initialized, and after setting the action of deployment increments, the reward component cells have to be re-evaluated again to take the updated/reset value into consideration when doing the formula calculation. Now the env.py's output rewards components are (almost) exactly the same as Excel's workbook's total capex opex revenue emissions: In 'Pathways to Net Zero - Simplified.xlsx' - 'Gale' spreadsheet, simply apply an increments of [1.0 1.0 2.0] to offshore wind, blue hydrogen, and green hydrogen on top of 2030 values and set the new values as 2031 values, then apply constant increments [2.0 2.0 5.0] (as in the test_reference_environment_direct_deployment.py) for all following years, then in the 'Outputs' and 'CCUS', the total capex opex revenue and emissions from 2031 to 2050 are exactly the same as env.py's outputs. Moreover, even in the original 'Pathways to Net Zero.xlsx', when doing the same operation, the total opex revenue and emissions from 2031 to 2050 are still exactly the same as env.py's outputs, whereas the total capex is different because in the original 'Pathways to Net Zero.xlsx', offshore wind's capex depends on some numbers that are weighted sum of future values, which is not removed and replaced by current year's values only in the 'Pathways to Net Zero - Simplified.xlsx'. * Update README.md Update readme for actions to control deployment directly (rather than indirectly via scenarios) * Added requirement * Update README.md Update for direct deployment * Fix typos * Updated the jobs number calculation using a simple formula: 0.25 * (capex + opex + decomm) / 0.05. * The training works successfully now (but needs to switch between different folders before and after creating the environment by running env = gym.make("reference_environment_direct_deployment:reference-environment-direct-deployment-v0")). The training takes about 1 sec per episode, and the 10 uploaded modes are trained by 1000 episodes per model. Please feel free to do some evaluations using the codes in evaluate.py. * Fixed inconsistence in def to_observation(self): and def observation_space(self): such that the training and evaluation now work properly. Updated some trained models with 100 episodes per model and the plot. * Implemented the randomization on prices and costs in spreadsheet 'CCUS' row 23,24,26 and spreadsheet 'Outputs' row 148, 149, 150, 153, 154, 155, 158, 159, 163, 164, 165, 166; Trained 10 models with 100 episodes per model. * Fixed the evaluation of trained models. * Clipped the multiplicative noise by max(noise, 0.1) such that the price/cost will not become negative or zero. * Clipped noise by max(noise, 0.001) in env.py, re-trained the models and evalute them by plotting. * fix: apply black formatter to all python files * fix: use relative paths when loading resources * fix: typo in comment * test: add demonstration of serialisation/de-serialisation * Update README.md * Revert "fix: apply black formatter to all python files" This reverts commit 87f9d59. Successfully implemented pycel serialization (for cells involved in env.py, but not necessarily for other cells) by using the evaluate_all_and_serialize.py in 'compiled_workbook_objects', and changed env.py to de-serialize the workbook, and also env.reset will also reset the env.param by self.param = Parameters(). * Implemented #3 of #36 and #1 of #5. Training and evaluation tests passed, and there is no longer the data type or array dimension issues experienced before with the def to_observation(self): and the def observation_space(self):. * step: test containerisation * step: use nztc:5000 in place of environment:5000 * fix: use evalai_rangl network * step: add base evaluation content * Updated README.md for #34 (comment); Fixed indexing of 'obs_high' in def observation_space(self):. * Implemented #35 (comment) to reset param when `env.reset()` is invoked, as an alternative to the current approach of creating a completely new `self.param = Parameters()` which will take 0.2~0.25 seconds to load the serialized .pkl/.yml; Correctly initialize the `state.randomized_costs` by setting them to fixed (non-randomized) 2030's values; Increased the upper bounds of costs/prices in the observation space by 10 times; Included non-independently randomized costs/prices in the `state.randomized_costs` and the observation space for #5 (comment). * step: add README for random agent submission * Update README.md Edit title from 'Random-agent...' to 'Agent...' because this workflow is the same for all agents. Move testing up before submission to reflect first-time use case. Summarise sample terminal output. * Update README.md Replace the challenge overview content with a link to the content on EvalAI. Add a note about the submission folder. * Add control of noise in Parameters() in env.py; Record deployment numbers (after applying actions of increments) to state.deployments, and add plotting of deployment numbers from 2031 to 2050. Add compiling and serialization of IEV model workbook in evaluate_all_and_serialize.py. Co-authored-by: Jia-Chen Hua <[email protected]> Co-authored-by: moriartyjm <[email protected]> Co-authored-by: jia-chenhua <[email protected]> Co-authored-by: Tomasz Kosmala <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the Koala Module koala\ast_init_.py line 9 it;
from openpyxl.compat import unicodeThe current version of openpyxl, openpyxl 3.0.0, does not have the module openpyxl.compat.unicode.
The text was updated successfully, but these errors were encountered: