From 82028a06ea74352b621d7724bdd25f8c5b5d954a Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo <22427519+sevisal@users.noreply.github.com> Date: Wed, 21 Jun 2023 12:03:11 +0200 Subject: [PATCH 01/48] Update functionSelection (#108) * Reupload integral plugin * argopt plugin * Css user interaction for Bayesian Optimisation (#91) * Display for less than 3 parameters * Added current selection * DoubleClick for editing * Change plugin readable name In order to avoid confusions with the Optimisation plugin, changed naming to indicate it referes to the ui. * Added icon to GUI window * Fix more than 2 variables error * Minor aesthetic changes * Css bayesian optimisation (#98) * added re-attempt of env and pybullet * ongoing pybullet/arg population changes * added pybullet stubs * added credit to stubs * added gui/headless choice function * added environment module tag to XML parser * added pybullet config file * renamed Environment module to remove plural * added types and ABC for env modules * removed renamed scripts * modified run pipeline for pybullet * moved relative to absolute converter to import helper * added half-cheetah model * pybullet loading and running working * added half cheetah example with mass * added option to set gravity * removed requirement for data * removed data loading for pybullet example script * added __getattr__ function to access all pybullet methods from ENV class * added option parsing * Change progressTracker to treeview * Correct icon * WIP: editing option parsing * option mods - not yet working * arg passing without kwords to pybul from xml * cleaned imports * renamed depreciated setup file * removed old setup file * cleaned up main __init__ file * added matplotlib to pyproject.toml * Add pytest structure (#56) Co-authored-by: Teemu Ruokolainen * Add cli (#57) * working cli * Update readme * Update readme Co-authored-by: Teemu Ruokolainen * Cm package restructuring (#54) * moved py files to src/vai_labs for packaging * added vscode config to gitignore * updated package name in run_pipeline * fixed pyproject matplotlib import * fixed path for import helper * updated project version for PYPI * updated readme and pyproject.toml * fixed README typo * removed depreciated setup script * fixed path in import_helper.py * Update README.md Updated repo name in README * moved README images * Update README image paths * Remove readme.md.bak (#58) Co-authored-by: Teemu Ruokolainen * Cm name updates (#59) * updated naming and paths for projects * merged README files * updated README naming * update import names in README * updated example links * updated missed readme names * find-replace aidesign refs -> vai_lab * renamed files containing old ai_design name * renamed test file * Css-gui_bugFixing (#60) * Adjust aidCanvas' buttons' size * Correctly resize lower buttons' sizes * Resize buttons, pad radio frame, correct click on in or out * Fix issue with data path * Fix upload data mattrix path * moved test script Co-authored-by: cam586 * Fix repository url in readme and pyproject.toml (#63) * added opencv & pytest (#62) * Clean up lingering/duplicate files. (#64) Co-authored-by: Teemu Ruokolainen * Added debug option (#65) * add debug option to core,GUI * Update test_launch.py Add test_launch() function for pytest discovery Co-authored-by: teemu * Added CI Test using Github Actions (#66) * add debug option to core,GUI * Update test_launch.py Add test_launch() function for pytest discovery * added initial actions yaml * testing now triggered on PR * removed testing for python3.7 * added option for manually running actions * added workflow_dispatch options * temp removal of PR trigger * testing using push trigger * removed incorrect pytest dir * added env variable for GUI testing * added virtual display for GUI testing * removed old yml tag * install virtual display with existing action * fixed yml syntax * fixed xvfb syntax * removed old xvfb tags * update to node.js 16 * test pip caching * removed cache test * fixed cov file path * removed testing for python3.7 * install virtual display with existing action * update to node.js 16 * fixed cov file path * changed trigger condition to pull request * renamed tests package * added programming language classifiers * changed trigger to PR * added test and version shields * fixed README typo * gitignore any venv * removed unneeded tk dep from toml * simplified test call * testing manual workflow trigger * added workflow_dispatch options * re-added pull_request trigger Co-authored-by: teemu * Remove flit from readme (#67) Co-authored-by: Teemu Ruokolainen * Cm fix readme img path (#68) * fixed readme image paths * updated version number * Css fix pluginarchitecture diagram (#69) * Include plugin diagram * Delete old verion * Update plugin diagram * Update plugin diagram * Add white filling to squares for dark mode * fixed contribute urls Co-authored-by: cam586 * Add rough sphinx documentation (#74) * Add autogen docsting sphinx docs * ReadTheDocs * Building version * Change index page title * Add github actions workflow * Update README * Add pip install . to documentation.yml * Add README.md as a page in documentation * Add sphinx-apidoc command to readme; gitignore generated rst files * Update readme sphinx commands * Update readme * Add comments * Update commands in documentation workflow * Fix typo in .gitignore * Remove _static folder from config to remove warning message * Specify python=3.10 for workflow * Remove pip install sphinx-apidoc, no such package * Update readme * added docs build badge to readme * fixed error-causing docstring Co-authored-by: Teemu Ruokolainen Co-authored-by: chris * Cm test docs (#76) * edited branch name for doc building * test actions * added jekyll options * removed settings * fixed gh pages deployment * Cm add docs (#77) * changed copyright year * updated readme headings * moved tutorials from readme to docs * removed tutorials from readme * added getting started page to index toc * Update python=3.8 to python=3.10 in readme * Cm feature add tests (#80) * bumped version number * add pilot example test * simplified core.py * improved naming and docs in core * progress tracker. -> * added example script iteration * fixed loop error msg in COre * updated progress tracker naming (#71) * fixed for loop logic and error handling in core (#72) * Cm fix tracker loops (#73) * fixed for loop logic and error handling in core * fixed tracker dict population for loops * Fix Progess Tracker naming Removing Initialiser name requirement from Progress Tracker * Remove Canvas if there are no ccordinates * Fixing last fix. Correctly read modules inside loop * Fix error with coordinates case * minor typo fix in core * renamed .s variable to .xml_handler for readability * fixed xml_handler naming in GUI_Core * Trying to fix coordinates case * moved avail_plugins to init * Display treeview for no-coordinates case (need fixes)) * FIX - Consistent input and output naming issue * Functioning progress tracker for coord and no coord * Fixed list plugin options issues * Minor fixes to test usability * changed avail plugins options to readable names * removed private and test files from avail_plugins * removed old xml files, mod pybullet * Fix parent issue * fixed pybullet and canvas gui-test issues * fixed pytest errors (except manual plugin) * changed sklearn->scikit-learn in toml * added pybullet to toml * Remove testing file * test fix with scalar * translate boolean literals * removed python 3.8,3.9 * added param to ridge example * removed closed check from core * switced core check to _debug --------- Co-authored-by: Carlos Sevilla Salcedo --------- Co-authored-by: Carlos Sevilla Salcedo * Fix examples path in README.md * Another bad examples path * update conda env part in README.md * fixed calls to undefined parent (#90) * Added Bayesian Optimisation plugin * Fix merge errors * Remove old run file * Modify plugin types * Addoptions to BO * Css feature trying to catch errors (#94) * Move Modelling plugins functions to template * Add clsutering methods to template * Change function doc * Move DataProcessing plugins functions to template * Update pipeline for testing * Update init * Fix conflicts * Add DecisionMaking plugin template * Fixing issues with Optimisation module * Fixed errors passing params to BO * Changed plugin Template to work for BO params Still need to fix issue when running the BO * BO suggests points correctly * Change xml_handler naming Change xml_handler naming to fit with the other UIs * Typo correction * Added bayes_opt optimisation plugin * Modify integral to simpson * Fixed issue with UI --------- Co-authored-by: cam586 Co-authored-by: teemu Co-authored-by: Teemu Ruokolainen Co-authored-by: Richard Darst * Fix issue with example * Added basic dataFlow selection * Fixed issue with module list * Remove option of data self-looping * Add plugin's data input to XML * Fixed error with ridge regression * Added new data reading option * Fix minor errors * Define data communication Redifined data passing and reading between different modules. `vai_lab_core` now stores all output data for each module and could be accessible by any module. Not tested on loops. * Fixed issue duplicated module entries Still need to fix issue with module name reading. * Fixed names used to define input * Clean some example files * Remove testing print * Add output module as editable * Include a checklist for output data * Fixed issue with output update * Write output data in XML file working * Include output path for outdata * Storing output in file * Fix space reading * Allow one or more stored module output * Update XML examples to dataflow * Store relative path * Add option of relative or absolute path * Allow relative path * Change input data definition to optionmenu * By default store output of last module * Update example coordinates * Improve look of output data's optionmenu --------- Co-authored-by: cam586 Co-authored-by: teemu Co-authored-by: Teemu Ruokolainen Co-authored-by: Richard Darst --- .github/workflows/documentation.yml | 26 ++ .github/workflows/pythonpackage.yml | 47 +++ src/vai_lab/Core/vai_lab_core.py | 42 ++- src/vai_lab/Data/Data_core.py | 4 + src/vai_lab/Data/xml_handler.py | 35 +- .../DataProcessing/DataProcessing_core.py | 3 +- src/vai_lab/DataProcessing/plugins/argopt.py | 52 +++ .../DataProcessing/plugins/integral.py | 32 ++ .../plugins/kbinsdiscretizer.py | 2 +- .../DecisionMaking/DecisionMaking_core.py | 36 +++ src/vai_lab/DecisionMaking/Modelling_core.py | 29 -- src/vai_lab/DecisionMaking/__init__.py | 2 +- .../plugins/BayesianOptimisation(GPy).py | 61 ++++ .../BayesianOptimisation(bayes_opt).py | 49 +++ src/vai_lab/GUI/GUI_core.py | 3 +- .../Modelling/plugins/knnclassifier.py | 2 +- .../Modelling/plugins/ridgeregression.py | 6 +- .../UserInteraction/plugins/CanvasInput.py | 2 +- .../UserInteraction/plugins/ManualInput.py | 2 +- .../plugins/OptimisationInput.py | 301 ++++++++++++++++++ src/vai_lab/_plugin_templates.py | 24 +- .../crystalDesign/phasestability/.DS_Store | Bin 0 -> 6148 bytes .../phasestability/CsFA/.DS_Store | Bin 0 -> 6148 bytes .../CsFA/fulldata/CsFA_T300_above.csv | 1 + .../phasestability/CsMA/.DS_Store | Bin 0 -> 6148 bytes .../CsMA/fulldata/CsMA_T300_above.csv | 1 + .../phasestability/FAMA/.DS_Store | Bin 0 -> 6148 bytes .../FAMA/fulldata/FAMA_T300_above.csv | 1 + .../crystalDesign/phasestability/Readme | 16 + .../crystalDesign/phasestability/template.csv | 16 + src/vai_lab/examples/optimisation/X.csv | 29 ++ src/vai_lab/examples/optimisation/Y.csv | 29 ++ .../xml_files/KNN-classification_demo.xml | 27 +- src/vai_lab/examples/xml_files/SVR_demo.xml | 25 +- .../examples/xml_files/canvas_demo.xml | 29 +- .../xml_files/k-mean_clustering_demo.xml | 27 +- .../xml_files/logistic_regression_demo.xml | 42 --- .../xml_files/pybullet_env_example.xml | 18 +- .../xml_files/random_forest_class_demo.xml | 25 +- .../examples/xml_files/regression_demo.xml | 20 +- .../xml_files/ridge-scalar-ridge_demo.xml | 35 +- .../xml_files/ridge_regression_demo.xml | 22 +- .../examples/xml_files/scalar_demo.xml | 27 +- .../examples/xml_files/scaler-lasso_demo.xml | 28 +- .../examples/xml_files/user_feedback_demo.xml | 31 -- .../xml_files/user_feedback_demo2.xml | 46 +++ src/vai_lab/run_pipeline.py | 7 +- src/vai_lab/utils/plugins/MainPage.py | 4 +- src/vai_lab/utils/plugins/aidCanvas.py | 14 +- src/vai_lab/utils/plugins/dataLoader.py | 8 + src/vai_lab/utils/plugins/pluginCanvas.py | 244 +++++++++++--- 51 files changed, 1269 insertions(+), 263 deletions(-) create mode 100644 .github/workflows/documentation.yml create mode 100644 .github/workflows/pythonpackage.yml create mode 100644 src/vai_lab/DataProcessing/plugins/argopt.py create mode 100644 src/vai_lab/DataProcessing/plugins/integral.py create mode 100644 src/vai_lab/DecisionMaking/DecisionMaking_core.py delete mode 100644 src/vai_lab/DecisionMaking/Modelling_core.py create mode 100644 src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(GPy).py create mode 100644 src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(bayes_opt).py create mode 100644 src/vai_lab/UserInteraction/plugins/OptimisationInput.py create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/.DS_Store create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/CsFA/.DS_Store create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/CsFA/fulldata/CsFA_T300_above.csv create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/CsMA/.DS_Store create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/CsMA/fulldata/CsMA_T300_above.csv create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/FAMA/.DS_Store create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/FAMA/fulldata/FAMA_T300_above.csv create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/Readme create mode 100644 src/vai_lab/examples/crystalDesign/phasestability/template.csv create mode 100644 src/vai_lab/examples/optimisation/X.csv create mode 100644 src/vai_lab/examples/optimisation/Y.csv delete mode 100644 src/vai_lab/examples/xml_files/logistic_regression_demo.xml delete mode 100644 src/vai_lab/examples/xml_files/user_feedback_demo.xml create mode 100644 src/vai_lab/examples/xml_files/user_feedback_demo2.xml diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml new file mode 100644 index 00000000..e592d3dd --- /dev/null +++ b/.github/workflows/documentation.yml @@ -0,0 +1,26 @@ +name: Docs +on: [push, pull_request, workflow_dispatch] +jobs: + docs: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + with: + python-version: '3.10' + - name: Install dependencies + run: | + python -m pip install . + python -m pip install sphinx sphinx-rtd-theme myst-parser + - name: Sphinx build + run: | + sphinx-apidoc --templatedir docs/templates/apidoc -o docs/source src/vai_lab + sphinx-build -M html docs/source docs/build + - name: Deploy + uses: peaceiris/actions-gh-pages@v3 + if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} + with: + publish_branch: gh-pages + github_token: ${{ secrets.GITHUB_TOKEN }} + publish_dir: docs/build/html + force_orphan: true \ No newline at end of file diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml new file mode 100644 index 00000000..e7337f16 --- /dev/null +++ b/.github/workflows/pythonpackage.yml @@ -0,0 +1,47 @@ +name: CI Tests + +on: + pull_request: + workflow_dispatch: + inputs: + logLevel: + description: 'Log level' + required: true + default: 'warning' + tags: + description: 'Test scenario tags' + +jobs: + build: + runs-on: ${{ matrix.os }} + strategy: + matrix: + os: [ubuntu-latest, macos-latest, windows-latest] + python-version: ["3.10", "3.11"] + defaults: + run: + shell: bash + steps: + - uses: actions/checkout@v3 + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v4 + with: + python-version: ${{ matrix.python-version }} + architecture: x64 + - name: Install dependencies + run: | + python -m pip install --upgrade pip + python -m pip install . + - name: Test with pytest + # uses: GabrielBB/xvfb-action@v1.6 + uses: hankolsen/xvfb-action@dcb076c1c3802845f73bb6fe14a009d8d3377255 + with: + run: | + pip install pytest + pip install pytest-cov + pytest --cov=./src/vai_lab -v --cov-report=xml:./coverage.xml --cov-report term-missing + + + # Note: hankolsen/xvfb-action@dcb076c1c3802845f73bb6fe14a009d8d3377255 is a + # PR waiting to be merged to GabrielBB/xvfb-action@v1.6 to update xvfb-action + # to node.js 16. After the merge, switch back to GabrielBB/xvfb-action@v1.6 \ No newline at end of file diff --git a/src/vai_lab/Core/vai_lab_core.py b/src/vai_lab/Core/vai_lab_core.py index fd65ea49..0cf8356c 100644 --- a/src/vai_lab/Core/vai_lab_core.py +++ b/src/vai_lab/Core/vai_lab_core.py @@ -2,8 +2,9 @@ from sys import exit from os.path import join from typing import Dict, List, Tuple, Union +import pickle -from vai_lab._import_helper import import_module +from vai_lab._import_helper import import_module, rel_to_abs from vai_lab._plugin_helpers import PluginSpecs from vai_lab._types import ModuleInterface, PluginSpecsInterface from vai_lab.GUI.GUI_core import GUI @@ -13,7 +14,8 @@ class Core: def __init__(self) -> None: - self.data = Data() + self.data = {} + self.data['Initialiser'] = Data() self._xml_handler = XML_handler() self._avail_plugins: PluginSpecsInterface = PluginSpecs() @@ -33,7 +35,7 @@ def _launch(self): self.load_config_file(gui_output["xml_filename"]) except: raise Exception("No XML File Selected. Cannot Run Pipeline") - self._load_data() + self._load_data('Initialiser') def load_config_file(self, filename: Union[str,List,Tuple]): """Loads XML file into XML_handler object. @@ -46,10 +48,15 @@ def load_config_file(self, filename: Union[str,List,Tuple]): self._xml_handler.load_XML(filedir) self._initialised = True - def _load_data(self) -> None: + def _load_data(self, module) -> None: """Loads data from XML file into Data object""" - init_data_fn = self._xml_handler.data_to_load - self.data.import_data_from_config(init_data_fn) + init_data_fn = self._xml_handler.data_to_load(module) + if module not in self.data.keys(): + self.data[module] = Data() + if isinstance(init_data_fn, str): + self.data[module].import_existing_data(init_data_fn, self.data) + elif isinstance(init_data_fn, dict): + self.data[module].import_data_from_config(init_data_fn) def _execute_module(self, specs): """Executes named module with given options @@ -60,7 +67,8 @@ def _execute_module(self, specs): mod: ModuleInterface = import_module(globals(), specs["module_type"]).__call__() mod._debug = self._debug mod.set_avail_plugins(self._avail_plugins) - mod.set_data_in(self.data) + self._load_data(specs["name"]) + mod.set_data_in(self.data[specs["name"]]) mod.set_options(specs) print("\t"*self.loop_level + specs["module_type"] @@ -68,7 +76,7 @@ def _execute_module(self, specs): + "processing..." ) mod.launch() - self.data = mod.get_result() + self.data[specs["name"]] = mod.get_result() def _execute_loop(self, specs): if hasattr(self,"_execute_{}_loop".format(specs["type"])): @@ -90,9 +98,17 @@ def _execute_entry_point(self, specs): pass def _execute_exit_point(self, specs): - """Placeholder: Will parse the Output module when ready""" - pass - + """ Runs the Output module """ + if all(k in specs['plugin']['options'] for k in ('outdata', 'outpath')): + data_out = {} + if type(specs['plugin']['options']['outdata']) is list: + for module in specs['plugin']['options']['outdata']: + data_out[module] = self.data[module] + elif type(specs['plugin']['options']['outdata']) is str: + data_out[specs['plugin']['options']['outdata']] = self.data[specs['plugin']['options']['outdata']] + + with open(rel_to_abs(specs['plugin']['options']['outpath']), 'wb') as handle: + pickle.dump(data_out, handle, protocol=pickle.HIGHEST_PROTOCOL) def _parse_loop_condition(self, condition): try: condition = int(condition) @@ -102,7 +118,7 @@ def _parse_loop_condition(self, condition): except: print("Condition \"{0}\" cannot be parsed".format(condition)) print( - "Other formats in in development. Only ranged for loops are working currently") + "Other formats in development. Only ranged for loops are working currently") def _execute_for_loop(self, specs): condition = self._parse_loop_condition(specs["condition"]) @@ -169,7 +185,7 @@ def run(self): if not self._initialised: self._initialise_with_gui() print("Running pipeline...") - self._load_data() + self._load_data('Initialiser') self._init_status(self._xml_handler.loaded_modules) self._execute(self._xml_handler.loaded_modules) diff --git a/src/vai_lab/Data/Data_core.py b/src/vai_lab/Data/Data_core.py index 327ba0fe..c072955a 100644 --- a/src/vai_lab/Data/Data_core.py +++ b/src/vai_lab/Data/Data_core.py @@ -101,6 +101,10 @@ def import_data_from_config(self: DataT, config: dict) -> None: for c in config.keys(): self.import_data(config[c], c) + def import_existing_data(self: DataT, config: dict, data: DataT) -> None: + for item in data[config].keys(): + self.data[item] = data[config][item] + def append_data_column(self: DataT, col_name: str, data=None) -> None: self.data[col_name] = data diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 6d2c323c..8025af2c 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -178,7 +178,7 @@ def _load_data(self, element: ET.Element, parent: dict) -> None: "class": "data", "to_load": {}} for child in element: - assert "file" in child.attrib or "folder" in child.attrib,\ + assert "file" in child.attrib or "folder" in child.attrib or "module" in child.attrib,\ str("XML Parse Error \ \n\tA path to data file must be specified. \ \n\t{0} does not contain the \"file\" tag. \ @@ -198,6 +198,9 @@ def _load_data(self, element: ET.Element, parent: dict) -> None: .format(self._check_filename_abs_rel(child.attrib["folder"]))) parent[data_name]["to_load"][child.tag] = child.attrib["folder"] + if "module" in child.attrib: + parent[data_name]["to_load"] = child.attrib["module"] + def _load_exit_point(self, element: ET.Element, parent: dict) -> None: """Parses tags associated with output and appends to parent dict :param elem: xml.etree.ElementTree.Element to be parsed @@ -267,11 +270,12 @@ def _parse_text_to_list(self, element: ET.Element) -> List: :returns out: list containing parsed text data """ if element.text is not None: - new = element.text.strip().replace(" ", "") + new = element.text.strip() out: List[Any] = new.split("\n") raw_elem_text = str() for idx in range(0, len(out)): + out[idx] = " ".join(out[idx].split()) raw_elem_text = (raw_elem_text+"\n{}").format(out[idx]) if "[" in out[idx] and "]" in out[idx]: out[idx] = literal_eval(out[idx]) @@ -499,15 +503,23 @@ def append_input_data(self, if input_data_elem is None: input_data_elem = ET.SubElement(xml_parent, "inputdata") + data_name_elem = input_data_elem.find("./"+data_name) + if data_name_elem is not None: + input_data_elem.remove(data_name_elem) + plugin_elem = ET.SubElement(input_data_elem, data_name) if save_dir_as_relative: data_dir = data_dir.replace(self.lib_base_path, "./") data_dir = data_dir.replace("\\", "/") - plugin_elem.set('file', data_dir) + if path.exists(path.dirname(data_dir)): + plugin_elem.set('file', data_dir) + else: + plugin_elem.set('module', data_dir) def append_plugin_to_module(self, plugin_type: str, plugin_options: dict, + plugin_data: str, xml_parent: Union[ET.Element, str], overwrite_existing: Union[bool, int] = False ): @@ -515,6 +527,7 @@ def append_plugin_to_module(self, :param plugin_type: string type of plugin to be loaded into module :param plugin_options: dict where keys & values are options & values + :param plugin_data: string type of data to be used as input :param xml_parent: dict OR str. If string given, parent elem is found via search, Otherwise, plugin appeneded directly @@ -531,6 +544,8 @@ def append_plugin_to_module(self, if plugin_elem is None: plugin_elem = ET.SubElement(xml_parent, "plugin") plugin_elem.set('type', plugin_type) + if plugin_data is not None and len(plugin_data) > 0: + self.append_input_data('X', plugin_data, xml_parent, False) self._add_plugin_options(plugin_elem, plugin_options) def append_pipeline_module(self, @@ -562,6 +577,7 @@ def append_pipeline_module(self, if plugin_type != None: self.append_plugin_to_module(plugin_type, plugin_options, + parents[0], new_mod, 0 ) @@ -609,9 +625,9 @@ def append_pipeline_loop(self, xml_parent_element.append(new_loop) - def _get_init_data_structure(self) -> Dict[str, Any]: + def _get_data_structure(self, module) -> Dict[str, Any]: data_struct = self._find_dict_with_key_val_pair( - self.loaded_modules, + self.loaded_modules[module], "class", "data") assert len(data_struct) < 2, \ @@ -624,9 +640,12 @@ def _get_init_data_structure(self) -> Dict[str, Any]: out = {"to_load": {}} return out - @property - def data_to_load(self) -> Dict[str, str]: - return (self._get_init_data_structure()["to_load"]) + #@property + def data_to_load(self, module=None) -> Dict[str, str]: + if module is None: + return self._get_data_structure('Initialiser')["to_load"] + else: + return self._get_data_structure(module)["to_load"] # Use case examples: diff --git a/src/vai_lab/DataProcessing/DataProcessing_core.py b/src/vai_lab/DataProcessing/DataProcessing_core.py index ae681db3..666e2a2d 100644 --- a/src/vai_lab/DataProcessing/DataProcessing_core.py +++ b/src/vai_lab/DataProcessing/DataProcessing_core.py @@ -32,8 +32,7 @@ def launch(self) -> None: self._plugin.set_data_in(self._data_in) self._plugin.configure(self._module_config["plugin"]) self._plugin.fit() - self.output_data = self._data_in.copy() - self.output_data = self._plugin.transform(self.output_data) + self.output_data = self._plugin.transform(self._data_in) def get_result(self) -> DataInterface: return self.output_data diff --git a/src/vai_lab/DataProcessing/plugins/argopt.py b/src/vai_lab/DataProcessing/plugins/argopt.py new file mode 100644 index 00000000..4cf7219e --- /dev/null +++ b/src/vai_lab/DataProcessing/plugins/argopt.py @@ -0,0 +1,52 @@ +from distutils.command.config import config +from numpy import argmin, argmax +from vai_lab._plugin_templates import DataProcessingT +import pandas as pd + +_PLUGIN_READABLE_NAMES = {"argopt": "default", + "argmax": "alias", + "argmin": "alias"} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"Type": "math operator"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {'min/max': "str"} # type:ignore +_PLUGIN_REQUIRED_DATA = {} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore + +class argopt(DataProcessingT): + """ + Calculate the optimum argument + """ + + def __init__(self): + """Initialises parent class. + Passes `globals` dict of all current variables + """ + super().__init__(globals()) + # self.proc = model() + + def configure(self, config: dict): + """Sets and parses plugin configurations options + :param config: dict of internal tags set in the XML config file + """ + # super().configure(config) + self.config = config + + def set_data_in(self, data_in): + """Sets and parses incoming data + :param data_in: saves data as class variable + expected type: aidesign.Data.Data_core.Data + """ + super().set_data_in(data_in) + + def fit(self): + self.cleaned_options = self._clean_solver_options() + # self.proc.set_params(**cleaned_options) + # self.proc.fit(self.X) + return + + def transform(self,data): + if config['min/max'] == 'max': + data.append_data_column("X", pd.DataFrame(argmax(self.X))) + else: + data.append_data_column("X", pd.DataFrame(argmin(self.X))) + return data \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/integral.py b/src/vai_lab/DataProcessing/plugins/integral.py new file mode 100644 index 00000000..6778dbec --- /dev/null +++ b/src/vai_lab/DataProcessing/plugins/integral.py @@ -0,0 +1,32 @@ +from scipy.integrate import simps as model +from vai_lab._plugin_templates import DataProcessingT +import pandas as pd + +_PLUGIN_READABLE_NAMES = {"Integral": "default", + "integral": "alias"} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"Type": "math operator"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"dx": "float", + "axis": "int", + "even": "str"} # type:ignore +_PLUGIN_REQUIRED_DATA = {} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore + +class Integral(DataProcessingT): + """ + Calculate integral of array using the composite trapezoidal rule + """ + + def __init__(self): + """Initialises parent class. + Passes `globals` dict of all current variables + """ + super().__init__(globals()) + self.proc = model() + + def fit(self): + return + + def transform(self,data): + data.append_data_column("X", pd.DataFrame(self.proc(self.X))) + return data \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py index bcc13b9a..a844db5e 100644 --- a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py +++ b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py @@ -8,7 +8,7 @@ _PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"n_bins": "int"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore -_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore class KBinsDiscretizer(DataProcessingT): """ diff --git a/src/vai_lab/DecisionMaking/DecisionMaking_core.py b/src/vai_lab/DecisionMaking/DecisionMaking_core.py new file mode 100644 index 00000000..88c8ea44 --- /dev/null +++ b/src/vai_lab/DecisionMaking/DecisionMaking_core.py @@ -0,0 +1,36 @@ +# -*- coding: utf-8 -*- +from vai_lab._import_helper import import_plugin_absolute +class DecisionMaking(object): + def __init__(self): + self.output_data = None + + def set_avail_plugins(self,avail_plugins): + self._avail_plugins = avail_plugins + + def set_data_in(self,data_in): + self._data_in = data_in + + def _load_plugin(self, plugin_name:str): + avail_plugins = self._avail_plugins.find_from_readable_name(plugin_name) + self._plugin_name = plugin_name + self._plugin = import_plugin_absolute(globals(),\ + avail_plugins["_PLUGIN_PACKAGE"],\ + avail_plugins["_PLUGIN_CLASS_NAME"])\ + .__call__() + + def set_options(self, module_config: dict): + """Send configuration arguments to plugin + + :param module_config: dict of settings to configure the plugin + """ + self._module_config = module_config + self._load_plugin(self._module_config["plugin"]["plugin_name"]) + + def launch(self): + self._plugin.set_data_in(self._data_in) + self._plugin.configure(self._module_config["plugin"]) + # self._plugin.optimise() + self.output_data = self._plugin.suggest_locations() + + def get_result(self): + return self.output_data \ No newline at end of file diff --git a/src/vai_lab/DecisionMaking/Modelling_core.py b/src/vai_lab/DecisionMaking/Modelling_core.py deleted file mode 100644 index 97a81d24..00000000 --- a/src/vai_lab/DecisionMaking/Modelling_core.py +++ /dev/null @@ -1,29 +0,0 @@ -# -*- coding: utf-8 -*- -class Modelling(object): - def __init__(self): - self.node_name = None - self.plugin_name = None - self.output_data = None - - def set_input_data(self, data): - self.input_data = data - - def set_target_data(self, data): - self.target_data = data - - def _load_plugin(self): - pass - - def set_plugin_name(self, plugin_name): - self.plugin_name = plugin_name - self._load_plugin() - - def set_options(self, options): - self.options = options - - def solve(self): - # send the input data to the modelling plugin and assign it to an output property - pass - - def get_result(self): - return self.output_data diff --git a/src/vai_lab/DecisionMaking/__init__.py b/src/vai_lab/DecisionMaking/__init__.py index 99e78a66..77990d09 100644 --- a/src/vai_lab/DecisionMaking/__init__.py +++ b/src/vai_lab/DecisionMaking/__init__.py @@ -4,4 +4,4 @@ Let's look for a more automatic way of doing this when we have more modules """ -from .Modelling_core import Modelling +from .DecisionMaking_core import DecisionMaking diff --git a/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(GPy).py b/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(GPy).py new file mode 100644 index 00000000..bf9ff291 --- /dev/null +++ b/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(GPy).py @@ -0,0 +1,61 @@ +from vai_lab._plugin_templates import DecisionMakingPluginT +from GPyOpt.methods import BayesianOptimization as model +from typing import Dict + +_PLUGIN_READABLE_NAMES = {"GPyOpt": "default", + "BayesianOptimisation": "alias", + "BayesianOptimisation_GPy": "alias",} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"Type": "decision making"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {"f": "function"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"domain": "list", + "constraints": "list", + "acquisition_type ": "str", + "files": "list", + "normalize_Y": "bool", + "evaluator_type": "str", + "batch_size": "int", + "acquisition_jitter": "float"} # type:ignore +_PLUGIN_REQUIRED_DATA = {} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y"} # type:ignore + +class GPyOpt(DecisionMakingPluginT): + """ + Bayesian optimisation model using GPyOpt. Compatible with no objective function using tabular data. + """ + + def __init__(self): + """Initialises parent class. + Passes `globals` dict of all current variables + """ + super().__init__(globals()) + self.model = model + + def _parse_options_dict(self, options_dict:Dict): + super()._parse_options_dict(options_dict) + if self.X is not None: + options_dict['X'] = self.X + if self.Y is not None: + options_dict['Y'] = self.Y.reshape(-1,1) + return options_dict + + def optimise(self): + """Sends parameters to optimizer, then runs Bayesian Optimization for a number 'max_iter' of iterations""" + try: + self.BO.run_optimization() + except Exception as exc: + print('The plugin encountered an error when running the optimization ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') + raise + + def suggest_locations(self): + """Run a single optimization step and return the next locations to evaluate the objective. + Number of suggested locations equals to batch_size. + :returns: array, shape (n_samples,) + Returns suggested values. + """ + try: + return self.BO.suggest_next_locations() + except Exception as exc: + print('The plugin encountered an error when suggesting points with ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') + raise \ No newline at end of file diff --git a/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(bayes_opt).py b/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(bayes_opt).py new file mode 100644 index 00000000..544bcdc9 --- /dev/null +++ b/src/vai_lab/DecisionMaking/plugins/BayesianOptimisation(bayes_opt).py @@ -0,0 +1,49 @@ +from vai_lab._plugin_templates import DecisionMakingPluginT +from bayes_opt import BayesianOptimization as model + +_PLUGIN_READABLE_NAMES = {"bayes_opt": "default", + "BayesOpt": "alias",} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"Type": "decision making"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {"f": "function", + "pbounds": "dict"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = { + # "constraint": "ConstraintModel", + "random_state ": "int", + "verbose": "bool", + # "bounds_transformer": "DomainTransformer", + "allow_duplicate_points": "str"} # type:ignore +_PLUGIN_REQUIRED_DATA = {} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y"} # type:ignore + +class bayes_opt(DecisionMakingPluginT): + """ + Bayesian optimisation model using bayes_opt. + """ + + def __init__(self): + """Initialises parent class. + Passes `globals` dict of all current variables + """ + super().__init__(globals()) + self.model = model + + def optimise(self): + """Probes the target space to find the parameters that yield the maximum value for the given function.""" + try: + self.BO.maximize() + except Exception as exc: + print('The plugin encountered an error when running the optimization ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') + raise + + def suggest_locations(self, utility_function): + """Run a single optimization step and return the next locations to evaluate the objective. + :returns: array, shape (n_samples,) + Returns suggested values. + """ + try: + return self.BO.suggest(utility_function) + except Exception as exc: + print('The plugin encountered an error when suggesting points with ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') + raise \ No newline at end of file diff --git a/src/vai_lab/GUI/GUI_core.py b/src/vai_lab/GUI/GUI_core.py index 3fa6f524..bb3f3c3b 100644 --- a/src/vai_lab/GUI/GUI_core.py +++ b/src/vai_lab/GUI/GUI_core.py @@ -11,7 +11,8 @@ class GUI(tk.Tk): """ TODO: This structure still needs serious overhaul. - TODO: By having the TKinter controller as in the main GUI module, we are locked to using TKinter which defeats the purpose of being modular. This class needs to only be calling the plugins themselves, not acting as a controller. + TODO: By having the TKinter controller as in the main GUI module, we are locked to using TKinter which defeats the purpose of being modular. + This class needs to only be calling the plugins themselves, not acting as a controller. """ def __init__(self, *args, **kwargs): diff --git a/src/vai_lab/Modelling/plugins/knnclassifier.py b/src/vai_lab/Modelling/plugins/knnclassifier.py index 9ae82293..3a56f8ec 100644 --- a/src/vai_lab/Modelling/plugins/knnclassifier.py +++ b/src/vai_lab/Modelling/plugins/knnclassifier.py @@ -11,7 +11,7 @@ _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore -class KNNclassifier(ModellingPluginTClass): +class KNNClassifier(ModellingPluginTClass): """ Classifier implementing the k-nearest neighbors vote """ diff --git a/src/vai_lab/Modelling/plugins/ridgeregression.py b/src/vai_lab/Modelling/plugins/ridgeregression.py index ee0a234b..6578c83d 100644 --- a/src/vai_lab/Modelling/plugins/ridgeregression.py +++ b/src/vai_lab/Modelling/plugins/ridgeregression.py @@ -1,8 +1,8 @@ from vai_lab._plugin_templates import ModellingPluginT from sklearn.linear_model import Ridge as model -_PLUGIN_READABLE_NAMES = {"Ridge": "default", - "RidgeRegression": "alias"} # type:ignore +_PLUGIN_READABLE_NAMES = {"RidgeRegression": "default", + "Ridge": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"alpha": "float"} # type:ignore @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.clf = model() diff --git a/src/vai_lab/UserInteraction/plugins/CanvasInput.py b/src/vai_lab/UserInteraction/plugins/CanvasInput.py index 30256eaa..0de202e7 100644 --- a/src/vai_lab/UserInteraction/plugins/CanvasInput.py +++ b/src/vai_lab/UserInteraction/plugins/CanvasInput.py @@ -11,7 +11,7 @@ import pandas as pd import math -_PLUGIN_READABLE_NAMES = {"canvas": "default", +_PLUGIN_READABLE_NAMES = {"CanvasInput": "default", "state-action": "alias", "robot": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"layer_priority": 2, "required_children": None} # type:ignore diff --git a/src/vai_lab/UserInteraction/plugins/ManualInput.py b/src/vai_lab/UserInteraction/plugins/ManualInput.py index 9e6a40e9..3e2a1000 100644 --- a/src/vai_lab/UserInteraction/plugins/ManualInput.py +++ b/src/vai_lab/UserInteraction/plugins/ManualInput.py @@ -12,7 +12,7 @@ from tkinter import messagebox, ttk from tkinter.filedialog import asksaveasfile -_PLUGIN_READABLE_NAMES = {"manual": "default", +_PLUGIN_READABLE_NAMES = {"ManualInput": "default", "binary": "alias", "classification": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"layer_priority": 2, diff --git a/src/vai_lab/UserInteraction/plugins/OptimisationInput.py b/src/vai_lab/UserInteraction/plugins/OptimisationInput.py new file mode 100644 index 00000000..d0027488 --- /dev/null +++ b/src/vai_lab/UserInteraction/plugins/OptimisationInput.py @@ -0,0 +1,301 @@ +from vai_lab._plugin_templates import UI +from vai_lab._import_helper import get_lib_parent_dir +from vai_lab._types import DictT, DataInterface, GUICoreInterface + +import os +import numpy as np +import pandas as pd +from typing import Tuple, List, Union +from PIL import Image, ImageTk, PngImagePlugin +import matplotlib.pyplot as plt +from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg + +import tkinter as tk +from tkinter import messagebox, ttk +from tkinter.filedialog import asksaveasfile + +_PLUGIN_READABLE_NAMES = {"OptimisationInput": "default", + "BOUI": "alias", + "optimisationUI": "alias"} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"layer_priority": 2, + "required_children": None} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"Bounds": "list"} # type:ignore +_PLUGIN_REQUIRED_DATA = {"X"} # type:ignore + + +class OptimisationInput(tk.Frame, UI): # type:ignore + """Method of user interaction for optimisation problems""" + + def __init__(self, parent, controller, config: DictT): + self.parent = parent + super().__init__(parent, bg=self.parent['bg']) + self.controller: GUICoreInterface = controller + self.controller.title('Optimisation Interaction') + + self.dirpath = get_lib_parent_dir() + self.tk.call('wm', 'iconphoto', self.controller._w, ImageTk.PhotoImage( + file=os.path.join(os.path.join( + self.dirpath, + 'utils', + 'resources', + 'Assets', + 'VAILabsIcon.ico')))) + + self.assets_path = os.path.join(self.dirpath, 'utils', 'resources', 'Assets') + + self._data_in: DataInterface + self._config = config + self.save_path = '' + self.saved = True + + + def _load_values_from_data(self): + + self.frame1 = tk.Frame(self, bg=self.parent['bg']) + frame4 = tk.Frame(self, bg=self.parent['bg']) + frame5 = tk.Frame(self, bg=self.parent['bg']) + # frame6 = tk.Frame(self, bg=self.parent['bg']) + + self.opt_var = list(self._data_in["X"].columns.values) + if len(self.opt_var) < 3: + figure = plt.Figure(figsize=(5, 4), dpi=100) + self.ax = figure.add_subplot(111) + + self.plot_points(self._data_in["X"], self.opt_var) + + self.canvas = FigureCanvasTkAgg(figure, self.frame1) + plot_frame = self.canvas.get_tk_widget() + plot_frame.grid(column=0, row=0, pady=10, padx=10, sticky="nsew") + self.frame1.grid_rowconfigure(0, weight=1) + self.frame1.grid_columnconfigure(0, weight=1) + + # Inital window + self.N = len(self._data_in["X"]) + + # Buttons + self.button_save = tk.Button( + frame4, text='Save', fg='white', bg=self.parent['bg'], height=3, + width=20, command=self.save_file) + self.button_save.grid(column=0, row=0, sticky="news", pady=2, padx=[2,0]) + + tk.Button( + frame5, text="Done", + fg='white', bg=self.parent['bg'], + height=3, width=20, + command=self.check_quit).grid(column=0, row=0, sticky="news", pady=2, padx=[0,2]) + + self.frame1.grid(row=0, column=0, sticky="nsew") + frame4.grid(row=1, column=0, sticky="nsew") + frame5.grid(row=1, column=1, sticky="nsew") + + frame4.grid_columnconfigure(0, weight=1) + frame5.grid_columnconfigure(0, weight=1) + self.grid_rowconfigure(0, weight=1) + self.grid_columnconfigure(tuple(range(2)), weight=1) + + def plot_points(self, data, labels, x=[None]): + """Plots points in pre-existing axis. If some extra points are given, + these are plotted with a different colour. + :param data: dict, dictionary to be plotted + :param labels: list, column and axis labels + :param x: array, extra points to be plotted + """ + + self.ax.clear() # clear axes from previous plot + self.ax.scatter(data[labels[0]], data[labels[1]]) + self.ax.set_xlabel(labels[0]) + self.ax.set_ylabel(labels[1]) + # self.ax.set_xlim(min(data[labels[0]]), max(data[labels[0]])) + # self.ax.set_ylim(min(data[labels[0]]), max(data[labels[0]])) + self.ax.set_title('Suggested points') + if None not in x: + self.ax.scatter(x[0], x[1], color='r') + + def set_data_in(self, data_in): + req_check = [ + r for r in _PLUGIN_REQUIRED_DATA if r not in data_in.keys()] + if len(req_check) > 0: + raise Exception("Minimal Data Requirements not met" + + "\n\t{0} ".format("OptimisationInput") + + "requires data: {0}".format(_PLUGIN_REQUIRED_DATA) + + "\n\tThe following data is missing:" + + "\n\t\u2022 {}".format(",\n\t\u2022 ".join([*req_check]))) + self._data_in = data_in + self._load_values_from_data() + self._load_classes_from_data() + + def class_list(self): + """Getter for required _class_list variable + + :return: list of class labels + :rtype: list of strings + """ + return self._class_list + + def _load_classes_from_data(self): + """Setter for required _class_list variable + + :param value: class labels for binary classification + :type value: list of strings + """ + self.out_data = self._data_in["X"] + + self.button_cl = {} + + # Tree defintion. Output display + style = ttk.Style() + style.configure( + "Treeview", background='white', foreground='white', + rowheight=25, fieldbackground='white', + font=self.controller.pages_font) + style.configure("Treeview.Heading", font=self.controller.pages_font) + style.map('Treeview', background=[('selected', 'grey')]) + + tree_frame = tk.Frame(self) + if len(self.opt_var) < 3: + tree_frame.grid(row=0, column=1, sticky="nsew", pady=10, padx=10) + else: + tree_frame.grid(row=0, column=0, columnspan = 2, sticky="nsew", pady=10, padx=10) + + tree_scrollx = tk.Scrollbar(tree_frame, orient='horizontal') + tree_scrollx.pack(side=tk.BOTTOM, fill=tk.X) + tree_scrolly = tk.Scrollbar(tree_frame) + tree_scrolly.pack(side=tk.RIGHT, fill=tk.Y) + + self.tree = ttk.Treeview(tree_frame, + yscrollcommand=tree_scrolly.set, + xscrollcommand=tree_scrollx.set) + self.tree.pack(fill='both', expand=True) + + tree_scrollx.config(command=self.tree.xview) + tree_scrolly.config(command=self.tree.yview) + + self.tree['columns'] = self.opt_var + + # Format columns + self.tree.column("#0", width=80, + minwidth=50) + for n, cl in enumerate(self.opt_var): + self.tree.column( + cl, width=int(self.controller.pages_font.measure(str(cl)))+20, + minwidth=50, anchor=tk.CENTER) + # Headings + self.tree.heading("#0", text="Sample", anchor=tk.CENTER) + for cl in self.opt_var: + self.tree.heading(cl, text=cl, anchor=tk.CENTER) + self.tree.tag_configure('odd', foreground='black', + background='#E8E8E8') + self.tree.tag_configure('even', foreground='black', + background='#DFDFDF') + # Add data + for n, sample in enumerate(self.out_data.values): + if n % 2 == 0: + self.tree.insert(parent='', index='end', iid=n, text=n+1, + values=tuple(sample.astype(float)), tags=('even',)) + else: + self.tree.insert(parent='', index='end', iid=n, text=n+1, + values=tuple(sample.astype(float)), tags=('odd',)) + + # Select the current row + self.tree.selection_set(str(int(0))) + + # Define click on row action + if len(self.opt_var) < 3: + self.tree.bind('', self.OnClick) + + # Define double-click on row action + self.tree.bind("", self.OnDoubleClick) + + def OnClick(self, event): + "Displays the corresponding ." + + item = self.tree.selection()[0] + x = [float(i) for i in self.tree.item(item)['values']] + if len(self.opt_var) < 3: + self.plot_points(self.out_data, self.opt_var, x = x) + self.canvas.draw() + + def OnDoubleClick(self, event): + """ Executed when a row is double clicked. + Opens an entry box to edit a cell and updates the plot and the + stored data. """ + + self.treerow = int(self.tree.identify_row(event.y)) + self.treecol = self.tree.identify_column(event.x) + + # get column position info + x, y, width, height = self.tree.bbox(self.treerow, self.treecol) + + # y-axis offset + pady = height // 2 + # pady = 0 + + if hasattr(self, 'entry'): + self.entry.destroy() + + self.entry = tk.Entry(self.tree, justify='center') + + if int(self.treecol[1:]) > 0: + self.entry.insert( + 0, self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1]) + self.entry['exportselection'] = False + + self.entry.focus_force() + self.entry.bind("", self.OnReturn) + self.entry.bind("", lambda *ignore: self.entry.destroy()) + + self.entry.place(x=x, + y=y + pady, + anchor=tk.W, width=width) + + def OnReturn(self, event): + """ Updates the stored data with the values in the entry. """ + val = self.tree.item(self.treerow)['values'] + val = [float(i) for i in val] + val[int(self.treecol[1:])-1] = float(self.entry.get()) + self.tree.item(self.treerow, values=val) + self.entry.destroy() + self.saved = False + + self.out_data.loc[self.treerow] = val + + self.OnClick(0) + self.saved = False + + def check_quit(self): + + if not self.saved: + response = messagebox.askokcancel( + "Exit?", + "Do you want to leave the program without saving?") + if response: + self.controller.destroy() + else: + response = messagebox.askokcancel( + "Exit?", + "Are you sure you are finished?") + self.controller.destroy() + + def save_file_as(self): + + self.save_path = asksaveasfile(mode='w') + self.save_file() + + def save_file(self): + + if self.save_path == '': + self.save_path = asksaveasfile(defaultextension='.txt', + filetypes=[('Text file', '.txt'), + ('CSV file', '.csv'), + ('All Files', '*.*')]) + # asksaveasfile return `None` if dialog closed with "cancel". + if self.save_path is not None: + filedata = pd.DataFrame( + self.out_data, columns=self.opt_var).to_string() + self.save_path.seek(0) # Move to the first row to overwrite it + self.save_path.write(filedata) + self.save_path.flush() # Save without closing + # typically the above line would do. however this is used to ensure that the file is written + os.fsync(self.save_path.fileno()) + self.saved = True \ No newline at end of file diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 2973574c..9d42a62f 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -113,6 +113,12 @@ def _parse_options_dict(self,options_dict:Dict): val = int(val) cleaned_opts.append(val) options_dict[key] = cleaned_opts + elif type(val) == str and val.lower() in ('y', 'yes', 't', 'true', 'on'): + options_dict[key] = True + elif type(val) == str and val.lower() in ('n', 'no', 'f', 'false', 'off'): + options_dict[key] = False + elif type(val) == str and val.lower() in ('none'): + options_dict[key] = None return options_dict def _clean_options(self): @@ -120,7 +126,7 @@ def _clean_options(self): and modifies DataInterface in-place str options which only contain numeric data are converted to float OR int """ - self._parse_options_dict(self._config["options"]) + return self._parse_options_dict(self._config["options"]) def _test(self, data: DataInterface) -> DataInterface: """Run debug tests on data operations @@ -280,6 +286,22 @@ def predict_proba(self, data): +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') raise +class DecisionMakingPluginT(PluginTemplate, ABC): + def __init__(self, plugin_globals: dict) -> None: + super().__init__(plugin_globals) + + def configure(self, config: dict): + """Extended from PluginTemplate.configure""" + super().configure(config) + try: + self.BO = self.model(**self._clean_options()) + except Exception as exc: + print('The plugin encountered an error on the parameters of ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') + raise + if type(self.X) is None and type(self.Y) is None: + print('Invalid Data name. Indicate whether to use `X` or `Y`') + class UI(PluginTemplate, ABC): @property diff --git a/src/vai_lab/examples/crystalDesign/phasestability/.DS_Store b/src/vai_lab/examples/crystalDesign/phasestability/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..0084bcb0d935f7fdeaf748e680dd4a5c5bad4435 GIT binary patch literal 6148 zcmeHKOHRW;47E!ZMQplcIalZnh7#DZ>ID#q1dBwe)OMev!7X|Wj==NS3ezUpB2?Lu z=S}=cJTFol6A^EY^_pl!L`$gP;ugzD~St7m!`&VVzJGa%oG02Pdey<+-wU`ZnY zuz)!U_A*OIPB4syy&^mi)>NRTvW*z5>97Zji-x_TrW4!n!S-af35C=SytH_pZi3%xb zH`<%?-K_6!Y}|m5 z?dvP;YlPF;(*$Q^KOITD>k7C6uE13)z??0X-79+Q3b+EUz(oQ1K18};ZrCcOPY0V+ z0ubjMPR72}5)u;)bHi4VGqj+gL=6pAVgwE6cr;#a*eYr`f)yXZRt{F7gtR*E9}^uR zSM=5ua0U7b-00;(^8e#>{ofDrD_6i3_*V*qd9f~5Jd)+s!Qmv=M*4;>CcIYh62eYO g#pFsVexN7gc$5w?H*6I-Li0ZYNd|9Rfj?E?2Xt;x>i_@% literal 0 HcmV?d00001 diff --git a/src/vai_lab/examples/crystalDesign/phasestability/CsFA/fulldata/CsFA_T300_above.csv b/src/vai_lab/examples/crystalDesign/phasestability/CsFA/fulldata/CsFA_T300_above.csv new file mode 100644 index 00000000..12a64d5c --- /dev/null +++ b/src/vai_lab/examples/crystalDesign/phasestability/CsFA/fulldata/CsFA_T300_above.csv @@ -0,0 +1 @@ +Cs,MA,FA,dEmix (ev/f.u.),TdSmix (ev/f.u.),dGmix (ev/f.u.) 1.00,0.00,0.00,0.000,0.000,0.000 0.92,0.00,0.08,0.043,0.007,0.036 0.92,0.00,0.08,0.053,0.007,0.046 0.88,0.00,0.13,0.055,0.010,0.045 0.88,0.00,0.13,0.033,0.010,0.023 0.83,0.00,0.17,0.090,0.012,0.078 0.83,0.00,0.17,0.089,0.012,0.077 0.75,0.00,0.25,0.041,0.015,0.026 0.75,0.00,0.25,0.038,0.015,0.023 0.73,0.00,0.28,0.020,0.015,0.005 0.73,0.00,0.28,0.017,0.016,0.001 0.67,0.00,0.33,0.011,0.016,-0.005 0.67,0.00,0.33,0.008,0.016,-0.008 0.65,0.00,0.35,0.000,0.017,-0.017 0.65,0.00,0.35,0.000,0.017,-0.017 0.63,0.00,0.38,-0.043,0.017,-0.060 0.63,0.00,0.38,-0.048,0.017,-0.065 0.58,0.00,0.42,-0.011,0.018,-0.029 0.58,0.00,0.42,-0.004,0.018,-0.022 0.50,0.00,0.50,-0.047,0.018,-0.065 0.50,0.00,0.50,-0.027,0.018,-0.045 0.42,0.00,0.58,-0.052,0.018,-0.070 0.42,0.00,0.58,-0.045,0.018,-0.063 0.38,0.00,0.63,-0.095,0.017,-0.112 0.38,0.00,0.63,-0.100,0.017,-0.117 0.33,0.00,0.67,-0.083,0.016,-0.099 0.33,0.00,0.67,-0.088,0.016,-0.104 0.25,0.00,0.75,-0.137,0.015,-0.152 0.25,0.00,0.75,-0.142,0.015,-0.157 0.17,0.00,0.83,-0.085,0.012,-0.097 0.17,0.00,0.83,-0.091,0.012,-0.103 0.13,0.00,0.88,-0.069,0.010,-0.079 0.13,0.00,0.88,-0.074,0.010,-0.084 0.08,0.00,0.92,-0.110,0.007,-0.117 0.08,0.00,0.92,-0.115,0.007,-0.122 0.00,0.00,1.00,0.000,0.000,0.000 \ No newline at end of file diff --git a/src/vai_lab/examples/crystalDesign/phasestability/CsMA/.DS_Store b/src/vai_lab/examples/crystalDesign/phasestability/CsMA/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..5bdbc9ea04ee6e5e413f9d38c05b7e4363b17af6 GIT binary patch literal 6148 zcmeHKJ5Iw;5S)b+mS|E^zANMgPf<7l7a(Azij)GAHDXxOnQv9s3#Y`S!)o}jda~7oV)1mI-=ZAW6BVU^6u4C2 zKDTS{|4;Nk=Ko8Qc2YnJ+>`>g+CA^Ke5LBGvzPN;+vqQJulb_8aUB$fXvf58$J}^3 eevG25Yrf|FUN|KNo%x^>^)ukQ$fUrv75D~yVHI@% literal 0 HcmV?d00001 diff --git a/src/vai_lab/examples/crystalDesign/phasestability/CsMA/fulldata/CsMA_T300_above.csv b/src/vai_lab/examples/crystalDesign/phasestability/CsMA/fulldata/CsMA_T300_above.csv new file mode 100644 index 00000000..f2358371 --- /dev/null +++ b/src/vai_lab/examples/crystalDesign/phasestability/CsMA/fulldata/CsMA_T300_above.csv @@ -0,0 +1 @@ +Cs,MA,FA,dEmix (ev/f.u.),TdSmix (ev/f.u.),dGmix (ev/f.u.) 1.00,0.00,0.00,0.000,0.000,0.000 0.92,0.08,0.00,0.054,0.007,0.047 0.88,0.12,0.00,0.043,0.010,0.033 0.88,0.12,0.00,0.058,0.010,0.048 0.75,0.25,0.00,0.044,0.015,0.029 0.67,0.33,0.00,0.053,0.016,0.037 0.63,0.37,0.00,0.024,0.017,0.007 0.63,0.37,0.00,0.026,0.017,0.009 0.50,0.50,0.00,0.016,0.018,-0.002 0.50,0.50,0.00,0.037,0.018,0.019 0.42,0.58,0.00,0.012,0.018,-0.006 0.42,0.58,0.00,0.027,0.018,0.009 0.38,0.62,0.00,0.001,0.017,-0.016 0.38,0.62,0.00,0.003,0.017,-0.014 0.33,0.67,0.00,0.029,0.016,0.013 0.33,0.67,0.00,0.031,0.016,0.015 0.25,0.75,0.00,0.005,0.015,-0.010 0.25,0.75,0.00,0.005,0.015,-0.010 0.17,0.83,0.00,-0.016,0.012,-0.028 0.17,0.83,0.00,-0.013,0.012,-0.025 0.13,0.87,0.00,-0.038,0.010,-0.048 0.13,0.87,0.00,-0.026,0.010,-0.036 0.08,0.92,0.00,-0.014,0.007,-0.021 0.08,0.92,0.00,-0.045,0.007,-0.052 0.00,1.00,0.00,0.000,0.000,0.000 \ No newline at end of file diff --git a/src/vai_lab/examples/crystalDesign/phasestability/FAMA/.DS_Store b/src/vai_lab/examples/crystalDesign/phasestability/FAMA/.DS_Store new file mode 100644 index 0000000000000000000000000000000000000000..9ffa3a33701ca322297e306fceeb140fe11e8206 GIT binary patch literal 6148 zcmeHKK~BRk5FD2t0$h6J=r8mKQ59a$4^R@Ihn7^LK^*f99>F8{1<&9K%&u2eq9hV0 z1ZcNfk7Liq_9V)70fbdkErA7qj4n7kXES0lFFvrDjYwS3d)(m#TQu15=k^yAkY{&+ z5;Y$2jL!4hJuKH{y+3pgG0NN5MNu`!68QLhe^vGbulg46upO+M;ojb<3L6};sh-N+ zZC&d-HkcgQ_<+;f^qX#agwxsZ1ZQNk9lgBk3b+EUz;9K6Ia@5dOz5pE;0m|`Ck5pD z5b1)sVVh7t9c;7&AkI0g#=g`N5)%z`!!{vjXd#6XQ)sXfBcyPSN8{y&ZNe0eV8utU zm4j6%L09MfqtX%bgx - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -19,24 +19,27 @@ - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0,350.0),2,{0:'d0-u2'}] - - - 50 - - - distance - - + + + + - - [(300.0,550),1,{2:'d2-u1'}] + [(350.0,650),1,{2:'d2-u1'}] + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/SVR_demo.xml b/src/vai_lab/examples/xml_files/SVR_demo.xml index 1323ee67..f94fa31e 100644 --- a/src/vai_lab/examples/xml_files/SVR_demo.xml +++ b/src/vai_lab/examples/xml_files/SVR_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -19,27 +19,34 @@ - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0,350.0),2,{0:'d0-u2'}] + + + - 0.01 + 0.1 - rbf + linear - - 10 - - - [(300.0,550),1,{2:'d2-u1'}] + [(350.0,650),1,{2:'d2-u1'}] + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/canvas_demo.xml b/src/vai_lab/examples/xml_files/canvas_demo.xml index 5ed1f55d..3966377a 100644 --- a/src/vai_lab/examples/xml_files/canvas_demo.xml +++ b/src/vai_lab/examples/xml_files/canvas_demo.xml @@ -1,30 +1,41 @@ - + + + [(350.0,50),0,{}] + - + - + - + + + [(350.0,350.0),2,{0:'d0-u2'}] + - - - all - - + + + Modelling + + + .\examples\results\output.pkl + + + [(350.0,650),1,{2:'d2-u1'}] + diff --git a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml index 7c0a369f..658fc9db 100644 --- a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml +++ b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -17,11 +17,14 @@ - [(301,201),2,{0:'d0-u2'}] + [(350,200),2,{0:'d0-u2'}] - + + + + - X + X @@ -31,17 +34,27 @@ - [(300.0,300.0),3,{2:'d2-u3'}] + [(350.0,350.0),3,{2:'d2-u3'}] + + + - - [(300.0,550),1,{3:'d3-u1'}] + [(350.0,650),1,{3:'d3-u1'}] + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/logistic_regression_demo.xml b/src/vai_lab/examples/xml_files/logistic_regression_demo.xml deleted file mode 100644 index 7606ece0..00000000 --- a/src/vai_lab/examples/xml_files/logistic_regression_demo.xml +++ /dev/null @@ -1,42 +0,0 @@ - - - - - - - [(300.0,50),0,{}] - - - - - - - - - - - - - - - [(300.0,300.0),2,{0:'d0-u2'}] - - - - l2 - - - 0.01 - - - - - - - - - - [(300.0,550),1,{2:'d2-u1'}] - - - diff --git a/src/vai_lab/examples/xml_files/pybullet_env_example.xml b/src/vai_lab/examples/xml_files/pybullet_env_example.xml index 886051e0..b12be5c7 100644 --- a/src/vai_lab/examples/xml_files/pybullet_env_example.xml +++ b/src/vai_lab/examples/xml_files/pybullet_env_example.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -13,7 +13,7 @@ - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0,350.0),2,{0:'d0-u2'}] @@ -35,14 +35,24 @@ 10 + + + - + + + Modelling + + + .\examples\results\output.pkl + + - [(300.0,550),1,{2:'d2-u1'}] + [(350.0,650),1,{2:'d2-u1'}] \ No newline at end of file diff --git a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml index e7c0d450..42d4e203 100644 --- a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml +++ b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml @@ -9,6 +9,9 @@ + + [(350.0,50),0,{}] + @@ -21,6 +24,9 @@ X + + + @@ -29,13 +35,28 @@ + + + + + [(350.0,350.0),2,{0:'d0-u2'}] + + + [(350.0,650),1,{2:'d2-u1'}] + - + + + My Modelling Module + + + .\examples\results\output.pkl + + - diff --git a/src/vai_lab/examples/xml_files/regression_demo.xml b/src/vai_lab/examples/xml_files/regression_demo.xml index a528722b..38faeca5 100644 --- a/src/vai_lab/examples/xml_files/regression_demo.xml +++ b/src/vai_lab/examples/xml_files/regression_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -17,18 +17,28 @@ - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0,350.0),2,{0:'d0-u2'}] + + + - - + + + + Modelling + + + .\examples\results\output.pkl + + - [(300.0,550),1,{2:'d2-u1'}] + [(350.0,650),1,{2:'d2-u1'}] diff --git a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml index 777ca31c..c98d1191 100644 --- a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml +++ b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -19,13 +19,16 @@ - [(300,180),2,{0:'d0-u2'}] + [(350,180),2,{0:'d0-u2'}] 0.01 + + + @@ -33,12 +36,15 @@ - [(300.0,300.0),3,{2:'d2-u3'}] + [(350.0,350.0),3,{2:'d2-u3'}] + + + - - True - + + X + @@ -47,8 +53,11 @@ - [(300,408),4,{3:'d3-u4'}] + [(350,408),4,{3:'d3-u4'}] + + + 0.01 @@ -56,12 +65,20 @@ - - [(300.0,550),1,{4:'d4-u1'}] + [(350.0,650),1,{4:'d4-u1'}] + + + Modelling + Modelling-1 + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/ridge_regression_demo.xml b/src/vai_lab/examples/xml_files/ridge_regression_demo.xml index 67b44bcd..58e182f7 100644 --- a/src/vai_lab/examples/xml_files/ridge_regression_demo.xml +++ b/src/vai_lab/examples/xml_files/ridge_regression_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -19,21 +19,31 @@ - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0,350.0),2,{0:'d0-u2'}] - 0.02 + 0.01 + + + - - [(300.0,550),1,{2:'d2-u1'}] - + [(350.0,650),1,{2:'d2-u1'}] + + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/scalar_demo.xml b/src/vai_lab/examples/xml_files/scalar_demo.xml index 10e3f914..5b2b1d9a 100644 --- a/src/vai_lab/examples/xml_files/scalar_demo.xml +++ b/src/vai_lab/examples/xml_files/scalar_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0, 50), 0, {}] @@ -14,26 +14,39 @@ + + + - [(300.0,300.0),2,{0:'d0-u2'}] + [(350.0, 350.0), 2, {0: 'd0-u2'}] - - True - + + X + - + + + - [(300.0,550),1,{2:'d2-u1'}] + [(350.0, 650), 1, {2: 'd2-u1'}] + + + Data Processing + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml b/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml index 45a93a90..a05d1bfa 100644 --- a/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml +++ b/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml @@ -4,7 +4,7 @@ - [(300.0,50),0,{}] + [(350.0,50),0,{}] @@ -19,8 +19,11 @@ - [(296,183),2,{0:'d0-u2'}] + [(350,183),2,{0:'d0-u2'}] + + + X @@ -33,21 +36,32 @@ - [(300,328),3,{2:'d2-u3'}] + [(350,328),3,{2:'d2-u3'}] - + + + + - 0.0002 + 0.2 - - [(300.0,550),1,{3:'d3-u1'}] + [(350.0,650),1,{3:'d3-u1'}] + + + Data Processing + Modelling + + + ./examples/results/test.pkl + + diff --git a/src/vai_lab/examples/xml_files/user_feedback_demo.xml b/src/vai_lab/examples/xml_files/user_feedback_demo.xml deleted file mode 100644 index 6d1a21a6..00000000 --- a/src/vai_lab/examples/xml_files/user_feedback_demo.xml +++ /dev/null @@ -1,31 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - all - - - - - - - - diff --git a/src/vai_lab/examples/xml_files/user_feedback_demo2.xml b/src/vai_lab/examples/xml_files/user_feedback_demo2.xml new file mode 100644 index 00000000..bfe76cc7 --- /dev/null +++ b/src/vai_lab/examples/xml_files/user_feedback_demo2.xml @@ -0,0 +1,46 @@ + + + + + + + + + + + [(350.0, 50), 0, {}] + + + + + + + + + + + + [(350.0, 350.0), 2, {0: 'd0-u2'}] + + + + + + + + + + + + [(350.0, 650), 1, {2: 'd2-u1'}] + + + + User Interaction + + + .\examples\results\output.pkl + + + + diff --git a/src/vai_lab/run_pipeline.py b/src/vai_lab/run_pipeline.py index 4aa9c9f8..f6eb4ea6 100644 --- a/src/vai_lab/run_pipeline.py +++ b/src/vai_lab/run_pipeline.py @@ -57,8 +57,11 @@ def main(): for i in range(0,len(args.file)): args.file[i] = abspath(args.file[i]) core.load_config_file(args.file) - - # Run pipeline + core.load_config_file(("./examples", + "xml_files", + # 'bayes_opt_demo.xml')) + 'dataFlowTest.xml')) + # Run pipeline core.run() if __name__=='__main__': diff --git a/src/vai_lab/utils/plugins/MainPage.py b/src/vai_lab/utils/plugins/MainPage.py index 92043453..e9b5dabe 100644 --- a/src/vai_lab/utils/plugins/MainPage.py +++ b/src/vai_lab/utils/plugins/MainPage.py @@ -320,7 +320,7 @@ def start_dataloader(self): # Infers by default, should it be None? data[variable] = pd.read_csv(filename) isVar[i] = 1 - self.controller.s.append_input_data(variable, rel_to_abs(filename)) + self.controller.xml_handler.append_input_data(variable, rel_to_abs(filename)) if i == 0: self.controller.Data.set(True) if any(isVar[1::2]) and ( @@ -343,7 +343,7 @@ def upload_data_path(self): title='Select a folder', mustexist=True) if folder is not None and len(folder) > 0: - self.controller.s.append_input_data('X', rel_to_abs(folder)) + self.controller.xml_handler.append_input_data('X', rel_to_abs(folder)) def upload_data_folder(self): """ Stores the directory containing the data that will be later loaded diff --git a/src/vai_lab/utils/plugins/aidCanvas.py b/src/vai_lab/utils/plugins/aidCanvas.py index ced0e04a..04171f0d 100644 --- a/src/vai_lab/utils/plugins/aidCanvas.py +++ b/src/vai_lab/utils/plugins/aidCanvas.py @@ -702,9 +702,9 @@ def save_file(self): # to avoid numpy bug during elementwise comparison of lists loop_modules.dtype = mn.dtype if len(loop_modules) == 0 else loop_modules.dtype - self.controller.s.update_module_coords(self.module_list[0],[self.canvas_startxy[0], 0, self.connections[0]]) - self.controller.s.append_module_relationships(self.module_list[0],list(mn[values[:,0]]),list(mn[values[0,:]])) - self.controller.s.filename = self.save_path.name + self.controller.xml_handler.update_module_coords(self.module_list[0],[self.canvas_startxy[0], 0, self.connections[0]]) + self.controller.xml_handler.append_module_relationships(self.module_list[0],list(mn[values[:,0]]),list(mn[values[0,:]])) + self.controller.xml_handler.filename = self.save_path.name for i, mnn in enumerate(mn_id): if (i > 1) and mnn: xml_parent = None @@ -713,7 +713,7 @@ def save_file(self): for x, loop in enumerate(self.loops): if mn[i] in loop['mod']: # Model in this loop if not out_loops[x]: # Loop already defined - self.controller.s.append_pipeline_loop(self.loops[x]['type'], + self.controller.xml_handler.append_pipeline_loop(self.loops[x]['type'], self.loops[x]['condition'], "loop" + str(x), @@ -728,7 +728,7 @@ def save_file(self): xml_parent = "loop"+str(x) parent_loops.append("loop"+str(x)) - self.controller.s.append_pipeline_module(self.module_list[i], + self.controller.xml_handler.append_pipeline_module(self.module_list[i], mn[i], "", {}, @@ -739,7 +739,7 @@ def save_file(self): xml_parent, [self.canvas_startxy[i], i, self.connections[i]]) - self.controller.s.append_pipeline_module(self.module_list[1], # Out + self.controller.xml_handler.append_pipeline_module(self.module_list[1], # Out mn[1], "", {}, @@ -747,7 +747,7 @@ def save_file(self): list(mn[values[1, :]]), None, [self.canvas_startxy[1], 1, self.connections[1]]) - self.controller.s.write_to_XML() + self.controller.xml_handler.write_to_XML() self.saved = True self.controller._append_to_output( "xml_filename", self.save_path.name) diff --git a/src/vai_lab/utils/plugins/dataLoader.py b/src/vai_lab/utils/plugins/dataLoader.py index bdff82a1..cd0559a4 100644 --- a/src/vai_lab/utils/plugins/dataLoader.py +++ b/src/vai_lab/utils/plugins/dataLoader.py @@ -6,6 +6,14 @@ from vai_lab._import_helper import get_lib_parent_dir # from ttkwidgets import CheckboxTreeview # from sys import platform +_PLUGIN_READABLE_NAMES = {"dataLoader": "default", + "data_Loader": "alias", + "data loader": "alias"} # type:ignore +_PLUGIN_MODULE_OPTIONS = {"layer_priority": 2, + "required_children": None} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore + class dataLoader(): """ Creates a window diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 5eb92fae..5c1e56c3 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -10,7 +10,7 @@ from PIL import Image, ImageTk import tkinter as tk from tkinter import messagebox, ttk - +from tkinter.filedialog import asksaveasfilename _PLUGIN_READABLE_NAMES = {"plugin_canvas": "default", "pluginCanvas": "alias", @@ -41,6 +41,8 @@ def __init__(self, parent, controller, config: dict): self.plugin: Dict[int, tk.StringVar] = {} self.dataType: Dict[int, tk.StringVar] = {} self.allWeHearIs: List[tk.Radiobutton] = [] + self.out_data_xml: List[int, tk.StringVar] = [] + self.out_data_list: List[int, tk.StringVar] = [] if not self.controller._debug: self._setup_frame() @@ -79,8 +81,8 @@ def _setup_frame(self): bg=self.bg, fg='white', anchor=tk.CENTER) - self.my_label.grid(column=5, - row=0, columnspan=2, padx=10) + self.my_label.grid(column=0, + row=0, columnspan=3, padx=10) self.back_img = ImageTk.PhotoImage(Image.open( os.path.join(script_dir, @@ -158,7 +160,7 @@ def select(self, x: float, y: float): self.check_updated() - if self.m in self.id_done and self.m > 1: + if self.m in self.id_done and self.m > 0: self.canvas.itemconfig('p'+str(self.m), fill='#46da63') else: self.canvas.itemconfig('p'+str(self.m), fill=self.bg) @@ -167,21 +169,32 @@ def select(self, x: float, y: float): if not (len(self.canvas.gettags(self.canvas_selected)[0].split('-')) > 1) and\ not (self.canvas.gettags(self.canvas_selected)[0].split('-')[0] == 'loop'): self.m = int(self.canvas.gettags(self.canvas_selected)[0][1:]) - if self.m > 1: - if self.m not in self.id_done and self.m > 1: + if self.m > 0: #> 1: + if self.m not in self.id_done: self.canvas.itemconfig('p'+str(self.m), fill='#dbaa21') for widget in self.allWeHearIs: widget.grid_remove() - - self.display_buttons() + if self.m > 1: + self.reset_sidecheck() + self.display_buttons() + else: + self.reset_sidepanel() + self.display_checklist() module_number = self.id_mod.index(self.m) if not hasattr(self, 'button_forw'): - if module_number == len(self.id_mod)-1: + if module_number == 1: self.button_forw = tk.Button( self.frame4, text='Finish', bg=self.bg, font=self.controller.pages_font, fg='white', height=3, width=15, command=self.finish, state=tk.NORMAL, image='') + elif module_number == len(self.id_mod)-1: + pCoord = self.canvas.coords( + 'p'+str(self.id_mod[1])) + self.button_forw = tk.Button( + self.frame4, image=self.forw_img, bg=self.bg, + command=lambda: self.select( + pCoord[0], pCoord[1]), state=tk.NORMAL) else: pCoord = self.canvas.coords( 'p'+str(self.id_mod[module_number+1])) @@ -191,10 +204,17 @@ def select(self, x: float, y: float): pCoord[0], pCoord[1]), state=tk.NORMAL) self.button_forw.grid( column=1, row=0, sticky="news", pady=(0, 10), padx=(0, 10)) - if module_number < 3: + if module_number == 1: + mCoord = self.canvas.coords( + 'p'+str(self.id_mod[-1])) self.button_back = tk.Button( self.frame4, image=self.back_img, bg=self.bg, - state=tk.DISABLED) + command=lambda: self.select( + mCoord[0], mCoord[1]), state=tk.NORMAL) + elif module_number < 3: + self.button_back = tk.Button( + self.frame4, image=self.back_img, bg=self.bg, + state=tk.DISABLED) else: mCoord = self.canvas.coords( 'p'+str(self.id_mod[module_number-1])) @@ -205,18 +225,30 @@ def select(self, x: float, y: float): self.button_back.grid( column=0, row=0, sticky="news", pady=(0, 10)) else: - if module_number == len(self.id_mod)-1: + if module_number == 1: self.button_forw.config(text='Finish', bg=self.bg, font=self.controller.pages_font, fg='white', height=3, width=15, command=self.finish, state=tk.NORMAL, image='') + elif module_number == len(self.id_mod)-1: + pCoord = self.canvas.coords( + 'p'+str(self.id_mod[1])) + self.button_forw.config(image=self.forw_img, bg=self.bg, + command=lambda: self.select( + pCoord[0], pCoord[1]), text='', state=tk.NORMAL) else: pCoord = self.canvas.coords( 'p'+str(self.id_mod[module_number+1])) self.button_forw.config(image=self.forw_img, bg=self.bg, command=lambda: self.select( pCoord[0], pCoord[1]), text='', state=tk.NORMAL) - if module_number < 3: + if module_number == 1: + mCoord = self.canvas.coords( + 'p'+str(self.id_mod[-1])) + self.button_back.config(image=self.back_img, bg=self.bg, + command=lambda: self.select( + mCoord[0], mCoord[1]), text='', state=tk.NORMAL) + elif module_number < 3: self.button_back.config(image=self.back_img, bg=self.bg, state=tk.DISABLED, text='') else: @@ -225,19 +257,30 @@ def select(self, x: float, y: float): self.button_back.config(image=self.back_img, bg=self.bg, command=lambda: self.select( mCoord[0], mCoord[1]), text='', state=tk.NORMAL) - else: # If user clicks on Initialiser or Output - self.my_label.config(text='') - for widget in self.allWeHearIs: - widget.grid_remove() - for widget in self.radio_label: - widget.grid_remove() + else: # If user clicks on Initialiser + self.reset_sidepanel() if hasattr(self, 'button_forw'): self.button_back.config(image=self.back_img, bg=self.bg, state=tk.DISABLED, text='') pCoord = self.canvas.coords('p'+str(self.id_mod[2])) self.button_forw.config(image=self.forw_img, bg=self.bg, command=lambda: self.select( - pCoord[0], pCoord[1]), text='', state=tk.NORMAL) + pCoord[0], pCoord[1]), text='', state=tk.NORMAL) + def reset_sidepanel(self): + """ Removes information from the panel to the side of the canvas. + """ + self.my_label.config(text='') + for widget in self.allWeHearIs: + widget.grid_remove() + for widget in self.radio_label: + widget.grid_remove() + self.reset_sidecheck() + + def reset_sidecheck(self): + if hasattr(self, 'frame_tree'): + self.frame_tree.grid_remove() + if hasattr(self, 'frame_path'): + self.frame_path.grid_remove() def finish(self): """ Calls function check_quit. @@ -248,17 +291,37 @@ def finish(self): self.check_quit() def check_updated(self): - """ Checks if the current plugin exists and - stores/updates the plugin options + """ Checks if the current plugin exists and stores/updates the plugin options. + It also cheks if the output data has been modfied and stores/updates the + information accordingly. """ if (self.m in self.plugin.keys()) and\ (self.plugin[self.m].get() != 'None'): # add self.id_done.append(self.m) - self.s.append_plugin_to_module(self.plugin[self.m].get(), - {**self.req_settings, ** - self.opt_settings}, - np.array(self.module_names)[ - self.m == np.array(self.id_mod)][0], + self.xml_handler.append_plugin_to_module( + self.plugin[self.m].get(), + {**self.req_settings, **self.opt_settings}, + self.plugin_inputData.get(), + np.array(self.module_names)[self.m == np.array(self.id_mod)][0], + True) + + elif len(set([elem.get() for elem in self.out_data_list if len(elem.get())>1]) + ^ set(self.out_data_xml)) > 0 : + self.out_data_xml = [elem.get() for elem in self.out_data_list if len(elem.get())>1] + self.id_done.append(self.m) + + if os.path.normpath(get_lib_parent_dir()) == os.path.normpath(os.path.commonpath([self.path_out, get_lib_parent_dir()])): + rel_path = os.path.join('.', os.path.relpath(self.path_out, + os.path.commonpath([self.path_out, get_lib_parent_dir()]))) + else: + rel_path = self.path_out + + self.xml_handler.append_plugin_to_module( + 'Output', + {'outdata': self.out_data_xml, + 'outpath': rel_path}, + None, + np.array(self.module_names)[self.m == np.array(self.id_mod)][0], True) def display_buttons(self): @@ -333,6 +396,81 @@ def display_buttons(self): self.CreateToolTip(rb, text=descriptions[p]) self.allWeHearIs.append(rb) + def display_checklist(self): + """ Updates the displayed tree to the left of canvas. + Used to specify the output data and files. """ + self.my_label.config( + text='Indicate which module\'s output data\nshould be saved:') + dataSources = [i for j, i in enumerate(self.module_names) if j != 1] + self.frame_tree = tk.Frame(self.frame_canvas, bg=self.bg) + self.frame_tree.grid(column=0, row=0, sticky="new", padx=(10, 0)) + for c, choice in enumerate(dataSources): + cb = tk.Checkbutton(self.frame_tree, var=self.out_data_list[c], text=choice, + onvalue=choice, offvalue="", + pady=10, + font=self.controller.pages_font, + bg=self.bg, + fg='white', + selectcolor=self.bg, + anchor='w', + width=20, + background=self.frame_canvas.cget("background") + ) + cb.grid(row=c, column=0) + if c == len(dataSources)-1: + cb.select() + self.frame_path = tk.Frame(self.frame_canvas, bg=self.bg) + self.frame_path.grid(column=0, row=1, sticky="new", padx=(10, 0)) + self.my_label = tk.Label(self.frame_path, + text='Indicate the output file name:', + pady=10, + font=self.controller.title_font, + bg=self.bg, + fg='white', + anchor=tk.CENTER) + self.my_label.grid(column=0, + row=0, columnspan=2, padx=10) + tk.Button(self.frame_path, + text="Browse", + command=self.upload_file + ).grid(column=0, row=1) + width = 63 + self.path_out = os.path.join(get_lib_parent_dir(),'examples','results','output.pkl') + filename = '...' + \ + self.path_out[-width + + 3:] if self.path_out and len(self.path_out) > width else self.path_out + self.label_list = tk.Label(self.frame_path, text=filename, + pady=10, + padx=10, + font=self.controller.pages_font, + fg='white', + bg=self.bg + ) + self.label_list.grid(column=1, row=1) + + def upload_file(self): + """ Asks for a file and stores the path and displays it. + """ + filename = asksaveasfilename(initialdir=os.path.join(get_lib_parent_dir(),'examples','results'), + title='Select an output file', + defaultextension='.pkl', + filetypes=[('Pickle file', '.pkl')]) + if filename is not None: + self.path_out = filename + width = 63 + filename = '...' + \ + filename[-width + + 3:] if filename and len(filename) > width else filename + self.label_list.config(text=filename) + + def getCheckedItems(self): + values = [] + for var in self.vars: + value = var.get() + if value: + values.append(value) + return values + def optionsWindow(self): """ Function to create a new window displaying the available options of the selected plugin.""" @@ -386,6 +524,21 @@ def optionsWindow(self): frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) frame2.grid_columnconfigure(tuple(range(2)), weight=1) + frame5 = tk.Frame(self.newWindow) + tk.Label(frame5, + text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) + + current = np.where(self.m == np.array(self.id_mod))[0][0] + dataSources = [i for j, i in enumerate(self.module_names) if j not in [1,current]] + + self.plugin_inputData = tk.StringVar(frame6) + dropDown = tk.ttk.OptionMenu(frame6, self.plugin_inputData, dataSources[current-2], *dataSources) + style.configure("TMenubutton", background="white") + dropDown["menu"].configure(bg="white") + dropDown.pack() + self.finishButton = tk.Button( frame4, text='Finish', command=self.removewindow) self.finishButton.grid( @@ -395,7 +548,10 @@ def optionsWindow(self): self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) frame1.grid(column=0, row=0, sticky="ew") - frame4.grid(column=0, row=2, sticky="se") + frame4.grid(column=0, row=20, sticky="se") + frame5.grid(column=0, row=2, sticky="ew") + frame6.grid(column=0, row=3) + self.newWindow.grid_rowconfigure(1, weight=2) self.newWindow.grid_columnconfigure(0, weight=1) @@ -537,7 +693,7 @@ def removewindow(self): self.focus() def get_all_children(self, item=""): - """ Iterates over the treeview to get all childer """ + """ Iterates over the treeview to get all children """ children = self.tree.get_children(item) for child in children: children += self.get_all_children(child) @@ -630,10 +786,12 @@ def add_module(self,boxName: str,x: float,y:float,ini = False,out = False, iid = :param out: bool type of whether the module corresponds to output. """ iid = self.modules if iid is None else iid - if not ini and not out: - tag = ('o'+str(iid),) - else: #Make initialisation and output unmoveable + if ini: #Make initialisation and output unmoveable tag = ('n0',) + elif out: + tag = ('n1',) + else: + tag = ('o'+str(iid),) text_w = self.controller.pages_font.measure(boxName+'-00') + 10 self.canvas.create_rectangle( x - text_w/2 , @@ -707,10 +865,10 @@ def upload(self): self.reset() - self.s = XML_handler() - self.s.load_XML(filename) - # self.s._print_pretty(self.s.loaded_modules) - modules = self.s.loaded_modules + self.xml_handler = XML_handler() + self.xml_handler.load_XML(filename) + # self.xml_handler._print_pretty(self.xml_handler.loaded_modules) + modules = self.xml_handler.loaded_modules modout = modules['Output'] # They are generated when resetting del modules['Initialiser'], modules['Output'] @@ -759,6 +917,12 @@ def upload(self): rowspan=4, pady=(0, 10)) self.framex.grid_columnconfigure(tuple(range(2)), weight=1) self.frame_canvas.pack(side=tk.LEFT, fill=tk.BOTH, expand=True) + + # Define modules for output data selection + for choice in self.module_names: + var = tk.StringVar(value=choice) + var.set(0) + self.out_data_list.append(var) def set_mousewheel(self, widget, command): """Activate / deactivate mousewheel scrolling when @@ -867,7 +1031,7 @@ def reset(self): self.loops = [] self.drawLoop = False self.l = 0 - self.id_done = [0, 1] + self.id_done = [0] self.plugin = {} for widget in self.allWeHearIs: widget.grid_remove() @@ -885,18 +1049,18 @@ def check_quit(self): self.canvas.delete(tk.ALL) self.frame_canvas.delete(tk.ALL) self.saved = True - self.s.write_to_XML() + self.xml_handler.write_to_XML() self.controller.Plugin.set(True) self.controller._show_frame("MainPage") # TODO: Check if loaded - elif len(self.s.loaded_modules) == 0: + elif len(self.xml_handler.loaded_modules) == 0: self.controller._show_frame("MainPage") self.controller.Plugin.set(False) else: self.reset() self.canvas.delete(tk.ALL) self.frame_canvas.delete(tk.ALL) - self.s.write_to_XML() + self.xml_handler.write_to_XML() self.controller.Plugin.set(True) self.controller._show_frame("MainPage") From a146b94390caf59a4cd7d4a9adc6d676ed578ca8 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Wed, 21 Jun 2023 13:07:26 +0200 Subject: [PATCH 02/48] Create a list with available functions --- src/vai_lab/utils/plugins/pluginCanvas.py | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 5c1e56c3..de35d34f 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -1,10 +1,11 @@ from vai_lab.Data.xml_handler import XML_handler from vai_lab._plugin_helpers import PluginSpecs -from vai_lab._import_helper import get_lib_parent_dir +from vai_lab._import_helper import get_lib_parent_dir, import_plugin_absolute import os import numpy as np import pandas as pd +from inspect import getmembers, isfunction from typing import Dict, List from PIL import Image, ImageTk @@ -481,6 +482,14 @@ def optionsWindow(self): )] self.req_settings = ps.required_settings[module][self.plugin[self.m].get( )] + + file_name = os.path.split(ps.find_from_class_name(self.plugin[self.m].get())['_PLUGIN_DIR'])[-1] + avail_plugins = ps.available_plugins[module][file_name] + plugin = import_plugin_absolute(globals(), + avail_plugins["_PLUGIN_PACKAGE"], + avail_plugins["_PLUGIN_CLASS_NAME"]) + function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): self.newWindow.destroy() From 8cd9dedaa8bb2a2e2f408a123650016796b4c4c6 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 7 Jul 2023 12:47:12 +0200 Subject: [PATCH 03/48] Fine-tuning merge --- .github/workflows/pytest.yml | 1 + .gitignore | 5 +- pyproject.toml | 3 +- src/vai_lab/Core/vai_lab_core.py | 6 +- src/vai_lab/Data/Data_core.py | 3 +- src/vai_lab/Data/xml_handler.py | 7 +- .../UserInteraction/plugins/CanvasInput.py | 163 ++++++++++-------- .../UserInteraction/plugins/ManualInput.py | 3 +- src/vai_lab/_import_helper.py | 2 +- src/vai_lab/_plugin_templates.py | 14 +- src/vai_lab/examples/results/output.pkl | Bin 0 -> 6665 bytes src/vai_lab/examples/state-action/X_data.csv | 122 ++++++------- .../examples/xml_files/canvas_demo.xml | 9 +- .../xml_files/pybullet_env_example.xml | 2 +- tests/test_launch.py | 1 + 15 files changed, 186 insertions(+), 155 deletions(-) create mode 100644 src/vai_lab/examples/results/output.pkl diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index ec4eaf1d..ea62fcce 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -23,6 +23,7 @@ jobs: matrix: os: [ubuntu-latest, macos-latest, windows-latest] python-version: ["3.10", "3.11"] + fail-fast: false steps: - uses: actions/checkout@v3 diff --git a/.gitignore b/.gitignore index 0e6e69c7..d80f36db 100644 --- a/.gitignore +++ b/.gitignore @@ -145,4 +145,7 @@ dmypy.json .pyre/ # pytype static type analyzer -.pytype/ \ No newline at end of file +.pytype/ + +# Ignore Mac DS_Store files +.DS_Store diff --git a/pyproject.toml b/pyproject.toml index 91fa8f7f..f1c9a7c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -28,8 +28,7 @@ dependencies = [ "matplotlib >= 3.5.0", "opencv-python >= 4.6.0.65", "pybullet >= 3.2.5", - "attrs >= 23.1.0", - "tk >= 0.1.0" + "attrs >= 23.1.0" ] [project.optional-dependencies] diff --git a/src/vai_lab/Core/vai_lab_core.py b/src/vai_lab/Core/vai_lab_core.py index 0cf8356c..a2aeca8b 100644 --- a/src/vai_lab/Core/vai_lab_core.py +++ b/src/vai_lab/Core/vai_lab_core.py @@ -35,7 +35,6 @@ def _launch(self): self.load_config_file(gui_output["xml_filename"]) except: raise Exception("No XML File Selected. Cannot Run Pipeline") - self._load_data('Initialiser') def load_config_file(self, filename: Union[str,List,Tuple]): """Loads XML file into XML_handler object. @@ -48,7 +47,7 @@ def load_config_file(self, filename: Union[str,List,Tuple]): self._xml_handler.load_XML(filedir) self._initialised = True - def _load_data(self, module) -> None: + def _load_data(self, module = 'Initialiser') -> None: """Loads data from XML file into Data object""" init_data_fn = self._xml_handler.data_to_load(module) if module not in self.data.keys(): @@ -185,7 +184,8 @@ def run(self): if not self._initialised: self._initialise_with_gui() print("Running pipeline...") - self._load_data('Initialiser') + if len(self._xml_handler.loaded_modules) > 0: + self._load_data() self._init_status(self._xml_handler.loaded_modules) self._execute(self._xml_handler.loaded_modules) diff --git a/src/vai_lab/Data/Data_core.py b/src/vai_lab/Data/Data_core.py index c072955a..fccb1c43 100644 --- a/src/vai_lab/Data/Data_core.py +++ b/src/vai_lab/Data/Data_core.py @@ -92,8 +92,7 @@ def import_data(self: DataT, :param filename: str, filename of file to be loaded :param data_name: str, name of class variable data will be loaded to """ - filename = rel_to_abs(filename).replace( - "\\", "/").replace("/", path.sep) + filename = rel_to_abs(filename) ext = self._get_ext(filename) getattr(self, "_import_{0}".format(ext))(filename, data_name) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 8025af2c..b3ebeae3 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -641,11 +641,8 @@ def _get_data_structure(self, module) -> Dict[str, Any]: return out #@property - def data_to_load(self, module=None) -> Dict[str, str]: - if module is None: - return self._get_data_structure('Initialiser')["to_load"] - else: - return self._get_data_structure(module)["to_load"] + def data_to_load(self, module='Initialiser') -> Dict[str, str]: + return self._get_data_structure(module)["to_load"] # Use case examples: diff --git a/src/vai_lab/UserInteraction/plugins/CanvasInput.py b/src/vai_lab/UserInteraction/plugins/CanvasInput.py index 0de202e7..e521ff37 100644 --- a/src/vai_lab/UserInteraction/plugins/CanvasInput.py +++ b/src/vai_lab/UserInteraction/plugins/CanvasInput.py @@ -32,7 +32,6 @@ def __init__(self, parent, controller, config: dict): self.notebook = ttk.Notebook(self) self.notebook.grid(row=0, column=0, columnspan=7, rowspan=12, pady=15) - self.frame: List[tk.Frame] = [] self.tree: List[ttk.Treeview] = [] self.canvas: List[tk.Canvas] = [] self.draw: List[tk.StringVar] = [] @@ -84,6 +83,8 @@ def _load_classes_from_data(self): if not all(islist): self._class_list = [list(self._class_list)] + frame: List[tk.Frame] = [] + self.out_data = {} self.out_data[0] = {} for ii in np.arange(len(self._class_list)): @@ -94,9 +95,8 @@ def _load_classes_from_data(self): self.out_data[ii][self._class_list[ii][k]] = [] for ii in np.arange(len(self.out_data)): - self.frame.append(tk.Frame(self, bg=self.parent['bg'])) - self.frame[-1].grid(row=0, column=0, sticky="nsew", pady=15) - # if self._class_list[ii][0].split('_')[1] == 'a': + frame.append(tk.Frame(self, bg=self.parent['bg'])) + frame[-1].grid(row=0, column=0, sticky="nsew", pady=(15,0)) if self._class_list[ii][0].split('_')[1] == 'a': self.type.append('Rotating') self.clock.append(tk.StringVar()) @@ -105,13 +105,18 @@ def _load_classes_from_data(self): self.type.append('Sliding') self.clock.append(-1) self.notebook.add( - self.frame[-1], + frame[-1], text='Object ' + str(ii) + ' - ' + self.type[-1]) - + + # Frames for canvas, treeview and buttons + frame1 = tk.Frame(frame[-1], bg=self.parent['bg']) + frame2 = tk.Frame(frame[-1], bg=self.parent['bg']) + frame3 = tk.Frame(frame[-1], bg=self.parent['bg']) + frame4 = tk.Frame(frame[-1], bg=self.parent['bg']) # Create a canvas widget self.width, self.height = 600, 600 self.canvas.append(tk.Canvas( - self.frame[-1], width=self.width, + frame1, width=self.width, height=self.height, background="white")) self.canvas[-1].grid(row=0, column=0, columnspan=4, rowspan=2) self.canvas[-1].bind('', self.draw_dot) @@ -124,64 +129,61 @@ def _load_classes_from_data(self): self.state.append(tk.StringVar()) self.state[ii].set('state') # Buttons under the canvas - self.button_draw = tk.Radiobutton( - self.frame[-1], text='Draw', fg='white', bg=self.parent['bg'], + tk.Radiobutton( + frame4, text='Draw', fg='white', bg=self.parent['bg'], height=3, width=20, var=self.draw[ii], selectcolor='black', value='draw').grid(column=4, row=3) - self.button_drag = tk.Radiobutton( - self.frame[-1], text='Move', fg='white', bg=self.parent['bg'], + tk.Radiobutton( + frame4, text='Move', fg='white', bg=self.parent['bg'], height=3, width=20, var=self.draw[ii], selectcolor='black', value='drag').grid(column=5, row=3) - self.clock_arc = tk.Radiobutton( - self.frame[-1], text='Clockwise', fg='white', bg=self.parent['bg'], + tk.Radiobutton( + frame4, text='Clockwise', fg='white', bg=self.parent['bg'], height=3, width=20, var=self.clock[ii], selectcolor='black', value='clock').grid(column=6, row=3) - self.countclock_arc = tk.Radiobutton( - self.frame[-1], text='Counterclockwise', fg='white', bg=self.parent['bg'], + tk.Radiobutton( + frame4, text='Counterclockwise', fg='white', bg=self.parent['bg'], height=3, width=20, var=self.clock[ii], selectcolor='black', value='countclock').grid(column=7, row=3) # self.button_edit = tk.Radiobutton( - # self.frame[-1], text = 'Edit', fg = 'white', bg = self.parent['bg'], + # frame[-1], text = 'Edit', fg = 'white', bg = self.parent['bg'], # height = 3, width = 20, var = self.draw[ii], # selectcolor = 'black', value = 'edit').grid(column = 6,row = 3) - self.button_save = tk.Button( - self.frame[-1], text='Save', fg='white', bg=self.parent['bg'], + tk.Button( + frame3, text='Save', fg='white', bg=self.parent['bg'], height=3, width=20, command=self.save_file).grid(column=1, row=3) - self.button_upload = tk.Button( - self.frame[-1], text='Upload coordinates', fg='white', + tk.Button( + frame3, text='Upload coordinates', fg='white', bg=self.parent['bg'], height=3, width=20, command=self.upload_sa).grid(column=0, row=3) - self.button_reset = tk.Button( - self.frame[-1], text='Reset', fg='white', + tk.Button( + frame3, text='Reset', fg='white', bg=self.parent['bg'], height=3, width=20, command=self.reset).grid(column=2, row=3) - button_main = tk.Button( - self.frame[-1], text="Done", fg='white', + tk.Button( + frame3, text="Done", fg='white', bg=self.parent['bg'], height=3, width=20, command=self.check_quit).grid(column=3, row=3) style = ttk.Style() style.configure( - "Treeview", background='white', foreground='white', + "Treeview", background='white', foreground='black', rowheight=25, fieldbackground='white', font=self.controller.pages_font) style.configure("Treeview.Heading", font=self.controller.pages_font) style.map('Treeview', background=[('selected', 'grey')]) - tree_frame = tk.Frame(self.frame[-1]) - tree_frame.grid(row=0, column=4, columnspan=3, rowspan=10) - - tree_scrollx = tk.Scrollbar(tree_frame, orient='horizontal') + tree_scrollx = tk.Scrollbar(frame2, orient='horizontal') tree_scrollx.pack(side=tk.BOTTOM, fill=tk.X) - tree_scrolly = tk.Scrollbar(tree_frame) + tree_scrolly = tk.Scrollbar(frame2) tree_scrolly.pack(side=tk.RIGHT, fill=tk.Y) self.tree.append(ttk.Treeview( - tree_frame, + frame2, yscrollcommand=tree_scrolly.set, xscrollcommand=tree_scrollx.set)) - self.tree[-1].pack() + self.tree[-1].pack(fill='both', expand=True) tree_scrollx.config(command=self.tree[-1].xview) tree_scrolly.config(command=self.tree[-1].yview) @@ -221,6 +223,22 @@ def _load_classes_from_data(self): # Define double-click on row action self.tree[-1].bind("", self.OnDoubleClick) + + frame1.grid(column=0, row=0, sticky="nsew", padx=2, pady=2) + frame2.grid(column=1, row=0, sticky="nsew", padx=2, pady=2) + frame3.grid(column=0, row=1, sticky="nswe") + frame4.grid(column=1, row=1, sticky="nsew") + + # self.frame2.grid_columnconfigure(tuple(range(2)), weight=1) + # self.frame2.grid_columnconfigure(1, weight=1) + # frame3.grid_columnconfigure(tuple(range(2)), weight=2) + # # frame3.grid_columnconfigure(2, weight=2) + # self.frame4.grid_columnconfigure(tuple(range(2)), weight=2) + # self.frame4.grid_columnconfigure(2, weight=2) + + # Load input data + if not self._data_in["X"].empty: + self.load_data(self._data_in["X"].values, True) def draw_dot(self, event): @@ -262,6 +280,7 @@ def draw_dot(self, event): self.out_data[ii])[n, :].astype(int)), tags=tag) self.tree[ii].selection_set(str(n)) + self.tree[ii].yview_moveto(int(n) / (len(self.tree[ii].get_children()))) else: self.tree[ii].insert( parent='', index='end', iid=0, text=1, @@ -269,7 +288,7 @@ def draw_dot(self, event): self.out_data[ii])[0, :].astype(int)), tags=('even',)) self.tree[ii].selection_set(str(0)) - + self.tree[ii].yview_moveto(0) elif self.state[ii].get() == 'action': if self.clock[ii].get() == 'clock': start = self.out_data[ii]['state_a'][-1] @@ -313,6 +332,7 @@ def draw_dot(self, event): self.out_data[ii])[n, :].astype(int)), tags=tag) self.tree[ii].selection_set(str(n)) + self.tree[ii].yview_moveto(int(n) / (len(self.tree[ii].get_children()))) else: self.tree[ii].insert( parent='', index='end', iid=0, text=1, @@ -320,7 +340,7 @@ def draw_dot(self, event): self.out_data[ii])[0, :].astype(int)), tags=('even',)) self.tree[ii].selection_set(str(0)) - + self.tree[ii].yview_moveto(0) elif self.state[ii].get() == 'action': self.canvas[ii].create_line( self.out_data[ii]['state_x'][-1], @@ -340,7 +360,7 @@ def draw_dot(self, event): def on_drag(self, event): ii = self.notebook.index(self.notebook.select()) - if self.draw[ii].get() == 'drag' and self.canvas[ii].selected: + if self.draw[ii].get() == 'drag' and self.canvas[ii].selected and len(self.canvas[ii].gettags("current")) > 0: # move the selected item n = int(self.canvas[ii].gettags("current")[0].split('-')[1]) @@ -416,6 +436,7 @@ def on_drag(self, event): values=tuple(self.dict2mat( self.out_data[ii])[n, :].astype(int))) self.tree[ii].selection_set(str(n)) + self.tree[ii].yview_moveto(int(n) / (len(self.tree[ii].get_children()))) def checkered(self, line_distance): @@ -554,9 +575,46 @@ def OnDoubleClick(self, event): y=y + pady, anchor=tk.W, width=width) - def upload_sa(self): + def load_data(self, data, read = False): ii = self.notebook.index(self.notebook.select()) + self.draw[ii].set('drag') + for point in data: + if read: # Not elegant at all, just to omit the header. + if isinstance(point, str): + sx, sy, ax, ay = point.split() + elif isinstance(point, (list, np.ndarray)): + sx, sy, ax, ay = point + # Draw an oval in the given coordinates + self.canvas[ii].create_oval( + float(sx)-3, float(sy)-3, float(sx)+3, float(sy)+3, + fill="black", width=0, + tags=("state"+str(ii)+"-" + str(len( + self.out_data[ii]['state_x'])))) + self.canvas[ii].create_line( + float(sx), float(sy), float(ax), float(ay), + fill="red", arrow=tk.LAST, + tags=("action"+str(ii)+"-" + str(len( + self.out_data[ii]['action_x'])))) + self.out_data[ii]['state_x'].append(sx) + self.out_data[ii]['state_y'].append(sy) + self.out_data[ii]['action_x'].append(ax) + self.out_data[ii]['action_y'].append(ay) + self.tree[ii].insert( + parent='', index='end', + iid=len(self.out_data[ii]['action_x'])-1, + text=len(self.out_data[ii]['action_x']), + values=tuple(self.dict2mat( + self.out_data[ii]) + [len(self.out_data[ii] + ['action_x'])-1, :].astype(int))) + sel_item = len(self.out_data[ii]['action_x'])-1 + self.tree[ii].selection_set(str(sel_item)) + self.tree[ii].yview_moveto(int(sel_item) / (len(self.tree[ii].get_children()))) + else: + read = True + + def upload_sa(self): filename = askopenfilename(initialdir=os.getcwd(), title='Select a file', defaultextension='.txt', @@ -565,38 +623,7 @@ def upload_sa(self): ('All Files', '*.*')]) if filename is not None: data = open(filename, 'r') - read = False - self.draw[ii].set('drag') - for n, point in enumerate(data): - if read: # Not elegant at all, just to omit the header. - i, sx, sy, ax, ay = point.split() - # Draw an oval in the given coordinates - self.canvas[ii].create_oval( - float(sx)-3, float(sy)-3, float(sx)+3, float(sy)+3, - fill="black", width=0, - tags=("state-" + str(len( - self.out_data[ii]['state_x'])))) - self.canvas[ii].create_line( - float(sx), float(sy), float(ax), float(ay), - fill="red", arrow=tk.LAST, - tags=("action"+str(ii)+"-" + str(len( - self.out_data[ii]['action_x'])))) - self.out_data[ii]['state_x'].append(sx) - self.out_data[ii]['state_y'].append(sy) - self.out_data[ii]['action_x'].append(ax) - self.out_data[ii]['action_y'].append(ay) - self.tree.insert( - parent='', index='end', - iid=len(self.out_data[ii]['action_x'])-1, - text=len(self.out_data[ii]['action_x']), - values=tuple(self.dict2mat( - self.out_data[ii]) - [len(self.out_data[ii] - ['action_x'])-1, :].astype(int))) - self.tree.selection_set(str(len( - self.out_data[ii]['action_x'])-1)) - else: - read = True + self.load_data(data) def reset(self): diff --git a/src/vai_lab/UserInteraction/plugins/ManualInput.py b/src/vai_lab/UserInteraction/plugins/ManualInput.py index 3e2a1000..b3d7ac1b 100644 --- a/src/vai_lab/UserInteraction/plugins/ManualInput.py +++ b/src/vai_lab/UserInteraction/plugins/ManualInput.py @@ -212,7 +212,7 @@ def _load_classes_from_data(self): # Select the current row self.tree.selection_set(str(int(0))) - + self.tree.yview_moveto(0) # Define double-click on row action self.tree.bind("", self.OnDoubleClick) @@ -302,6 +302,7 @@ def forward_back(self, image_number): " Forward button to continue to the next image in the folder." self.tree.selection_set(str(int(image_number-1))) + self.tree.yview_moveto(int(image_number-1) / (len(self.tree.get_children()))) # Print the corresponding image self.resizing((0, 0)) diff --git a/src/vai_lab/_import_helper.py b/src/vai_lab/_import_helper.py index e083a007..e56d2118 100644 --- a/src/vai_lab/_import_helper.py +++ b/src/vai_lab/_import_helper.py @@ -40,7 +40,7 @@ def rel_to_abs(filename: str) -> str: If relative, converts path to absolute by appending to base directory """ if filename[0] == ".": - filename = path.join(get_lib_parent_dir(), filename) + filename = path.normpath(path.join(get_lib_parent_dir(), filename)) elif filename[0] == "/" or (filename[0].isalpha() and filename[0].isupper()): filename = filename return filename \ No newline at end of file diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 9d42a62f..98d289d4 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -191,13 +191,13 @@ def fit(self): self.proc.set_params(**cleaned_options) except Exception as exc: print('The plugin encountered an error on the parameters of ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: self.proc.fit(self.X) except Exception as exc: print('The plugin encountered an error when fitting ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise def transform(self, data: DataInterface) -> DataInterface: @@ -205,14 +205,14 @@ def transform(self, data: DataInterface) -> DataInterface: data.append_data_column("X", pd.DataFrame(self.proc.transform(self.X))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise if self.X_tst is not None: try: data.append_data_column("X_test", pd.DataFrame(self.proc.transform(self.X_tst))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise return data @@ -227,13 +227,13 @@ def solve(self): self.clf.set_params(**self._config["options"]) except Exception as exc: print('The plugin encountered an error on the parameters of ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: self.clf.fit(self.X, self.Y) except Exception as exc: print('The plugin encountered an error when fitting ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise def predict(self, data): @@ -248,7 +248,7 @@ def predict(self, data): return self.clf.predict(data) except Exception as exc: print('The plugin encountered an error when predicting with ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+exc+'.') + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise def score(self, X, Y, sample_weight): diff --git a/src/vai_lab/examples/results/output.pkl b/src/vai_lab/examples/results/output.pkl new file mode 100644 index 0000000000000000000000000000000000000000..c89ae4a91cc3acd3c4ce0c1ee30b78cbfe3ae253 GIT binary patch literal 6665 zcmcIod0b3w-!Cnsv{(`mEhuSG)-2!ALVK17l`+-S4AYcZN|8wRDPm+TOGFU~Mam?! zXpuH)->1E#&35;m;klppzTfA4-p}*Tdp@6Y&UKyZ`d#O@e1GTq{?0+E^M6be`*j5I zc`|lB9#jT{&h+B*l>I1lGK1o#Z%*M*eqG4!J}fGqCn(ah=0c(CQP^xcn?v=` z^PsZnUQ9ht7R8(D@59>d!e+U<{Nvp&e4Z?MuQ!8OnMJI~=c)WnDbWax!t`K>hGe(e z+5WTlPwgiBzbYrs@S%86J;>fZ9$W^MP56V**N?)WdyqL4FHv8cH_n&tOC?-Ncuv-X z>dv6BC>**Elh2#LqB4Fp*)+N@n@^j{llEnBz35CnO@$}x?&HJqpff2PVp*OfgYXq0 zZ^fi@=*05Bd?8DClodes^`SF~F&R43m&@@Gg@z~j`zGzc6-2SI zVsfY~iu-Rxq<^c==gEsAyPoCaP9=gvV1TCaS44<bMO*@P{2l->WB zE^EQ`qqBUNqG*f6Wkqv0aeP=5FRG|7$?);<<d9BNALI!o+_gO_J>GMFd*yuhR+JsVpXi!PfVtFo~D22ye-mGJM>3+x_kn;ymSl zNQjWi*8isnpQk|PGJWapyBSonn@C19c_NeR?Hll08KOyL(cMK+^YyKy}|3803v8F@HE!dmFzBh0iCrTF15E`+Ua=%k`}|yPt8mHop*g4#GjDpf>o+l$(sYzJsjN zrW-T*noyY#wBMmB1Mi9}%o278;5I6MD0N*Ka-A#d+tO}hMm6`f*KjVz*s{_kFyg85b&}2arZZ)AYJDNjN4_$epZpk3g8Q3acJ^T7`WJWN zy@YT(DRwSp)oX;YE`$|v>o7i8!*w{a9mb5Kp>Isi)tPoUYnlm7}x;&n9h~Y2!Ch}r`KBL)Ps7tTTvj>i318RChkEeE|&#X zN$^^+$t3ahg3RaWy>#q;u0=D>^HxXi*KR@Fokg%t>Bc>yIfrItb;7O%jpe>iAvof> zD$=42MF|%1{h3Kvs8(t+V|Ew9a^$vHkF;X;m5rerO+R8gwK-AwT_+aYD7Q(>d;|YE zAk8ugVlzfE8)z_gsRym$# zTHg@%cY<0SP(JS9Ekp*z@5s5a18uq;6?R<6;0^`eNKSMs?nfsj=XSkQ~I4L}NVP7d~1?&RX zwlg4=C?2d|-;IE43l-FfJTJ<;Jj3B$Bc2|QF|(@Az*UQg>8`3}_{lVSt=rIrfFt@6 zY|UKQTJO^41wO*gw4R{pZOyo*_hZ3!S`Xwlnds-3m&3nF<$%-LHUx{kam`C^z>oGM za^lGvoY)p(E)md*x37J+9~`bk>YJaLHBm)a^`j^As&5WXw@N3Lzo`W0?jq}e%q}pe z4A0*FvJI9}^v^##3$U+b>)f$7dg0_Oxo=rP2NIRZpZpCzKsqjCZgyfj=n77@G3FVF zAer#C##dp9R!qislNPLdG+Ax*T^=?DDOpKAYrtgZa3kiO7*wB}?*jYL-8SlGmPI%0)Y+Z};yKtC81!abatrpQ6)fn|>w!u4H_d|J zY*bz`)88~Z8+}&%IXybP=(9a`P_L{T9rCs6Wfq;#RPa?R)ht3>ZJS0;LOA$ZO~Qmf8Z~CkBpwf+ZNjiFxzi)d(jTn_6bGpI5T7xw?(QP^_40996>J(#eKC&RcUa6 zrti=2k2vHMyFo3s8pWy6yMs?PV0z!ART~4#F)u6liM??%8Y<;h&(*6%$%%#=jS&^N z?*F3o6!!z3WPQK-MY0`@A7wgklosK6$dg;9Tk7GdnYMMvrVc5lloPvDJ7IlN*}#bZ z3aRN9uNZSP@!Y{=&YWGfIDfJ4!q3bS+^l@yHuR_iE;0*8A8u%Y$tqpx2k8|EizQ2q z5ICC8UZ8T}TN$b&cZtuaegir8+je$@{hncQ4hegDFiqx==~zNfo>k3$)sw||yz{_^ z!9$fO)|A=0;_e6ZzYA(oS=ojcPvYXIKl}*$L)Fex-x7KKoY)KnN1(AT&H)S~+5*2m83c^JMTEFTGN#FOTycg9U` z!8zW2ueh)dsD^tvG$#=DPB_gF$ah1G&o?xR?84T)c%?zhLzzEkjoFLipt8D^WA3&g zbufB%v~L$=Zsb1BCGg+#S>sB>!Pl_5d*Q_G`JGrk+Ujnt)`^Ewf~v^@`RJi6h|pB1 z#WCtq1%n&!5j`!`FPYN{IpwZ}-Lf60 zC-}#vx8LX9uZMl|h4o@jd*NnUF}|;<5ud&<(!NKEhEiVSo+Z7#*sK#>|6o%h9c7Jh}^zXs`)3i~|m4x4O9%_!C-3rFB1xC7a>Y=x9$WgqZ7#1r+ z7={CFs4-*+HJm@-!`yV{KJOYdvaM|fZ`Z(~bD&(kqXNes$&cvq8)0?XWnlmI*DznY ze#wN5g^=11km1?XgtuP%Paiof#CylUjYpM*@J~^;`K;cF%-+J*iHbeo^Q3<9G2-sQ z3GfxMLEae8W&9JzkMTG1wQVGL;w$;~_V(7J#PRE5BgWyo*+|)}Jj&;ZoAEisG~PHj zE}g-lGYN7i#b(gmMa$TT5tGD4tJsVYU61fx2|lXwcXH`L-AiTbi;5e9MU$CSf3g*m zct(iRHt?kWY!gJ5rpBA{7vkv6C0<8iu_yrq9VN4GwExX27KKTWa^CojqWfP>@wO+t#G$Y_gmOGdHiuyBHY-I9D$x-CwI)@x zFi(<9goNPg9t2naled$-sT_*v0pU)b6otbf7&%cENs~Ppzbhi%R0@OPZP-pP{OJ(52(i7*fBD@L2+J#>UrkRG}9qL<;_I1Io0AMg@i}%M`gNEC_l3^@%@Nk-zsyu zrTL)1Z_{ams zWLMeo-iZ@Q35_;O94$Vh^H}Lc?LZIk{WE*SiHOL_G_K7`n%`f6Oxs7>PYa?QpaloI2-#+aqt}5iDo(tNxMgnb>TZZl7k&FFkm<3-=W-mP*CFL-dYTYZdWNV8s`&b|Tov z8KtjXiuc=4th?}!Y2mkFJ*7O~Preum>!Ut9wlrd{NzjyVpK4r`GTSn?FA3Qdv*tS}P;IQVSVvHs=UG6#Wz-L2r+)%amSUylu|g)NMgVD>xAc5 zf6}3LB2FC}63)%5f}=#3M~iwao z$@;i)QWkiX;}r}~zlBR%uyQsx13FjMw|_V7K+7b$y)d~QL&@|QN^%`~3mK%s{xbN7 zF~_vz)PYtaGw!`t=j2d1>XensYcZjNEq0Mc5iD$=ZeTwIJ9FRPyEb}aW+d5a_QY~cfkN{k)7v40@u83GbKUEZjd;F-pXrcWPkp^#r#p&8MF z;&Hy}skhTn_g4LFno|v)nW~OTskGy{%1b6t>))jrmA^*tmeleo)Q zbs^O*Hmz`M5AH}0^tou4VTH;Svo%OGxxJXI^BCI0vGc{x-odL^>T4p-w_r1EOuc_vCg#^_ zjq(OL?-Px) z;I-k}wq2Q>VBBpxQ+i7X^8(6MQ z?m~VK-(#^ ze|7_#i_HQ@7e-JT` zh#ccoYZ;PKY>rQw%vh4u3Mp@mFMkN;>-m{X+nA53X`{BwA2;G%;gVe^ABdC2S=Fe} zXcDBx88&ws1N-p7-|fV-Z(ksPcx${@*AU!xZ6Hk!=*N~Dmkw99whDH|TyeKOEJaF; zkPfInUMA4$);$wd(kFOher2!C@;?OYH*UEvC7eY17GdM~B3psf@j~O2!46qcWYpJG zgKc7@+NJ5=NqgI{CBEBw-}Esg@d=49jAJH|2ER1jC(V;0JvWm-83FZ=&<}!2f>`o&PA_#LsxP{(k|?Yf`rW literal 0 HcmV?d00001 diff --git a/src/vai_lab/examples/state-action/X_data.csv b/src/vai_lab/examples/state-action/X_data.csv index a33f8ae8..b10f8298 100644 --- a/src/vai_lab/examples/state-action/X_data.csv +++ b/src/vai_lab/examples/state-action/X_data.csv @@ -1,61 +1,61 @@ -State_x ,State_y ,Action_x ,Action_y -0.9671880089819392 ,0.4380968121803849 ,0.4984887549541376 ,0.9322726736376392 -0.7643188319722541 ,0.01231887020330591 ,0.9830931007621688 ,0.8522744809601737 -0.9237525748244838 ,0.11898284551274751 ,0.9924417893483648 ,0.25218680355486556 -0.9792179211485214 ,0.6255946427530268 ,0.3431932557012749 ,0.22506274930650727 -0.7535567846092962 ,0.24123313933735335 ,0.018579652407582636 ,0.8800662109032867 -0.3437925770776318 ,0.07699836482205524 ,0.4728350987688088 ,0.7662442031942271 -0.9172917672442188 ,0.9662772820792992 ,0.18452917862354012 ,0.24925132769402703 -0.3558655691713499 ,0.7314046169290214 ,0.6292458753640825 ,0.7281748531672392 -0.05455147830497775 ,0.15032095601880535 ,0.000742940145887161 ,0.0263197539225396 -0.12794977066232505 ,0.49156583621314887 ,0.10202991264566497 ,0.5301707346161213 -0.536257172265748 ,0.7996784991300683 ,0.9172832910459607 ,0.8776049102983675 -0.6599930891873356 ,0.09321141488037277 ,0.1482680061282743 ,0.3125012812415898 -0.23424261012475522 ,0.55511448517851 ,0.013046984000703254 ,0.34234482093755003 -0.7101280593659418 ,0.37479280138602833 ,0.27362008914570524 ,0.682871585689911 -0.421235878074406 ,0.3116126920265224 ,0.02583910127018263 ,0.5547669223178248 -0.34286708741990113 ,0.21179048991267835 ,0.32716029003021707 ,0.4890251672215029 -0.5189154708727883 ,0.5461829716414196 ,0.016511407532504907 ,0.9121837468923465 -0.5631894552042326 ,0.3325024218489049 ,0.9161226750644361 ,0.22983791939614862 -0.49074103400728974 ,0.524449286501952 ,0.7100987717394524 ,0.21311227550583367 -0.42488315106570407 ,0.8449397402408293 ,0.5834117468530966 ,0.535672046560473 -0.3192056373096188 ,0.851494347175034 ,0.13184467127491017 ,0.23424731183882164 -0.41787827702876124 ,0.7045916830745 ,0.3297951468894895 ,0.806647258078522 -0.017145552002915787 ,0.10061912324474898 ,0.9024866878930592 ,0.3440758374198163 -0.438299588339767 ,0.6335190171928938 ,0.19639432578427918 ,0.9921209657903454 -0.7419383944302453 ,0.6611922350606557 ,0.06827617632578975 ,0.1127204989170465 -0.6272401825306418 ,0.9286242708669664 ,0.6533329448337888 ,0.3681024165243707 -0.9924588758607049 ,0.867934944378779 ,0.9502383116354627 ,0.44763906453714963 -0.6814344435878125 ,0.08375304244682025 ,0.3479481173938417 ,0.5482785322145338 -0.1050339978746333 ,0.7584944915035335 ,0.7585866415821404 ,0.351215896269197 -0.12658825210891933 ,0.0732871690467427 ,0.5535637694702159 ,0.6312631154416452 -0.7422778633830742 ,0.8789197110172094 ,0.4615899023589518 ,0.47150569972602174 -0.060643318283553405 ,0.7178088837302014 ,0.702272065932264 ,0.38390413101856535 -0.8191895410512178 ,0.3634229938710849 ,0.42135628539750447 ,0.2727994683499393 -0.9999199411408117 ,0.8684190230298257 ,0.2596353916370152 ,0.7818544450839995 -0.4309294603116195 ,0.1621914263062164 ,0.19093665617217148 ,0.2629780574128501 -0.6769153420880742 ,0.25354748987282205 ,0.41826407613880523 ,0.5105298249146336 -0.6973003602191449 ,0.6705192894977414 ,0.8987394761400211 ,0.4890113703300386 -0.24062711159874495 ,0.5047811254637201 ,0.7732363361216537 ,0.847096699066319 -0.7652673533252651 ,0.3180290726747005 ,0.233764403225033 ,0.13733508395405847 -0.20599356053214612 ,0.4693193853964881 ,0.9884067295994179 ,0.14179763257150835 -0.6230605269306876 ,0.37226905005157285 ,0.8375513460804126 ,0.47522858272099777 -0.13169282511385816 ,0.3507415100597824 ,0.2685222265975922 ,0.30877628171800475 -0.12253840357227408 ,0.8734014654383687 ,0.8028293377767859 ,0.8342239310648731 -0.3325029621446359 ,0.04003932588687398 ,0.12709466197531627 ,0.6662216171464865 -0.04245689148699905 ,0.9570047234482667 ,0.43227208799454175 ,0.3865771351092523 -0.2028627113720355 ,0.42487299763806974 ,0.6634501230665879 ,0.42138481640595493 -0.7302574619774882 ,0.7445376224867151 ,0.21713246315756352 ,0.3434198127813938 -0.6365439014447486 ,0.25606444854559063 ,0.5537129809489818 ,0.1629672057071535 -0.7000014791771763 ,0.5979649835252617 ,0.03838676802768348 ,0.9121462360882062 -0.3886002445513097 ,0.977242303405838 ,0.2705691672546615 ,0.9513765865234916 -0.6473604202902449 ,0.28350244814093783 ,0.08775351900953965 ,0.6312290476710444 -0.6990859986824999 ,0.032529525745985444 ,0.5598806306599382 ,0.2896794051531735 -0.9730538815643153 ,0.5161662313731986 ,0.8374231998423672 ,0.41418214281430754 -0.31942545883706497 ,0.47600176099408975 ,0.9147675682128449 ,0.2758930549358044 -0.9906583523353286 ,0.08568211061940656 ,0.18753616812160567 ,0.7573360865775085 -0.06608602930280472 ,0.740352078456626 ,0.6784709775636907 ,0.37006251016903113 -0.48699741570742805 ,0.5692196155586737 ,0.9431057810851509 ,0.4807794949611165 -0.6052645403989522 ,0.1991049312114247 ,0.18358768708081097 ,0.3520418987936338 -0.4376150973891353 ,0.07995788691173977 ,0.18067397841443167 ,0.23594549880692017 -0.31292316838092515 ,0.45870340272609433 ,0.973868440052966 ,0.280301692602244 +State_x,State_y,Action_x,Action_y +580.3128053891636,262.85808730823095,299.09325297248256,559.3636041825836 +458.5912991833525,7.39132212198354,589.8558604573013,511.3646885761042 +554.2515448946903,71.38970730764851,595.4650736090189,151.3120821329193 +587.5307526891128,375.3567856518161,205.91595342076494,135.0376495839043 +452.13407076557775,144.73988360241196,11.147791444549561,528.0397265419721 +206.27554624657907,46.199018893233124,283.7010592612853,459.74652191653627 +550.3750603465313,579.7663692475795,110.71750717412405,149.5507966164162 +213.51934150280997,438.8427701574128,377.5475252184495,436.9049119003435 +32.730886982986625,90.19257361128318,0.44576408753226,15.791852353523762 +76.76986239739499,294.9395017278893,61.21794758739894,318.1024407696728 +321.7543033594488,479.807099478041,550.3699746275765,526.5629461790205 +395.99585351240137,55.92684892822362,88.96080367696457,187.5007687449539 +140.54556607485313,333.068691107106,7.828190400421921,205.40689256252998 +426.0768356195651,224.87568083161696,164.1720534874231,409.7229514139466 +252.7415268446436,186.96761521591344,15.503460762109562,332.86015339069485 +205.72025245194064,127.07429394760698,196.29617401813022,293.4151003329018 +311.349282523673,327.70978298485176,9.90684451950294,547.3102481354078 +337.91367312253954,199.50145310934292,549.6736050386617,137.90275163768916 +294.44462040437384,314.6695719011712,426.0592630436714,127.86736530350015 +254.9298906394224,506.96384414449756,350.047048111858,321.40322793628377 +191.5233823857713,510.89660830502044,79.10680276494605,140.54838710329295 +250.72696621725672,422.7550098447,197.87708813369372,483.9883548471132 +10.28733120174942,60.37147394684934,541.4920127358355,206.44550245188978 +262.9797530038602,380.1114103157363,117.83659547056746,595.2725794742072 +445.1630366581472,396.71534103639345,40.965705795473816,67.6322993502279 +376.3441095183851,557.1745625201798,391.99976690027324,220.86144991462243 +595.4753255164229,520.7609666272674,570.1429869812777,268.58343872228977 +408.8606661526875,50.25182546809212,208.768870436305,328.9671193287203 +63.02039872477998,455.09669490212013,455.1519849492842,210.7295377615182 +75.95295126535159,43.972301428045625,332.13826168212955,378.7578692649871 +445.36671802984455,527.3518266103257,276.9539414153711,282.903419835613 +36.38599097013204,430.6853302381208,421.3632395593584,230.34247861113917 +491.5137246307307,218.05379632265095,252.81377123850265,163.6796810099636 +599.9519646844869,521.0514138178954,155.78123498220913,469.1126670503997 +258.5576761869717,97.31485578372985,114.56199370330283,157.78683444771005 +406.1492052528445,152.12849392369318,250.95844568328312,306.3178949487802 +418.3802161314869,402.31157369864485,539.2436856840127,293.4068221980232 +144.37626695924695,302.86867527823205,463.9418016729922,508.2580194397914 +459.16041199515905,190.81744360482028,140.2586419350198,82.40105037243505 +123.59613631928765,281.59163123789284,593.0440377596508,85.07857954290498 +373.83631615841256,223.36143003094367,502.53080764824756,285.1371496325986 +79.01569506831486,210.44490603586942,161.1133359585553,185.26576903080283 +73.52304214336439,524.0408792630212,481.69760266607153,500.5343586389239 +199.50177728678156,24.02359553212434,76.25679718518971,399.73297028789193 +25.4741348921994,574.2028340689601,259.363252796725,231.94628106555137 +121.7176268232213,254.9237985828418,398.0700738399527,252.83088984357292 +438.1544771864929,446.722573492029,130.2794778945381,206.05188766883626 +381.92634086684916,153.63866912735435,332.2277885693891,97.7803234242921 +420.00088750630573,358.77899011515706,23.032060816610038,547.2877416529237 +233.1601467307858,586.3453820435028,162.3415003527969,570.825951914095 +388.4162521741469,170.10146888456268,52.65211140572376,378.7374286026266 +419.45159920949993,19.51771544759124,335.9283783959629,173.80764309190408 +583.8323289385891,309.6997388239192,502.4539199054203,248.5092856885845 +191.65527530223895,285.6010565964538,548.8605409277069,165.53583296148264 +594.3950114011972,51.4092663716439,112.52170087296335,454.4016519465051 +39.65161758168282,444.2112470739756,407.0825865382144,222.03750610141864 +292.1984494244568,341.5317693352042,565.8634686510904,288.4676969766699 +363.1587242393713,119.46295872685482,110.15261224848655,211.22513927618027 +262.5690584334812,47.974732147043824,108.40438704865896,141.56729928415206 +187.75390102855505,275.2220416356566,584.3210640317797,168.1810155613464 diff --git a/src/vai_lab/examples/xml_files/canvas_demo.xml b/src/vai_lab/examples/xml_files/canvas_demo.xml index 3966377a..d9d4859d 100644 --- a/src/vai_lab/examples/xml_files/canvas_demo.xml +++ b/src/vai_lab/examples/xml_files/canvas_demo.xml @@ -11,7 +11,7 @@ - + @@ -20,12 +20,15 @@ [(350.0,350.0),2,{0:'d0-u2'}] - + + + + - Modelling + My First UserFeedback Module .\examples\results\output.pkl diff --git a/src/vai_lab/examples/xml_files/pybullet_env_example.xml b/src/vai_lab/examples/xml_files/pybullet_env_example.xml index b12be5c7..53629908 100644 --- a/src/vai_lab/examples/xml_files/pybullet_env_example.xml +++ b/src/vai_lab/examples/xml_files/pybullet_env_example.xml @@ -42,7 +42,7 @@ - Modelling + MyEnv .\examples\results\output.pkl diff --git a/tests/test_launch.py b/tests/test_launch.py index 07d70d5e..88866127 100644 --- a/tests/test_launch.py +++ b/tests/test_launch.py @@ -7,3 +7,4 @@ def test_launch(): core = ai.Core() core._debug = True core.run() + \ No newline at end of file From 50727af1cf4ce6cbe96a254788872ba2e0a1788b Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 7 Jul 2023 14:54:07 +0200 Subject: [PATCH 04/48] Change model name inside class --- .../DataProcessing/plugins/binarizer.py | 2 +- .../DataProcessing/plugins/integral.py | 2 +- .../plugins/kbinsdiscretizer.py | 2 +- .../DataProcessing/plugins/labelbinarizer.py | 2 +- .../DataProcessing/plugins/labelencoder.py | 2 +- .../DataProcessing/plugins/maxabsscaler.py | 2 +- .../DataProcessing/plugins/minmaxscaler.py | 2 +- .../plugins/multilabelbinarizer.py | 2 +- .../DataProcessing/plugins/normalizer.py | 2 +- .../DataProcessing/plugins/onehotencoder.py | 2 +- .../DataProcessing/plugins/ordinalencoder.py | 2 +- .../plugins/polynomialfeatures.py | 2 +- .../plugins/quantiletransformer.py | 2 +- .../DataProcessing/plugins/standardscaler.py | 2 +- .../Modelling/plugins/affinitypropagation.py | 2 +- .../Modelling/plugins/bayesianridge.py | 2 +- src/vai_lab/Modelling/plugins/birch.py | 2 +- .../plugins/decisiontreeclassifier.py | 2 +- .../plugins/decisiontreeregressor.py | 2 +- src/vai_lab/Modelling/plugins/elasticnet.py | 2 +- src/vai_lab/Modelling/plugins/gpclassifier.py | 2 +- src/vai_lab/Modelling/plugins/gpregressor.py | 2 +- src/vai_lab/Modelling/plugins/kernelridge.py | 2 +- src/vai_lab/Modelling/plugins/kmeans.py | 2 +- .../Modelling/plugins/knnclassifier.py | 2 +- src/vai_lab/Modelling/plugins/knnregressor.py | 2 +- src/vai_lab/Modelling/plugins/lasso.py | 2 +- .../Modelling/plugins/linearregression.py | 2 +- .../Modelling/plugins/logisticregression.py | 2 +- src/vai_lab/Modelling/plugins/meanshift.py | 2 +- .../plugins/passiveaggressiveclassifier.py | 2 +- src/vai_lab/Modelling/plugins/perceptron.py | 2 +- .../plugins/randomforestclassifier.py | 2 +- .../plugins/randomforestregressor.py | 2 +- .../Modelling/plugins/ridgeregression.py | 2 +- src/vai_lab/Modelling/plugins/svc.py | 2 +- src/vai_lab/Modelling/plugins/svr.py | 2 +- src/vai_lab/examples/results/output.pkl | Bin 6665 -> 6601 bytes 38 files changed, 37 insertions(+), 37 deletions(-) diff --git a/src/vai_lab/DataProcessing/plugins/binarizer.py b/src/vai_lab/DataProcessing/plugins/binarizer.py index 14d0e285..f364bdea 100644 --- a/src/vai_lab/DataProcessing/plugins/binarizer.py +++ b/src/vai_lab/DataProcessing/plugins/binarizer.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/integral.py b/src/vai_lab/DataProcessing/plugins/integral.py index 6778dbec..e2ec4993 100644 --- a/src/vai_lab/DataProcessing/plugins/integral.py +++ b/src/vai_lab/DataProcessing/plugins/integral.py @@ -22,7 +22,7 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() + self.model = model() def fit(self): return diff --git a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py index a844db5e..e305f911 100644 --- a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py +++ b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/labelbinarizer.py b/src/vai_lab/DataProcessing/plugins/labelbinarizer.py index b9858065..ef6bd839 100644 --- a/src/vai_lab/DataProcessing/plugins/labelbinarizer.py +++ b/src/vai_lab/DataProcessing/plugins/labelbinarizer.py @@ -18,4 +18,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/labelencoder.py b/src/vai_lab/DataProcessing/plugins/labelencoder.py index 84bed864..c735b262 100644 --- a/src/vai_lab/DataProcessing/plugins/labelencoder.py +++ b/src/vai_lab/DataProcessing/plugins/labelencoder.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/maxabsscaler.py b/src/vai_lab/DataProcessing/plugins/maxabsscaler.py index d8e700ce..7239c582 100644 --- a/src/vai_lab/DataProcessing/plugins/maxabsscaler.py +++ b/src/vai_lab/DataProcessing/plugins/maxabsscaler.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py index d3010e3b..e8cb1098 100644 --- a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py +++ b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py b/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py index 24be8036..db25d2e2 100644 --- a/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py +++ b/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/normalizer.py b/src/vai_lab/DataProcessing/plugins/normalizer.py index 5b92cfe6..afc29fdf 100644 --- a/src/vai_lab/DataProcessing/plugins/normalizer.py +++ b/src/vai_lab/DataProcessing/plugins/normalizer.py @@ -23,4 +23,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/onehotencoder.py b/src/vai_lab/DataProcessing/plugins/onehotencoder.py index 259574bc..b20b97c8 100644 --- a/src/vai_lab/DataProcessing/plugins/onehotencoder.py +++ b/src/vai_lab/DataProcessing/plugins/onehotencoder.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/ordinalencoder.py b/src/vai_lab/DataProcessing/plugins/ordinalencoder.py index 558083cf..037bcf43 100644 --- a/src/vai_lab/DataProcessing/plugins/ordinalencoder.py +++ b/src/vai_lab/DataProcessing/plugins/ordinalencoder.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py index deae011c..a176bf26 100644 --- a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py +++ b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py @@ -24,4 +24,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/quantiletransformer.py b/src/vai_lab/DataProcessing/plugins/quantiletransformer.py index 1e6b1c5b..c10ecd76 100644 --- a/src/vai_lab/DataProcessing/plugins/quantiletransformer.py +++ b/src/vai_lab/DataProcessing/plugins/quantiletransformer.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/standardscaler.py b/src/vai_lab/DataProcessing/plugins/standardscaler.py index f44c7d7b..c60ea60d 100644 --- a/src/vai_lab/DataProcessing/plugins/standardscaler.py +++ b/src/vai_lab/DataProcessing/plugins/standardscaler.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.proc = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/affinitypropagation.py b/src/vai_lab/Modelling/plugins/affinitypropagation.py index b3dbc666..648a2091 100644 --- a/src/vai_lab/Modelling/plugins/affinitypropagation.py +++ b/src/vai_lab/Modelling/plugins/affinitypropagation.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/bayesianridge.py b/src/vai_lab/Modelling/plugins/bayesianridge.py index 9844f610..c8c034f4 100644 --- a/src/vai_lab/Modelling/plugins/bayesianridge.py +++ b/src/vai_lab/Modelling/plugins/bayesianridge.py @@ -19,4 +19,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/birch.py b/src/vai_lab/Modelling/plugins/birch.py index c1bd4a01..86325a30 100644 --- a/src/vai_lab/Modelling/plugins/birch.py +++ b/src/vai_lab/Modelling/plugins/birch.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py b/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py index 32a8b3cc..899cd1e7 100644 --- a/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py +++ b/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/decisiontreeregressor.py b/src/vai_lab/Modelling/plugins/decisiontreeregressor.py index d3b071f6..4665cd54 100644 --- a/src/vai_lab/Modelling/plugins/decisiontreeregressor.py +++ b/src/vai_lab/Modelling/plugins/decisiontreeregressor.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/elasticnet.py b/src/vai_lab/Modelling/plugins/elasticnet.py index 0e85f49d..4a3f53ee 100644 --- a/src/vai_lab/Modelling/plugins/elasticnet.py +++ b/src/vai_lab/Modelling/plugins/elasticnet.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/gpclassifier.py b/src/vai_lab/Modelling/plugins/gpclassifier.py index b332f673..cefcf61a 100644 --- a/src/vai_lab/Modelling/plugins/gpclassifier.py +++ b/src/vai_lab/Modelling/plugins/gpclassifier.py @@ -22,4 +22,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/gpregressor.py b/src/vai_lab/Modelling/plugins/gpregressor.py index edf23cb6..add31a24 100644 --- a/src/vai_lab/Modelling/plugins/gpregressor.py +++ b/src/vai_lab/Modelling/plugins/gpregressor.py @@ -22,4 +22,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/kernelridge.py b/src/vai_lab/Modelling/plugins/kernelridge.py index 95cbdb20..28e65b4b 100644 --- a/src/vai_lab/Modelling/plugins/kernelridge.py +++ b/src/vai_lab/Modelling/plugins/kernelridge.py @@ -23,4 +23,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/kmeans.py b/src/vai_lab/Modelling/plugins/kmeans.py index 960512ce..7a7bc6dd 100644 --- a/src/vai_lab/Modelling/plugins/kmeans.py +++ b/src/vai_lab/Modelling/plugins/kmeans.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/knnclassifier.py b/src/vai_lab/Modelling/plugins/knnclassifier.py index 3a56f8ec..6fdfc9c9 100644 --- a/src/vai_lab/Modelling/plugins/knnclassifier.py +++ b/src/vai_lab/Modelling/plugins/knnclassifier.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/knnregressor.py b/src/vai_lab/Modelling/plugins/knnregressor.py index 8be1bcc8..656d052c 100644 --- a/src/vai_lab/Modelling/plugins/knnregressor.py +++ b/src/vai_lab/Modelling/plugins/knnregressor.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/lasso.py b/src/vai_lab/Modelling/plugins/lasso.py index 10ccbfed..bfb391b4 100644 --- a/src/vai_lab/Modelling/plugins/lasso.py +++ b/src/vai_lab/Modelling/plugins/lasso.py @@ -18,4 +18,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/linearregression.py b/src/vai_lab/Modelling/plugins/linearregression.py index c25523cf..25facd00 100644 --- a/src/vai_lab/Modelling/plugins/linearregression.py +++ b/src/vai_lab/Modelling/plugins/linearregression.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/logisticregression.py b/src/vai_lab/Modelling/plugins/logisticregression.py index 84390752..8d722aef 100644 --- a/src/vai_lab/Modelling/plugins/logisticregression.py +++ b/src/vai_lab/Modelling/plugins/logisticregression.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/meanshift.py b/src/vai_lab/Modelling/plugins/meanshift.py index d3d8c207..28d1327b 100644 --- a/src/vai_lab/Modelling/plugins/meanshift.py +++ b/src/vai_lab/Modelling/plugins/meanshift.py @@ -18,4 +18,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/passiveaggressiveclassifier.py b/src/vai_lab/Modelling/plugins/passiveaggressiveclassifier.py index 772ef0e6..dc01447a 100644 --- a/src/vai_lab/Modelling/plugins/passiveaggressiveclassifier.py +++ b/src/vai_lab/Modelling/plugins/passiveaggressiveclassifier.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/perceptron.py b/src/vai_lab/Modelling/plugins/perceptron.py index 63155166..93f35f40 100644 --- a/src/vai_lab/Modelling/plugins/perceptron.py +++ b/src/vai_lab/Modelling/plugins/perceptron.py @@ -21,4 +21,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/randomforestclassifier.py b/src/vai_lab/Modelling/plugins/randomforestclassifier.py index 39850f41..4075920d 100644 --- a/src/vai_lab/Modelling/plugins/randomforestclassifier.py +++ b/src/vai_lab/Modelling/plugins/randomforestclassifier.py @@ -22,4 +22,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/randomforestregressor.py b/src/vai_lab/Modelling/plugins/randomforestregressor.py index bea9acbb..19cc9914 100644 --- a/src/vai_lab/Modelling/plugins/randomforestregressor.py +++ b/src/vai_lab/Modelling/plugins/randomforestregressor.py @@ -22,4 +22,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/ridgeregression.py b/src/vai_lab/Modelling/plugins/ridgeregression.py index 6578c83d..d354170c 100644 --- a/src/vai_lab/Modelling/plugins/ridgeregression.py +++ b/src/vai_lab/Modelling/plugins/ridgeregression.py @@ -20,4 +20,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() + self.model = model() diff --git a/src/vai_lab/Modelling/plugins/svc.py b/src/vai_lab/Modelling/plugins/svc.py index 7862ae70..7fd4d700 100644 --- a/src/vai_lab/Modelling/plugins/svc.py +++ b/src/vai_lab/Modelling/plugins/svc.py @@ -23,4 +23,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/Modelling/plugins/svr.py b/src/vai_lab/Modelling/plugins/svr.py index 9519bf7e..282fb622 100644 --- a/src/vai_lab/Modelling/plugins/svr.py +++ b/src/vai_lab/Modelling/plugins/svr.py @@ -23,4 +23,4 @@ def __init__(self): Passes `globals` dict of all current variables """ super().__init__(globals()) - self.clf = model() \ No newline at end of file + self.model = model() \ No newline at end of file diff --git a/src/vai_lab/examples/results/output.pkl b/src/vai_lab/examples/results/output.pkl index c89ae4a91cc3acd3c4ce0c1ee30b78cbfe3ae253..ea411b5c3df0709e68a01e804ce75d8c8c6fa2f0 100644 GIT binary patch delta 4434 zcmai%c{J7E*T*SCNKptyrEgN|+i3ctT}Yu2B_%^57iA_DKKW*hLT*Yl|70qKG*C9l zP{<{j#l5ci;$C+!^Z5PNv!1n{Kfcd%);jC#v-a70ueH~Duh%}&`@Zd6dtLBuu^iuj zRxmksrNKGBa~sTkJkLA(`sPZjE+w)ClFt}qO@W_&ca%^V1I}t(+%BXw0b;g?k9nS& zhW8bxIJ-=Tp+Vimt+0ZIIkc|@f*m~YUT9mo#hr@T+rIHLUQFQ!k;GoR2re$O9~$jt zy@Q{w<|ffq-7strNqHnYh8t>@iT1hoK}ToT+$p6-yptR6XiudP=rZ=>sImP+IHYm+ zo_kUy6#ItitMTU{f0L-WmHAJ!*nVkU;>})M7}?j9-P4F3&zfcIOJ`8Cvg1di2?eR^ zoP^#k48ondEZVNUZE$e^i=`qmGqCy3r7s$G_khgCUb+s2hihjZZ=X!&qQ6iM`-;dA zMDJLwu5Ct4;j{0m>pM@e;J(AxAkWYlSo8Us{vJ*}zO)xP@!>TM9^Z+uxj#*Zk7f;r zC=R12AgFQR@t~F@nVrY}Oc8M4rFGh_W z5j;7KYlS^E1k72OMSM|qdp!WI9}kT65XK#FcT?Jk&5>SskU^}xa-|(-4@3*(ef$lf z=BkB!fA&JwU%EbT=cgdHh>s7#dXW{fVOn8!8uU}1gwXh=kew2E@1lJHhMp`h0x%7_=jR9I6yqP6$) zAgXC74_ET@P>$oaeN*5J^vT{-`Bck*s(t%4{6<(PUXxwkI`RfCf37@ew09gR=36#* z$aArAvt;L}eFGRD&HrFj`U|t_Ok&OrP*JMdvA1{W2*wHMtX^}T0_UGkH9z0hg+ike zBjs|HKzySSvds4!$f@vhF_Gp%#QRIu&Q7M6PYeRd+YknjhoA2fGaCof4A>;nD9TInyBEsAyrR zS&k<8yJ$`Nqu@Rz)bd_t0A((SY~M534nfMNvdpAr!H}-duwJAWs|dZXH#T3JM8~pM zH>X_7ApcO(k(KwOK>10BgWRH=TIc#tr`C)>OuFdxh#SAa^`*zlgO^6pJ>&xCNi!FF zE*ZP8dy#<*4)@HiQ|}?GZ2sA=v?-_;cx!H=Fb&cQ6%#*HTd_Y=|2#82_8ZP@Nu`cQ{U7YJcNCS{UuF>(=WBrKU%+RvXLk z^=|oI>k&FS6>4q~`!o@NZR4NAzk2YofX;$|7#kH6^M&89X@FI(?;JfG#vx4) ztx9!xaA{xX>v3W|0~6iLj-}A(Ks!+*{`Y}PaTZRb@6NlvVgh7J{TsG6k6?NYJ5caB6LYOyJibsS;g?t0?f0oHXzCCm4y%vi zM8L6){Vt;rRL1zwtv-j_>cnj#9Lk}7hV&Of%b0`=RW)ed*N0wP%%oc`(Qv@zspxd} z03=)68togRfKSl~O*gtE6kL zV4~QkGq;RYIl%N>a<){Bh8}eX(p3i*z&x*d^%|VO;b?}#HLKsKdgBStI zu3urGY;60U%FcIik#~SApz95XdFi7M3wdCqbCxjO#lx$zEy_yY z%}2lLLnnPA^_0XA7pfX&)?SP61J0g+WxAQI*!A|~NK7jQ*J%A(xzvP)rY1zbNCF3i zqf%>vTNZUL5GyEoVh(%+{f7RfM1vC?#qr~^gV?C2TWj{R0ds5!J>ljdyxOy1?7W-> z-!?kj_HXXQO{c$&D=mA7;uydA`Ccko5G5UbZoOleaBa49$B`i%IHlB4jeWqKb$+{g z=^P$%WH`4ke+9x}#U)dQgE(@3%=A3d2>Hgee$)<1Rw?{4082uDOeXNX!-Iw2 zpQUpQgsO9Eyv=;IrD(nTILIbTmD0>>dX zrc3>q#soY!-JolEkAk`v_@l~myFpvDgD46ozMzs#d(oa42FBao+qJoB2xcghtEctG z&~(N@#3=A58k8FD=*elo)(d}V?5Lt5f9|!Ov_k1ITk4Vy|gO=96{YtE>% zXFU)TRTQ7JUJ$Vm$t&GJhfYVW!~{bYSQsCelDf_Va;%_-|0S<0i>H%c;;ZGcOJd~` zCB>x@XT)W7V+B0UU2-Oy#tQoSo%8V{oAAX7`1*Nzkxc~uZ8$F){QePS3o^-)WED(S zAz7P}PMh$N&Y0+sZ1Q)lh+LBRm-s1u{=_@ts(P{fyU6GK$c~GT{xRmi(>ebq=XsJ7 z$(eM4&$8VIi~fS>JQ@5gavpBV+lI3@OiXJowV?tX_2`q%qnT zW)xU>a8-Cz$e}8@vGZ2i!$>ZcJ*v9a@`?*fX5W=8lVuZFSRS7n)l3K7*`-w_qKh5B zu9zA&$%Ba_B1v+gw*&Z={$-_}ft779Y0(|c-h-b7^HB5o2X=d=cz#pB`b-2-|e_XqHbnGQ4a-V7QRsgYz9AL8jDsbwio zT0r8$It|CbMR7^&=v2MTfjxO6)1eRAaN~nM&)ESAWLZj8)Jk#T%djLtTNKHlSP*^5 zHXcgnITHo7V;HL_#~LG4si5p}yjc4;4>~)?wX>CaAt8Bwn+J6QOTXp(^ww#Dq%A?e zUoQF+vjP>P;<}Oh;dDrxKNmJo-xs%~_JjW=l}*pDmLumxB~;QTFt&HIf62E-XuJE6 z+!Is_8CTh{S5u~;)+On+kN-EY&VI}%k;g<<#nSZzG0end_n$BNmDpGiDP42cjt$h6 zmOybEhq?T{=0l!u!AXp+R@mHuC!H$O-MkCH>bHHM#jjr=u_Nkox>*^1ELQD*XVZhm z`?Xr+1Das%pkL(ucPuayjFGjyF#*5l1ZC6p8-Mzgsn%269xiL5=`{vN`0HkD zdzfJA5h%|0trv}8AtR)>7}tr&t6Se8V^r~S^9S;DT;UfR-&V~5`MNUE!|W~?I=S-c zEujYBme_kfQs#nrHt{f=zN-(R|?O$x=+9c-Q=yS-n8TBQ9n*Ts}c4Z z{{od2JV4PH`lCzo%jc%4ymNpZYNl5QVljGu#fZ)DmtA?RjDSt#^XFJAP?>W#M z!9Zr(OZRXZ8`?LXiP2%OaV&3GYg<1XFZ9I8^axCXWSXYS)h}bP=7WD=YFj@N_9fE> zk&P92c>TWkF-tZCf0YYA>B7N`^`Di5ghsH*+E~HEX&QyaNXJSj3}`9j8R;9{$I6$N zU046C0O<^>YGy4LWyo>Z#2&_n?2GwcJLllojR#w~O}XfHrbszjoq-O~>%W&4c0*Cu zsm`u@Oe}5)+FrhgN5Ftl>+P-a75GH=(cPN|Tfy9|EbjKAOn%G829r0^G4I9Tl9=Rj zBy_3%>kI3_{U($6oE!;bL_u3`^(?aiqwSUR zRk%oMz9m$5jRgXFW~EAcoshEOOmEiuJUGGY2%7ep!`Ym)8hZv4`q zjW1Nlo0qW9ss0Kn64N1T-*7Odppl59&Em_24MGPD$5DYk=t3@F;TC-@wIZUT6UCSz z``eGvQ2&u&Nnm&ro~FaJP&)%{9IFh*YJ0%mQuMpZ9}F;zbl6s6IEgP!JrC9SPGEPJ z`uiV?pWj-eJ8@-C#zCs{Zg^;eB@P(5}e@8CzQN@Jcgn#i86>Uo<#r~fZ{C}%N`T9Hh`#bvn F`Cl(8H2nYo delta 4491 zcma)9c{mi@_vbB2QQD-GvXwQWRm(@D>{^tvM$srqSteT(DJe@bsfhB5qGXM-gpn=V z*oLe#!^~iqk%YJ9^_%y7zW;uo@9#d(dG5W>eeOB;o_jv$b8ekZ__kFyL>~UVM(E!b zM2-+MzkJTc-91M}Z5~NM$>-p?)1sbXBY-wEux*G$07IDv9Y-ZtY1Jz(v0qnd^h=kR_t-fMKj;M%J*<@mltv2nf%1MFA%ZRxTs zI64n|r2lM$ziq$2!v{9*HZQRX``(S3nU4B9Upq)h(_E7@6;FqG$(5m05hf1%%&asg z48XMD=^Ep$J@|BPZ$RIrc5oJ)4GaC)jm>9Bq47~M2+_MzxJIHEHd@5E@1-0FKcNZP zEFZ#*Nz?k>6h7($v=c0wSh(?>wk%|@2(}F#yBBXOhe_kL;L>sqG|!OMybvd4;l8o( zuZA6?u+%78duomk)tMsY)8rmBeUPv>NoV1ws#TZK5jGwKy5H-(9x_pzVN7l2o1y=qHSfp_Gz882zVWza-xRwr=_;HeCnHU$>mliL- z0xse^{Cnm_uyI^ByHcI<0sJRnZ|V$(@GxS3QlK&m=`Xjzq;MRswUvT5RC8g@0+Z(X z4rRAo_r)0wgPLiWF-ghAR@rvlbsNVJSu1|hWR{JM3CBW@>371$Whh(fGZ*T~G}CO# zClHlL(A%I(MT0G|f5@8!?k24=)y;GisF0t=*|cMJ!Kk{GSU)ywZ@qt@trO>-R)$7g zrQsdLB$+eGg-fp=ZQ+db;dvO$7(M=)|DeqoBM^&iP;>)3Qq!+AMP4(@E*`?`22K;we&W`S! z&gQ^KR(^^2P9_fO&D&)dSO=XR>r&Zk`MBwMG}Ohm50fLm&g|n1fMOb#PV(VHnj2}f zN`QhFzII*o=?=u@blKk%jYWg*c_p5HKMD`N`I?qg15(!0ANy(+;cK@$38Uthq$ zuaR8G?7MvkKOJ&#o*x@km6vU9%ygsZ({D;&A{G07@hDF{YZ1*B&8B_ohWD#&CVrGL zc&wV)X!CIxMj~!se{)HV2xvRCX+bg{*0%Elb~TP7TgveZQL_c2>E)YhvPa;y)Y>fN zU^!y+bR$k>^k9c#O8HM+77pjEl%4xrk7Gg0jOV{+V5M!8w#Um9^xjpq7Czb{c&Y;* z=Mf8_rBq*gca#COkOie`4WlSEOZ>Cia2)1xUap!X;aUXv2Yp&tz(PPtqxzTz54z(& z6dFl2=uR-$a(rVACXC5SJQY4B%))MHbc|zkNxxi&Ar}fuJ!RV!s7UW0*63+!KxyK^ z#2YGOFcQ+dzx>uP)TD`TiJWGX^KPErZqC6*#Zz_nzr|zzhUP@&89M4sV}?j4SB)V< z_=BT}P&-mRuIc?4?m*P9Um>fySqR>LCpxX10kgV=JGIOPa8N>KHbH9~ho>HGx4PJf zcYA0j4G!0%)lV&dw;}_J7gP>V1-YK;ATuIC9l`x+gY=UkG|;;XiQZ*=XbF2N>-Cht z4ho*XqdF06oklt;o7RigqNIzYdkn0dSh4SzKMk9!N%J@o zib@SD+&5A1!BSUA=|VpqKAw2=o6?3qyGd`Ir*lSOC$@Di>nIDl`_x3=mUbaB%~51l z5KO9G>e7#Xbf7oxg7CWDPY^$wYHlv@fNNyBW#$zg)`$h`FAy9w!niL`=58zU&RlQ# z9^8#q1+i0mUbSHIbI_pl-eDByr)R9q>V!pbukGq80la_R`Pdys!(XJPU^8ExCfv5q z3h|w2!6y6Sq})~x8UlG!laxPU;GC8^{*8g-QukN<{jMFcX^Fe0G#d~Uapf?raR3?h z&)mIO!+1NDZXEHe4M#rLE>T%s1now*1I~W^=>2^v&sMG;GYK5pY$y}?L+@TLT+2di z#GCW!k)t5VM4h)B$`ssKBHF!d$vA|_WG(HuF`T-F3VA|3I*8r}42ti->w2`nl;19To+TjtITu!&$#;(ZnDVUw&>=eyx{;W%Y4acJTQ)p^`*@dpsLiUcm;6 zB0LZzj_cfN zO%0<@%bg={+k%!&r5*v7`oQ!uG5wy}2TSf0O>VRcVL3}?HONdD$J+En7;&LcY{ zjx|B#s9(A3;2^5b2S(qz$w9M~|FPRr91shoOux!;LE$&CmrL?Mju7$ix#a05=+Ak_ z=7~qdYDu0F7Ksg)R8v-t5Y{3`2yYdLwm@_QvW@KGO?Hl$@8NXGg>2z-n|zE+I35d$ zE$a!FEFwg_yq&IklTC#p=6iWx_9UC`{b%6fNzTiV2>avAi6F8vL6&fuU~4LLi%hT& z6YN$t6*d(XOwNYelL_aY1j+@9{-c+0^`8xwEC}AFLIfYvy#!xV;eVH0{WJ0h&ZpKbaZYCWNUGqSut`Ye;vc%OQEja6F6PH{sNRTHgCTDFZIAK(3QKHR9 zT8`wpAgu(1rr~YK#yU8x3(ndgNkyWX_2l1^m1xUL9OJvyKuXO_i6F*Av@2W9%i!ekY#!FNe3mL5s`YQ-Wku}L>* zRn+BPJQguHxnLp}HC-FrQUd8PxYu}7A*U5{xr6nJx5?OOP4a7AMaR<%mA{sM8ARxc zGJhit8YsYwVWEpvw#a--8VF|B=BFHGqWH(vv=G7i zLo0`browjEuFLgm_T|F$FQQ)Xh~TM4k7mYh?t#_3$a5^YG>GHVixcaZ;OW~%cI{)p z@}#O#yE_k2dl@gXztOR-ue$TtifTl3FIq~{imrm)FiEP$ryME?M@D|?kAk(r&4N=f zg6RUc6sLj#@SEKAnkG9yjP&?}RXYGeo7lo;eJ0kc^Lx$^d*GSrc#kR5hIyJ6q2s5S z;O>blg5?O5sEM!lq>drGf8$`@{(d-9Mt5J5uEMPhFV0)bZY-EfCIwEVyoX<=tKBEL zHoTYLGx(+D1)9iBT?#QQv@Z0ND@rZJK$TooiFF^|>&wiENRQxy*v9sO&JjeJxCM(Z zqe8F6E%rfF4^*;==Qv4yn634jo>go?nSV;>q0?j7_x(+%tilkAQi5K5<}kn)`yx+_ z9mC=@l5*JK(`L+DspqqM-x!L_(@L5a@bFS}YQj#r11EJhh~0H$LRBjE$Y$0U%o`&Q z-r6^c3w;+a>c}x*GjVBl-*6Y2Sh-Inb=uJR+_QAdiXBpsVFq+CEsd(I`pWSsZ-eoxEWt<0 zbN1V@urZu_*%Re=nSvMM4?b*de~FOxTAjx^92Cgcbmt!JKxkvB^I#(ti)OEE)qg#L zD(|RX50z@n-}tOKTc;Z5k3Ktnfx?CRtKs|YFF24RIa*ezwt-Y{sbXAc-HRu6OZTXp z9YLMMh?|dA6{JP&RLPN445UwrS!~Wlvi}`_LpB>mZKegrUHwS+pxTj_FhDA;6DqW& zphwnEVwZFaoYSXvD?9SgNXs)GSksTqyPo@9pQd72pRMnru0bf6QCnSE4G8Ey+xvkg zppaZE+g#f-y;vTxu$bJ$#O(H8#%<3ku*u-=g*5RF)c4HB-y!wFs%Gahem4Ui%QRgL zE(~DnO8uEubsm(hDQ)+3>X9)`Ip*Kp3dxM2YNLq;gwz|;A0MrTkeyMpuX-j{aPq1c zIRf}f+v!I>=)m>;5}mnS0>yOx_vxe*2A1g-=Wq*nh>j`JEVZZxBKr3>3h?{4_`>>{ zdn`QB4nMJLB@@))S@XoqL`c8!%JAoA5jUktWGUaDo;Jf?&b@4$jrc&km z_T;x>GF-XDY>tgXLbf+*PK-eBFS(Ge=O3|-zNaEZr2=YSHq2T+q9AhwN7fE;v0w7> z-cThD`19_*(cba}_KP$_J|=PS%qyq;mdp@*x)k;IULS|N>yEyxLlkJY2Kbqsn`J;W z<19%zupj1YZ8A?53=4vzqGE*$9pjSchhv;75OL;W;+OX$m>BpO+;cqx_ Date: Fri, 7 Jul 2023 15:27:44 +0200 Subject: [PATCH 05/48] Display all available class optional parameters --- src/vai_lab/utils/plugins/pluginCanvas.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index de35d34f..935f11a5 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -5,7 +5,7 @@ import os import numpy as np import pandas as pd -from inspect import getmembers, isfunction +from inspect import getmembers, isfunction, getfullargspec from typing import Dict, List from PIL import Image, ImageTk @@ -489,6 +489,9 @@ def optionsWindow(self): avail_plugins["_PLUGIN_PACKAGE"], avail_plugins["_PLUGIN_CLASS_NAME"]) function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + func_opt = getfullargspec(plugin().model.__init__).kwonlydefaults + func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} + self.opt_settings = {**self.opt_settings, **func_opt} if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): From 489719a7421ac680ec76cbce3289dbe12dbc4995 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 7 Jul 2023 15:49:52 +0200 Subject: [PATCH 06/48] Read required settings from the class --- src/vai_lab/Modelling/plugins/randomforestclassifier.py | 3 +-- src/vai_lab/Modelling/plugins/randomforestregressor.py | 3 +-- src/vai_lab/utils/plugins/pluginCanvas.py | 5 +++++ 3 files changed, 7 insertions(+), 4 deletions(-) diff --git a/src/vai_lab/Modelling/plugins/randomforestclassifier.py b/src/vai_lab/Modelling/plugins/randomforestclassifier.py index 4075920d..a8dc60cd 100644 --- a/src/vai_lab/Modelling/plugins/randomforestclassifier.py +++ b/src/vai_lab/Modelling/plugins/randomforestclassifier.py @@ -6,8 +6,7 @@ "RFC": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "classification"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"max_depth": "int", - "n_estimators": "int"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/randomforestregressor.py b/src/vai_lab/Modelling/plugins/randomforestregressor.py index 19cc9914..85cf9c10 100644 --- a/src/vai_lab/Modelling/plugins/randomforestregressor.py +++ b/src/vai_lab/Modelling/plugins/randomforestregressor.py @@ -6,8 +6,7 @@ "RFR": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"max_depth": "int", - "n_estimators": "int"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 935f11a5..361f53c5 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -489,6 +489,11 @@ def optionsWindow(self): avail_plugins["_PLUGIN_PACKAGE"], avail_plugins["_PLUGIN_CLASS_NAME"]) function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + # Update required and optional settings with the actual class + func_req = getfullargspec(plugin().model.__init__).args + func_req = {p: '' for p in func_req} + func_req.pop('self', None) + self.req_settings = {**self.req_settings, **func_req} func_opt = getfullargspec(plugin().model.__init__).kwonlydefaults func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} self.opt_settings = {**self.opt_settings, **func_opt} From cff2d6f533b74b13a6aa6a3719fcb4ba9fb3d50b Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 7 Jul 2023 16:27:12 +0200 Subject: [PATCH 07/48] Remove some unrequired parameters --- src/vai_lab/DataProcessing/plugins/argopt.py | 52 ------------------- .../DataProcessing/plugins/integral.py | 32 ------------ .../plugins/kbinsdiscretizer.py | 4 +- .../DataProcessing/plugins/minmaxscaler.py | 2 +- .../DataProcessing/plugins/normalizer.py | 2 +- .../plugins/polynomialfeatures.py | 3 +- .../plugins/decisiontreeclassifier.py | 2 +- src/vai_lab/Modelling/plugins/elasticnet.py | 3 +- src/vai_lab/Modelling/plugins/kernelridge.py | 3 +- src/vai_lab/Modelling/plugins/kmeans.py | 3 +- .../Modelling/plugins/knnclassifier.py | 3 +- src/vai_lab/Modelling/plugins/knnregressor.py | 3 +- .../Modelling/plugins/logisticregression.py | 2 +- .../Modelling/plugins/ridgeregression.py | 4 +- 14 files changed, 14 insertions(+), 104 deletions(-) delete mode 100644 src/vai_lab/DataProcessing/plugins/argopt.py delete mode 100644 src/vai_lab/DataProcessing/plugins/integral.py diff --git a/src/vai_lab/DataProcessing/plugins/argopt.py b/src/vai_lab/DataProcessing/plugins/argopt.py deleted file mode 100644 index 4cf7219e..00000000 --- a/src/vai_lab/DataProcessing/plugins/argopt.py +++ /dev/null @@ -1,52 +0,0 @@ -from distutils.command.config import config -from numpy import argmin, argmax -from vai_lab._plugin_templates import DataProcessingT -import pandas as pd - -_PLUGIN_READABLE_NAMES = {"argopt": "default", - "argmax": "alias", - "argmin": "alias"} # type:ignore -_PLUGIN_MODULE_OPTIONS = {"Type": "math operator"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {'min/max': "str"} # type:ignore -_PLUGIN_REQUIRED_DATA = {} # type:ignore -_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore - -class argopt(DataProcessingT): - """ - Calculate the optimum argument - """ - - def __init__(self): - """Initialises parent class. - Passes `globals` dict of all current variables - """ - super().__init__(globals()) - # self.proc = model() - - def configure(self, config: dict): - """Sets and parses plugin configurations options - :param config: dict of internal tags set in the XML config file - """ - # super().configure(config) - self.config = config - - def set_data_in(self, data_in): - """Sets and parses incoming data - :param data_in: saves data as class variable - expected type: aidesign.Data.Data_core.Data - """ - super().set_data_in(data_in) - - def fit(self): - self.cleaned_options = self._clean_solver_options() - # self.proc.set_params(**cleaned_options) - # self.proc.fit(self.X) - return - - def transform(self,data): - if config['min/max'] == 'max': - data.append_data_column("X", pd.DataFrame(argmax(self.X))) - else: - data.append_data_column("X", pd.DataFrame(argmin(self.X))) - return data \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/integral.py b/src/vai_lab/DataProcessing/plugins/integral.py deleted file mode 100644 index e2ec4993..00000000 --- a/src/vai_lab/DataProcessing/plugins/integral.py +++ /dev/null @@ -1,32 +0,0 @@ -from scipy.integrate import simps as model -from vai_lab._plugin_templates import DataProcessingT -import pandas as pd - -_PLUGIN_READABLE_NAMES = {"Integral": "default", - "integral": "alias"} # type:ignore -_PLUGIN_MODULE_OPTIONS = {"Type": "math operator"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"dx": "float", - "axis": "int", - "even": "str"} # type:ignore -_PLUGIN_REQUIRED_DATA = {} # type:ignore -_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore - -class Integral(DataProcessingT): - """ - Calculate integral of array using the composite trapezoidal rule - """ - - def __init__(self): - """Initialises parent class. - Passes `globals` dict of all current variables - """ - super().__init__(globals()) - self.model = model() - - def fit(self): - return - - def transform(self,data): - data.append_data_column("X", pd.DataFrame(self.proc(self.X))) - return data \ No newline at end of file diff --git a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py index e305f911..66af2f5a 100644 --- a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py +++ b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py @@ -6,9 +6,9 @@ _PLUGIN_READABLE_NAMES = {"KBinsDiscretizer":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"n_bins": "int"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore -_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore +_PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore class KBinsDiscretizer(DataProcessingT): """ diff --git a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py index e8cb1098..9a241fcc 100644 --- a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py +++ b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py @@ -5,7 +5,7 @@ _PLUGIN_READABLE_NAMES = {"MinMaxScaler":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"feature_range": "tuple"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/normalizer.py b/src/vai_lab/DataProcessing/plugins/normalizer.py index afc29fdf..d2c9ca57 100644 --- a/src/vai_lab/DataProcessing/plugins/normalizer.py +++ b/src/vai_lab/DataProcessing/plugins/normalizer.py @@ -8,7 +8,7 @@ "Norm": "alias", "normalizer": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"norm": "str"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py index a176bf26..a0d6926b 100644 --- a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py +++ b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py @@ -7,8 +7,7 @@ "polynomialfeatures": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "Other"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"degree": "int", - "interaction_only": "bool", +_PLUGIN_OPTIONAL_SETTINGS = {"interaction_only": "bool", "include_bias": "bool"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py b/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py index 899cd1e7..797de431 100644 --- a/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py +++ b/src/vai_lab/Modelling/plugins/decisiontreeclassifier.py @@ -1,7 +1,7 @@ from vai_lab._plugin_templates import ModellingPluginTClass from sklearn.tree import DecisionTreeClassifier as model -_PLUGIN_READABLE_NAMES = {"DecissionTreeClassifier": "default", +_PLUGIN_READABLE_NAMES = {"DecisionTreeClassifier": "default", "DTClassifier": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "classification"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/elasticnet.py b/src/vai_lab/Modelling/plugins/elasticnet.py index 4a3f53ee..eabd33bf 100644 --- a/src/vai_lab/Modelling/plugins/elasticnet.py +++ b/src/vai_lab/Modelling/plugins/elasticnet.py @@ -4,8 +4,7 @@ _PLUGIN_READABLE_NAMES = {"ElasticNet": "default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"alpha": "float", - "l1_ratio": "float"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"l1_ratio": "float"} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/kernelridge.py b/src/vai_lab/Modelling/plugins/kernelridge.py index 28e65b4b..fcac937a 100644 --- a/src/vai_lab/Modelling/plugins/kernelridge.py +++ b/src/vai_lab/Modelling/plugins/kernelridge.py @@ -5,8 +5,7 @@ "KR": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"alpha": "float", - "kernel": "str", +_PLUGIN_OPTIONAL_SETTINGS = {"kernel": "str", "gamma": "float", "degree": "int"} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/kmeans.py b/src/vai_lab/Modelling/plugins/kmeans.py index 7a7bc6dd..11e3e71e 100644 --- a/src/vai_lab/Modelling/plugins/kmeans.py +++ b/src/vai_lab/Modelling/plugins/kmeans.py @@ -4,8 +4,7 @@ _PLUGIN_READABLE_NAMES = {"KMeans": "default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "clustering"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"n_clusters": "int", - "n_init": "int"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {"X"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/knnclassifier.py b/src/vai_lab/Modelling/plugins/knnclassifier.py index 6fdfc9c9..2c880e95 100644 --- a/src/vai_lab/Modelling/plugins/knnclassifier.py +++ b/src/vai_lab/Modelling/plugins/knnclassifier.py @@ -5,8 +5,7 @@ "KNN-C": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "classification"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"n_neighbors": "int", - "weights": "str"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"weights": "str"} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/knnregressor.py b/src/vai_lab/Modelling/plugins/knnregressor.py index 656d052c..64982701 100644 --- a/src/vai_lab/Modelling/plugins/knnregressor.py +++ b/src/vai_lab/Modelling/plugins/knnregressor.py @@ -5,8 +5,7 @@ "KNN-R": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"n_neighbors": "int", - "weights": "str"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {"weights": "str"} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/logisticregression.py b/src/vai_lab/Modelling/plugins/logisticregression.py index 8d722aef..25eed6c2 100644 --- a/src/vai_lab/Modelling/plugins/logisticregression.py +++ b/src/vai_lab/Modelling/plugins/logisticregression.py @@ -6,7 +6,7 @@ "MaxEnt": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "classification"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"penalty": "str", "C": "float"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/Modelling/plugins/ridgeregression.py b/src/vai_lab/Modelling/plugins/ridgeregression.py index d354170c..0ee2f965 100644 --- a/src/vai_lab/Modelling/plugins/ridgeregression.py +++ b/src/vai_lab/Modelling/plugins/ridgeregression.py @@ -2,10 +2,10 @@ from sklearn.linear_model import Ridge as model _PLUGIN_READABLE_NAMES = {"RidgeRegression": "default", - "Ridge": "alias"} # type:ignore + "Ridge": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "regression"} # type:ignore _PLUGIN_REQUIRED_SETTINGS = {} # type:ignore -_PLUGIN_OPTIONAL_SETTINGS = {"alpha": "float"} # type:ignore +_PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {"X", "Y"} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X_tst", 'Y_tst'} # type:ignore From 9dc4f0cadf46b2f4ca131a033a3af6c84871e55d Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 7 Jul 2023 16:27:45 +0200 Subject: [PATCH 08/48] Set arguments with default values as optional --- src/vai_lab/utils/plugins/pluginCanvas.py | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 361f53c5..b7464ab9 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -490,13 +490,21 @@ def optionsWindow(self): avail_plugins["_PLUGIN_CLASS_NAME"]) function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] # Update required and optional settings with the actual class - func_req = getfullargspec(plugin().model.__init__).args - func_req = {p: '' for p in func_req} - func_req.pop('self', None) - self.req_settings = {**self.req_settings, **func_req} + func_args = getfullargspec(plugin().model.__init__).args + if func_args is not None: + func_args.remove('self') + func_def = getfullargspec(plugin().model.__init__).defaults + if func_def is None: + func_def = [] + func_req = {p: '' for p in func_args[:(len(func_args)-len(func_def))]} + func_opt = {p: type(func_def[i]).__name__ for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} + self.req_settings = {**self.req_settings, **func_req} + if func_opt is not None: + self.opt_settings = {**self.opt_settings, **func_opt} func_opt = getfullargspec(plugin().model.__init__).kwonlydefaults - func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} - self.opt_settings = {**self.opt_settings, **func_opt} + if func_opt is not None: + func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} + self.opt_settings = {**self.opt_settings, **func_opt} if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): From 40fd90252562d90f361c337edf8993bca2794043 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Mon, 10 Jul 2023 09:02:13 +0200 Subject: [PATCH 09/48] Define function to get arguments from function --- src/vai_lab/utils/plugins/pluginCanvas.py | 46 ++++++++++++++++------- 1 file changed, 32 insertions(+), 14 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index b7464ab9..cdb817ed 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -488,24 +488,15 @@ def optionsWindow(self): plugin = import_plugin_absolute(globals(), avail_plugins["_PLUGIN_PACKAGE"], avail_plugins["_PLUGIN_CLASS_NAME"]) - function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] - # Update required and optional settings with the actual class - func_args = getfullargspec(plugin().model.__init__).args - if func_args is not None: - func_args.remove('self') - func_def = getfullargspec(plugin().model.__init__).defaults - if func_def is None: - func_def = [] - func_req = {p: '' for p in func_args[:(len(func_args)-len(func_def))]} - func_opt = {p: type(func_def[i]).__name__ for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} + # Update required and optional settings for the plugin + func_req, func_opt = self.getArgs(plugin().model.__init__) + if func_req is not None: self.req_settings = {**self.req_settings, **func_req} - if func_opt is not None: - self.opt_settings = {**self.opt_settings, **func_opt} - func_opt = getfullargspec(plugin().model.__init__).kwonlydefaults if func_opt is not None: - func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} self.opt_settings = {**self.opt_settings, **func_opt} + function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): self.newWindow.destroy() @@ -580,6 +571,33 @@ def optionsWindow(self): self.newWindow.grid_rowconfigure(1, weight=2) self.newWindow.grid_columnconfigure(0, weight=1) + def getArgs(self, f): + """ Get required and optional arguments from function. + + Parameters + ---------- + f : function + function to extract arguments from + + :returns out: two dictionaries with arguments and default value (if optional) + """ + + func_args = getfullargspec(f).args + if func_args is not None: + func_args.remove('self') + func_def = getfullargspec(f).defaults + if func_def is None: + func_def = [] + func_req = {p: '' for p in func_args[:(len(func_args)-len(func_def))]} + func_r_opt = {p: type(func_def[i]).__name__ for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} + + func_opt = getfullargspec(f).kwonlydefaults + if func_opt is not None: + func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} + if func_r_opt is not None: + func_opt = {**func_r_opt, **func_opt} + return func_req, func_opt + def create_treeView(self, tree_frame): """ Function to create a new tree view in the given frame From 09fc1d183411a39327e769f02dc63efb9ba47743 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Mon, 10 Jul 2023 11:25:06 +0200 Subject: [PATCH 10/48] Updated settings display --- src/vai_lab/_plugin_templates.py | 18 +++---- src/vai_lab/utils/plugins/pluginCanvas.py | 66 +++++++++++++---------- 2 files changed, 48 insertions(+), 36 deletions(-) diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 98d289d4..697672c4 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -188,13 +188,13 @@ def _clean_solver_options(self): def fit(self): cleaned_options = self._clean_solver_options() try: - self.proc.set_params(**cleaned_options) + self.model.set_params(**cleaned_options) except Exception as exc: print('The plugin encountered an error on the parameters of ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: - self.proc.fit(self.X) + self.model.fit(self.X) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -202,14 +202,14 @@ def fit(self): def transform(self, data: DataInterface) -> DataInterface: try: - data.append_data_column("X", pd.DataFrame(self.proc.transform(self.X))) + data.append_data_column("X", pd.DataFrame(self.model.transform(self.X))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise if self.X_tst is not None: try: - data.append_data_column("X_test", pd.DataFrame(self.proc.transform(self.X_tst))) + data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -224,13 +224,13 @@ def __init__(self, plugin_globals: dict) -> None: def solve(self): """Sends params to solver, then runs solver""" try: - self.clf.set_params(**self._config["options"]) + self.model.set_params(**self._config["options"]) except Exception as exc: print('The plugin encountered an error on the parameters of ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: - self.clf.fit(self.X, self.Y) + self.model.fit(self.X, self.Y) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -245,7 +245,7 @@ def predict(self, data): Returns predicted values. """ try: - return self.clf.predict(data) + return self.model.predict(data) except Exception as exc: print('The plugin encountered an error when predicting with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -261,7 +261,7 @@ def score(self, X, Y, sample_weight): :returns: score : float of ``self.predict(X)`` wrt. `y`. """ try: - return self.clf.score(X, Y, sample_weight) + return self.model.score(X, Y, sample_weight) except Exception as exc: print('The plugin encountered an error when calculating the score with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') @@ -280,7 +280,7 @@ def predict_proba(self, data): Returns predicted values. """ try: - return self.clf.predict_proba(data) + return self.model.predict_proba(data) except Exception as exc: print('The plugin encountered an error when predicting the probability with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index cdb817ed..64f6de24 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -478,17 +478,14 @@ def optionsWindow(self): module = np.array(self.module_list)[self.m == np.array(self.id_mod)][0] ps = PluginSpecs() - self.opt_settings = ps.optional_settings[module][self.plugin[self.m].get( - )] - self.req_settings = ps.required_settings[module][self.plugin[self.m].get( - )] - file_name = os.path.split(ps.find_from_class_name(self.plugin[self.m].get())['_PLUGIN_DIR'])[-1] avail_plugins = ps.available_plugins[module][file_name] plugin = import_plugin_absolute(globals(), - avail_plugins["_PLUGIN_PACKAGE"], - avail_plugins["_PLUGIN_CLASS_NAME"]) + avail_plugins["_PLUGIN_PACKAGE"], + avail_plugins["_PLUGIN_CLASS_NAME"]) # Update required and optional settings for the plugin + self.req_settings = ps.required_settings[module][self.plugin[self.m].get()] + self.opt_settings = ps.optional_settings[module][self.plugin[self.m].get()] func_req, func_opt = self.getArgs(plugin().model.__init__) if func_req is not None: self.req_settings = {**self.req_settings, **func_req} @@ -589,11 +586,11 @@ def getArgs(self, f): if func_def is None: func_def = [] func_req = {p: '' for p in func_args[:(len(func_args)-len(func_def))]} - func_r_opt = {p: type(func_def[i]).__name__ for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} + func_r_opt = {p: func_def[i] for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} func_opt = getfullargspec(f).kwonlydefaults if func_opt is not None: - func_opt = {p: type(func_opt[p]).__name__ for p in func_opt} + func_opt = {p: func_opt[p] for p in func_opt} if func_r_opt is not None: func_opt = {**func_r_opt, **func_opt} return func_req, func_opt @@ -622,7 +619,7 @@ def create_treeView(self, tree_frame): tree_scrollx.config(command=self.tree.xview) tree_scrolly.config(command=self.tree.yview) - columns_names = ['Name', 'Type', 'Value'] + columns_names = ['Name', 'Value'] self.tree['columns'] = columns_names # Format columns @@ -689,9 +686,9 @@ def on_return(self, event): val = self.tree.item(self.treerow)['values'] val[int(self.treecol[1:])-1] = self.entry.get() if self.entry.get() != '': - self.tree.item(self.treerow, values=tuple([val[0], val[1], self.entry.get()])) - elif val[2] == '': - self.tree.item(self.treerow, values=tuple([val[0], val[1], 'default'])) + self.tree.item(self.treerow, values=tuple([val[0], self.entry.get()])) + elif val[1] == '': + self.tree.item(self.treerow, values=tuple([val[0], 'default'])) else: self.tree.item(self.treerow, values=val) self.entry.destroy() @@ -704,22 +701,22 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): :param parent: string type of parent name """ self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', - values=tuple(['Required settings', '', '']), tags=('type',)) + values=tuple(['Required settings', '']), tags=('type',)) self.r+=1 for arg, val in req_settings.items(): if arg == 'Data': self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val, 'Choose X or Y']), tags=('req',)) + values=tuple([arg, 'Choose X or Y']), tags=('req',)) else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val, '']), tags=('req',)) + values=tuple([arg, val]), tags=('req',)) self.r+=1 self.tree.insert(parent=parent, index='end', iid=parent+'_opt', text='', - values=tuple(['Optional settings', '', '']), tags=('type',)) + values=tuple(['Optional settings', '']), tags=('type',)) self.r+=1 for arg, val in opt_settings.items(): self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, val, 'default']), tags=('opt',)) + values=tuple([arg, val]), tags=('opt',)) self.r+=1 def removewindow(self): @@ -745,15 +742,15 @@ def get_all_children(self, item=""): def settingOptions(self, tag, val): """ Identifies how the data should be stored """ if val[0] == 'Data': - if val[2] == 'Choose X or Y' or len(val[2]) == 0: + if val[1] == 'Choose X or Y' or len(val[1]) == 0: self.updateSettings(tag, val[0], 'X') else: - self.updateSettings(tag, val[0], val[2]) + self.updateSettings(tag, val[0], val[1]) else: - if val[2] == 'default' or len(str(val[2])) == 0: + if val[1] == 'default' or len(str(val[1])) == 0: self.updateSettings(tag, val[0]) else: - self.updateSettings(tag, val[0], val[2]) + self.updateSettings(tag, val[0], val[1]) def updateSettings(self, tag, key, value = None): """ Return the selected settings @@ -763,17 +760,32 @@ def updateSettings(self, tag, key, value = None): tag : str tag for the settings """ + + value = self.str_to_bool(value) if tag == 'req': - if value is not None or self.req_settings[key] != value: + if value is not None or key == 'Data' or self.req_settings[key] != value: self.req_settings[key] = value else: self.req_settings.pop(key, None) elif tag == 'opt': - if value is not None or self.opt_settings[key] != value: + if self.opt_settings[key] != value: self.opt_settings[key] = value else: - self.opt_settings.pop(key, None) - + self.opt_settings.pop(key, None) + + def str_to_bool(self, s): + if type(s) is str: + if s == 'True': + return True + elif s == 'False': + return False + elif s == 'None': + return None + else: + return s + else: + return s + def on_return_entry(self, r): """ Changes focus to the next available entry. When no more, focuses on the finish button. @@ -1096,7 +1108,7 @@ def check_quit(self): self.controller.Plugin.set(True) self.controller._show_frame("MainPage") # TODO: Check if loaded - elif len(self.xml_handler.loaded_modules) == 0: + elif not hasattr(self,"xml_handler") or len(self.xml_handler.loaded_modules) == 0: self.controller._show_frame("MainPage") self.controller.Plugin.set(False) else: From 5f7ef77a23a4d4754aeec0da9b03e3e6aafc2963 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 11 Jul 2023 08:25:40 +0200 Subject: [PATCH 11/48] Add treview for function selection --- src/vai_lab/utils/plugins/pluginCanvas.py | 50 +++++++++++++++-------- 1 file changed, 32 insertions(+), 18 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 64f6de24..fb59cf41 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -511,11 +511,15 @@ def optionsWindow(self): # self.newWindow.geometry("350x400") frame1 = tk.Frame(self.newWindow) + frame11 = tk.Frame(self.newWindow) frame4 = tk.Frame(self.newWindow) # Print settings tk.Label(frame1, - text="Please indicate your desired options for the "+self.plugin[self.m].get()+" plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + text="Please indicate your desired options for the plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + # Print settings + tk.Label(frame11, + text="Please indicate your desired functions for the "+self.plugin[self.m].get()+" plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) style = ttk.Style() style.configure( @@ -528,11 +532,21 @@ def optionsWindow(self): ) style.map('Treeview', background=[('selected', 'grey')]) - frame2 = tk.Frame(self.newWindow, bg='green') + frame12 = tk.Frame(self.newWindow) + self.create_treeView(frame12, ['Name']) + # self.fill_treeview(self.req_settings, self.opt_settings) + frame12.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + + # frame12.grid_rowconfigure(tuple(range(self.r)), weight=1) + # frame12.grid_columnconfigure(tuple(range(2)), weight=1) + + frame2 = tk.Frame(self.newWindow) self.r = 1 - self.create_treeView(frame2) - self.fill_treeview(self.req_settings, self.opt_settings) - frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + self.create_treeView(frame2, ['Name', 'Value']) + self.tree.insert(parent='', index='end', iid='init', text='', values=tuple(['__init__', '']), + tags=('func','init'), open=True) + self.fill_treeview(self.req_settings, self.opt_settings, 'init') + frame2.grid(column=1, row=1, sticky="nswe", pady=10, padx=10) frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) frame2.grid_columnconfigure(tuple(range(2)), weight=1) @@ -560,13 +574,14 @@ def optionsWindow(self): "", lambda event: self.removewindow()) self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) - frame1.grid(column=0, row=0, sticky="ew") - frame4.grid(column=0, row=20, sticky="se") - frame5.grid(column=0, row=2, sticky="ew") - frame6.grid(column=0, row=3) + frame11.grid(column=0, row=0, sticky="ew") + frame1.grid(column=1, row=0, sticky="ew") + frame4.grid(column=1, row=20, sticky="se") + frame5.grid(column=1, row=2, sticky="ew") + frame6.grid(column=1, row=3) self.newWindow.grid_rowconfigure(1, weight=2) - self.newWindow.grid_columnconfigure(0, weight=1) + self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) def getArgs(self, f): """ Get required and optional arguments from function. @@ -595,7 +610,7 @@ def getArgs(self, f): func_opt = {**func_r_opt, **func_opt} return func_req, func_opt - def create_treeView(self, tree_frame): + def create_treeView(self, tree_frame, columns_names): """ Function to create a new tree view in the given frame Parameters @@ -619,11 +634,10 @@ def create_treeView(self, tree_frame): tree_scrollx.config(command=self.tree.xview) tree_scrolly.config(command=self.tree.yview) - columns_names = ['Name', 'Value'] self.tree['columns'] = columns_names # Format columns - self.tree.column("#0", width=20, + self.tree.column("#0", width=40, minwidth=0, stretch=tk.NO) for n, cl in enumerate(columns_names): self.tree.column( @@ -701,22 +715,22 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): :param parent: string type of parent name """ self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', - values=tuple(['Required settings', '']), tags=('type',)) + values=tuple(['Required settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in req_settings.items(): if arg == 'Data': self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, 'Choose X or Y']), tags=('req',)) + values=tuple([arg, 'Choose X or Y']), tags=('req',parent)) else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val]), tags=('req',)) + values=tuple([arg, val]), tags=('req',parent)) self.r+=1 self.tree.insert(parent=parent, index='end', iid=parent+'_opt', text='', - values=tuple(['Optional settings', '']), tags=('type',)) + values=tuple(['Optional settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in opt_settings.items(): self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, val]), tags=('opt',)) + values=tuple([arg, val]), tags=('opt',parent)) self.r+=1 def removewindow(self): From 98631f1c27b0f366e247fff9e9696c1635c92354 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 11 Jul 2023 10:52:32 +0200 Subject: [PATCH 12/48] GUI for function definition --- src/vai_lab/utils/plugins/pluginCanvas.py | 149 ++++++++++++++-------- 1 file changed, 97 insertions(+), 52 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index fb59cf41..7bbba3a5 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -5,7 +5,7 @@ import os import numpy as np import pandas as pd -from inspect import getmembers, isfunction, getfullargspec +from inspect import getmembers, isfunction, ismethod, getfullargspec from typing import Dict, List from PIL import Image, ImageTk @@ -484,17 +484,24 @@ def optionsWindow(self): avail_plugins["_PLUGIN_PACKAGE"], avail_plugins["_PLUGIN_CLASS_NAME"]) # Update required and optional settings for the plugin - self.req_settings = ps.required_settings[module][self.plugin[self.m].get()] - self.opt_settings = ps.optional_settings[module][self.plugin[self.m].get()] - func_req, func_opt = self.getArgs(plugin().model.__init__) + self.req_settings = {'__init__': ps.required_settings[module][self.plugin[self.m].get()]} + self.opt_settings = {'__init__': ps.optional_settings[module][self.plugin[self.m].get()]} + self.model = plugin().model + func_req, func_opt = self.getArgs(self.model.__init__) if func_req is not None: - self.req_settings = {**self.req_settings, **func_req} + self.req_settings['__init__'] = {**self.req_settings['__init__'], **func_req} if func_opt is not None: - self.opt_settings = {**self.opt_settings, **func_opt} + self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **func_opt} - function_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + # Find functions defined for the module + plugin_func_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + # Find available methods for the model + model_func_list = [func[0] for func in getmembers(self.model, ismethod) if func[0][0] != '_'] + # List intersection + func_list = list(set(plugin_func_list) & set(model_func_list))[::-1] + self.funcs_sort = [] - if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): + if (len(self.opt_settings['__init__']) != 0) or (len(self.req_settings['__init__']) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): self.newWindow.destroy() self.newWindow = tk.Toplevel(self.controller) @@ -510,17 +517,21 @@ def optionsWindow(self): 'VAILabsIcon.ico')))) # self.newWindow.geometry("350x400") + frame1 = tk.Frame(self.newWindow) - frame11 = tk.Frame(self.newWindow) + frame2 = tk.Frame(self.newWindow) + frame21 = tk.Frame(self.newWindow) + frame3 = tk.Frame(self.newWindow) frame4 = tk.Frame(self.newWindow) + frame5 = tk.Frame(self.newWindow) + frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) + frameDrop = tk.Frame(frame3, highlightbackground="black", highlightthickness=1) + frameButt = tk.Frame(frame3) # Print settings tk.Label(frame1, text="Please indicate your desired options for the plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - # Print settings - tk.Label(frame11, - text="Please indicate your desired functions for the "+self.plugin[self.m].get()+" plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - + style = ttk.Style() style.configure( "Treeview", background='white', foreground='white', @@ -532,31 +543,29 @@ def optionsWindow(self): ) style.map('Treeview', background=[('selected', 'grey')]) - frame12 = tk.Frame(self.newWindow) - self.create_treeView(frame12, ['Name']) - # self.fill_treeview(self.req_settings, self.opt_settings) - frame12.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) - - # frame12.grid_rowconfigure(tuple(range(self.r)), weight=1) - # frame12.grid_columnconfigure(tuple(range(2)), weight=1) + tk.Label(frame21, + text="Add your desired functions in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + self.func2add = tk.StringVar(frameDrop) + dropDown = tk.ttk.OptionMenu(frameDrop, self.func2add, func_list[0], *func_list) + style.configure("TMenubutton", background="white") + dropDown["menu"].configure(bg="white") + dropDown.grid(row=0,column=0) + + tk.Button(frameButt, text='Add', command=self.addFunc).grid(row=0,column=0) + tk.Button(frameButt, text='Delete', command=self.deleteFunc).grid(row=0,column=1) + tk.Button(frameButt, text='Up', command=lambda: self.moveFunc(-1)).grid(row=0,column=2) + tk.Button(frameButt, text='Down', command=lambda: self.moveFunc(+1)).grid(row=0,column=3) - frame2 = tk.Frame(self.newWindow) self.r = 1 - self.create_treeView(frame2, ['Name', 'Value']) + self.tree = self.create_treeView(frame2, ['Name', 'Value']) self.tree.insert(parent='', index='end', iid='init', text='', values=tuple(['__init__', '']), - tags=('func','init'), open=True) - self.fill_treeview(self.req_settings, self.opt_settings, 'init') - frame2.grid(column=1, row=1, sticky="nswe", pady=10, padx=10) - - frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) - frame2.grid_columnconfigure(tuple(range(2)), weight=1) + tags=('func','init')) + self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], 'init') - frame5 = tk.Frame(self.newWindow) tk.Label(frame5, text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) - current = np.where(self.m == np.array(self.id_mod))[0][0] dataSources = [i for j, i in enumerate(self.module_names) if j not in [1,current]] @@ -574,12 +583,18 @@ def optionsWindow(self): "", lambda event: self.removewindow()) self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) - frame11.grid(column=0, row=0, sticky="ew") - frame1.grid(column=1, row=0, sticky="ew") - frame4.grid(column=1, row=20, sticky="se") - frame5.grid(column=1, row=2, sticky="ew") - frame6.grid(column=1, row=3) - + frameDrop.grid(column=0, row=0) + frameButt.grid(column=1, row=0, sticky="w") + frame1.grid(column=0, row=0, sticky="ew") + frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + frame21.grid(column=0, row=2, sticky="ew") + frame3.grid(column=0, row=3, pady=10, padx=10) + frame4.grid(column=0, row=20, sticky="se") + frame5.grid(column=0, row=4, sticky="ew") + frame6.grid(column=0, row=5) + + frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) + frame2.grid_columnconfigure(tuple(range(2)), weight=1) self.newWindow.grid_rowconfigure(1, weight=2) self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) @@ -608,7 +623,35 @@ def getArgs(self, f): func_opt = {p: func_opt[p] for p in func_opt} if func_r_opt is not None: func_opt = {**func_r_opt, **func_opt} - return func_req, func_opt + return func_req, func_opt + else: + return func_req, func_r_opt + + def addFunc(self): + """ Adds selected function in dropdown menu to the plugin tree """ + func = self.func2add.get() + self.funcs_sort.append(func) + self.tree.insert(parent='', index='end', iid=func, text='', values=tuple([func, '']), + tags=('func',func)) + # TODO: Remove X and y? + self.req_settings[func], self.opt_settings[func] = self.getArgs(getattr(self.model, func)) + self.fill_treeview(self.req_settings[func], self.opt_settings[func], func) + + def deleteFunc(self): + """ Deletes selected function in dropdown menu from the plugin tree """ + func = self.func2add.get() + if func in self.funcs_sort: + self.funcs_sort.remove(func) + del self.req_settings[func] + del self.opt_settings[func] + self.tree.delete(func) + + def moveFunc(self, m): + func = self.func2add.get() + if func in self.funcs_sort and self.tree.index(func)+m > 0: + idx = self.funcs_sort.index(func) + self.funcs_sort.insert(idx+m, self.funcs_sort.pop(idx)) + self.tree.move(func, self.tree.parent(func), self.tree.index(func)+m) def create_treeView(self, tree_frame, columns_names): """ Function to create a new tree view in the given frame @@ -626,36 +669,37 @@ def create_treeView(self, tree_frame, columns_names): tree_scrolly = tk.Scrollbar(tree_frame) tree_scrolly.pack(side=tk.RIGHT, fill=tk.Y) - self.tree = ttk.Treeview(tree_frame, + tree = ttk.Treeview(tree_frame, yscrollcommand=tree_scrolly.set, xscrollcommand=tree_scrollx.set) - self.tree.pack(fill='both', expand=True) + tree.pack(fill='both', expand=True) - tree_scrollx.config(command=self.tree.xview) - tree_scrolly.config(command=self.tree.yview) + tree_scrollx.config(command=tree.xview) + tree_scrolly.config(command=tree.yview) - self.tree['columns'] = columns_names + tree['columns'] = columns_names # Format columns - self.tree.column("#0", width=40, + tree.column("#0", width=40, minwidth=0, stretch=tk.NO) for n, cl in enumerate(columns_names): - self.tree.column( + tree.column( cl, width=int(self.controller.pages_font.measure(str(cl)))+20, minwidth=50, anchor=tk.CENTER) # Headings for cl in columns_names: - self.tree.heading(cl, text=cl, anchor=tk.CENTER) - self.tree.tag_configure('req', foreground='black', + tree.heading(cl, text=cl, anchor=tk.CENTER) + tree.tag_configure('req', foreground='black', background='#9fc5e8') - self.tree.tag_configure('opt', foreground='black', + tree.tag_configure('opt', foreground='black', background='#cfe2f3') - self.tree.tag_configure('type', foreground='black', + tree.tag_configure('type', foreground='black', background='#E8E8E8') - self.tree.tag_configure('func', foreground='black', + tree.tag_configure('func', foreground='black', background='#DFDFDF') # Define double-click on row action - self.tree.bind("", self.OnDoubleClick) + tree.bind("", self.OnDoubleClick) + return tree def OnDoubleClick(self, event): """ Executed when a row of the treeview is double clicked. @@ -741,7 +785,8 @@ def removewindow(self): tag = self.tree.item(child)["tags"][0] if tag in ['req', 'opt']: val = self.tree.item(child)["values"] - self.settingOptions(tag, val) + self.settingOptions(tag, val) + del self.model self.newWindow.destroy() self.newWindow = None self.focus() From 7a4cded1bf79a1a4fd8f77dee9218370f07742dd Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 11 Jul 2023 15:42:01 +0200 Subject: [PATCH 13/48] Changed solve method to fit --- src/vai_lab/Modelling/Modelling_core.py | 2 +- src/vai_lab/_plugin_templates.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/vai_lab/Modelling/Modelling_core.py b/src/vai_lab/Modelling/Modelling_core.py index 81695629..8a4c475e 100644 --- a/src/vai_lab/Modelling/Modelling_core.py +++ b/src/vai_lab/Modelling/Modelling_core.py @@ -29,7 +29,7 @@ def set_options(self, module_config: dict): def launch(self): self._plugin.set_data_in(self._data_in) self._plugin.configure(self._module_config["plugin"]) - self._plugin.solve() + self._plugin.fit() self.output_data = self._data_in.copy() # self.output_data = self._plugin._test(self.output_data) diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 697672c4..18aeaf91 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -221,7 +221,7 @@ class ModellingPluginT(PluginTemplate, ABC): def __init__(self, plugin_globals: dict) -> None: super().__init__(plugin_globals) - def solve(self): + def fit(self): """Sends params to solver, then runs solver""" try: self.model.set_params(**self._config["options"]) From 11b46e5ee10aef98f68061d48f746c77c329b162 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 11 Jul 2023 15:42:40 +0200 Subject: [PATCH 14/48] Save methods in XML --- src/vai_lab/Data/xml_handler.py | 43 +++++- src/vai_lab/utils/plugins/pluginCanvas.py | 170 +++++++++++----------- 2 files changed, 126 insertions(+), 87 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index b3ebeae3..58ba365e 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -453,12 +453,12 @@ def update_plugin_options(self, else: xml_parent = xml_parent_name plugin_elem: ET.Element = xml_parent.find("./plugin") - self._add_plugin_options(plugin_elem, options) + self._add_options(plugin_elem, options) self._parse_XML() if save_changes: self.write_to_XML() - def _add_plugin_options(self, + def _add_options(self, plugin_elem: ET.Element, options ): @@ -476,7 +476,7 @@ def _add_plugin_options(self, new_option.text = "{0} {1}".format( text_lead, str(options[key])) elif isinstance(options[key], (dict)): - self._add_plugin_options(plugin_elem, options[key]) + self._add_options(plugin_elem, options[key]) def append_input_data(self, data_name: str, @@ -516,9 +516,39 @@ def append_input_data(self, else: plugin_elem.set('module', data_dir) + def append_method_to_plugin(self, + method_type: str, + method_options: dict, + xml_parent: Union[ET.Element, str], + overwrite_existing: Union[bool, int] = False + ): + """Appened method as subelement to existing plugin element + + :param method_type: string type of method to be loaded into plugin + :param method_options: dict where keys & values are options & values + :param xml_parent: dict OR str. + If string given, parent elem is found via search, + Otherwise, method appeneded directly + """ + if isinstance(xml_parent, str): + xml_parent = self._get_element_from_name(xml_parent) + + method_elem = xml_parent.find("./method") + + if method_elem is not None and overwrite_existing: + if method_elem.attrib['type'] == method_type: + xml_parent.remove(method_elem) + method_elem = None + + if method_elem is None: + method_elem = ET.SubElement(xml_parent, "method") + method_elem.set('type', method_type) + self._add_options(method_elem, method_options) + def append_plugin_to_module(self, plugin_type: str, plugin_options: dict, + method_list: list, plugin_data: str, xml_parent: Union[ET.Element, str], overwrite_existing: Union[bool, int] = False @@ -546,7 +576,12 @@ def append_plugin_to_module(self, plugin_elem.set('type', plugin_type) if plugin_data is not None and len(plugin_data) > 0: self.append_input_data('X', plugin_data, xml_parent, False) - self._add_plugin_options(plugin_elem, plugin_options) + self._add_options(plugin_elem, plugin_options['__init__']) + for f in method_list: + self.append_method_to_plugin(f, + plugin_options[f], + plugin_elem, + overwrite_existing) def append_pipeline_module(self, module_type: str, diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 7bbba3a5..7484f170 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -302,6 +302,7 @@ def check_updated(self): self.xml_handler.append_plugin_to_module( self.plugin[self.m].get(), {**self.req_settings, **self.opt_settings}, + self.meths_sort, self.plugin_inputData.get(), np.array(self.module_names)[self.m == np.array(self.id_mod)][0], True) @@ -319,8 +320,9 @@ def check_updated(self): self.xml_handler.append_plugin_to_module( 'Output', - {'outdata': self.out_data_xml, - 'outpath': rel_path}, + {'__init__': {'outdata': self.out_data_xml, + 'outpath': rel_path}}, + [], None, np.array(self.module_names)[self.m == np.array(self.id_mod)][0], True) @@ -487,19 +489,20 @@ def optionsWindow(self): self.req_settings = {'__init__': ps.required_settings[module][self.plugin[self.m].get()]} self.opt_settings = {'__init__': ps.optional_settings[module][self.plugin[self.m].get()]} self.model = plugin().model - func_req, func_opt = self.getArgs(self.model.__init__) - if func_req is not None: - self.req_settings['__init__'] = {**self.req_settings['__init__'], **func_req} - if func_opt is not None: - self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **func_opt} + meth_req, meth_opt = self.getArgs(self.model.__init__) + if meth_req is not None: + self.req_settings['__init__'] = {**self.req_settings['__init__'], **meth_req} + if meth_opt is not None: + self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **meth_opt} # Find functions defined for the module - plugin_func_list = [func[0] for func in getmembers(plugin, isfunction) if func[0][0] != '_'] + plugin_meth_list = [meth[0] for meth in getmembers(plugin, isfunction) if meth[0][0] != '_'] # Find available methods for the model - model_func_list = [func[0] for func in getmembers(self.model, ismethod) if func[0][0] != '_'] + model_meth_list = [meth[0] for meth in getmembers(self.model, ismethod) if meth[0][0] != '_'] # List intersection - func_list = list(set(plugin_func_list) & set(model_func_list))[::-1] - self.funcs_sort = [] + # TODO: use only methods from the model + meth_list = list(set(plugin_meth_list) & set(model_meth_list))[::-1] + self.meths_sort = [] if (len(self.opt_settings['__init__']) != 0) or (len(self.req_settings['__init__']) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): @@ -544,24 +547,24 @@ def optionsWindow(self): style.map('Treeview', background=[('selected', 'grey')]) tk.Label(frame21, - text="Add your desired functions in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + text="Add your desired methods in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - self.func2add = tk.StringVar(frameDrop) - dropDown = tk.ttk.OptionMenu(frameDrop, self.func2add, func_list[0], *func_list) + self.meth2add = tk.StringVar(frameDrop) + dropDown = tk.ttk.OptionMenu(frameDrop, self.meth2add, meth_list[0], *meth_list) style.configure("TMenubutton", background="white") dropDown["menu"].configure(bg="white") dropDown.grid(row=0,column=0) - tk.Button(frameButt, text='Add', command=self.addFunc).grid(row=0,column=0) - tk.Button(frameButt, text='Delete', command=self.deleteFunc).grid(row=0,column=1) - tk.Button(frameButt, text='Up', command=lambda: self.moveFunc(-1)).grid(row=0,column=2) - tk.Button(frameButt, text='Down', command=lambda: self.moveFunc(+1)).grid(row=0,column=3) + tk.Button(frameButt, text='Add', command=self.addMeth).grid(row=0,column=0) + tk.Button(frameButt, text='Delete', command=self.deleteMeth).grid(row=0,column=1) + tk.Button(frameButt, text='Up', command=lambda: self.moveMeth(-1)).grid(row=0,column=2) + tk.Button(frameButt, text='Down', command=lambda: self.moveMeth(+1)).grid(row=0,column=3) self.r = 1 self.tree = self.create_treeView(frame2, ['Name', 'Value']) - self.tree.insert(parent='', index='end', iid='init', text='', values=tuple(['__init__', '']), - tags=('func','init')) - self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], 'init') + self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), + tags=('meth','__init__')) + self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], '__init__') tk.Label(frame5, text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) @@ -599,59 +602,59 @@ def optionsWindow(self): self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) def getArgs(self, f): - """ Get required and optional arguments from function. + """ Get required and optional arguments from method. Parameters ---------- - f : function - function to extract arguments from + f : method + method to extract arguments from :returns out: two dictionaries with arguments and default value (if optional) """ - func_args = getfullargspec(f).args - if func_args is not None: - func_args.remove('self') - func_def = getfullargspec(f).defaults - if func_def is None: - func_def = [] - func_req = {p: '' for p in func_args[:(len(func_args)-len(func_def))]} - func_r_opt = {p: func_def[i] for i,p in enumerate(func_args[(len(func_args)-len(func_def)):])} - - func_opt = getfullargspec(f).kwonlydefaults - if func_opt is not None: - func_opt = {p: func_opt[p] for p in func_opt} - if func_r_opt is not None: - func_opt = {**func_r_opt, **func_opt} - return func_req, func_opt + meth_args = getfullargspec(f).args + if meth_args is not None: + meth_args.remove('self') + meth_def = getfullargspec(f).defaults + if meth_def is None: + meth_def = [] + meth_req = {p: '' for p in meth_args[:(len(meth_args)-len(meth_def))]} + meth_r_opt = {p: meth_def[i] for i,p in enumerate(meth_args[(len(meth_args)-len(meth_def)):])} + + meth_opt = getfullargspec(f).kwonlydefaults + if meth_opt is not None: + meth_opt = {p: meth_opt[p] for p in meth_opt} + if meth_r_opt is not None: + meth_opt = {**meth_r_opt, **meth_opt} + return meth_req, meth_opt else: - return func_req, func_r_opt + return meth_req, meth_r_opt - def addFunc(self): - """ Adds selected function in dropdown menu to the plugin tree """ - func = self.func2add.get() - self.funcs_sort.append(func) - self.tree.insert(parent='', index='end', iid=func, text='', values=tuple([func, '']), - tags=('func',func)) + def addMeth(self): + """ Adds selected method in dropdown menu to the plugin tree """ + meth = self.meth2add.get() + self.meths_sort.append(meth) + self.tree.insert(parent='', index='end', iid=meth, text='', values=tuple([meth, '']), + tags=('meth',meth)) # TODO: Remove X and y? - self.req_settings[func], self.opt_settings[func] = self.getArgs(getattr(self.model, func)) - self.fill_treeview(self.req_settings[func], self.opt_settings[func], func) - - def deleteFunc(self): - """ Deletes selected function in dropdown menu from the plugin tree """ - func = self.func2add.get() - if func in self.funcs_sort: - self.funcs_sort.remove(func) - del self.req_settings[func] - del self.opt_settings[func] - self.tree.delete(func) + self.req_settings[meth], self.opt_settings[meth] = self.getArgs(getattr(self.model, meth)) + self.fill_treeview(self.req_settings[meth], self.opt_settings[meth], meth) + + def deleteMeth(self): + """ Deletes selected method in dropdown menu from the plugin tree """ + meth = self.meth2add.get() + if meth in self.meths_sort: + self.meths_sort.remove(meth) + del self.req_settings[meth] + del self.opt_settings[meth] + self.tree.delete(meth) - def moveFunc(self, m): - func = self.func2add.get() - if func in self.funcs_sort and self.tree.index(func)+m > 0: - idx = self.funcs_sort.index(func) - self.funcs_sort.insert(idx+m, self.funcs_sort.pop(idx)) - self.tree.move(func, self.tree.parent(func), self.tree.index(func)+m) + def moveMeth(self, m): + meth = self.meth2add.get() + if meth in self.meths_sort and self.tree.index(meth)+m > 0: + idx = self.meths_sort.index(meth) + self.meths_sort.insert(idx+m, self.meths_sort.pop(idx)) + self.tree.move(meth, self.tree.parent(meth), self.tree.index(meth)+m) def create_treeView(self, tree_frame, columns_names): """ Function to create a new tree view in the given frame @@ -695,7 +698,7 @@ def create_treeView(self, tree_frame, columns_names): background='#cfe2f3') tree.tag_configure('type', foreground='black', background='#E8E8E8') - tree.tag_configure('func', foreground='black', + tree.tag_configure('meth', foreground='black', background='#DFDFDF') # Define double-click on row action tree.bind("", self.OnDoubleClick) @@ -779,13 +782,14 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): def removewindow(self): """ Stores settings options and closes window """ - self.req_settings.pop("Data", None) - children = self.get_all_children() - for child in children: - tag = self.tree.item(child)["tags"][0] - if tag in ['req', 'opt']: - val = self.tree.item(child)["values"] - self.settingOptions(tag, val) + for f in self.tree.get_children(): + self.req_settings[f].pop("Data", None) + for c in self.tree.get_children(f): + for child in self.tree.get_children(c): + tag = self.tree.item(child)["tags"][0] + if tag in ['req', 'opt']: + val = self.tree.item(child)["values"] + self.settingOptions(tag, f, val) del self.model self.newWindow.destroy() self.newWindow = None @@ -798,20 +802,20 @@ def get_all_children(self, item=""): children += self.get_all_children(child) return children - def settingOptions(self, tag, val): + def settingOptions(self, tag, f, val): """ Identifies how the data should be stored """ if val[0] == 'Data': if val[1] == 'Choose X or Y' or len(val[1]) == 0: - self.updateSettings(tag, val[0], 'X') + self.updateSettings(tag, f, val[0], 'X') else: - self.updateSettings(tag, val[0], val[1]) + self.updateSettings(tag, f, val[0], val[1]) else: if val[1] == 'default' or len(str(val[1])) == 0: - self.updateSettings(tag, val[0]) + self.updateSettings(tag, f, val[0]) else: - self.updateSettings(tag, val[0], val[1]) + self.updateSettings(tag, f, val[0], val[1]) - def updateSettings(self, tag, key, value = None): + def updateSettings(self, tag, f, key, value = None): """ Return the selected settings Parameters @@ -822,15 +826,15 @@ def updateSettings(self, tag, key, value = None): value = self.str_to_bool(value) if tag == 'req': - if value is not None or key == 'Data' or self.req_settings[key] != value: - self.req_settings[key] = value + if value is not None or key == 'Data' or self.req_settings[f][key] != value: + self.req_settings[f][key] = value else: - self.req_settings.pop(key, None) + self.req_settings[f].pop(key, None) elif tag == 'opt': - if self.opt_settings[key] != value: - self.opt_settings[key] = value + if self.opt_settings[f][key] != value: + self.opt_settings[f][key] = value else: - self.opt_settings.pop(key, None) + self.opt_settings[f].pop(key, None) def str_to_bool(self, s): if type(s) is str: From c3dac48938ec00fcc68de47704687cdf503a7d7f Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Mon, 17 Jul 2023 11:48:20 +0200 Subject: [PATCH 15/48] Add tag "options" to plugins --- src/vai_lab/Data/xml_handler.py | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 58ba365e..fcddbb15 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -30,6 +30,7 @@ def __init__(self, filename: str = None): "pipeline": "declaration", "relationships": "relationships", "plugin": "plugin", + "method": "method", "coordinates": "list", "Initialiser": "entry_point", "inputdata": "data", @@ -462,21 +463,24 @@ def _add_options(self, plugin_elem: ET.Element, options ): + opt_elem = plugin_elem.find("./options") + if opt_elem is None: + opt_elem = ET.SubElement(plugin_elem, "options") for key in options.keys(): if isinstance(options[key], list): - new_option = ET.SubElement(plugin_elem, key) + new_option = ET.SubElement(opt_elem, key) option_text = ("\n{}".format( "\n".join([*options[key]]))) new_option.text = option_text elif isinstance(options[key], (int, float, str)): - new_option = plugin_elem.find(str("./" + key)) + new_option = opt_elem.find(str("./" + key)) if new_option is None: - new_option = ET.SubElement(plugin_elem, key) + new_option = ET.SubElement(opt_elem, key) text_lead = "\n" if "\n" not in str(options[key]) else "" new_option.text = "{0} {1}".format( text_lead, str(options[key])) elif isinstance(options[key], (dict)): - self._add_options(plugin_elem, options[key]) + self._add_options(opt_elem, options[key]) def append_input_data(self, data_name: str, From 4ecc32e9185d25a1faafe77fbbac0a1f26374544 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 18 Jul 2023 13:16:58 +0200 Subject: [PATCH 16/48] Load options from XML --- src/vai_lab/Data/xml_handler.py | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index fcddbb15..85b8b033 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -31,6 +31,7 @@ def __init__(self, filename: str = None): "relationships": "relationships", "plugin": "plugin", "method": "method", + "options": "options", "coordinates": "list", "Initialiser": "entry_point", "inputdata": "data", @@ -145,18 +146,36 @@ def _load_plugin(self, element: ET.Element, parent: dict) -> None: """ parent["plugin"] = {} parent["plugin"]["plugin_name"] = element.attrib["type"] + parent["plugin"]["methods"] = {"_order" : []} parent["plugin"]["options"] = {} + self._parse_tags(element, parent["plugin"]) + + def _load_method(self, element: ET.Element, parent: dict) -> None: + """Parses tags associated with methods and appends to parent dict + :param elem: xml.etree.ElementTree.Element to be parsed + :param parent: dict or dict fragment parsed tags will be appened to + """ + for key in element.attrib: + parent["methods"]["_order"].append(element.attrib[key]) + parent["methods"][element.attrib[key]] = {'options': {}} + self._parse_tags(element, parent["methods"][element.attrib[key]]) + + def _load_options(self, element: ET.Element, parent: dict) -> None: + """Parses tags associated with options and appends to parent dict + :param elem: xml.etree.ElementTree.Element to be parsed + :param parent: dict or dict fragment parsed tags will be appened to + """ for child in element: if child.text is not None: val = self._parse_text_to_list(child) val = (val[0] if len(val) == 1 else val) - parent["plugin"]["options"][child.tag] = val + parent["options"][child.tag] = val + for key in child.attrib: if key == "val": - parent["plugin"]["options"][child.tag] = child.attrib[key] + parent["options"][child.tag] = child.attrib[key] else: - parent["plugin"]["options"][child.tag] = { - key: child.attrib[key]} + parent["options"][child.tag] = {key: child.attrib[key]} def _load_entry_point(self, element: ET.Element, parent: dict) -> None: """Parses tags associated with initialiser and appends to parent dict From 863731750f11799624b54268dbb2f0673d169ff8 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Wed, 19 Jul 2023 11:20:04 +0200 Subject: [PATCH 17/48] Using methods and options defined in XML --- src/vai_lab/Modelling/Modelling_core.py | 4 ++- src/vai_lab/_plugin_templates.py | 35 +++++++++++++------------ 2 files changed, 21 insertions(+), 18 deletions(-) diff --git a/src/vai_lab/Modelling/Modelling_core.py b/src/vai_lab/Modelling/Modelling_core.py index 8a4c475e..a6486126 100644 --- a/src/vai_lab/Modelling/Modelling_core.py +++ b/src/vai_lab/Modelling/Modelling_core.py @@ -29,7 +29,9 @@ def set_options(self, module_config: dict): def launch(self): self._plugin.set_data_in(self._data_in) self._plugin.configure(self._module_config["plugin"]) - self._plugin.fit() + self._plugin.init() + for method in self._module_config["plugin"]["methods"]["_order"]: + getattr(self._plugin, "{}".format(method))#(self._module_config["plugin"]["methods"][method]["options"]) self.output_data = self._data_in.copy() # self.output_data = self._plugin._test(self.output_data) diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 18aeaf91..c71c5535 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -185,7 +185,7 @@ def _clean_solver_options(self): _cleaned[key] = False return _cleaned - def fit(self): + def fit(self, **args): cleaned_options = self._clean_solver_options() try: self.model.set_params(**cleaned_options) @@ -194,22 +194,22 @@ def fit(self): +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: - self.model.fit(self.X) + self.model.fit(self.X, **args) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def transform(self, data: DataInterface) -> DataInterface: + def transform(self, data: DataInterface, **args) -> DataInterface: try: - data.append_data_column("X", pd.DataFrame(self.model.transform(self.X))) + data.append_data_column("X", pd.DataFrame(self.model.transform(self.X, **args))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise if self.X_tst is not None: try: - data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst))) + data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst, **args))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -220,23 +220,26 @@ def transform(self, data: DataInterface) -> DataInterface: class ModellingPluginT(PluginTemplate, ABC): def __init__(self, plugin_globals: dict) -> None: super().__init__(plugin_globals) - - def fit(self): - """Sends params to solver, then runs solver""" + + def init(self): + """Sends params to model""" try: self.model.set_params(**self._config["options"]) except Exception as exc: print('The plugin encountered an error on the parameters of ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise + + def fit(self, **args): + """Sends params to fit, then runs fit""" try: - self.model.fit(self.X, self.Y) + self.model.fit(self.X, self.Y, **args) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def predict(self, data): + def predict(self, data, **args): """Uses fitted model to predict output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -245,23 +248,21 @@ def predict(self, data): Returns predicted values. """ try: - return self.model.predict(data) + return self.model.predict(data, **args) except Exception as exc: print('The plugin encountered an error when predicting with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def score(self, X, Y, sample_weight): + def score(self, X, Y, **args): """Return the coefficient of determination :param X : array-like of shape (n_samples, n_features) :param Y : array-like of shape (n_samples,) or (n_samples, n_outputs) - :param sample_weight : array-like of shape (n_samples,), default=None - Sample weights. :returns: score : float of ``self.predict(X)`` wrt. `y`. """ try: - return self.model.score(X, Y, sample_weight) + return self.model.score(X, Y, **args) except Exception as exc: print('The plugin encountered an error when calculating the score with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') @@ -271,7 +272,7 @@ class ModellingPluginTClass(ModellingPluginT, ABC): def __init__(self, plugin_globals: dict) -> None: super().__init__(plugin_globals) - def predict_proba(self, data): + def predict_proba(self, data, **args): """Uses fitted model to predict the probability of the output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -280,7 +281,7 @@ def predict_proba(self, data): Returns predicted values. """ try: - return self.model.predict_proba(data) + return self.model.predict_proba(data, **args) except Exception as exc: print('The plugin encountered an error when predicting the probability with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') From b5309650e990c40afdaa2a2cc6ec2b44be11dae3 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Wed, 19 Jul 2023 11:20:36 +0200 Subject: [PATCH 18/48] Fix issue with canvas interaction --- src/vai_lab/Data/xml_handler.py | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 85b8b033..ac73fa6e 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -599,12 +599,13 @@ def append_plugin_to_module(self, plugin_elem.set('type', plugin_type) if plugin_data is not None and len(plugin_data) > 0: self.append_input_data('X', plugin_data, xml_parent, False) - self._add_options(plugin_elem, plugin_options['__init__']) + if '__init__' in plugin_options.keys(): + self._add_options(plugin_elem, plugin_options['__init__']) for f in method_list: self.append_method_to_plugin(f, - plugin_options[f], - plugin_elem, - overwrite_existing) + plugin_options[f], + plugin_elem, + overwrite_existing) def append_pipeline_module(self, module_type: str, @@ -635,6 +636,7 @@ def append_pipeline_module(self, if plugin_type != None: self.append_plugin_to_module(plugin_type, plugin_options, + [], parents[0], new_mod, 0 From 66c932a77ff0a629e818bb35d0347a5ea2a8ebc7 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 20 Jul 2023 19:02:05 +0200 Subject: [PATCH 19/48] Allow having methods without options --- src/vai_lab/Modelling/Modelling_core.py | 8 ++++++-- src/vai_lab/_plugin_templates.py | 26 ++++++++++++------------- 2 files changed, 19 insertions(+), 15 deletions(-) diff --git a/src/vai_lab/Modelling/Modelling_core.py b/src/vai_lab/Modelling/Modelling_core.py index a6486126..1c6b64f3 100644 --- a/src/vai_lab/Modelling/Modelling_core.py +++ b/src/vai_lab/Modelling/Modelling_core.py @@ -31,9 +31,13 @@ def launch(self): self._plugin.configure(self._module_config["plugin"]) self._plugin.init() for method in self._module_config["plugin"]["methods"]["_order"]: - getattr(self._plugin, "{}".format(method))#(self._module_config["plugin"]["methods"][method]["options"]) + if "options" in self._module_config["plugin"]["methods"][method].keys(): + getattr(self._plugin, "{}".format(method))(self._plugin._parse_options_dict(self._module_config["plugin"]["methods"][method]["options"])) + else: + getattr(self._plugin, "{}".format(method))() + self.output_data = self._data_in.copy() # self.output_data = self._plugin._test(self.output_data) def get_result(self): - return self.output_data + return self.output_data \ No newline at end of file diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index c71c5535..b9e0ed0f 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -185,7 +185,7 @@ def _clean_solver_options(self): _cleaned[key] = False return _cleaned - def fit(self, **args): + def fit(self, options={}): cleaned_options = self._clean_solver_options() try: self.model.set_params(**cleaned_options) @@ -194,22 +194,22 @@ def fit(self, **args): +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: - self.model.fit(self.X, **args) + self.model.fit(self.X, **options) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def transform(self, data: DataInterface, **args) -> DataInterface: + def transform(self, data: DataInterface, options={}) -> DataInterface: try: - data.append_data_column("X", pd.DataFrame(self.model.transform(self.X, **args))) + data.append_data_column("X", pd.DataFrame(self.model.transform(self.X, **options))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise if self.X_tst is not None: try: - data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst, **args))) + data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst, **options))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -230,16 +230,16 @@ def init(self): +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def fit(self, **args): + def fit(self, options={}): """Sends params to fit, then runs fit""" try: - self.model.fit(self.X, self.Y, **args) + self.model.fit(self.X, self.Y, **options) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def predict(self, data, **args): + def predict(self, data, options={}): """Uses fitted model to predict output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -248,13 +248,13 @@ def predict(self, data, **args): Returns predicted values. """ try: - return self.model.predict(data, **args) + return self.model.predict(data, **options) except Exception as exc: print('The plugin encountered an error when predicting with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def score(self, X, Y, **args): + def score(self, X, Y, options={}): """Return the coefficient of determination :param X : array-like of shape (n_samples, n_features) :param Y : array-like of shape (n_samples,) or (n_samples, n_outputs) @@ -262,7 +262,7 @@ def score(self, X, Y, **args): :returns: score : float of ``self.predict(X)`` wrt. `y`. """ try: - return self.model.score(X, Y, **args) + return self.model.score(X, Y, **options) except Exception as exc: print('The plugin encountered an error when calculating the score with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') @@ -272,7 +272,7 @@ class ModellingPluginTClass(ModellingPluginT, ABC): def __init__(self, plugin_globals: dict) -> None: super().__init__(plugin_globals) - def predict_proba(self, data, **args): + def predict_proba(self, data, options={}): """Uses fitted model to predict the probability of the output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -281,7 +281,7 @@ def predict_proba(self, data, **args): Returns predicted values. """ try: - return self.model.predict_proba(data, **args) + return self.model.predict_proba(data, **options) except Exception as exc: print('The plugin encountered an error when predicting the probability with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') From c2c454c6efd8025aa490c06c4c745b2b6974a690 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 20 Jul 2023 19:13:44 +0200 Subject: [PATCH 20/48] Indicate method's data --- src/vai_lab/utils/plugins/pluginCanvas.py | 80 ++++++++++++++++------- 1 file changed, 55 insertions(+), 25 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 7484f170..86fa31db 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -715,31 +715,56 @@ def OnDoubleClick(self, event): if len(tags) > 0 and tags[0] in ['opt', 'req']: # get column position info x, y, width, height = self.tree.bbox(self.treerow, self.treecol) - # y-axis offset pady = height // 2 - # pady = 0 - - if hasattr(self, 'entry'): - self.entry.destroy() - - self.entry = tk.Entry(self.tree, justify='center') - - if int(self.treecol[1:]) > 0: - value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1] - value = str(value) if str(value) not in ['default', 'Choose X or Y'] else '' - self.entry.insert(0, value) - # self.entry['selectbackground'] = '#123456' - self.entry['exportselection'] = False - - self.entry.focus_force() - self.entry.bind("", self.on_return) - self.entry.bind("", lambda *ignore: self.entry.destroy()) - - self.entry.place(x=x, + if tags[-1] != 'data': + if hasattr(self, 'entry'): + self.entry.destroy() + self.entry = tk.Entry(self.tree, justify='center') + if int(self.treecol[1:]) > 0: + value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1] + value = str(value) if str(value) not in ['default', 'Choose X or Y'] else '' + self.entry.insert(0, value) + # self.entry['selectbackground'] = '#123456' + self.entry['exportselection'] = False + + self.entry.focus_force() + self.entry.bind("", self.on_return) + self.entry.bind("", lambda *ignore: self.entry.destroy()) + + self.entry.place(x=x, + y=y + pady, + anchor=tk.W, width=width) + else: + value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] + if tags[1]+str(value) not in self.method_inputData.keys(): + self.method_inputData[tags[1]+str(value)] = tk.StringVar(self.tree) + self.method_inputData[tags[1]+str(value)].set(self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1]) + self.method_inputData[tags[1]+str(value)].trace("w", self.on_changeOption) + self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData[tags[1]+str(value)], + self.method_inputData[tags[1]+str(value)].get(), *['X','Y','X_tst','Y_tst']) + bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' + self.dropDown["menu"].configure(bg=bg) + style = ttk.Style() + style.configure("new.TMenubutton", background=bg, highlightbackground="black", highlightthickness=1) + self.dropDown.configure(style="new.TMenubutton") + self.dropDown.place(x=x, y=y + pady, anchor=tk.W, width=width) - + + def on_changeOption(self, *args): + """ Executed when the optionmenu is selected and pressed enter. + Saves the value""" + if hasattr(self, 'dropDown'): + value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] + tags = self.tree.item(self.treerow)["tags"] + val = self.tree.item(self.treerow)['values'] + new_val = self.method_inputData[tags[1]+str(value)].get() + val[int(self.treecol[1:])-1] = new_val + self.tree.item(self.treerow, values=tuple([val[0], new_val])) + self.dropDown.destroy() + self.saved = False + def on_return(self, event): """ Executed when the entry is edited and pressed enter. Saves the edited value""" @@ -761,13 +786,14 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): :param opt_settings: dict type of plugin optional setting options :param parent: string type of parent name """ + self.method_inputData = {} self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', values=tuple(['Required settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in req_settings.items(): - if arg == 'Data': + if arg.lower() in ['x', 'y']: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, 'Choose X or Y']), tags=('req',parent)) + values=tuple([arg, np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0]]), tags=('req',parent,'data')) else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('req',parent)) @@ -776,8 +802,12 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): values=tuple(['Optional settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in opt_settings.items(): - self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, val]), tags=('opt',parent)) + if arg.lower() in ['x', 'y']: + self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', + values=tuple([arg, np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0]]), tags=('opt',parent,'data')) + else: + self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', + values=tuple([arg, val]), tags=('opt',parent)) self.r+=1 def removewindow(self): From 2d4706fb288ab8253800d0714aab7c5b9b54395a Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 21 Jul 2023 11:02:39 +0200 Subject: [PATCH 21/48] Remove 'Data' from DataProcessing modules --- src/vai_lab/DataProcessing/plugins/binarizer.py | 2 +- src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py | 2 +- src/vai_lab/DataProcessing/plugins/labelbinarizer.py | 2 +- src/vai_lab/DataProcessing/plugins/labelencoder.py | 2 +- src/vai_lab/DataProcessing/plugins/maxabsscaler.py | 2 +- src/vai_lab/DataProcessing/plugins/minmaxscaler.py | 2 +- src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py | 2 +- src/vai_lab/DataProcessing/plugins/normalizer.py | 2 +- src/vai_lab/DataProcessing/plugins/onehotencoder.py | 2 +- src/vai_lab/DataProcessing/plugins/ordinalencoder.py | 2 +- src/vai_lab/DataProcessing/plugins/polynomialfeatures.py | 2 +- src/vai_lab/DataProcessing/plugins/quantiletransformer.py | 2 +- src/vai_lab/DataProcessing/plugins/standardscaler.py | 2 +- 13 files changed, 13 insertions(+), 13 deletions(-) diff --git a/src/vai_lab/DataProcessing/plugins/binarizer.py b/src/vai_lab/DataProcessing/plugins/binarizer.py index f364bdea..71e334ed 100644 --- a/src/vai_lab/DataProcessing/plugins/binarizer.py +++ b/src/vai_lab/DataProcessing/plugins/binarizer.py @@ -5,7 +5,7 @@ _PLUGIN_READABLE_NAMES = {"Binarizer":"default","binarizer":"alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"threshold": "float"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py index 66af2f5a..4fc7023e 100644 --- a/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py +++ b/src/vai_lab/DataProcessing/plugins/kbinsdiscretizer.py @@ -5,7 +5,7 @@ _PLUGIN_READABLE_NAMES = {"KBinsDiscretizer":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst","Y_tst"} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/labelbinarizer.py b/src/vai_lab/DataProcessing/plugins/labelbinarizer.py index ef6bd839..811e1774 100644 --- a/src/vai_lab/DataProcessing/plugins/labelbinarizer.py +++ b/src/vai_lab/DataProcessing/plugins/labelbinarizer.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"LabelBinarizer":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"neg_label": "int", "pos_label": "int"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/labelencoder.py b/src/vai_lab/DataProcessing/plugins/labelencoder.py index c735b262..a7166062 100644 --- a/src/vai_lab/DataProcessing/plugins/labelencoder.py +++ b/src/vai_lab/DataProcessing/plugins/labelencoder.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"LabelEncoder":"default","LE":"alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/maxabsscaler.py b/src/vai_lab/DataProcessing/plugins/maxabsscaler.py index 7239c582..58def4b4 100644 --- a/src/vai_lab/DataProcessing/plugins/maxabsscaler.py +++ b/src/vai_lab/DataProcessing/plugins/maxabsscaler.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"MaxAbsScaler":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py index 9a241fcc..c7e2d170 100644 --- a/src/vai_lab/DataProcessing/plugins/minmaxscaler.py +++ b/src/vai_lab/DataProcessing/plugins/minmaxscaler.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"MinMaxScaler":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py b/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py index db25d2e2..4be65da8 100644 --- a/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py +++ b/src/vai_lab/DataProcessing/plugins/multilabelbinarizer.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"MultiLabelBinarizer":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"classes": "array-like"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/normalizer.py b/src/vai_lab/DataProcessing/plugins/normalizer.py index d2c9ca57..6c0cbb8c 100644 --- a/src/vai_lab/DataProcessing/plugins/normalizer.py +++ b/src/vai_lab/DataProcessing/plugins/normalizer.py @@ -7,7 +7,7 @@ _PLUGIN_READABLE_NAMES = {"Normalizer": "default", "Norm": "alias", "normalizer": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/onehotencoder.py b/src/vai_lab/DataProcessing/plugins/onehotencoder.py index b20b97c8..73d8eb2c 100644 --- a/src/vai_lab/DataProcessing/plugins/onehotencoder.py +++ b/src/vai_lab/DataProcessing/plugins/onehotencoder.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"OneHotEncoder":"default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"categories": "array-like"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X","Y","X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/ordinalencoder.py b/src/vai_lab/DataProcessing/plugins/ordinalencoder.py index 037bcf43..a9095ccf 100644 --- a/src/vai_lab/DataProcessing/plugins/ordinalencoder.py +++ b/src/vai_lab/DataProcessing/plugins/ordinalencoder.py @@ -4,7 +4,7 @@ _PLUGIN_READABLE_NAMES = {"OrdinalEncoder": "default"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"categories": "array-like"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py index a0d6926b..c40c5ea7 100644 --- a/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py +++ b/src/vai_lab/DataProcessing/plugins/polynomialfeatures.py @@ -6,7 +6,7 @@ "polyfeat": "alias", "polynomialfeatures": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "Other"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"interaction_only": "bool", "include_bias": "bool"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/quantiletransformer.py b/src/vai_lab/DataProcessing/plugins/quantiletransformer.py index c10ecd76..91085593 100644 --- a/src/vai_lab/DataProcessing/plugins/quantiletransformer.py +++ b/src/vai_lab/DataProcessing/plugins/quantiletransformer.py @@ -5,7 +5,7 @@ _PLUGIN_READABLE_NAMES = { "QuantileTransformer": "default", "Quantile": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "encoder"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"n_quantiles": "int"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore diff --git a/src/vai_lab/DataProcessing/plugins/standardscaler.py b/src/vai_lab/DataProcessing/plugins/standardscaler.py index c60ea60d..cda34552 100644 --- a/src/vai_lab/DataProcessing/plugins/standardscaler.py +++ b/src/vai_lab/DataProcessing/plugins/standardscaler.py @@ -5,7 +5,7 @@ _PLUGIN_READABLE_NAMES = { "StandardScaler": "default", "standardscaler": "alias"} # type:ignore _PLUGIN_MODULE_OPTIONS = {"Type": "scaler"} # type:ignore -_PLUGIN_REQUIRED_SETTINGS = {"Data": "str"} # type:ignore +_PLUGIN_REQUIRED_SETTINGS = {} # type:ignore _PLUGIN_OPTIONAL_SETTINGS = {"with_mean": "bool"} # type:ignore _PLUGIN_REQUIRED_DATA = {} # type:ignore _PLUGIN_OPTIONAL_DATA = {"X", "Y", "X_tst", 'Y_tst'} # type:ignore From b3d0f98d7d8e6d343b0765d9f3af162ebbf60aac Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 21 Jul 2023 11:05:52 +0200 Subject: [PATCH 22/48] Store options - Fixed issue storing options from methods - Removed 'Data' for DataProcessing modules - Make correct comparison to determine if options are updated - Correctly store method's data input. Stores name of variable. --- src/vai_lab/utils/plugins/pluginCanvas.py | 71 +++++++++++++++-------- 1 file changed, 47 insertions(+), 24 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 86fa31db..88ab8d15 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -6,6 +6,7 @@ import numpy as np import pandas as pd from inspect import getmembers, isfunction, ismethod, getfullargspec +from functools import reduce from typing import Dict, List from PIL import Image, ImageTk @@ -301,7 +302,7 @@ def check_updated(self): self.id_done.append(self.m) self.xml_handler.append_plugin_to_module( self.plugin[self.m].get(), - {**self.req_settings, **self.opt_settings}, + self.merge_dicts(self.req_settings, self.opt_settings), self.meths_sort, self.plugin_inputData.get(), np.array(self.module_names)[self.m == np.array(self.id_mod)][0], @@ -327,6 +328,21 @@ def check_updated(self): np.array(self.module_names)[self.m == np.array(self.id_mod)][0], True) + def merge_dicts(self, a, b, path = None): + "merges b into a" + if path is None: path = [] + for key in b: + if key in a: + if isinstance(a[key], dict) and isinstance(b[key], dict): + self.merge_dicts(a[key], b[key], path + [str(key)]) + elif a[key] == b[key]: + pass # same leaf value + else: + raise Exception('Conflict at %s' % '.'.join(path + [str(key)])) + else: + a[key] = b[key] + return a + def display_buttons(self): """ Updates the displayed radiobuttons and the description windows. It loads the information corresponding to the selected module (self.m) @@ -736,13 +752,8 @@ def OnDoubleClick(self, event): y=y + pady, anchor=tk.W, width=width) else: - value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] - if tags[1]+str(value) not in self.method_inputData.keys(): - self.method_inputData[tags[1]+str(value)] = tk.StringVar(self.tree) - self.method_inputData[tags[1]+str(value)].set(self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1]) - self.method_inputData[tags[1]+str(value)].trace("w", self.on_changeOption) - self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData[tags[1]+str(value)], - self.method_inputData[tags[1]+str(value)].get(), *['X','Y','X_tst','Y_tst']) + self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags)], + self.method_inputData['_'.join(tags)].get(), *['X','Y','X_tst','Y_tst']) bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' self.dropDown["menu"].configure(bg=bg) style = ttk.Style() @@ -759,7 +770,7 @@ def on_changeOption(self, *args): value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] tags = self.tree.item(self.treerow)["tags"] val = self.tree.item(self.treerow)['values'] - new_val = self.method_inputData[tags[1]+str(value)].get() + new_val = self.method_inputData['_'.join(tags)].get() val[int(self.treecol[1:])-1] = new_val self.tree.item(self.treerow, values=tuple([val[0], new_val])) self.dropDown.destroy() @@ -792,8 +803,12 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.r+=1 for arg, val in req_settings.items(): if arg.lower() in ['x', 'y']: + value = np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0] self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0]]), tags=('req',parent,'data')) + values=tuple([arg, value]), tags=('req',parent,arg,'data')) + self.method_inputData['req_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) + self.method_inputData['req_'+parent+'_'+str(arg)].set(value) + self.method_inputData['req_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('req',parent)) @@ -803,8 +818,12 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.r+=1 for arg, val in opt_settings.items(): if arg.lower() in ['x', 'y']: + value = np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0] self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0]]), tags=('opt',parent,'data')) + values=tuple([arg, value]), tags=('opt',parent,arg,'data')) + self.method_inputData['opt_'+parent+'_'+str(value)] = tk.StringVar(self.tree) + self.method_inputData['opt_'+parent+'_'+str(value)].set(arg) + self.method_inputData['opt_'+parent+'_'+str(value)].trace("w", self.on_changeOption) else: self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('opt',parent)) @@ -812,8 +831,10 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): def removewindow(self): """ Stores settings options and closes window """ + for data in self.method_inputData.keys(): + tags = data.split('_') + self.updateSettings(tags[0], tags[1], tags[2], self.method_inputData[data].get()) for f in self.tree.get_children(): - self.req_settings[f].pop("Data", None) for c in self.tree.get_children(f): for child in self.tree.get_children(c): tag = self.tree.item(child)["tags"][0] @@ -834,16 +855,10 @@ def get_all_children(self, item=""): def settingOptions(self, tag, f, val): """ Identifies how the data should be stored """ - if val[0] == 'Data': - if val[1] == 'Choose X or Y' or len(val[1]) == 0: - self.updateSettings(tag, f, val[0], 'X') - else: - self.updateSettings(tag, f, val[0], val[1]) + if val[1] == 'default' or len(str(val[1])) == 0: + self.updateSettings(tag, f, val[0]) else: - if val[1] == 'default' or len(str(val[1])) == 0: - self.updateSettings(tag, f, val[0]) - else: - self.updateSettings(tag, f, val[0], val[1]) + self.updateSettings(tag, f, val[0], val[1]) def updateSettings(self, tag, f, key, value = None): """ Return the selected settings @@ -856,15 +871,23 @@ def updateSettings(self, tag, f, key, value = None): value = self.str_to_bool(value) if tag == 'req': - if value is not None or key == 'Data' or self.req_settings[f][key] != value: + if value is not None or self.isClose(self.req_settings[f][key], value): self.req_settings[f][key] = value else: self.req_settings[f].pop(key, None) elif tag == 'opt': - if self.opt_settings[f][key] != value: + if self.isClose(self.opt_settings[f][key], value): self.opt_settings[f][key] = value else: - self.opt_settings[f].pop(key, None) + self.opt_settings[f].pop(key, None) + + def isClose(self, a, b, rel_tol=1e-09, abs_tol=0.0): + a = self.xml_handler._str_to_num(a) if isinstance(a, (str)) else a + b = self.xml_handler._str_to_num(b) if isinstance(b, (str)) else b + if isinstance(a, (int, float)) and isinstance(b, (int, float)): + return abs(a-b) > max(rel_tol * max(abs(a), abs(b)), abs_tol) + else: + return a != b def str_to_bool(self, s): if type(s) is str: From d0d3e1a841f96542c66f726ac41fbb1b53d9a2c0 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 21 Jul 2023 11:24:41 +0200 Subject: [PATCH 23/48] Reset XML if reset canvas --- src/vai_lab/utils/plugins/MainPage.py | 1 - src/vai_lab/utils/plugins/aidCanvas.py | 8 ++++---- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/src/vai_lab/utils/plugins/MainPage.py b/src/vai_lab/utils/plugins/MainPage.py index e9b5dabe..db905590 100644 --- a/src/vai_lab/utils/plugins/MainPage.py +++ b/src/vai_lab/utils/plugins/MainPage.py @@ -5,7 +5,6 @@ from tkinter.filedialog import askopenfilename, askdirectory import pandas as pd -from vai_lab.Data.xml_handler import XML_handler from vai_lab.utils.plugins.dataLoader import dataLoader from vai_lab._import_helper import get_lib_parent_dir, rel_to_abs diff --git a/src/vai_lab/utils/plugins/aidCanvas.py b/src/vai_lab/utils/plugins/aidCanvas.py index 04171f0d..a0efe4a2 100644 --- a/src/vai_lab/utils/plugins/aidCanvas.py +++ b/src/vai_lab/utils/plugins/aidCanvas.py @@ -762,10 +762,9 @@ def upload(self): if filename is not None and len(filename) > 0: self.reset() - s = XML_handler() - s.load_XML(filename) + self.controller.xml_handler.load_XML(filename) # s._print_pretty(s.loaded_modules) - modules = s.loaded_modules + modules = self.controller.xml_handler.loaded_modules modout = modules['Output'] del modules['Initialiser'], modules['Output'] # They are generated when resetting disp_mod = ['Initialiser', 'Output'] @@ -897,13 +896,14 @@ def reset(self): if hasattr(self, 'entry2'): self.entry2.destroy() + self.controller.xml_handler.new_config_file() + self.canvas_startxy = [] self.out_data = pd.DataFrame() self.connections = {} self.modules = 0 self.module_list = [] self.module_names = [] - self.add_module('Initialiser', self.width/2, self.h, ini=True) self.add_module('Output', self.width/2, self.height - self.h, out=True) From f38c37bcbb5f062d42b1fb9cf4b377095773a24d Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Mon, 24 Jul 2023 12:04:21 +0200 Subject: [PATCH 24/48] Change default values for input data list --- src/vai_lab/utils/plugins/pluginCanvas.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 88ab8d15..b8390ac6 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -33,7 +33,6 @@ def __init__(self, parent, controller, config: dict): self.bg = parent['bg'] self.parent = parent self.controller = controller - self.s = XML_handler() self.m: int self.w, self.h = 100, 50 self.cr = 4 @@ -752,8 +751,10 @@ def OnDoubleClick(self, event): y=y + pady, anchor=tk.W, width=width) else: + data_list = [self.method_inputData['_'.join(tags)].get()] + list( + set(['X','Y','X_tst','Y_tst']) - set([self.method_inputData['_'.join(tags)].get()])) self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags)], - self.method_inputData['_'.join(tags)].get(), *['X','Y','X_tst','Y_tst']) + self.method_inputData['_'.join(tags)].get(), *data_list) bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' self.dropDown["menu"].configure(bg=bg) style = ttk.Style() @@ -822,7 +823,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', values=tuple([arg, value]), tags=('opt',parent,arg,'data')) self.method_inputData['opt_'+parent+'_'+str(value)] = tk.StringVar(self.tree) - self.method_inputData['opt_'+parent+'_'+str(value)].set(arg) + self.method_inputData['opt_'+parent+'_'+str(value)].set(val) self.method_inputData['opt_'+parent+'_'+str(value)].trace("w", self.on_changeOption) else: self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', From dcf19c1cee6465f88c67059f00ee87cd62fd4ccd Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Wed, 26 Jul 2023 12:54:33 +0200 Subject: [PATCH 25/48] Load data before XML --- src/vai_lab/Data/xml_handler.py | 38 ++++++++++++++++++++++++-- src/vai_lab/utils/plugins/MainPage.py | 17 ++++-------- src/vai_lab/utils/plugins/aidCanvas.py | 2 -- 3 files changed, 42 insertions(+), 15 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index ac73fa6e..a9398c78 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -93,9 +93,43 @@ def load_XML(self, filename: str) -> None: """ if filename != None: self.set_filename(filename) - self.tree = ET.parse(self.filename) + if hasattr(self, 'tree'): + prev_tree = self.tree + self.tree = self._combine_XML(ET.parse(self.filename).getroot(), prev_tree.getroot()) + else: + self.tree = ET.parse(self.filename) self._parse_XML() - + + def _combine_XML(self, tree1, tree2): + """ + This function recursively updates either the text or the children + of an element if another element is found in `tree1`, or adds it + from `tree2` if not found. + """ + # Create a mapping from tag name to element, as that's what we are fltering with + mapping = {el.tag: el for el in tree1} + for el in tree2: + if len(el) == 0: + # Not nested + try: + # Update the text + mapping[el.tag].text = el.text + except KeyError: + # An element with this name is not in the mapping + mapping[el.tag] = el + # Add it + tree1.append(el) + else: + try: + # Recursively process the element, and update it in the same way + self._combine_XML(mapping[el.tag], el) + except KeyError: + # Not in the mapping + mapping[el.tag] = el + # Just add it + tree1.append(el) + return ET.ElementTree(tree1) + def _parse_XML(self) -> None: self.root = self.tree.getroot() self._parse_tags(self.root, self.loaded_modules) diff --git a/src/vai_lab/utils/plugins/MainPage.py b/src/vai_lab/utils/plugins/MainPage.py index db905590..ad30fc60 100644 --- a/src/vai_lab/utils/plugins/MainPage.py +++ b/src/vai_lab/utils/plugins/MainPage.py @@ -216,6 +216,9 @@ def upload_xml(self): ('All Files', '*.*')]) if filename is not None and len(filename) > 0: self.controller._append_to_output("xml_filename", filename) + self.controller.xml_handler.filename = filename + self.controller.xml_handler.load_XML(filename) + self.controller.xml_handler.write_to_XML() self.controller.XML.set(True) def upload_data_file(self): @@ -298,17 +301,6 @@ def start_dataloader(self): """ Reads all the selected files, loads the data and passes it to dataLoader. """ - - # s = XML_handler() - # s.new_config_file(self.save_path.name) - # s.filename = self.save_path.name - - # self.s = XML_handler() - # self.s.new_config_file() - - # self.s._print_xml_config() - # self.s.load_XML(self.controller.output["xml_filename"]) - data = {} isVar = [0] * len(self.var) if len(self.label_list[0].cget("text")) > 0: @@ -329,6 +321,9 @@ def start_dataloader(self): for i in data[variable].to_numpy().flatten()]) if not any(isVar[1::2]): self.controller.output_type = 'unsupervised' + if hasattr(self.controller.xml_handler, 'filename'): + self.controller.xml_handler._parse_XML() + self.controller.xml_handler.write_to_XML() self.newWindow.destroy() dataLoader(self.controller, data) else: diff --git a/src/vai_lab/utils/plugins/aidCanvas.py b/src/vai_lab/utils/plugins/aidCanvas.py index a0efe4a2..8e4d03a9 100644 --- a/src/vai_lab/utils/plugins/aidCanvas.py +++ b/src/vai_lab/utils/plugins/aidCanvas.py @@ -1,4 +1,3 @@ -from vai_lab.Data.xml_handler import XML_handler from vai_lab._types import DictT import os @@ -763,7 +762,6 @@ def upload(self): self.reset() self.controller.xml_handler.load_XML(filename) - # s._print_pretty(s.loaded_modules) modules = self.controller.xml_handler.loaded_modules modout = modules['Output'] del modules['Initialiser'], modules['Output'] # They are generated when resetting From d376bcb1e0272129370ff6fac202559068d78c6f Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 27 Jul 2023 12:08:06 +0200 Subject: [PATCH 26/48] Correctly update settings dictionary --- src/vai_lab/utils/plugins/pluginCanvas.py | 63 +++++++++++++++-------- 1 file changed, 42 insertions(+), 21 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index b8390ac6..7f7d519b 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -751,10 +751,11 @@ def OnDoubleClick(self, event): y=y + pady, anchor=tk.W, width=width) else: - data_list = [self.method_inputData['_'.join(tags)].get()] + list( - set(['X','Y','X_tst','Y_tst']) - set([self.method_inputData['_'.join(tags)].get()])) - self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags)], - self.method_inputData['_'.join(tags)].get(), *data_list) + data_list = self.default_inputData['_'.join(tags[:-1])] + list( + set(['X','Y','X_tst','Y_tst']) - set([self.method_inputData['_'.join(tags[:-1])].get()]) + - set(self.default_inputData['_'.join(tags[:-1])])) + self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags[:-1])], + self.method_inputData['_'.join(tags[:-1])].get(), *data_list) bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' self.dropDown["menu"].configure(bg=bg) style = ttk.Style() @@ -771,7 +772,7 @@ def on_changeOption(self, *args): value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] tags = self.tree.item(self.treerow)["tags"] val = self.tree.item(self.treerow)['values'] - new_val = self.method_inputData['_'.join(tags)].get() + new_val = self.method_inputData['_'.join(tags[:-1])].get() val[int(self.treecol[1:])-1] = new_val self.tree.item(self.treerow, values=tuple([val[0], new_val])) self.dropDown.destroy() @@ -799,6 +800,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): :param parent: string type of parent name """ self.method_inputData = {} + self.default_inputData = {} self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', values=tuple(['Required settings', '']), tags=('type',parent), open=True) self.r+=1 @@ -810,6 +812,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.method_inputData['req_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) self.method_inputData['req_'+parent+'_'+str(arg)].set(value) self.method_inputData['req_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) + self.default_inputData['req_'+parent+'_'+str(arg)] = [value] else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('req',parent)) @@ -819,12 +822,12 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.r+=1 for arg, val in opt_settings.items(): if arg.lower() in ['x', 'y']: - value = np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0] self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, value]), tags=('opt',parent,arg,'data')) - self.method_inputData['opt_'+parent+'_'+str(value)] = tk.StringVar(self.tree) - self.method_inputData['opt_'+parent+'_'+str(value)].set(val) - self.method_inputData['opt_'+parent+'_'+str(value)].trace("w", self.on_changeOption) + values=tuple([arg, val]), tags=('opt',parent,arg,'data')) + self.method_inputData['opt_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) + self.method_inputData['opt_'+parent+'_'+str(arg)].set(val) + self.method_inputData['opt_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) + self.default_inputData['opt_'+parent+'_'+str(arg)] = [str(val)] else: self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('opt',parent)) @@ -832,16 +835,27 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): def removewindow(self): """ Stores settings options and closes window """ - for data in self.method_inputData.keys(): - tags = data.split('_') - self.updateSettings(tags[0], tags[1], tags[2], self.method_inputData[data].get()) + # Updates the tree with any unclosed dropDown menu + if hasattr(self, 'dropDown'): + for data in self.method_inputData.keys(): + tags = data.split('_') + el = self.get_element_from_tags(*tags) + val = self.tree.item(el)['values'] + new_val = self.method_inputData[data].get() + val[int(self.treecol[1:])-1] = new_val + self.tree.item(el, values=tuple([val[0], new_val])) + self.dropDown.destroy() + # Updates the modified options and removes the ones that are not for f in self.tree.get_children(): for c in self.tree.get_children(f): for child in self.tree.get_children(c): - tag = self.tree.item(child)["tags"][0] - if tag in ['req', 'opt']: - val = self.tree.item(child)["values"] - self.settingOptions(tag, f, val) + tags = self.tree.item(child)["tags"] + if tags[0] in ['req', 'opt']: + if tags[-1] == 'data': + self.updateSettings(tags[0], tags[1], tags[2], self.method_inputData['_'.join(tags[:-1])].get()) + else: + val = self.tree.item(child)["values"] + self.settingOptions(tags[0], f, val) del self.model self.newWindow.destroy() self.newWindow = None @@ -854,6 +868,13 @@ def get_all_children(self, item=""): children += self.get_all_children(child) return children + def get_element_from_tags(self, *args): + """ Finds item in tree with specified tags """ + el = set(self.tree.tag_has(args[0])) + for arg in args[1:]: + el = set.intersection(el, set(self.tree.tag_has(arg))) + return list(el)[0] + def settingOptions(self, tag, f, val): """ Identifies how the data should be stored """ if val[1] == 'default' or len(str(val[1])) == 0: @@ -872,17 +893,17 @@ def updateSettings(self, tag, f, key, value = None): value = self.str_to_bool(value) if tag == 'req': - if value is not None or self.isClose(self.req_settings[f][key], value): + if value is not None or self.isNotClose(self.req_settings[f][key], value): self.req_settings[f][key] = value else: self.req_settings[f].pop(key, None) elif tag == 'opt': - if self.isClose(self.opt_settings[f][key], value): + if self.isNotClose(self.opt_settings[f][key], value): self.opt_settings[f][key] = value else: - self.opt_settings[f].pop(key, None) + self.opt_settings[f].pop(key, None) - def isClose(self, a, b, rel_tol=1e-09, abs_tol=0.0): + def isNotClose(self, a, b, rel_tol=1e-09, abs_tol=0.0): a = self.xml_handler._str_to_num(a) if isinstance(a, (str)) else a b = self.xml_handler._str_to_num(b) if isinstance(b, (str)) else b if isinstance(a, (int, float)) and isinstance(b, (int, float)): From 397e3acb689f135f2645de3c8dc7856841d30ce7 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 27 Jul 2023 12:21:25 +0200 Subject: [PATCH 27/48] Method-wise data --- .../DataProcessing/DataProcessing_core.py | 12 +- src/vai_lab/Modelling/Modelling_core.py | 2 +- src/vai_lab/_plugin_templates.py | 104 ++++++++++++------ 3 files changed, 82 insertions(+), 36 deletions(-) diff --git a/src/vai_lab/DataProcessing/DataProcessing_core.py b/src/vai_lab/DataProcessing/DataProcessing_core.py index 666e2a2d..ba3716d6 100644 --- a/src/vai_lab/DataProcessing/DataProcessing_core.py +++ b/src/vai_lab/DataProcessing/DataProcessing_core.py @@ -31,8 +31,14 @@ def set_options(self, module_config: dict) -> None: def launch(self) -> None: self._plugin.set_data_in(self._data_in) self._plugin.configure(self._module_config["plugin"]) - self._plugin.fit() - self.output_data = self._plugin.transform(self._data_in) + self._plugin.init() + for method in self._module_config["plugin"]["methods"]["_order"]: + if "options" in self._module_config["plugin"]["methods"][method].keys(): + getattr(self._plugin, "{}".format(method))(self._plugin._parse_options_dict(self._module_config["plugin"]["methods"][method]["options"])) + else: + getattr(self._plugin, "{}".format(method))() + + self.output_data = self._data_in.copy() def get_result(self) -> DataInterface: - return self.output_data + return self.output_data \ No newline at end of file diff --git a/src/vai_lab/Modelling/Modelling_core.py b/src/vai_lab/Modelling/Modelling_core.py index 1c6b64f3..fa19e436 100644 --- a/src/vai_lab/Modelling/Modelling_core.py +++ b/src/vai_lab/Modelling/Modelling_core.py @@ -37,7 +37,7 @@ def launch(self): getattr(self._plugin, "{}".format(method))() self.output_data = self._data_in.copy() - # self.output_data = self._plugin._test(self.output_data) + self.output_data = self._plugin._test(self.output_data) def get_result(self): return self.output_data \ No newline at end of file diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index b9e0ed0f..5c0b0d3f 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -79,8 +79,7 @@ def _parse_config(self): self.X = np.array(self._get_data_if_exist(self._data_in, "X")) self.Y = np.array(self._get_data_if_exist(self._data_in, "Y")).ravel() self.X_tst = self._get_data_if_exist(self._data_in, "X_test") - self.Y_tst = np.array(self._get_data_if_exist( - self._data_in, "Y_test")).ravel() + self.Y_tst = np.array(self._get_data_if_exist(self._data_in, "Y_test")).ravel() self._clean_options() def _get_data_if_exist(self, data_dict: dict, key: str, default=None): @@ -113,12 +112,20 @@ def _parse_options_dict(self,options_dict:Dict): val = int(val) cleaned_opts.append(val) options_dict[key] = cleaned_opts - elif type(val) == str and val.lower() in ('y', 'yes', 't', 'true', 'on'): + elif type(val) == str and val.lower() in ('yes', 'true'): options_dict[key] = True - elif type(val) == str and val.lower() in ('n', 'no', 'f', 'false', 'off'): + elif type(val) == str and val.lower() in ('no', 'false'): options_dict[key] = False elif type(val) == str and val.lower() in ('none'): options_dict[key] = None + elif val == 'X': + options_dict[key] = self.X + elif val == 'Y': + options_dict[key] = self.Y + elif val == 'X': + options_dict[key] = self.X_ts + elif val == 'Y_tst': + options_dict[key] = self.Y_tst return options_dict def _clean_options(self): @@ -134,26 +141,26 @@ def _test(self, data: DataInterface) -> DataInterface: """ if self._PLUGIN_MODULE_OPTIONS['Type'] == 'classification': print('Training accuracy: %.2f%%' % - (self.score(self.X, self.Y)*100)) # type: ignore + (self.score([self.X, self.Y])*100)) # type: ignore if self.Y_tst is not None: print('Test accuracy: %.2f%%' % - (self.score(self.X_tst, self.Y_tst)*100)) + (self.score([self.X_tst, self.Y_tst])*100)) if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict(self.X_tst)) + data.append_data_column("Y_pred", self.predict([self.X_tst])) return data elif self._PLUGIN_MODULE_OPTIONS['Type'] == 'regression': print('Training R2 score: %.3f' % - (self.score(self.X, self.Y))) # type: ignore + (self.score([self.X, self.Y]))) # type: ignore if self.Y_tst is not None: - print('Training R2 score: %.3f' % - (self.score(self.X_tst, self.Y_tst))) + print('Test R2 score: %.3f' % + (self.score([self.X_tst, self.Y_tst]))) if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict(self.X_tst)) + data.append_data_column("Y_pred", self.predict([self.X_tst])) return data elif self._PLUGIN_MODULE_OPTIONS['Type'] == 'clustering': print('Clustering completed') if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict(self.X_tst)) + data.append_data_column("Y_pred", self.predict([self.X_tst])) return data else: return data @@ -183,18 +190,40 @@ def _clean_solver_options(self): _cleaned[key] = True elif val == 'False': _cleaned[key] = False + elif val == 'X': + _cleaned[key] = self.X + elif val == 'Y': + _cleaned[key] = self.Y + elif val == 'X': + _cleaned[key] = self.X_tst + elif val == 'Y_tst': + _cleaned[key] = self.Y_tst return _cleaned - + + def init(self): + """Sends params to model""" + try: + self.model.set_params(**self._config["options"]) + except Exception as exc: + print('The plugin encountered an error on the parameters of ' + +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') + raise + def fit(self, options={}): - cleaned_options = self._clean_solver_options() try: - self.model.set_params(**cleaned_options) + if type(self._clean_solver_options()) == list: + self.model.set_params(*self._clean_solver_options()) + else: + self.model.set_params(**self._clean_solver_options()) except Exception as exc: print('The plugin encountered an error on the parameters of ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise try: - self.model.fit(self.X, **options) + if type(options) == list: + return self.model.fit(*options) + else: + return self.model.fit(**options) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') @@ -202,18 +231,14 @@ def fit(self, options={}): def transform(self, data: DataInterface, options={}) -> DataInterface: try: - data.append_data_column("X", pd.DataFrame(self.model.transform(self.X, **options))) + if type(options) == list: + data.append_data_column("X", pd.DataFrame(self.model.transform(*options))) + else: + data.append_data_column("X", pd.DataFrame(self.model.transform(**options))) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - if self.X_tst is not None: - try: - data.append_data_column("X_test", pd.DataFrame(self.model.transform(self.X_tst, **options))) - except Exception as exc: - print('The plugin encountered an error when transforming the data with ' - +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') - raise return data @@ -233,13 +258,16 @@ def init(self): def fit(self, options={}): """Sends params to fit, then runs fit""" try: - self.model.fit(self.X, self.Y, **options) + if type(options) == list: + return self.model.fit(*options) + else: + return self.model.fit(**options) except Exception as exc: print('The plugin encountered an error when fitting ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def predict(self, data, options={}): + def predict(self, options={}): """Uses fitted model to predict output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -248,13 +276,16 @@ def predict(self, data, options={}): Returns predicted values. """ try: - return self.model.predict(data, **options) + if type(options) == list: + return self.model.predict(*options) + else: + return self.model.predict(**options) except Exception as exc: print('The plugin encountered an error when predicting with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def score(self, X, Y, options={}): + def score(self, options={}): """Return the coefficient of determination :param X : array-like of shape (n_samples, n_features) :param Y : array-like of shape (n_samples,) or (n_samples, n_outputs) @@ -262,7 +293,10 @@ def score(self, X, Y, options={}): :returns: score : float of ``self.predict(X)`` wrt. `y`. """ try: - return self.model.score(X, Y, **options) + if type(options) == list: + return self.model.score(*options) + else: + return self.model.score(**options) except Exception as exc: print('The plugin encountered an error when calculating the score with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') @@ -272,7 +306,7 @@ class ModellingPluginTClass(ModellingPluginT, ABC): def __init__(self, plugin_globals: dict) -> None: super().__init__(plugin_globals) - def predict_proba(self, data, options={}): + def predict_proba(self, options={}): """Uses fitted model to predict the probability of the output of a given Y :param data: array-like or sparse matrix, shape (n_samples, n_features) Samples @@ -281,7 +315,10 @@ def predict_proba(self, data, options={}): Returns predicted values. """ try: - return self.model.predict_proba(data, **options) + if type(options) == list: + return self.model.predict_proba(*options) + else: + return self.model.predict_proba(**options) except Exception as exc: print('The plugin encountered an error when predicting the probability with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') @@ -295,7 +332,10 @@ def configure(self, config: dict): """Extended from PluginTemplate.configure""" super().configure(config) try: - self.BO = self.model(**self._clean_options()) + if type(self._clean_options()) == list: + self.BO = self.model(*self._clean_options()) + else: + self.BO = self.model(**self._clean_options()) except Exception as exc: print('The plugin encountered an error on the parameters of ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+'.') From 1667f4023905a187192e0402678494e2ec71a1cf Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 15 Sep 2023 09:43:26 +0200 Subject: [PATCH 28/48] Fix issue with stored values and changed test data --- src/vai_lab/Core/vai_lab_core.py | 2 +- src/vai_lab/_plugin_templates.py | 58 +++++++++++++---------- src/vai_lab/utils/plugins/pluginCanvas.py | 19 ++++---- 3 files changed, 43 insertions(+), 36 deletions(-) diff --git a/src/vai_lab/Core/vai_lab_core.py b/src/vai_lab/Core/vai_lab_core.py index a2aeca8b..3eb4ad19 100644 --- a/src/vai_lab/Core/vai_lab_core.py +++ b/src/vai_lab/Core/vai_lab_core.py @@ -108,6 +108,7 @@ def _execute_exit_point(self, specs): with open(rel_to_abs(specs['plugin']['options']['outpath']), 'wb') as handle: pickle.dump(data_out, handle, protocol=pickle.HIGHEST_PROTOCOL) + def _parse_loop_condition(self, condition): try: condition = int(condition) @@ -169,7 +170,6 @@ def _execute(self, specs): if not _tracker['terminate']: self.load_config_file(self._xml_handler.filename) - # pass else: print('Pipeline terminated') exit() diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 5c0b0d3f..2cf0ad72 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -18,8 +18,8 @@ def __init__(self, plugin_globals: dict) -> None: """ self.X = None self.Y = None - self.X_tst = None - self.Y_tst = None + self.X_test = None + self.Y_test = None self._PLUGIN_READABLE_NAMES: dict self._PLUGIN_MODULE_OPTIONS: dict @@ -78,8 +78,8 @@ def _parse_config(self): """Parse incoming data and args, sets them as class variables""" self.X = np.array(self._get_data_if_exist(self._data_in, "X")) self.Y = np.array(self._get_data_if_exist(self._data_in, "Y")).ravel() - self.X_tst = self._get_data_if_exist(self._data_in, "X_test") - self.Y_tst = np.array(self._get_data_if_exist(self._data_in, "Y_test")).ravel() + self.X_test = self._get_data_if_exist(self._data_in, "X_test") + self.Y_test = np.array(self._get_data_if_exist(self._data_in, "Y_test")).ravel() self._clean_options() def _get_data_if_exist(self, data_dict: dict, key: str, default=None): @@ -122,10 +122,14 @@ def _parse_options_dict(self,options_dict:Dict): options_dict[key] = self.X elif val == 'Y': options_dict[key] = self.Y - elif val == 'X': - options_dict[key] = self.X_ts - elif val == 'Y_tst': - options_dict[key] = self.Y_tst + elif val == 'X_test': + options_dict[key] = self.X_test + elif val == 'Y_test': + options_dict[key] = self.Y_test + elif key.lower() == 'x': + options_dict[key] = self.X + elif key.lower() == 'y': + options_dict[key] = self.Y return options_dict def _clean_options(self): @@ -142,25 +146,25 @@ def _test(self, data: DataInterface) -> DataInterface: if self._PLUGIN_MODULE_OPTIONS['Type'] == 'classification': print('Training accuracy: %.2f%%' % (self.score([self.X, self.Y])*100)) # type: ignore - if self.Y_tst is not None: + if self.Y_test is not None: print('Test accuracy: %.2f%%' % - (self.score([self.X_tst, self.Y_tst])*100)) - if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict([self.X_tst])) + (self.score([self.X_test, self.Y_test])*100)) + if self.X_test is not None: + data.append_data_column("Y_pred", self.predict([self.X_test])) return data elif self._PLUGIN_MODULE_OPTIONS['Type'] == 'regression': print('Training R2 score: %.3f' % (self.score([self.X, self.Y]))) # type: ignore - if self.Y_tst is not None: + if self.Y_test is not None: print('Test R2 score: %.3f' % - (self.score([self.X_tst, self.Y_tst]))) - if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict([self.X_tst])) + (self.score([self.X_test, self.Y_test]))) + if self.X_test is not None: + data.append_data_column("Y_pred", self.predict([self.X_test])) return data elif self._PLUGIN_MODULE_OPTIONS['Type'] == 'clustering': print('Clustering completed') - if self.X_tst is not None: - data.append_data_column("Y_pred", self.predict([self.X_tst])) + if self.X_test is not None: + data.append_data_column("Y_pred", self.predict([self.X_test])) return data else: return data @@ -194,10 +198,14 @@ def _clean_solver_options(self): _cleaned[key] = self.X elif val == 'Y': _cleaned[key] = self.Y - elif val == 'X': - _cleaned[key] = self.X_tst - elif val == 'Y_tst': - _cleaned[key] = self.Y_tst + elif val == 'X_test': + _cleaned[key] = self.X_test + elif val == 'Y_test': + _cleaned[key] = self.Y_test + elif key.lower() == 'x': + _cleaned[key] = self.X + elif key.lower() == 'y': + _cleaned[key] = self.Y return _cleaned def init(self): @@ -229,12 +237,12 @@ def fit(self, options={}): +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') raise - def transform(self, data: DataInterface, options={}) -> DataInterface: + def transform(self, options={}) -> DataInterface: try: if type(options) == list: - data.append_data_column("X", pd.DataFrame(self.model.transform(*options))) + return pd.DataFrame(self.model.transform(*options)) else: - data.append_data_column("X", pd.DataFrame(self.model.transform(**options))) + return pd.DataFrame(self.model.transform(**options)) except Exception as exc: print('The plugin encountered an error when transforming the data with ' +str(list(self._PLUGIN_READABLE_NAMES.keys())[list(self._PLUGIN_READABLE_NAMES.values()).index('default')])+': '+str(exc)+'.') diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 7f7d519b..268d1e92 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -516,9 +516,11 @@ def optionsWindow(self): model_meth_list = [meth[0] for meth in getmembers(self.model, ismethod) if meth[0][0] != '_'] # List intersection # TODO: use only methods from the model - meth_list = list(set(plugin_meth_list) & set(model_meth_list))[::-1] + set_2 = frozenset(model_meth_list) + meth_list = [x for x in plugin_meth_list if x in set_2] self.meths_sort = [] - + self.method_inputData = {} + self.default_inputData = {} if (len(self.opt_settings['__init__']) != 0) or (len(self.req_settings['__init__']) != 0): if hasattr(self, 'newWindow') and (self.newWindow != None): self.newWindow.destroy() @@ -751,9 +753,9 @@ def OnDoubleClick(self, event): y=y + pady, anchor=tk.W, width=width) else: - data_list = self.default_inputData['_'.join(tags[:-1])] + list( - set(['X','Y','X_tst','Y_tst']) - set([self.method_inputData['_'.join(tags[:-1])].get()]) - - set(self.default_inputData['_'.join(tags[:-1])])) + data_list = ['X','Y','X_test','Y_test'] # TODO: Substitute with loaded data + data_list.insert(0,self.default_inputData['_'.join(tags[:-1])]) + data_list = list(np.unique(data_list)) self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags[:-1])], self.method_inputData['_'.join(tags[:-1])].get(), *data_list) bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' @@ -799,8 +801,6 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): :param opt_settings: dict type of plugin optional setting options :param parent: string type of parent name """ - self.method_inputData = {} - self.default_inputData = {} self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', values=tuple(['Required settings', '']), tags=('type',parent), open=True) self.r+=1 @@ -812,7 +812,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.method_inputData['req_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) self.method_inputData['req_'+parent+'_'+str(arg)].set(value) self.method_inputData['req_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) - self.default_inputData['req_'+parent+'_'+str(arg)] = [value] + self.default_inputData['req_'+parent+'_'+str(arg)] = value else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('req',parent)) @@ -827,7 +827,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.method_inputData['opt_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) self.method_inputData['opt_'+parent+'_'+str(arg)].set(val) self.method_inputData['opt_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) - self.default_inputData['opt_'+parent+'_'+str(arg)] = [str(val)] + self.default_inputData['opt_'+parent+'_'+str(arg)] = str(val) else: self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', values=tuple([arg, val]), tags=('opt',parent)) @@ -1060,7 +1060,6 @@ def upload(self): self.xml_handler = XML_handler() self.xml_handler.load_XML(filename) - # self.xml_handler._print_pretty(self.xml_handler.loaded_modules) modules = self.xml_handler.loaded_modules modout = modules['Output'] # They are generated when resetting From aedc426178a1f09ddcbf137f7ff235b594bfe104 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 15 Sep 2023 09:44:03 +0200 Subject: [PATCH 29/48] Combine Initialiser if previous XML existed --- src/vai_lab/Data/xml_handler.py | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index a9398c78..30a84252 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -95,11 +95,37 @@ def load_XML(self, filename: str) -> None: self.set_filename(filename) if hasattr(self, 'tree'): prev_tree = self.tree - self.tree = self._combine_XML(ET.parse(self.filename).getroot(), prev_tree.getroot()) + self.tree = self._combine_Initialiser(ET.parse(self.filename).getroot(), prev_tree.getroot(), False) else: self.tree = ET.parse(self.filename) self._parse_XML() + def _combine_Initialiser(self, tree1, tree2, module = True): + + # Create a mapping from tag name to element, as that's what we are fltering with + mapping = {el.tag: el for el in tree1} + for el in tree2: + if el.tag == 'Initialiser' or module: + if len(el) == 0: + # Not nested + try: + # Update the text + mapping[el.tag].text = el.text + except KeyError: + # An element with this name is not in the mapping + mapping[el.tag] = el + # Add it + tree1.append(el) + else: + try: + # Recursively process the element, and update it in the same way + self._combine_XML(mapping[el.tag], el) + except KeyError: + # Not in the mapping + mapping[el.tag] = el + tree1.append(el) + return ET.ElementTree(tree1) + def _combine_XML(self, tree1, tree2): """ This function recursively updates either the text or the children @@ -119,14 +145,13 @@ def _combine_XML(self, tree1, tree2): mapping[el.tag] = el # Add it tree1.append(el) - else: + else : try: # Recursively process the element, and update it in the same way self._combine_XML(mapping[el.tag], el) except KeyError: # Not in the mapping mapping[el.tag] = el - # Just add it tree1.append(el) return ET.ElementTree(tree1) From 16774ebfaafda1b29e202d378a4fed146510b2ce Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 19 Sep 2023 12:44:07 +0200 Subject: [PATCH 30/48] Fixed issue with correctly updating loaded data --- src/vai_lab/Data/xml_handler.py | 28 +--------------------------- 1 file changed, 1 insertion(+), 27 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 30a84252..79d45292 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -95,37 +95,11 @@ def load_XML(self, filename: str) -> None: self.set_filename(filename) if hasattr(self, 'tree'): prev_tree = self.tree - self.tree = self._combine_Initialiser(ET.parse(self.filename).getroot(), prev_tree.getroot(), False) + self.tree = self._combine_XML(prev_tree.getroot(), ET.parse(self.filename).getroot()) else: self.tree = ET.parse(self.filename) self._parse_XML() - - def _combine_Initialiser(self, tree1, tree2, module = True): - # Create a mapping from tag name to element, as that's what we are fltering with - mapping = {el.tag: el for el in tree1} - for el in tree2: - if el.tag == 'Initialiser' or module: - if len(el) == 0: - # Not nested - try: - # Update the text - mapping[el.tag].text = el.text - except KeyError: - # An element with this name is not in the mapping - mapping[el.tag] = el - # Add it - tree1.append(el) - else: - try: - # Recursively process the element, and update it in the same way - self._combine_XML(mapping[el.tag], el) - except KeyError: - # Not in the mapping - mapping[el.tag] = el - tree1.append(el) - return ET.ElementTree(tree1) - def _combine_XML(self, tree1, tree2): """ This function recursively updates either the text or the children From 337491b02301c6242247efae663a66bfeaa08cc7 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 19 Sep 2023 13:16:13 +0200 Subject: [PATCH 31/48] ProgressTracking working for functions Still need to fill the treeview with the previously set options and functions. --- src/vai_lab/utils/plugins/progressTracker.py | 394 ++++++++++++++----- 1 file changed, 296 insertions(+), 98 deletions(-) diff --git a/src/vai_lab/utils/plugins/progressTracker.py b/src/vai_lab/utils/plugins/progressTracker.py index fbb02e27..7baf48e6 100644 --- a/src/vai_lab/utils/plugins/progressTracker.py +++ b/src/vai_lab/utils/plugins/progressTracker.py @@ -3,11 +3,14 @@ from typing import Dict from tkinter import ttk +from inspect import getmembers, isfunction, ismethod, getfullargspec +from vai_lab._plugin_helpers import PluginSpecs + import numpy as np import pandas as pd from PIL import Image, ImageTk from vai_lab.Data.xml_handler import XML_handler -from vai_lab._import_helper import get_lib_parent_dir +from vai_lab._import_helper import get_lib_parent_dir, import_plugin_absolute _PLUGIN_READABLE_NAMES = {"progress_tracker":"default", "progressTracker":"alias", @@ -266,20 +269,44 @@ def add_module(self,boxName: str,x: float,y:float,ini = False,out = False): def optionsWindow(self): """ Function to create a new window displaying the available options of the selected plugin.""" - - mod_idx = np.where(self.m == np.array(self.id_mod))[0][0] - self.module = np.array(self.module_list)[mod_idx] - self.plugin = np.array(self.plugin_list)[mod_idx] - module_type = np.array(self.type_list)[mod_idx] - - self.opt_settings = self.controller._avail_plugins.optional_settings[module_type][self.plugin] - self.req_settings = self.controller._avail_plugins.required_settings[module_type][self.plugin] - if (len(self.opt_settings) != 0) or (len(self.req_settings) != 0): - if hasattr(self, 'newWindow') and (self.newWindow!= None): + + if self.m < len(self.module_list)-1: + module = np.array(self.module_list)[self.m-1 == np.array(self.id_mod)][0] + else: + module = np.array(self.module_list)[1 == np.array(self.id_mod)][0] + ps = PluginSpecs() + file_name = os.path.split(ps.find_from_class_name(self.plugin_list[self.m])['_PLUGIN_DIR'])[-1] + avail_plugins = ps.available_plugins[module][file_name] + plugin = import_plugin_absolute(globals(), + avail_plugins["_PLUGIN_PACKAGE"], + avail_plugins["_PLUGIN_CLASS_NAME"]) + # Update required and optional settings for the plugin + self.req_settings = {'__init__': ps.required_settings[module][self.plugin_list[self.m]]} + self.opt_settings = {'__init__': ps.optional_settings[module][self.plugin_list[self.m]]} + self.model = plugin().model + meth_req, meth_opt = self.getArgs(self.model.__init__) + if meth_req is not None: + self.req_settings['__init__'] = {**self.req_settings['__init__'], **meth_req} + if meth_opt is not None: + self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **meth_opt} + + # Find functions defined for the module + plugin_meth_list = [meth[0] for meth in getmembers(plugin, isfunction) if meth[0][0] != '_'] + # Find available methods for the model + model_meth_list = [meth[0] for meth in getmembers(self.model, ismethod) if meth[0][0] != '_'] + # List intersection + # TODO: use only methods from the model + set_2 = frozenset(model_meth_list) + meth_list = [x for x in plugin_meth_list if x in set_2] + self.meths_sort = [] + self.method_inputData = {} + self.default_inputData = {} + if (len(self.opt_settings['__init__']) != 0) or (len(self.req_settings['__init__']) != 0): + if hasattr(self, 'newWindow') and (self.newWindow != None): self.newWindow.destroy() self.newWindow = tk.Toplevel(self.controller) # Window options - self.newWindow.title(self.plugin+' plugin options') + self.newWindow.title(self.plugin_list[self.m]+' plugin options') script_dir = get_lib_parent_dir() self.tk.call('wm', 'iconphoto', self.newWindow, ImageTk.PhotoImage( file=os.path.join(os.path.join( @@ -288,15 +315,21 @@ def optionsWindow(self): 'resources', 'Assets', 'VAILabsIcon.ico')))) - self.newWindow.geometry("350x400") - self.raise_above_all(self.newWindow) + # self.newWindow.geometry("350x400") frame1 = tk.Frame(self.newWindow) + frame2 = tk.Frame(self.newWindow) + frame21 = tk.Frame(self.newWindow) + frame3 = tk.Frame(self.newWindow) frame4 = tk.Frame(self.newWindow) - + frame5 = tk.Frame(self.newWindow) + frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) + frameDrop = tk.Frame(frame3, highlightbackground="black", highlightthickness=1) + frameButt = tk.Frame(frame3) + # Print settings tk.Label(frame1, - text ="Please indicate your desired options for the "+self.plugin+" plugin.", anchor = tk.N, justify=tk.LEFT).pack(expand = True) - + text="Please indicate your desired options for the plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + style = ttk.Style() style.configure( "Treeview", background='white', foreground='white', @@ -308,14 +341,37 @@ def optionsWindow(self): ) style.map('Treeview', background=[('selected', 'grey')]) - frame2 = tk.Frame(self.newWindow, bg='green') + tk.Label(frame21, + text="Add your desired methods in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + self.meth2add = tk.StringVar(frameDrop) + dropDown = tk.ttk.OptionMenu(frameDrop, self.meth2add, meth_list[0], *meth_list) + style.configure("TMenubutton", background="white") + dropDown["menu"].configure(bg="white") + dropDown.grid(row=0,column=0) + + tk.Button(frameButt, text='Add', command=self.addMeth).grid(row=0,column=0) + tk.Button(frameButt, text='Delete', command=self.deleteMeth).grid(row=0,column=1) + tk.Button(frameButt, text='Up', command=lambda: self.moveMeth(-1)).grid(row=0,column=2) + tk.Button(frameButt, text='Down', command=lambda: self.moveMeth(+1)).grid(row=0,column=3) + self.r = 1 - self.create_treeView(frame2, ['Name', 'Type', 'Value']) - self.fill_treeview(frame2, self.req_settings, self.opt_settings) - frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + self.tree = self.create_treeView(frame2, ['Name', 'Value']) + self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), + tags=('meth','__init__')) + self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], '__init__') - frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) - frame2.grid_columnconfigure(tuple(range(2)), weight=1) + tk.Label(frame5, + text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + current = np.where(self.m == np.array(self.id_mod))[0][0] + dataSources = [i for j, i in enumerate(self.module_names) if j not in [1,current]] + + self.plugin_inputData = tk.StringVar(frame6) + dropDown = tk.ttk.OptionMenu(frame6, self.plugin_inputData, dataSources[current-2], *dataSources) + style.configure("TMenubutton", background="white") + dropDown["menu"].configure(bg="white") + dropDown.pack() self.finishButton = tk.Button( frame4, text='Finish', command=self.removewindow) @@ -325,14 +381,75 @@ def optionsWindow(self): "", lambda event: self.removewindow()) self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) + frameDrop.grid(column=0, row=0) + frameButt.grid(column=1, row=0, sticky="w") frame1.grid(column=0, row=0, sticky="ew") - frame4.grid(column=0, row=2, sticky="se") + frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + frame21.grid(column=0, row=2, sticky="ew") + frame3.grid(column=0, row=3, pady=10, padx=10) + frame4.grid(column=0, row=20, sticky="se") + frame5.grid(column=0, row=4, sticky="ew") + frame6.grid(column=0, row=5) + + frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) + frame2.grid_columnconfigure(tuple(range(2)), weight=1) self.newWindow.grid_rowconfigure(1, weight=2) - self.newWindow.grid_columnconfigure(0, weight=1) + self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) - def raise_above_all(self, window): - window.attributes('-topmost', 1) - window.attributes('-topmost', 0) + def getArgs(self, f): + """ Get required and optional arguments from method. + + Parameters + ---------- + f : method + method to extract arguments from + + :returns out: two dictionaries with arguments and default value (if optional) + """ + + meth_args = getfullargspec(f).args + if meth_args is not None: + meth_args.remove('self') + meth_def = getfullargspec(f).defaults + if meth_def is None: + meth_def = [] + meth_req = {p: '' for p in meth_args[:(len(meth_args)-len(meth_def))]} + meth_r_opt = {p: meth_def[i] for i,p in enumerate(meth_args[(len(meth_args)-len(meth_def)):])} + + meth_opt = getfullargspec(f).kwonlydefaults + if meth_opt is not None: + meth_opt = {p: meth_opt[p] for p in meth_opt} + if meth_r_opt is not None: + meth_opt = {**meth_r_opt, **meth_opt} + return meth_req, meth_opt + else: + return meth_req, meth_r_opt + + def addMeth(self): + """ Adds selected method in dropdown menu to the plugin tree """ + meth = self.meth2add.get() + self.meths_sort.append(meth) + self.tree.insert(parent='', index='end', iid=meth, text='', values=tuple([meth, '']), + tags=('meth',meth)) + # TODO: Remove X and y? + self.req_settings[meth], self.opt_settings[meth] = self.getArgs(getattr(self.model, meth)) + self.fill_treeview(self.req_settings[meth], self.opt_settings[meth], meth) + + def deleteMeth(self): + """ Deletes selected method in dropdown menu from the plugin tree """ + meth = self.meth2add.get() + if meth in self.meths_sort: + self.meths_sort.remove(meth) + del self.req_settings[meth] + del self.opt_settings[meth] + self.tree.delete(meth) + + def moveMeth(self, m): + meth = self.meth2add.get() + if meth in self.meths_sort and self.tree.index(meth)+m > 0: + idx = self.meths_sort.index(meth) + self.meths_sort.insert(idx+m, self.meths_sort.pop(idx)) + self.tree.move(meth, self.tree.parent(meth), self.tree.index(meth)+m) def create_treeView(self, tree_frame, columns_names): """ Function to create a new tree view in the given frame @@ -350,72 +467,97 @@ def create_treeView(self, tree_frame, columns_names): tree_scrolly = tk.Scrollbar(tree_frame) tree_scrolly.pack(side=tk.RIGHT, fill=tk.Y) - self.tree = ttk.Treeview(tree_frame, + tree = ttk.Treeview(tree_frame, yscrollcommand=tree_scrolly.set, xscrollcommand=tree_scrollx.set) - self.tree.pack(fill='both', expand=True) + tree.pack(fill='both', expand=True) - tree_scrollx.config(command=self.tree.xview) - tree_scrolly.config(command=self.tree.yview) + tree_scrollx.config(command=tree.xview) + tree_scrolly.config(command=tree.yview) - self.tree['columns'] = columns_names + tree['columns'] = columns_names # Format columns - self.tree.column("#0", width=20, + tree.column("#0", width=40, minwidth=0, stretch=tk.NO) for n, cl in enumerate(columns_names): - self.tree.column( + tree.column( cl, width=int(self.controller.pages_font.measure(str(cl)))+20, minwidth=50, anchor=tk.CENTER) # Headings for cl in columns_names: - self.tree.heading(cl, text=cl, anchor=tk.CENTER) - self.tree.tag_configure('req', foreground='black', + tree.heading(cl, text=cl, anchor=tk.CENTER) + tree.tag_configure('req', foreground='black', background='#9fc5e8') - self.tree.tag_configure('opt', foreground='black', + tree.tag_configure('opt', foreground='black', background='#cfe2f3') - self.tree.tag_configure('type', foreground='black', + tree.tag_configure('type', foreground='black', background='#E8E8E8') - self.tree.tag_configure('func', foreground='black', + tree.tag_configure('meth', foreground='black', background='#DFDFDF') # Define double-click on row action - self.tree.bind("", self.OnDoubleClick) + tree.bind("", self.OnDoubleClick) + return tree def OnDoubleClick(self, event): """ Executed when a row of the treeview is double clicked. Opens an entry box to edit a cell. """ + # ii = self.notebook.index(self.notebook.select()) self.treerow = self.tree.identify_row(event.y) self.treecol = self.tree.identify_column(event.x) tags = self.tree.item(self.treerow)["tags"] if len(tags) > 0 and tags[0] in ['opt', 'req']: # get column position info x, y, width, height = self.tree.bbox(self.treerow, self.treecol) - # y-axis offset pady = height // 2 - # pady = 0 - - if hasattr(self, 'entry'): - self.entry.destroy() - - self.entry = tk.Entry(self.tree, justify='center') - - if int(self.treecol[1:]) > 0: - value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1] - value = str(value) if str(value) not in ['default', 'Choose X or Y'] else '' - self.entry.insert(0, value) - # self.entry['selectbackground'] = '#123456' - self.entry['exportselection'] = False - - self.entry.focus_force() - self.entry.bind("", self.on_return) - self.entry.bind("", lambda *ignore: self.entry.destroy()) - - self.entry.place(x=x, + if tags[-1] != 'data': + if hasattr(self, 'entry'): + self.entry.destroy() + self.entry = tk.Entry(self.tree, justify='center') + if int(self.treecol[1:]) > 0: + value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-1] + value = str(value) if str(value) not in ['default', 'Choose X or Y'] else '' + self.entry.insert(0, value) + # self.entry['selectbackground'] = '#123456' + self.entry['exportselection'] = False + + self.entry.focus_force() + self.entry.bind("", self.on_return) + self.entry.bind("", lambda *ignore: self.entry.destroy()) + + self.entry.place(x=x, + y=y + pady, + anchor=tk.W, width=width) + else: + data_list = ['X','Y','X_test','Y_test'] # TODO: Substitute with loaded data + data_list.insert(0,self.default_inputData['_'.join(tags[:-1])]) + data_list = list(np.unique(data_list)) + self.dropDown = tk.ttk.OptionMenu(self.tree, self.method_inputData['_'.join(tags[:-1])], + self.method_inputData['_'.join(tags[:-1])].get(), *data_list) + bg = '#9fc5e8' if tags[0] == 'req' else '#cfe2f3' + self.dropDown["menu"].configure(bg=bg) + style = ttk.Style() + style.configure("new.TMenubutton", background=bg, highlightbackground="black", highlightthickness=1) + self.dropDown.configure(style="new.TMenubutton") + self.dropDown.place(x=x, y=y + pady, anchor=tk.W, width=width) + def on_changeOption(self, *args): + """ Executed when the optionmenu is selected and pressed enter. + Saves the value""" + if hasattr(self, 'dropDown'): + value = self.tree.item(self.treerow)['values'][int(str(self.treecol[1:]))-2] + tags = self.tree.item(self.treerow)["tags"] + val = self.tree.item(self.treerow)['values'] + new_val = self.method_inputData['_'.join(tags[:-1])].get() + val[int(self.treecol[1:])-1] = new_val + self.tree.item(self.treerow, values=tuple([val[0], new_val])) + self.dropDown.destroy() + self.saved = False + def on_return(self, event): """ Executed when the entry is edited and pressed enter. Saves the edited value""" @@ -423,73 +565,106 @@ def on_return(self, event): val = self.tree.item(self.treerow)['values'] val[int(self.treecol[1:])-1] = self.entry.get() if self.entry.get() != '': - self.tree.item(self.treerow, values=tuple([val[0], val[1], self.entry.get()])) - elif val[2] == '': - self.tree.item(self.treerow, values=tuple([val[0], val[1], 'default'])) + self.tree.item(self.treerow, values=tuple([val[0], self.entry.get()])) + elif val[1] == '': + self.tree.item(self.treerow, values=tuple([val[0], 'default'])) else: self.tree.item(self.treerow, values=val) self.entry.destroy() self.saved = False - def fill_treeview(self, frame, req_settings, opt_settings, parent = ''): + def fill_treeview(self, req_settings, opt_settings, parent = ''): """ Adds an entry for each setting. Displays it in the specified row. :param req_settings: dict type of plugin required setting options :param opt_settings: dict type of plugin optional setting options :param parent: string type of parent name """ self.tree.insert(parent=parent, index='end', iid=parent+'_req', text='', - values=tuple(['Required settings', '', '']), tags=('type',)) + values=tuple(['Required settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in req_settings.items(): - if arg == 'Data': + if arg.lower() in ['x', 'y']: + value = np.array(['X', 'Y'])[arg.lower() == np.array(['x', 'y'])][0] self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val, 'Choose X or Y']), tags=('req',)) + values=tuple([arg, value]), tags=('req',parent,arg,'data')) + self.method_inputData['req_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) + self.method_inputData['req_'+parent+'_'+str(arg)].set(value) + self.method_inputData['req_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) + self.default_inputData['req_'+parent+'_'+str(arg)] = value else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val, '']), tags=('req',)) + values=tuple([arg, val]), tags=('req',parent)) self.r+=1 self.tree.insert(parent=parent, index='end', iid=parent+'_opt', text='', - values=tuple(['Optional settings', '', '']), tags=('type',)) + values=tuple(['Optional settings', '']), tags=('type',parent), open=True) self.r+=1 for arg, val in opt_settings.items(): - self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, val, 'default']), tags=('opt',)) + if arg.lower() in ['x', 'y']: + self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', + values=tuple([arg, val]), tags=('opt',parent,arg,'data')) + self.method_inputData['opt_'+parent+'_'+str(arg)] = tk.StringVar(self.tree) + self.method_inputData['opt_'+parent+'_'+str(arg)].set(val) + self.method_inputData['opt_'+parent+'_'+str(arg)].trace("w", self.on_changeOption) + self.default_inputData['opt_'+parent+'_'+str(arg)] = str(val) + else: + self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', + values=tuple([arg, val]), tags=('opt',parent)) self.r+=1 + def raise_above_all(self, window): + window.attributes('-topmost', 1) + window.attributes('-topmost', 0) + def removewindow(self): """ Stores settings options and closes window """ - self.req_settings.pop("Data", None) - children = self.get_all_children() - for child in children: - tag = self.tree.item(child)["tags"][0] - if tag in ['req', 'opt']: - val = self.tree.item(child)["values"] - self.settingOptions(tag, val) + # Updates the tree with any unclosed dropDown menu + if hasattr(self, 'dropDown'): + for data in self.method_inputData.keys(): + tags = data.split('_') + el = self.get_element_from_tags(*tags) + val = self.tree.item(el)['values'] + new_val = self.method_inputData[data].get() + val[int(self.treecol[1:])-1] = new_val + self.tree.item(el, values=tuple([val[0], new_val])) + self.dropDown.destroy() + # Updates the modified options and removes the ones that are not + for f in self.tree.get_children(): + for c in self.tree.get_children(f): + for child in self.tree.get_children(c): + tags = self.tree.item(child)["tags"] + if tags[0] in ['req', 'opt']: + if tags[-1] == 'data': + self.updateSettings(tags[0], tags[1], tags[2], self.method_inputData['_'.join(tags[:-1])].get()) + else: + val = self.tree.item(child)["values"] + self.settingOptions(tags[0], f, val) + del self.model self.newWindow.destroy() self.newWindow = None self.focus() def get_all_children(self, item=""): - """ Iterates over the treeview to get all childer """ + """ Iterates over the treeview to get all children """ children = self.tree.get_children(item) for child in children: children += self.get_all_children(child) return children - def settingOptions(self, tag, val): + def get_element_from_tags(self, *args): + """ Finds item in tree with specified tags """ + el = set(self.tree.tag_has(args[0])) + for arg in args[1:]: + el = set.intersection(el, set(self.tree.tag_has(arg))) + return list(el)[0] + + def settingOptions(self, tag, f, val): """ Identifies how the data should be stored """ - if val[0] == 'Data': - if val[2] == 'Choose X or Y' or len(val[2]) == 0: - self.updateSettings(tag, val[0], 'X') - else: - self.updateSettings(tag, val[0], val[2]) + if val[1] == 'default' or len(str(val[1])) == 0: + self.updateSettings(tag, f, val[0]) else: - if val[2] == 'default' or len(str(val[2])) == 0: - self.updateSettings(tag, val[0]) - else: - self.updateSettings(tag, val[0], val[2]) + self.updateSettings(tag, f, val[0], val[1]) - def updateSettings(self, tag, key, value = None): + def updateSettings(self, tag, f, key, value = None): """ Return the selected settings Parameters @@ -497,17 +672,40 @@ def updateSettings(self, tag, key, value = None): tag : str tag for the settings """ + + value = self.str_to_bool(value) if tag == 'req': - if value is not None or self.req_settings[key] != value: - self.req_settings[key] = value + if value is not None or self.isNotClose(self.req_settings[f][key], value): + self.req_settings[f][key] = value else: - self.req_settings.pop(key, None) + self.req_settings[f].pop(key, None) elif tag == 'opt': - if value is not None or self.opt_settings[key] != value: - self.opt_settings[key] = value + if self.isNotClose(self.opt_settings[f][key], value): + self.opt_settings[f][key] = value else: - self.opt_settings.pop(key, None) + self.opt_settings[f].pop(key, None) + def isNotClose(self, a, b, rel_tol=1e-09, abs_tol=0.0): + a = self.xml_handler._str_to_num(a) if isinstance(a, (str)) else a + b = self.xml_handler._str_to_num(b) if isinstance(b, (str)) else b + if isinstance(a, (int, float)) and isinstance(b, (int, float)): + return abs(a-b) > max(rel_tol * max(abs(a), abs(b)), abs_tol) + else: + return a != b + + def str_to_bool(self, s): + if type(s) is str: + if s == 'True': + return True + elif s == 'False': + return False + elif s == 'None': + return None + else: + return s + else: + return s + def on_return_entry(self, r): """ Changes focus to the next available entry. When no more, focuses on the finish button. @@ -790,7 +988,7 @@ def reset(self): def check_quit(self): self.controller.destroy() - + class CanvasTooltip: ''' It creates a tooltip for a given canvas tag or id as the mouse is From bf45c1508de1143255bd8ee28c0d9c0e985ec612 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 10:20:38 +0200 Subject: [PATCH 32/48] Progress tracker loads the plugin's options --- src/vai_lab/utils/plugins/progressTracker.py | 29 ++++++++++++++++---- 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/src/vai_lab/utils/plugins/progressTracker.py b/src/vai_lab/utils/plugins/progressTracker.py index 7baf48e6..4ba46144 100644 --- a/src/vai_lab/utils/plugins/progressTracker.py +++ b/src/vai_lab/utils/plugins/progressTracker.py @@ -270,10 +270,9 @@ def optionsWindow(self): """ Function to create a new window displaying the available options of the selected plugin.""" - if self.m < len(self.module_list)-1: - module = np.array(self.module_list)[self.m-1 == np.array(self.id_mod)][0] - else: - module = np.array(self.module_list)[1 == np.array(self.id_mod)][0] + m = self.m -1 if self.m < len(self.module_list)-1 else 1 + + module = np.array(self.module_list)[m == np.array(self.id_mod)][0] ps = PluginSpecs() file_name = os.path.split(ps.find_from_class_name(self.plugin_list[self.m])['_PLUGIN_DIR'])[-1] avail_plugins = ps.available_plugins[module][file_name] @@ -358,8 +357,9 @@ def optionsWindow(self): self.r = 1 self.tree = self.create_treeView(frame2, ['Name', 'Value']) self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), - tags=('meth','__init__')) - self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], '__init__') + tags=('meth','__init__')) + self.fill_treeview(self.update_options(self.req_settings['__init__'], self.p_list[m]['options']), + self.update_options(self.opt_settings['__init__'], self.p_list[m]['options']), '__init__') tk.Label(frame5, text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) @@ -396,6 +396,23 @@ def optionsWindow(self): self.newWindow.grid_rowconfigure(1, weight=2) self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) + def update_options(self, base, new): + """ Updates the values in a dictionary with the keys in the new dictionary. + + Parameters + ---------- + base : dict + dictionary that needs to be updated + new : dict + dictionary with keys that will be updated if they exist in base + + :returns out: the updated dictionary + """ + for key in base: + if key in new: + base[key] = new[key] + return base + def getArgs(self, f): """ Get required and optional arguments from method. From a1228d69d6478f641ef7b4bc45e96ae49ee60092 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 10:34:30 +0200 Subject: [PATCH 33/48] Add methods and options to Progress tracker --- src/vai_lab/utils/plugins/progressTracker.py | 25 ++++++++++++++++---- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/src/vai_lab/utils/plugins/progressTracker.py b/src/vai_lab/utils/plugins/progressTracker.py index 4ba46144..22fab3fe 100644 --- a/src/vai_lab/utils/plugins/progressTracker.py +++ b/src/vai_lab/utils/plugins/progressTracker.py @@ -270,9 +270,9 @@ def optionsWindow(self): """ Function to create a new window displaying the available options of the selected plugin.""" - m = self.m -1 if self.m < len(self.module_list)-1 else 1 + self.mt = self.m -1 if self.m < len(self.module_list)-1 else 1 - module = np.array(self.module_list)[m == np.array(self.id_mod)][0] + module = np.array(self.module_list)[self.mt == np.array(self.id_mod)][0] ps = PluginSpecs() file_name = os.path.split(ps.find_from_class_name(self.plugin_list[self.m])['_PLUGIN_DIR'])[-1] avail_plugins = ps.available_plugins[module][file_name] @@ -358,8 +358,14 @@ def optionsWindow(self): self.tree = self.create_treeView(frame2, ['Name', 'Value']) self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), tags=('meth','__init__')) - self.fill_treeview(self.update_options(self.req_settings['__init__'], self.p_list[m]['options']), - self.update_options(self.opt_settings['__init__'], self.p_list[m]['options']), '__init__') + self.fill_treeview(self.update_options(self.req_settings['__init__'], self.p_list[self.mt]['options']), + self.update_options(self.opt_settings['__init__'], self.p_list[self.mt]['options']), + '__init__') + meth2add = self.meth2add.get() + for meth in self.p_list[self.mt]['methods']['_order']: + self.meth2add.set(meth) + self.addMeth() + self.meth2add.set(meth2add) tk.Label(frame5, text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) @@ -450,7 +456,16 @@ def addMeth(self): tags=('meth',meth)) # TODO: Remove X and y? self.req_settings[meth], self.opt_settings[meth] = self.getArgs(getattr(self.model, meth)) - self.fill_treeview(self.req_settings[meth], self.opt_settings[meth], meth) + if meth in self.p_list[self.mt]['methods']['_order']: + self.fill_treeview( + self.update_options(self.req_settings[meth], self.p_list[self.mt]['methods'][meth]['options']), + self.update_options(self.opt_settings[meth], self.p_list[self.mt]['methods'][meth]['options']), + meth) + else: + self.fill_treeview( + self.req_settings[meth], + self.opt_settings[meth], + meth) def deleteMeth(self): """ Deletes selected method in dropdown menu from the plugin tree """ From 9fc60998d6e3871c1ce7fe798cdccd04b8e42800 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 11:00:20 +0200 Subject: [PATCH 34/48] Fixed error with plugin trereview It now displays the options as a string. For exmaple, it now shows "(0, 1)" for a tuple instead of "0 1" --- src/vai_lab/utils/plugins/pluginCanvas.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 268d1e92..424881b1 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -313,7 +313,7 @@ def check_updated(self): self.id_done.append(self.m) if os.path.normpath(get_lib_parent_dir()) == os.path.normpath(os.path.commonpath([self.path_out, get_lib_parent_dir()])): - rel_path = os.path.join('.', os.path.relpath(self.path_out, + rel_path = os.path.join('.', os.path.relpath(self.path_out, os.path.commonpath([self.path_out, get_lib_parent_dir()]))) else: rel_path = self.path_out @@ -815,7 +815,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.default_inputData['req_'+parent+'_'+str(arg)] = value else: self.tree.insert(parent=parent+'_req', index='end', iid=str(self.r), text='', - values=tuple([arg, val]), tags=('req',parent)) + values=tuple([arg, str(val)]), tags=('req',parent)) self.r+=1 self.tree.insert(parent=parent, index='end', iid=parent+'_opt', text='', values=tuple(['Optional settings', '']), tags=('type',parent), open=True) @@ -830,7 +830,7 @@ def fill_treeview(self, req_settings, opt_settings, parent = ''): self.default_inputData['opt_'+parent+'_'+str(arg)] = str(val) else: self.tree.insert(parent=parent+'_opt', index='end', iid=str(self.r), text='', - values=tuple([arg, val]), tags=('opt',parent)) + values=tuple([arg, str(val)]), tags=('opt',parent)) self.r+=1 def removewindow(self): From ac3fe9a5051d84494bdef9188d45f0d807dbc768 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 11:21:43 +0200 Subject: [PATCH 35/48] Remove reuirement of having methods and options --- src/vai_lab/utils/plugins/pluginCanvas.py | 196 ++++++++++++---------- 1 file changed, 103 insertions(+), 93 deletions(-) diff --git a/src/vai_lab/utils/plugins/pluginCanvas.py b/src/vai_lab/utils/plugins/pluginCanvas.py index 424881b1..62dd8e54 100644 --- a/src/vai_lab/utils/plugins/pluginCanvas.py +++ b/src/vai_lab/utils/plugins/pluginCanvas.py @@ -503,68 +503,76 @@ def optionsWindow(self): # Update required and optional settings for the plugin self.req_settings = {'__init__': ps.required_settings[module][self.plugin[self.m].get()]} self.opt_settings = {'__init__': ps.optional_settings[module][self.plugin[self.m].get()]} - self.model = plugin().model - meth_req, meth_opt = self.getArgs(self.model.__init__) - if meth_req is not None: - self.req_settings['__init__'] = {**self.req_settings['__init__'], **meth_req} - if meth_opt is not None: - self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **meth_opt} - - # Find functions defined for the module - plugin_meth_list = [meth[0] for meth in getmembers(plugin, isfunction) if meth[0][0] != '_'] - # Find available methods for the model - model_meth_list = [meth[0] for meth in getmembers(self.model, ismethod) if meth[0][0] != '_'] - # List intersection - # TODO: use only methods from the model - set_2 = frozenset(model_meth_list) - meth_list = [x for x in plugin_meth_list if x in set_2] + # Tries to upload the settings from the actual library + try: + self.model = plugin().model + meth_req, meth_opt = self.getArgs(self.model.__init__) + if meth_req is not None: + self.req_settings['__init__'] = {**self.req_settings['__init__'], **meth_req} + if meth_opt is not None: + self.opt_settings['__init__'] = {**self.opt_settings['__init__'], **meth_opt} + # Find functions defined for the module + plugin_meth_list = [meth[0] for meth in getmembers(plugin, isfunction) if meth[0][0] != '_'] + # Find available methods for the model + model_meth_list = [meth[0] for meth in getmembers(self.model, ismethod) if meth[0][0] != '_'] + # List intersection + # TODO: use only methods from the model + set_2 = frozenset(model_meth_list) + meth_list = [x for x in plugin_meth_list if x in set_2] + except Exception as exc: + meth_list = [] + + self.meths_sort = [] self.method_inputData = {} self.default_inputData = {} - if (len(self.opt_settings['__init__']) != 0) or (len(self.req_settings['__init__']) != 0): - if hasattr(self, 'newWindow') and (self.newWindow != None): - self.newWindow.destroy() - self.newWindow = tk.Toplevel(self.controller) - # Window options - self.newWindow.title(self.plugin[self.m].get()+' plugin options') - script_dir = get_lib_parent_dir() - self.tk.call('wm', 'iconphoto', self.newWindow, ImageTk.PhotoImage( - file=os.path.join(os.path.join( - script_dir, - 'utils', - 'resources', - 'Assets', - 'VAILabsIcon.ico')))) - # self.newWindow.geometry("350x400") - - - frame1 = tk.Frame(self.newWindow) - frame2 = tk.Frame(self.newWindow) - frame21 = tk.Frame(self.newWindow) - frame3 = tk.Frame(self.newWindow) - frame4 = tk.Frame(self.newWindow) - frame5 = tk.Frame(self.newWindow) - frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) - frameDrop = tk.Frame(frame3, highlightbackground="black", highlightthickness=1) - frameButt = tk.Frame(frame3) - # Print settings - tk.Label(frame1, - text="Please indicate your desired options for the plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - - style = ttk.Style() - style.configure( - "Treeview", background='white', foreground='white', - rowheight=25, fieldbackground='white', - # font=self.controller.pages_font - ) - style.configure("Treeview.Heading", + if hasattr(self, 'newWindow') and (self.newWindow != None): + self.newWindow.destroy() + self.newWindow = tk.Toplevel(self.controller) + # Window options + self.newWindow.title(self.plugin[self.m].get()+' plugin options') + script_dir = get_lib_parent_dir() + self.tk.call('wm', 'iconphoto', self.newWindow, ImageTk.PhotoImage( + file=os.path.join(os.path.join( + script_dir, + 'utils', + 'resources', + 'Assets', + 'VAILabsIcon.ico')))) + # self.newWindow.geometry("350x400") + + + frame1 = tk.Frame(self.newWindow) + frame2 = tk.Frame(self.newWindow) + frame3 = tk.Frame(self.newWindow) + frame4 = tk.Frame(self.newWindow) + frame5 = tk.Frame(self.newWindow) + frame6 = tk.Frame(self.newWindow, highlightbackground="black", highlightthickness=1) + + # Print settings + tk.Label(frame1, + text="Please indicate your desired options for the plugin.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + style = ttk.Style() + style.configure( + "Treeview", background='white', foreground='white', + rowheight=25, fieldbackground='white', # font=self.controller.pages_font ) - style.map('Treeview', background=[('selected', 'grey')]) + style.configure("Treeview.Heading", + # font=self.controller.pages_font + ) + style.map('Treeview', background=[('selected', 'grey')]) + + # Show method selection if there is any + if len(meth_list) > 0: + frame21 = tk.Frame(self.newWindow) + frameButt = tk.Frame(frame3) + frameDrop = tk.Frame(frame3, highlightbackground="black", highlightthickness=1) tk.Label(frame21, - text="Add your desired methods in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + text="Add your desired methods in your required order.", anchor=tk.N, justify=tk.LEFT).pack(expand=True) self.meth2add = tk.StringVar(frameDrop) dropDown = tk.ttk.OptionMenu(frameDrop, self.meth2add, meth_list[0], *meth_list) @@ -577,46 +585,47 @@ def optionsWindow(self): tk.Button(frameButt, text='Up', command=lambda: self.moveMeth(-1)).grid(row=0,column=2) tk.Button(frameButt, text='Down', command=lambda: self.moveMeth(+1)).grid(row=0,column=3) - self.r = 1 - self.tree = self.create_treeView(frame2, ['Name', 'Value']) - self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), - tags=('meth','__init__')) - self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], '__init__') - - tk.Label(frame5, - text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) - - current = np.where(self.m == np.array(self.id_mod))[0][0] - dataSources = [i for j, i in enumerate(self.module_names) if j not in [1,current]] - - self.plugin_inputData = tk.StringVar(frame6) - dropDown = tk.ttk.OptionMenu(frame6, self.plugin_inputData, dataSources[current-2], *dataSources) - style.configure("TMenubutton", background="white") - dropDown["menu"].configure(bg="white") - dropDown.pack() + frame21.grid(column=0, row=2, sticky="ew") + frameButt.grid(column=1, row=0, sticky="w") + frameDrop.grid(column=0, row=0) - self.finishButton = tk.Button( - frame4, text='Finish', command=self.removewindow) - self.finishButton.grid( - column=1, row=0, sticky="es", pady=(0, 10), padx=(0, 10)) - self.finishButton.bind( - "", lambda event: self.removewindow()) - self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) + self.r = 1 + self.tree = self.create_treeView(frame2, ['Name', 'Value']) + self.tree.insert(parent='', index='end', iid='__init__', text='', values=tuple(['__init__', '']), + tags=('meth','__init__')) + self.fill_treeview(self.req_settings['__init__'], self.opt_settings['__init__'], '__init__') - frameDrop.grid(column=0, row=0) - frameButt.grid(column=1, row=0, sticky="w") - frame1.grid(column=0, row=0, sticky="ew") - frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) - frame21.grid(column=0, row=2, sticky="ew") - frame3.grid(column=0, row=3, pady=10, padx=10) - frame4.grid(column=0, row=20, sticky="se") - frame5.grid(column=0, row=4, sticky="ew") - frame6.grid(column=0, row=5) - - frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) - frame2.grid_columnconfigure(tuple(range(2)), weight=1) - self.newWindow.grid_rowconfigure(1, weight=2) - self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) + tk.Label(frame5, + text="Indicate which plugin's output data should be used as input", anchor=tk.N, justify=tk.LEFT).pack(expand=True) + + current = np.where(self.m == np.array(self.id_mod))[0][0] + dataSources = [i for j, i in enumerate(self.module_names) if j not in [1,current]] + + self.plugin_inputData = tk.StringVar(frame6) + dropDown = tk.ttk.OptionMenu(frame6, self.plugin_inputData, dataSources[current-2], *dataSources) + style.configure("TMenubutton", background="white") + dropDown["menu"].configure(bg="white") + dropDown.pack() + + self.finishButton = tk.Button( + frame4, text='Finish', command=self.removewindow) + self.finishButton.grid( + column=1, row=0, sticky="es", pady=(0, 10), padx=(0, 10)) + self.finishButton.bind( + "", lambda event: self.removewindow()) + self.newWindow.protocol('WM_DELETE_WINDOW', self.removewindow) + + frame1.grid(column=0, row=0, sticky="ew") + frame2.grid(column=0, row=1, sticky="nswe", pady=10, padx=10) + frame3.grid(column=0, row=3, pady=10, padx=10) + frame4.grid(column=0, row=20, sticky="se") + frame5.grid(column=0, row=4, sticky="ew") + frame6.grid(column=0, row=5) + + frame2.grid_rowconfigure(tuple(range(self.r)), weight=1) + frame2.grid_columnconfigure(tuple(range(2)), weight=1) + self.newWindow.grid_rowconfigure(1, weight=2) + self.newWindow.grid_columnconfigure(tuple(range(2)), weight=1) def getArgs(self, f): """ Get required and optional arguments from method. @@ -856,7 +865,8 @@ def removewindow(self): else: val = self.tree.item(child)["values"] self.settingOptions(tags[0], f, val) - del self.model + if hasattr(self, 'model'): + del self.model self.newWindow.destroy() self.newWindow = None self.focus() From c9eca5e63b941b744d85f7e0e8604e1218253359 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 12:59:32 +0200 Subject: [PATCH 36/48] Update example XML files --- .../xml_files/KNN-classification_demo.xml | 68 +++++++--- src/vai_lab/examples/xml_files/SVR_demo.xml | 71 ++++++---- .../examples/xml_files/canvas_demo.xml | 55 ++++---- .../xml_files/k-mean_clustering_demo.xml | 81 ++++++++---- .../xml_files/pybullet_env_example.xml | 54 ++++---- .../xml_files/random_forest_class_demo.xml | 116 ++++++++++++----- .../examples/xml_files/regression_demo.xml | 44 ------- .../xml_files/ridge-scalar-ridge_demo.xml | 123 +++++++++++++----- .../xml_files/ridge_regression_demo.xml | 49 ------- .../examples/xml_files/scalar_demo.xml | 52 -------- .../examples/xml_files/scaler-lasso_demo.xml | 67 ---------- ...dback_demo2.xml => user_feedback_demo.xml} | 25 ++-- 12 files changed, 403 insertions(+), 402 deletions(-) delete mode 100644 src/vai_lab/examples/xml_files/regression_demo.xml delete mode 100644 src/vai_lab/examples/xml_files/ridge_regression_demo.xml delete mode 100644 src/vai_lab/examples/xml_files/scalar_demo.xml delete mode 100644 src/vai_lab/examples/xml_files/scaler-lasso_demo.xml rename src/vai_lab/examples/xml_files/{user_feedback_demo2.xml => user_feedback_demo.xml} (74%) diff --git a/src/vai_lab/examples/xml_files/KNN-classification_demo.xml b/src/vai_lab/examples/xml_files/KNN-classification_demo.xml index e6bc3fb5..d017339f 100644 --- a/src/vai_lab/examples/xml_files/KNN-classification_demo.xml +++ b/src/vai_lab/examples/xml_files/KNN-classification_demo.xml @@ -4,42 +4,76 @@ - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] - - - - + + + + + + + - [(350.0,350.0),2,{0:'d0-u2'}] + [(350.0, 350.0), 2, {0: 'd0-u2'}] - - - - + + + + distance + + + 7 + + + + + + X + + + Y + + + + + + + X + + + Y + + + + + + + - [(350.0,650),1,{2:'d2-u1'}] + [(350.0, 650), 1, {2: 'd2-u1'}] - - Modelling - - - .\examples\results\output.pkl - + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/SVR_demo.xml b/src/vai_lab/examples/xml_files/SVR_demo.xml index f94fa31e..5d38db16 100644 --- a/src/vai_lab/examples/xml_files/SVR_demo.xml +++ b/src/vai_lab/examples/xml_files/SVR_demo.xml @@ -4,49 +4,76 @@ - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] - - - - + + + + + + + - [(350.0,350.0),2,{0:'d0-u2'}] + [(350.0, 350.0), 2, {0: 'd0-u2'}] - - - - - 0.1 - - - linear - + + + 0.01 + + + linear + + + + + + X + + + Y + + + + + + + X + + + Y + + + + + + - [(350.0,650),1,{2:'d2-u1'}] + [(350.0, 650), 1, {2: 'd2-u1'}] - - Modelling - - - .\examples\results\output.pkl - + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/canvas_demo.xml b/src/vai_lab/examples/xml_files/canvas_demo.xml index d9d4859d..439fbedc 100644 --- a/src/vai_lab/examples/xml_files/canvas_demo.xml +++ b/src/vai_lab/examples/xml_files/canvas_demo.xml @@ -1,44 +1,49 @@ - - - - + - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] + + + - - - - + + + + - + - [(350.0,350.0),2,{0:'d0-u2'}] - + [(350.0, 350.0), 2, {0: 'd0-u2'}] + + + + + + - + - - - - - My First UserFeedback Module - - - .\examples\results\output.pkl - - - + - [(350.0,650),1,{2:'d2-u1'}] + [(350.0, 650), 1, {2: 'd2-u1'}] + + + + User Interaction + + + .\examples\results\output.pkl + + + diff --git a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml index 658fc9db..778d8f54 100644 --- a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml +++ b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml @@ -4,57 +4,94 @@ - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] - - + + + + + - [(350,200),2,{0:'d0-u2'}] + [(227, 254), 2, {0: 'd0-u2'}] - - - - - X - + + + (0, 1) + + + + + + X + + + + + + - [(350.0,350.0),3,{2:'d2-u3'}] + [(474, 412), 3, {2: 'd2-u3'}] - - - - + + + + 4 + + + 500 + + + + + + X + + + + + + + X + + + + + + + - [(350.0,650),1,{3:'d3-u1'}] + [(350.0, 650), 1, {3: 'd3-u1'}] - - Modelling - - - .\examples\results\output.pkl - + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/pybullet_env_example.xml b/src/vai_lab/examples/xml_files/pybullet_env_example.xml index 53629908..41e8a999 100644 --- a/src/vai_lab/examples/xml_files/pybullet_env_example.xml +++ b/src/vai_lab/examples/xml_files/pybullet_env_example.xml @@ -15,25 +15,27 @@ [(350.0,350.0),2,{0:'d0-u2'}] - - - plane.urdf - ./Environment/resources/models/half_cheetah_with_mass.xml - - - False - - - 0.0 - 0.0 - -9.81 - - - 0.01 - - - 10 - + + + + plane.urdf + ./Environment/resources/models/half_cheetah_with_mass.xml + + + False + + + 0.0 + 0.0 + -9.81 + + + 0.01 + + + 10 + + @@ -41,12 +43,14 @@ - - MyEnv - - - .\examples\results\output.pkl - + + + MyEnv + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml index 42d4e203..aa4d8ab0 100644 --- a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml +++ b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml @@ -1,62 +1,108 @@ - + - - - - - - - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] + + + + + + - - + + + + - + + + [(349, 207), 2, {0: 'd0-u2'}] + - - X - + + + + + X + + + + + + + X + + + + + - + - - - - + - - - - - [(350.0,350.0),2,{0:'d0-u2'}] + [(349, 419), 3, {2: 'd2-u3'}] + + + + 50 + + + 500 + + + + + + X + + + Y + + + + + + + X + + + Y + + + + - - - [(350.0,650),1,{2:'d2-u1'}] - + + + - + + + [(350.0, 650), 1, {3: 'd3-u1'}] + - - My Modelling Module - - - .\examples\results\output.pkl - + + + Modelling + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/regression_demo.xml b/src/vai_lab/examples/xml_files/regression_demo.xml deleted file mode 100644 index 38faeca5..00000000 --- a/src/vai_lab/examples/xml_files/regression_demo.xml +++ /dev/null @@ -1,44 +0,0 @@ - - - - - - - [(350.0,50),0,{}] - - - - - - - - - - - - - [(350.0,350.0),2,{0:'d0-u2'}] - - - - - - - - - - - Modelling - - - .\examples\results\output.pkl - - - - - - - [(350.0,650),1,{2:'d2-u1'}] - - - diff --git a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml index c98d1191..c3544328 100644 --- a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml +++ b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml @@ -4,81 +4,138 @@ - [(350.0,50),0,{}] + [(350.0, 50), 0, {}] - - - - + + + + + + + - [(350,180),2,{0:'d0-u2'}] + [(178, 218), 2, {0: 'd0-u2'}] - - 0.01 - + + + 1e-3 + + + + + + X + + + Y + + + + + + + X + + + Y + + + + + - - - [(350.0,350.0),3,{2:'d2-u3'}] + [(450, 219), 3, {2: 'r2-l3'}] - - - - - X - + + + + + X + + + + + + + X + + + + + + - [(350,408),4,{3:'d3-u4'}] + [(350.0, 350.0), 4, {3: 'd3-u4'}] - - - - - 0.01 - + + + + + X + + + Y + + + + + + + X + + + Y + + + + + + - [(350.0,650),1,{4:'d4-u1'}] + [(350.0, 650), 1, {4: 'd4-u1'}] - - Modelling - Modelling-1 - - - .\examples\results\output.pkl - + + + Modelling + Modelling-1 + + + .\examples\results\output.pkl + + diff --git a/src/vai_lab/examples/xml_files/ridge_regression_demo.xml b/src/vai_lab/examples/xml_files/ridge_regression_demo.xml deleted file mode 100644 index 58e182f7..00000000 --- a/src/vai_lab/examples/xml_files/ridge_regression_demo.xml +++ /dev/null @@ -1,49 +0,0 @@ - - - - - - - [(350.0,50),0,{}] - - - - - - - - - - - - - - - [(350.0,350.0),2,{0:'d0-u2'}] - - - - 0.01 - - - - - - - - - - - - [(350.0,650),1,{2:'d2-u1'}] - - - - Modelling - - - .\examples\results\output.pkl - - - - diff --git a/src/vai_lab/examples/xml_files/scalar_demo.xml b/src/vai_lab/examples/xml_files/scalar_demo.xml deleted file mode 100644 index 5b2b1d9a..00000000 --- a/src/vai_lab/examples/xml_files/scalar_demo.xml +++ /dev/null @@ -1,52 +0,0 @@ - - - - - - - [(350.0, 50), 0, {}] - - - - - - - - - - - - - - - - - - [(350.0, 350.0), 2, {0: 'd0-u2'}] - - - - X - - - - - - - - - - - - [(350.0, 650), 1, {2: 'd2-u1'}] - - - - Data Processing - - - .\examples\results\output.pkl - - - - diff --git a/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml b/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml deleted file mode 100644 index a05d1bfa..00000000 --- a/src/vai_lab/examples/xml_files/scaler-lasso_demo.xml +++ /dev/null @@ -1,67 +0,0 @@ - - - - - - - [(350.0,50),0,{}] - - - - - - - - - - - - - - - [(350,183),2,{0:'d0-u2'}] - - - - - - - X - - - - - - - - - - [(350,328),3,{2:'d2-u3'}] - - - - - - - 0.2 - - - - - - - - - [(350.0,650),1,{3:'d3-u1'}] - - - - Data Processing - Modelling - - - ./examples/results/test.pkl - - - - diff --git a/src/vai_lab/examples/xml_files/user_feedback_demo2.xml b/src/vai_lab/examples/xml_files/user_feedback_demo.xml similarity index 74% rename from src/vai_lab/examples/xml_files/user_feedback_demo2.xml rename to src/vai_lab/examples/xml_files/user_feedback_demo.xml index bfe76cc7..7b1ca210 100644 --- a/src/vai_lab/examples/xml_files/user_feedback_demo2.xml +++ b/src/vai_lab/examples/xml_files/user_feedback_demo.xml @@ -1,15 +1,15 @@ - - - - [(350.0, 50), 0, {}] + + + + @@ -22,7 +22,8 @@ [(350.0, 350.0), 2, {0: 'd0-u2'}] - + + @@ -35,12 +36,14 @@ [(350.0, 650), 1, {2: 'd2-u1'}] - - User Interaction - - - .\examples\results\output.pkl - + + + User Interaction + + + .\examples\results\output.pkl + + From 0840e2e1f5377a6d335f65d97d4858279eca9bdc Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 13:02:01 +0200 Subject: [PATCH 37/48] Remove tests --- src/vai_lab/run_pipeline.py | 6 ------ 1 file changed, 6 deletions(-) diff --git a/src/vai_lab/run_pipeline.py b/src/vai_lab/run_pipeline.py index f6eb4ea6..a2c0537e 100644 --- a/src/vai_lab/run_pipeline.py +++ b/src/vai_lab/run_pipeline.py @@ -9,7 +9,6 @@ import vai_lab as ai - def parse_args(): """ Parse command line arguments @@ -57,13 +56,8 @@ def main(): for i in range(0,len(args.file)): args.file[i] = abspath(args.file[i]) core.load_config_file(args.file) - core.load_config_file(("./examples", - "xml_files", - # 'bayes_opt_demo.xml')) - 'dataFlowTest.xml')) # Run pipeline core.run() if __name__=='__main__': - main() \ No newline at end of file From 1e48acb078525805416e1ed18d7e24b17daf5117 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 22 Sep 2023 15:16:33 +0200 Subject: [PATCH 38/48] Fix error with user interaction demo --- src/vai_lab/examples/results/output.pkl | Bin 6601 -> 601 bytes ...dback_demo2.xml => user_feedback_demo.xml} | 25 ++++++++++-------- 2 files changed, 14 insertions(+), 11 deletions(-) rename src/vai_lab/examples/xml_files/{user_feedback_demo2.xml => user_feedback_demo.xml} (79%) diff --git a/src/vai_lab/examples/results/output.pkl b/src/vai_lab/examples/results/output.pkl index ea411b5c3df0709e68a01e804ce75d8c8c6fa2f0..1a58e9b136be0a5de98826b888d6fad4b351da99 100644 GIT binary patch literal 601 zcmZo*o$ANL00y;FdIUm?Q;QTl^GZ^S5|c|Z^Yf;}PU#WKE6pva)Jx7UO4S20Q;RZ_ zr}S{grxoSrCY7e8r4~(@JVj&L0Y-=^l#w6oDbqiyo6-=N(!&alZy=pgQdy7+jNBfk yG>a*n9qm(srf7IG_b}Q_@$>WZ`VRzP!keLFN|HCbH%Hr)9!BRWB|v3~#d-iB1w9i0 literal 6601 zcmcIocUV)~vZqT?0YwBAMQm8GD|!Sc0v14u4N=jUKp=sTU2F8RJ@1@*?tS0==k4#y-fQidwbrbe`DOiPA5cC0(oo^Y z6(SU9+faBEV`mQAoyO%dSYASbzCVRQW>Tng<&qyavOAkY6AD!1FNCLrGY<+wgrOGQ3!3o*arVEr8AOapQ8_-F{!&O(@VJ2l_IJnK{IaLV>|wjFL;}6qW~5 zF4A&!SpP@w59>z!zZy4^$)rY`aJjgtXm%MK(Q1@f_(U=St z;W;f2nmd!iq3{@NmQXN)Lu3AEa_I~|u8=-fpytQqdofr-x`9B;oz3QWFjy2GF|9y_ zN%)EwZ_8rv7{v5He4#~nloLeuV>4KUOoPGlU+k?g=%=Z$?d$#gLe#D>XjkKY;Gq}WF z94J1&D`>4|`7=0dmOR?>0a|k1O*}S-;zg79RhVqHpO8M2&!^HS%DtoLDirAY5iz82 z=gQM-?vFemwj&?7MxGP@n`tU!UoRri6aTDE?48D8QJCDhz7!U*2#4_2a4RO;-N)f) zpBOIC|9yl!Q@L~h7(!${na}cLxce|^WU73OTzM3W@9P)z(-@*j<1pOiQT61oeW`p; zPa@S$2~A^nYy5;xLPOkkn}iRx5b~pBc{fN3$xS z*(M_Il0iEf=csGe`P4yq#kYa2CIxsh`2>|*B|@+EkL&Ho7h$#e=`%jDnedW((0YmD zeN-&ebarw6j2q|pPB?X}7Kd6F7v8BZK-RS)eR6sqF3T+cSYV%rRTDha9t<_Y$=n3d z{6(d(YUNF34gEftHdpy({(@@IpIlpGktf6PeV6BakCP%_EvYj|qX~|hjb6IKsRysU zpFOE!s|3#OeiO_-*au@@?6Y3j^$8Qn8k?W|CW6Z+!*`tRt%0Xb`K$AGx1oxvIcdw~ z7U1QDlysO#(96r=RfJUue71qZA3Qy|5Ih0ZI|v}%wU*}hJ7OD~Qe#x_@R zmS95YtLfgqH2~u&slD2+98OQU(Yj+@Eu4!F)eZ_O!+z3HmHSV>!a?WRsS0yzA>kKG z&V#`o5Ij>*fWx&Y*>Buy)ZYu%=dbJ+DfD3H`M@(w@wpMPk9ti>zMd94y0k#py5N$Wibc#*lvc2Ni9InS6@K2C}S({wA^ z$oXKmKIMsR`WH;dwU44SRH0rrwYFBd6{A%wMvryR1NZAaMb~G2L$x-Y*0+Y45c*DJ zAb#8hNf~}#_G6?F{@B;G<5MH-F%+%dp4^4)zmzPeHdevgce4d|2WkM`YPpt>dtk6~ zMHz2M3bOrDqf=Lcov}mc2#$l=-4U z>o)M|Q7e9|-+=m_8gmvlmO=3JtqD$g{jjFSD1VYhEoNE037fX>J5pcYJ=Vi`4JoT* z*J+=Pgy~nxcN-EoDjBHXb|I$~qHbv(2oL)Lj6_!AD&IEr+3(qPrAUg^zIHwnZpLFv zmvsC5t&bt$_29KHH+tZc$^&P6qh1(el+pchb_v$swssfqm*LqReMQ6b!yzPZS4K9o z1rtmp>2oCCpvvQtb2+se`Fmd7J+rw3LnfaotaFn=lz5nDnyMH#y=oe|_Ev`D`few5 zL{wwCp=6Aii43*Zt<7IgE=T96k}pidgU(MwRos*)@~6z)nb?_DSd~j zmYarWEGtHzK3JzGYJueHb^D77@*um)T$3Dj7B|{1$gQ-h2b01}1}>9d<8KSb)w;IU zphxPm8Cvg}@%H-R*Zg+A!4`eLPZG8i{G!Cmf?V4mc~s5i!-wi|dXfK=XN^6OoSiDv zrFOuYS%L*CG;2_H*jbHwQw-Tl-4{FQNpRv^>73?cFR(q{L^qXM4O;6LI-i`~hyym; z8UxDnAjfRV=fZ-9Qyp_`#sCr$D$*x|kS9#yzi{I;CA779_VYyo9+NEvS?e8$T z-m49QU$;D|TsnZWa!2e4-~ARo_0fM3#eawR*-M~kaUJ^2a2iwWE5Zi*tD3!c8X(Sf zr|sgVJm5TQ6l)zM1-td(yiGIL*RTrd-PcB#^$3fyZ5<##o1w3yq)N8a9-EucwW>; zXnRi1^>nPlOljIdoFanEG<58~aStaX_YLq}N>S5gh-b5+8DrviW?YKvfao6nvjagb zXdozCm|5`%m@<-7#c~g(Imc!d9IoDP-mMjo){o4!Rclt6SbK!EXF z{VTty0;g+pZnmxo8@8I1XJZ{m`)LnGD-Ymm zY74Dw_+1!w_+?tpnnr9r+wS=MXb1M@Ymn~^yv0uo7g1tQHNg?#gPm7nnsAS|_p=1s z9voE`mr@#(i?46*Ua~`}6U}Q^?%Qj`A!&Zls>Rep$sh2 zGA1U|Ww>gI%4K~Q37lNL`us?e1ZM4(curw-V0`$?fe+@N@VxHJrOgV&`M7=bz>{xP z;QD-8!^696u=s1J>e_<0Xwszauj88!yp-^91vlz(MR1&H>Z=A&I{5K>jKU*amHPhL z1y&E3teEeVb?prV*rt?cc@N;Pz2=X+3VUEu#d6g&|2_IfePyKQ!#5Ku+fgxqjx|CLV@ZJK1Mt}ctL)0HYiZ?TqZmk z86n&#G__Y2C|U>|9Ubk4i0jAAUV$g1+N;`IgbM|V)PE|cLd zpT=H+P*Rl7Vy`5>9}(^(_^82O$)yJ^kj9-WFK!4HO=i&o$hIuvo!kn;f2YZzun3|n z(A+40{!>-Xb5&S!D(grr&!cd7#Nq-KE{~wX_Nwv*jc6$H!wDReWDmX{v3;VbkXLpzCRZ5ATU+l2JFKg&7#0{UXQIsF&Y*hP;Lt(w~CfU6XTo`|RywW2*(jaY~+vqdeq-XVyvptM*fO|P{%ls*KP;%^1 zzCUdU;&X~#lp1$Hd`+~aj=yr6Y~G5E6n2Dcc(`gz%2hH5118MC8LEcysf zK2{u?S|LTnEnAGYhJQoh{DpBbHZ`cd+f2PiR|4bp54bfYE0Hda)iGU>cn`KyFC|XD zrJQ!pciS%k|M(6Yrk*pmyQ@xeUgF%>LutUdDTSi0;mvq$Xv~I?#nqsq=r*K}`XqC0 z`x!}gVz}z^<*cr)BI)c**DF?s#4yVsW?hSCKdv(yn-Fqn6sgNc?^=M5D#`R%oPbu_ zg2rpmGxLl>nrld{jJ#ThwAGPR8aJs2&jt&6{d8(Edn)sJ(spIiXroh+*@?;|Ux}0X z^^2oPeGVP>?RO}oiNiv7-abh%+S9Co=T`!fTDc-g`)+dXkVY$f9BI^iFZVm{ zv$SEXvyr0zrs#Q(u4|AgoC4H#>*nHm&o=}qeyo}_DeUfmOyhv!CBM29w59uF2;yd2eGHk#m#LaNsk*mw%+A+fsnq+ zUQvz~`f52==yn7!6#oVcA}==xRR5IG^ofEof2x#x`2@Zw91bOjsK3Y@nme1t<#G7# zJOV0o3&Aj=!22PT?n(Cw5z-mN&*w+JtGSYivYIGR|CaP5wVz44_y6a0@v1yBf?Zd84 zY#F{TUXR^@s}gs)S!HL!jh3>ZJ|hXP8g(Rc|LQCVn|J)i#Rw_BzLd4E_^uR``X8mK z8+2mo+Y`x=MKxgAubh>pN&Iadn0VpvcNuiA(}*=Z*ntw8S@hCJ&*94O;^Zw6A|#v8 zOX&se78uSeSA3O0MMcMo zJ0`Ud6E`@ERn?8@?~*?6u_%Pt8Npu@3Hhmnz=@I3l_-6(ZGSXh3dU8BUzT2|2fnZA zlpmrKR?Og$sp1vro4*me~dpiYt&-cOyuh`!>{CD7b=dyy%eKATh zlqZEYi!sjU^UZpbPJ9?KCdX}8Csb)~f;{gI7)V*<+{AtW9$Ga^Qj5xQi$~@y?>!H} zV-^{3%jq>feL1`S(T-}gTe-Y=oPQyVZ{$UseIx-V)hL6VVcqa`K-J)u z^*9)Lqrcy1k_7n`%Qxt`W}{W`rvtyXcj3W9cDH2;RcKhTYv`qi6rYE68XscUfa$jU zmk0kL!5#dX9>?WZan2C~`(=LZuud_zZ)dd_99e-Q6yDXMEeyr)uYHLVG{!A;Jt@S< zm&2XUjjO?tJi&?5>@FCW`&x5t=Qn8DqJ8zaT0ThA$m~ngrQm$$;*py9b?C;t^W5M{ z7i`cTHo)-d24l;(nWOHNVcU9MSBj(n7OnYZisnrz1efJ(YCb4|+-2L#CA;2(LhFq2 zdyuZ=Klf9X`&`Sc;miFBCS0)WAmixNFTuVvKdY zc(9c#flKFN!j46XVNb?W^~lKx%ZRG^b+H(~mOop1=IR5mi_J7m8ZCta?@d*b6=GD^ z8DnLV@L+aE=HA9g}N~_VlG8DKjBo&3Xp!i0j&k<24luh0qWzo`!?f09P&#Lc4 z&+2IXYL)Mxdt;f`-dF7~_6a}mLTNpc(|TIu(KI2b~01^ z<_&;D*twa~!esQ`{%rcur7gJo=%n}Qsg>~T+t!M2XTap)cYhpgr_qX>E2t z+_7FkP!I_o-)wq?9MpghC2D^APl|E*^}&|i#huu(vyd}=O&>;E4EGK+NQZ*Mb>rU{ zRO4r-U1#SNzrqpzG4rBT@?cnLLe#0--B5OvJUzu!iu9u6YPtI)pkn2eZempd=Z&}5 zCQP~yn`Py}y_^B;Pr8vqZV_X>dV%ypq z1)`Lp6{$X}&j|+5&4LcF#&%^Bof%*{gH8-B|f;>En;YXQ~gjC!=3q=>WZoEkoum zHDJ5v`N%w?JSsWre&oDYb||eP zzk+|X;xKXGU}bkR$`4gld(*!hs()vje15mXA5XU5>6}MQ z)}mB94U+E7EcSDaVIG4`9EWona}o|#0=+oeT{z1WjFZ<_+? ztNDvQQeXqPB(&P?8>d849C7d2qU)nc($4Bc(hOD7Q|nPtw)vkSQp04F!*E4XV4UXC zN`4PsJa^^cS>nUht8%xsEV2xAb&igr)^vgS^!PGLU_R<-j{T^-vH>GXUTk{Ou0#r3 znJ{2tB!Qm$43)NGRgwy=$-)1#BFT0uy;YCWhV}je>8cE+H2a47jgJi!NPE>H?zimd w{1I8je~IkBrAvF|G=3s|KmCB5@z4+bneRjrT!t?AKbg#b6>s7*I(P2>0ItH)jQ{`u diff --git a/src/vai_lab/examples/xml_files/user_feedback_demo2.xml b/src/vai_lab/examples/xml_files/user_feedback_demo.xml similarity index 79% rename from src/vai_lab/examples/xml_files/user_feedback_demo2.xml rename to src/vai_lab/examples/xml_files/user_feedback_demo.xml index bfe76cc7..8fe8f048 100644 --- a/src/vai_lab/examples/xml_files/user_feedback_demo2.xml +++ b/src/vai_lab/examples/xml_files/user_feedback_demo.xml @@ -1,15 +1,15 @@ - - - - [(350.0, 50), 0, {}] + + + + @@ -22,7 +22,8 @@ [(350.0, 350.0), 2, {0: 'd0-u2'}] - + + @@ -35,12 +36,14 @@ [(350.0, 650), 1, {2: 'd2-u1'}] - - User Interaction - - - .\examples\results\output.pkl - + + + User Interaction + + + .\examples\results\output.pkl + + From d1f23c20453077719deaae6351661797582d3013 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 26 Sep 2023 10:06:20 +0300 Subject: [PATCH 39/48] Modify example to use test data --- src/vai_lab/examples/results/output.pkl | Bin 601 -> 7076 bytes .../xml_files/KNN-classification_demo.xml | 4 ++-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/vai_lab/examples/results/output.pkl b/src/vai_lab/examples/results/output.pkl index 1a58e9b136be0a5de98826b888d6fad4b351da99..8bbc6a98a533d17e7e9260a66b7d5592d7b320bf 100644 GIT binary patch literal 7076 zcmcIpd0bOh(`FR}MQ}kylv+hV1r_UBeX|aOBp?rFHc(K?tK2J^Fz>nse{lE|9 zoAPu#7<5k$Pf7sI!=J(-_&odNi`_N{(AWWP0klAR0EOYUg5|~aqcJ%FZnkvJYObdn zg>Fs>2%raWXjF45Er9OBH1}px{AfWe_71lIwwK!7qGVk~cnKZEN-XYz@0ydi!x4#A@G ziSaxQmcLwqlrZ3Fd$Cw-DxFE;khAgB7~~d`!`d|j>)01DPEt~H{`Qbe4dWn;p^EfFB<7H zGBk*(e-0EmXD6TJQ&^Ck&I+0rJ%C*CI?9ee#I;v41LE(%5t_xwF05EI&`Kw>KGq(R|ZW+lPM6rE^Kc;~hFh=$jvu&{Wli z6F(WRHF09%uw+bes$a5yUo|E;B{zg+b%8qVyV!@W zUDz4++S8Ga^P+yBePBz?lt(je8~(;R9=6-ikH}MCU>$Zx@#dDu(-E!bX^$|| zvaNT0xDa{k9Gw?ewV?I#{8pK;1FL_jl}(DOLB&IpJNgYxu=`EB>fBusHeVX^gT;0k zJQ_B$EAQ4odsY2}3R)^sGt0u2w$~xpeWlN|)ly8|s5v-jK6QJ4 z>|irA7idh+*jRv3DnFmO^B4R}l!@MjN(4BOEbud5AVv4={y-nA4p_E`ChHCBK~rRf$jG}E z!>*~{UAjqz^)5QXBZc)ac1xLSE0ds0Gk5Nk{w_EzeH1PNl0q;4?52JFLQF`!79 zJlhGaRVU|JyVQeu+#u%tZ>5Nx{LqfLCd1p4qYv~RX~PUl4XS&c2%`@#yYRiB6P#z7 zn~xmMz>C=0lpNoanBO8TOx%(N7mvn$9c$`wNMIxLauj1t$cw|>nPt#eVW>1gy#ZhQ z&6>NCT<_>LZ@Sz$LbN@wRrNeng`rks8?0Xv$+S z$6PfLI$Xt&w#*36;oV>lj;q^Ghs4SnARjgN^s>@U&N^mJh{3p7o1-^n~g@k zM91aHUdDMnm>(W0*j1AamzKVazCIBonm^yzQvVDVZ{44!v_D3Y)A_ZwTU*gvTG1t5 z+J-C4v5u3KBuJOZ*jvP6SoyyEpnIbVRLur{!NwlE6&4;|<=2f5+iWIH3Jt?{nMPPp zwFq;4A4FEp?!bw_7tEI#WvC~dPW~F(jB49Qp~9yx(Yx(l_l2$wH0o-lJ(`dUUgAv0 zc_VvZ;GehPX!=9kFfQ0a3v9vDXN+wv@ny)>+uX04Dnd9dHO)k$1wk{pI&M{QuuB(= zKcQKQtKr67#=>Nb+-$bBZb21v;*SyTW-*93R^~W;bt|~VyVQ4|OhVw4k+MD6B0Tqf zXt1=Z30Kzm?3}k-g#EgcR8>xO<4TQ|ZO#c9h7@i$ta{LaZKZBk%`?PMCN7Q04JpAZ z)6@k96{RT9ovTf&DZ*;q5b25jMpzsjAF<|Y6Dssxhre_A2{W1V^?H}J!Dz{z?^}0N zA!DPz_1j4j=tYFpN>pB;m(^svNz{#TPCX&jlwz=uz}u8o3Ds34dza3T;k0r`Ug#k) z<`TcO^yq!{NHCciKH0?C8Scq-a4t6dZDo{e~qT4jJnOe&6eSe-X;^Z%S~uS+Q#?yq?qt}>|N%y z=g7KgZFl-rEsAw_tgE(uiKjab+~*c|LF=@I?xBrh3_JH$kZWIyBKiw@;N)hs8ffaP zG_>LUkB`}vZ@Q5rJy*+@^&oNWga>!%HL&yYU0Bui7|oj;3bjU6<9*`c{B3!)sK|-g zy=Zg>`iK7FY%wJXHV2cxPmjHWvmqy63{31o=A1Rb!PCMqV%lh`3oRe@dc$3+DD}Q$cVoEk#ENO6Y)myi*wB?M#?=!4V{@iRvF5j$fL~9gK;_-B2iIP8 zA^EjRx{_HBu0)N<>6D2;bEJev)mDKoD;)jIM1mOKVQOPNTQJN_QnMkv8l$y#z5ZBT z02P5+x%(qA%#YXW&OOnM5lfzIa~0o3+wjv*JhyjZ>80u4+YCsNS73G1esU}3r)3WL zjU~gHyOx6FvX}S}nSa{t*A8%g?_Heust3RR;I?m6trS}xWcF!}EXIvlt!=LPh0spd z)wjvXhu!P%_(xMa@#LY{v&ggpR<><3XE}>N*qxQ=X|}_5+^w>=c}>_PSa51>c_k)J zNx1#$P$#I_3FYP~5(w1eMp#DGW7E*a4olSzgkH=~@QDz>aIazS#urs^ji1p}v9Jlg ztua@AyZ;<1ogW%A$#aAHu0uFIsT{1hlU@s5+EL;2Qfp;<6$Ilfi}KFiIy?--*&>+$JIHtGZUs;Ua4v4OJV6NezHEZ1YeDDr>?s! z!SVe4?{&%FXJ6l;texXZk>~z!nfj8Ki1GB*SaLoUO0O#Jj-++taFTO2x4sRHZ_3Ym z?rX=H0_KIe#UeaI*CuP#9&A+BHye9Pf-Mazo5##3!K$@J&Sx$2(7WMgeE9r!d|Sfi z`i`uC{*WD^-R{p|);C@vV>P2!iLQFzN`zy%cQ-BH*^c>no6D1@cH`DmyLWkU<>){B zG&d|pj9a#SW0qC_glW5?->%+Yi+g3yFV1`+1ap?bk_@33C8L8D2iE_D{r(HgGp4np z`NXod-!2oPqGw)osc8YI(n~gxhK*QhnHUm%Tnau<{SzJ|DU}oKFGqts74L=o$U_JD z-|*D%AWks<|x~%NAQWI zJQc^kn7Dh8i6BjklrL9#HfiJ>^0p-Jx#ax~pGNla)R=PQ?7%%h8b)m9jsL%_VpEtT zDCcQ?C#Qclh2=o*5{JU(kc;D~1#n2jZf_-jp^-1jpT?xg2j;1HkS-yyI+eug3f%5N z)-x1&1G$|%bqa?=B66}?8sg#2_*^LQ#!?szR#1QkmFrKsf-G?4#V3ssz&{kg_2enL z@`>4>D^6lAF`rmOSec$`R{o5(6;B2GCHi-AgZuxEA3TpjUO@-<59a@=++aNg|0R7z z``=N;IRBI?##6{E_`y`sKbR`y2DcC92g`j5efpn5pIGsKaOfhdL2T z>>>6N`-mSx`9wH*M|=v_Aub+deNI-o{}!=JhkUlp`Q2fsoC+zV0|F~8zdS7JkGx28 zY(s28kc}Wlg!@)=XDpiDji$c&)5>*ov9SK0`3CbAIA8Zs8FA_*UOX7sbKboS@i}j{ zMjdFvtNt@znPjFTYW0?x-IeLE4*JS%D!&^6Hq(rVwIX~Sy>-gBO%f~|v%E8FPA!_c znsuU=X5qCU?o{H&Zj_z!aoRmwikZg@r=L;l!NQrxlvgY(g`aS2;skydB-OHI*++#C zzVkmW*-zF3QR{vfkMy3QqV#HXT^sy3~WAom~9>|bz{m#4m^fI_N5A>_vsmB|egOL+@ z8=(L7`Lmrpb;t?d8Z-5k2#Xd7B74&=;d$#tx4Tw-kZ9H>L|-qAe?3)2zYmU>qsqzn{~Un(Y_m0_jN zh0^wSFG0_`6*8kjh@_MEM&xOTaH&51T4s9_^rzo%F`m@~&w|}|1l#H`wSoWpG_z_@ zUJMJd_^}u{xo>Lb^);XvmirKran@y-qg4=ilV;5FC z=DB6xay0M3`eyq+>w`iB&M9g(Un|Ax$grD<4pKZFA?3bZB85>-R)%O*1-88jnc`}g zjmN9Kimq4|!kTt9t(~6=J6dE(lTI62FLWPJo|%r7!tw#3ffQNUaT!ZoN-*HZN_U!? zf^Bp54fy<6hl7DJ3(t`8c0S(mOw`!tSRH6;w&+m-o|+VXjJFn|c8qw-7*7duCr?_u z;K~zhaW5zu5-7roF01KtY`bv!6z`17gH!0AVE@e-VGcyj3(j@yYeB}33!ao(im;Y_ zS95lR7-_#<|Im826)|ToDAi`agh7#Sc)Wf)hF|jRy4F#S<7VgAzQ5Cs2xXTC^pa*Q z-Q2?QJlp`B*r>}XBI8;}=`)Nifqq&?_@qQJmY3@cJ3PJ_oHy~JryHJP*mdF(cWV`b zGh2y~-a=GWoP3fyryJIlg;EXoG@SaqnHucXfuF+)CrLDJMGxp1ny;$I{JWGiEl*IM~4Jn4lo1D8UIgglTo6g9e9^ght!2EdY zOUP;%I#VF(gzWkZi@uZw%-%aZ^FTrsUSH=P(d(*&XPGD=VxkyF7ThwDx)?+3SI zKKY>N^>M8aOJRPruWq}i5Sj6*No_q+>n~`^#bPn4E9W69-OKgSFszHZFf3 zG<`+97>{#2-;3;PAk*r-T>i8GcDw5=-`4Ma{@X9{&``7XN-i2T5lXXtZPJMeW>9-jLvIJK$jtss%9+%pV z-!l@A%(E0jg=KCNzVRW#ypLPhB$0J(X^DFBw>>!MG;C6O?p+lA&hC}I?0{GCy7mh4 zJHYk<=StC`=Lqp{I27@7DLM>oi>K5zVZG7JA7ht^AXX)U^2T<f8 zk?NJuxooO!X3_?M>b=R85mF?ZOx3X*Xn?uSyo-X+ay+CM8Xi!UptbF_le1ALf_5bO z9?I*2_pyxc>KsMbMBl2JaF(nq?JsK9Karw1yx>63vUcPyyV0i6Dg&QbPQp~vznpjf z4qN}9^G=Q@<>#F#K_`5P9R!0Q0owm_@R|OzgU|Jg2j3rO{NEk*ivGb6K*9eVRp<|< zgY^{s|Fr!}a*A=jL=}2p>i#yHh*>ee*@_JV}JXzY2?2YBp6@%hban+zGV5} z_QAYDUqQdr|D|??o`Ndm6jagvpHhXsg8ozeFUc#~2UEF+P08&11Z2rSL;@`S2T7d? ACjbBd delta 124 zcmZ2tev?J9fpw}M69X93PU#T{Elw>`@XRYoElNx-$;{825<6L)S%dMw - X + X_test - Y + Y_test From 3d6d78305c2d2f291473a945797c195ad266a5b9 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 29 Sep 2023 11:43:28 +0300 Subject: [PATCH 40/48] Fix path issue with xml --- src/vai_lab/examples/results/output.pkl | Bin 7076 -> 5769 bytes .../xml_files/KNN-classification_demo.xml | 8 ++++---- src/vai_lab/examples/xml_files/SVR_demo.xml | 8 ++++---- .../examples/xml_files/canvas_demo.xml | 2 +- .../xml_files/k-mean_clustering_demo.xml | 4 ++-- .../xml_files/random_forest_class_demo.xml | 8 ++++---- .../xml_files/ridge-scalar-ridge_demo.xml | 8 ++++---- 7 files changed, 19 insertions(+), 19 deletions(-) diff --git a/src/vai_lab/examples/results/output.pkl b/src/vai_lab/examples/results/output.pkl index 8bbc6a98a533d17e7e9260a66b7d5592d7b320bf..37f85b484926d421d1e5b3e3af1eec68b9db0c9a 100644 GIT binary patch delta 4276 zcmY*c2{aU3*v=Io zZOA$^X2CEc318~-Pv<|K^Z)NT_dNH$_r34A_r34)o_kZaKhPAqCG_;BWUkOrL879v z_x>&4xY)P#>?bqtVHke32tOM>4*4vHwu<9CSYFnVQT%%t1#|ULc?&2Qyqjj|WiyW8 zeb#4h)le|v_+UVt`vyHQy#D%31->s+YFemD1M77`d%6r04vxd_>2po+vmNj|{egvt zEz7L#eeXf-OlQOWubo7sX{}9|ilM^1^eQ7&h>nBaGpo#9263n`Zmr4YUVJ*g`$qre z4zL%UyBGYi2OG~2gJZ%X5Tt*#XsuWuY_ti{-^1?YN>BC%|M)f~>51^%F z$a?Ml5pEey%ZP&u8Tg^PdT%kc8Be1tb<_&;VO{?A_(u*GM-P|0%DBLWIuLysy9`htt(ct}N(`3xgE`(h$Zoh68gZ=oF84D8J*pC)#KjTCNm3p* z%XH|imm5Q9o#+YESr+7Cj|Lw#=z@(CBS-Qx2O3F~Lph{RASe@|FRMpJqb*^8;l%`J zgLb*vMk)$bqvE1%IxxFvRKrSS0J2-#9`9@K!uhzW;E-z+yd#+=u_rlj>hq;6zLkRL zfQ-|1iCjd~c;1%bC++CE3y<~N*)W=0qGWfD4C#1LpL+^JFvxhD7GcDL^o<%%wo)5< zh(rSE-K&?V4nDi|N-!1ag)7&PeFk85dHR4M?;TdK!n>w(*f5q+SSGrij{W+AJB^#oJK6SQa^&ZkJ?ue{4n?OEy?K!0gc`5rvmnLCuA4gDiRj#J`$xi& zXwHHPciudPUl|1&2L4ICX;pT}B43v8|*$+R4e!KWB>fv$tCT?CX z&*wXt6tmv)4IS_9CmWhjD-dTCvCdw)6TjSbs?=#?MEKs>BQw?Dj|NkI8(|E zSU1c-qUKKx8y7A_kLhiy+fM;uNb08bfng9Ae6nvSq~YgCoU#pNC{9g1Y64-9MeHaDUyet-Ne)iD?^(0aV$&M?%a2w4Pn z3o5v`Pi-}4Lr&>r{o`*jSSZ^PuQEeL!=VTU@xK-Nn_udartDwQGe(`o~vq9_^lb(&$9>>9{XIrf!_q}n zgJgcKC!dxc5hIV_@w8$32_XuoJw*hsavrn=JXG|1OJN5^k6&S32((Tk9+64wLtAme zCE_C*)=jM3bJUN5jWxsq3ta|iJ)(OzXbzzL0WFCh(T$gc64oQ{Rut6C#eG{i0(zGS zC#i#s4?zVf1}CU+RxCX^eP|Fx24^2!l;*(np`?~hR3(bbj4ItWknq7$Pg(im0G>Xd zc=n6bj(>WHS&q}Wqp%a%JfD4p3B5h)!dYeA2u*Vkn&l^xnx}@;vmc%4i@qqZzV8!6 z&!w81^CRFKnr@kSm5a3^fd-5C8x1k(_m_UyhWs-(TfYbPpiNQa$1CG5!h$DrkYRhpZ7oZ+@s)MVsoIGk8U&W+Ghv(OtfNyeMv%I8yk)O+^I>@ zKQMGmOCA44!!gOnEB|}ffylJ@9aCD32ne})kkT}WjD|!vPv$VPrqWG9ezxQA=elL8 zYl@-Md*qk!@5)#7zbM25V*kRCb;qIB2=OGYOD#A4xY z<;%vgASy~*CwdGgucK1Ir2(A;uYHCk_u+JHTr&CfFp9q?$R&7;K_sa@zn;$m?pK9a z+N~;>yngl|RgHt4^Q?2GG8|+JmG`dlZ3OqMMug(>0o-%izFaG*1qo||eG0u;5S1L; zJic_42xnPNZc6(g#Qd(+&j>T1#fblFGO-geMfAN==lBnjW7TsT-cn&v`0Vh4cRV;6 zbT640qT|cl7L_;p36N-rzPgRa!*SIFYSytFWP1i0DhYAnB6dvoP7?>}=%_0bsnEPJZ6!c!gVC-aH|?om^lQ7Z6>MA4x}nVd#^rv{ zJxvdNPwj^#XNn>}+Kqd;%VssB=rD<~oAS4*!v5`tw}~BXhR_k;3g@9ARA2B9zjK?7 z7AwD_cO}^%6iFWXD$fCl*UVZW&P7y+kh}L~4_`i>_sahy$`FV=E}=l&C%pEFbbS*F za%Bi>gj#XV!6rJXfnR2hS5D5_kZ|0%%%W6>hqPSr^#R(k2u?#*kX$`ZuMfFP>#JuJ7zfl8s}}PIPkLDpmos43oA7ksfiZ~Q zHyoGMWulu|zQlY}8eGh-FHAl{N6C+CX+iw?r&gz9n~OSNyFSme#fJmue+l}5BmAox zJ(3x@u@_c?q346vMU(;hD!y|JBqhPLdwO|*HV7kyX`E21Jc+GD5&6AxVgu4I5tQ&+& zyU5}e13LcJ;Psv%^ui<4;SpWB9fDdG!Q-ds;OvSnhUEy9$?&YTb|b2Ga9FQX}{vBG)n4HG(ix*FaGTGW1(rBcFuzLN$kQo}JK-**f3pS*2E# z`z3cBI5mbn-`@tyC^Ar-9FX#vO#@Hlivl5X3`^69D))xsS|GSe-+SktF%+Apl{PQp z;+62!gq=z!PUy;tJanK#O)~QEM&=mIn?m>B*)xiZ{g*E3%F|#oad~#na5tKnd2!;p z?dW>xQ8xdiA31udTOCyMp)!-ECh+7HDl6spMLcEVxXWKuLMaJq14{EDTSjr3_^#%D zYWxV4IweEz8B#&FG_JPlufV5#S(De<{O^dz?6;%$#&G^sZiB;eAwLa3PBxp zy3cdjC{(EJ$ve`C;HEOip(Zkx%wFAW@MZ+nUSWOisx??Bm)Me{TZ0Qn5>H(uap3lP z_;E)H8}dX4%SyF&5F0F2O^U4h@S=YCF4c1*s23Y?_13P2l(3yzR45sP>60QB8}pFl zci+#5#e#ABp+b}H0i?T=?V^^^KrE|YP-II&uZ*wQ4yjf+rcdouap0nfl5aA&b^sf9 zy!5>}O@>6jtx^Z?T>UDkTFd<>etf- z@eD?d@kAqn8ce9qkJMm+opFngMkZFW^Q&pOeELe+8H7IR#Lc~8U3uMn$5h_;>4an& zB=kyhIfYz=M-*$7S=0a_1G}5}{QXaKaYOARCSK?~IKE>Q9pvFz^Z3koNY=}?8@04R zz0^bMawHkoI;X?#xUt~8le)B-MTgI%x2KVPEgt-1TVBFS##r}^>Z#gBD5=xjm`j_{ z75TI$miV9t45!#~mBeoF+K=62kX zX282!$zb=*aVR)%>(4$wf==5F-$UnTX%NmhM^y12fcZL`%oByf{N$*tTb49*4i-pl~L9=p`8JBlB{f4iMEY{SxwHAOFV+an9t#1cKOIh~Jk z*{u@&a>4(KARqqXx8L|j1o)lb?YI87{IU6?|BZiizw!6@xBThLjZrT8A%D4UiF_+AyG-Ryl*>V{sH0Kd-j1G|{s)fTAX@+c literal 7076 zcmcIpd0bOh(`FR}MQ}kylv+hV1r_UBeX|aOBp?rFHc(K?tK2J^Fz>nse{lE|9 zoAPu#7<5k$Pf7sI!=J(-_&odNi`_N{(AWWP0klAR0EOYUg5|~aqcJ%FZnkvJYObdn zg>Fs>2%raWXjF45Er9OBH1}px{AfWe_71lIwwK!7qGVk~cnKZEN-XYz@0ydi!x4#A@G ziSaxQmcLwqlrZ3Fd$Cw-DxFE;khAgB7~~d`!`d|j>)01DPEt~H{`Qbe4dWn;p^EfFB<7H zGBk*(e-0EmXD6TJQ&^Ck&I+0rJ%C*CI?9ee#I;v41LE(%5t_xwF05EI&`Kw>KGq(R|ZW+lPM6rE^Kc;~hFh=$jvu&{Wli z6F(WRHF09%uw+bes$a5yUo|E;B{zg+b%8qVyV!@W zUDz4++S8Ga^P+yBePBz?lt(je8~(;R9=6-ikH}MCU>$Zx@#dDu(-E!bX^$|| zvaNT0xDa{k9Gw?ewV?I#{8pK;1FL_jl}(DOLB&IpJNgYxu=`EB>fBusHeVX^gT;0k zJQ_B$EAQ4odsY2}3R)^sGt0u2w$~xpeWlN|)ly8|s5v-jK6QJ4 z>|irA7idh+*jRv3DnFmO^B4R}l!@MjN(4BOEbud5AVv4={y-nA4p_E`ChHCBK~rRf$jG}E z!>*~{UAjqz^)5QXBZc)ac1xLSE0ds0Gk5Nk{w_EzeH1PNl0q;4?52JFLQF`!79 zJlhGaRVU|JyVQeu+#u%tZ>5Nx{LqfLCd1p4qYv~RX~PUl4XS&c2%`@#yYRiB6P#z7 zn~xmMz>C=0lpNoanBO8TOx%(N7mvn$9c$`wNMIxLauj1t$cw|>nPt#eVW>1gy#ZhQ z&6>NCT<_>LZ@Sz$LbN@wRrNeng`rks8?0Xv$+S z$6PfLI$Xt&w#*36;oV>lj;q^Ghs4SnARjgN^s>@U&N^mJh{3p7o1-^n~g@k zM91aHUdDMnm>(W0*j1AamzKVazCIBonm^yzQvVDVZ{44!v_D3Y)A_ZwTU*gvTG1t5 z+J-C4v5u3KBuJOZ*jvP6SoyyEpnIbVRLur{!NwlE6&4;|<=2f5+iWIH3Jt?{nMPPp zwFq;4A4FEp?!bw_7tEI#WvC~dPW~F(jB49Qp~9yx(Yx(l_l2$wH0o-lJ(`dUUgAv0 zc_VvZ;GehPX!=9kFfQ0a3v9vDXN+wv@ny)>+uX04Dnd9dHO)k$1wk{pI&M{QuuB(= zKcQKQtKr67#=>Nb+-$bBZb21v;*SyTW-*93R^~W;bt|~VyVQ4|OhVw4k+MD6B0Tqf zXt1=Z30Kzm?3}k-g#EgcR8>xO<4TQ|ZO#c9h7@i$ta{LaZKZBk%`?PMCN7Q04JpAZ z)6@k96{RT9ovTf&DZ*;q5b25jMpzsjAF<|Y6Dssxhre_A2{W1V^?H}J!Dz{z?^}0N zA!DPz_1j4j=tYFpN>pB;m(^svNz{#TPCX&jlwz=uz}u8o3Ds34dza3T;k0r`Ug#k) z<`TcO^yq!{NHCciKH0?C8Scq-a4t6dZDo{e~qT4jJnOe&6eSe-X;^Z%S~uS+Q#?yq?qt}>|N%y z=g7KgZFl-rEsAw_tgE(uiKjab+~*c|LF=@I?xBrh3_JH$kZWIyBKiw@;N)hs8ffaP zG_>LUkB`}vZ@Q5rJy*+@^&oNWga>!%HL&yYU0Bui7|oj;3bjU6<9*`c{B3!)sK|-g zy=Zg>`iK7FY%wJXHV2cxPmjHWvmqy63{31o=A1Rb!PCMqV%lh`3oRe@dc$3+DD}Q$cVoEk#ENO6Y)myi*wB?M#?=!4V{@iRvF5j$fL~9gK;_-B2iIP8 zA^EjRx{_HBu0)N<>6D2;bEJev)mDKoD;)jIM1mOKVQOPNTQJN_QnMkv8l$y#z5ZBT z02P5+x%(qA%#YXW&OOnM5lfzIa~0o3+wjv*JhyjZ>80u4+YCsNS73G1esU}3r)3WL zjU~gHyOx6FvX}S}nSa{t*A8%g?_Heust3RR;I?m6trS}xWcF!}EXIvlt!=LPh0spd z)wjvXhu!P%_(xMa@#LY{v&ggpR<><3XE}>N*qxQ=X|}_5+^w>=c}>_PSa51>c_k)J zNx1#$P$#I_3FYP~5(w1eMp#DGW7E*a4olSzgkH=~@QDz>aIazS#urs^ji1p}v9Jlg ztua@AyZ;<1ogW%A$#aAHu0uFIsT{1hlU@s5+EL;2Qfp;<6$Ilfi}KFiIy?--*&>+$JIHtGZUs;Ua4v4OJV6NezHEZ1YeDDr>?s! z!SVe4?{&%FXJ6l;texXZk>~z!nfj8Ki1GB*SaLoUO0O#Jj-++taFTO2x4sRHZ_3Ym z?rX=H0_KIe#UeaI*CuP#9&A+BHye9Pf-Mazo5##3!K$@J&Sx$2(7WMgeE9r!d|Sfi z`i`uC{*WD^-R{p|);C@vV>P2!iLQFzN`zy%cQ-BH*^c>no6D1@cH`DmyLWkU<>){B zG&d|pj9a#SW0qC_glW5?->%+Yi+g3yFV1`+1ap?bk_@33C8L8D2iE_D{r(HgGp4np z`NXod-!2oPqGw)osc8YI(n~gxhK*QhnHUm%Tnau<{SzJ|DU}oKFGqts74L=o$U_JD z-|*D%AWks<|x~%NAQWI zJQc^kn7Dh8i6BjklrL9#HfiJ>^0p-Jx#ax~pGNla)R=PQ?7%%h8b)m9jsL%_VpEtT zDCcQ?C#Qclh2=o*5{JU(kc;D~1#n2jZf_-jp^-1jpT?xg2j;1HkS-yyI+eug3f%5N z)-x1&1G$|%bqa?=B66}?8sg#2_*^LQ#!?szR#1QkmFrKsf-G?4#V3ssz&{kg_2enL z@`>4>D^6lAF`rmOSec$`R{o5(6;B2GCHi-AgZuxEA3TpjUO@-<59a@=++aNg|0R7z z``=N;IRBI?##6{E_`y`sKbR`y2DcC92g`j5efpn5pIGsKaOfhdL2T z>>>6N`-mSx`9wH*M|=v_Aub+deNI-o{}!=JhkUlp`Q2fsoC+zV0|F~8zdS7JkGx28 zY(s28kc}Wlg!@)=XDpiDji$c&)5>*ov9SK0`3CbAIA8Zs8FA_*UOX7sbKboS@i}j{ zMjdFvtNt@znPjFTYW0?x-IeLE4*JS%D!&^6Hq(rVwIX~Sy>-gBO%f~|v%E8FPA!_c znsuU=X5qCU?o{H&Zj_z!aoRmwikZg@r=L;l!NQrxlvgY(g`aS2;skydB-OHI*++#C zzVkmW*-zF3QR{vfkMy3QqV#HXT^sy3~WAom~9>|bz{m#4m^fI_N5A>_vsmB|egOL+@ z8=(L7`Lmrpb;t?d8Z-5k2#Xd7B74&=;d$#tx4Tw-kZ9H>L|-qAe?3)2zYmU>qsqzn{~Un(Y_m0_jN zh0^wSFG0_`6*8kjh@_MEM&xOTaH&51T4s9_^rzo%F`m@~&w|}|1l#H`wSoWpG_z_@ zUJMJd_^}u{xo>Lb^);XvmirKran@y-qg4=ilV;5FC z=DB6xay0M3`eyq+>w`iB&M9g(Un|Ax$grD<4pKZFA?3bZB85>-R)%O*1-88jnc`}g zjmN9Kimq4|!kTt9t(~6=J6dE(lTI62FLWPJo|%r7!tw#3ffQNUaT!ZoN-*HZN_U!? zf^Bp54fy<6hl7DJ3(t`8c0S(mOw`!tSRH6;w&+m-o|+VXjJFn|c8qw-7*7duCr?_u z;K~zhaW5zu5-7roF01KtY`bv!6z`17gH!0AVE@e-VGcyj3(j@yYeB}33!ao(im;Y_ zS95lR7-_#<|Im826)|ToDAi`agh7#Sc)Wf)hF|jRy4F#S<7VgAzQ5Cs2xXTC^pa*Q z-Q2?QJlp`B*r>}XBI8;}=`)Nifqq&?_@qQJmY3@cJ3PJ_oHy~JryHJP*mdF(cWV`b zGh2y~-a=GWoP3fyryJIlg;EXoG@SaqnHucXfuF+)CrLDJMGxp1ny;$I{JWGiEl*IM~4Jn4lo1D8UIgglTo6g9e9^ght!2EdY zOUP;%I#VF(gzWkZi@uZw%-%aZ^FTrsUSH=P(d(*&XPGD=VxkyF7ThwDx)?+3SI zKKY>N^>M8aOJRPruWq}i5Sj6*No_q+>n~`^#bPn4E9W69-OKgSFszHZFf3 zG<`+97>{#2-;3;PAk*r-T>i8GcDw5=-`4Ma{@X9{&``7XN-i2T5lXXtZPJMeW>9-jLvIJK$jtss%9+%pV z-!l@A%(E0jg=KCNzVRW#ypLPhB$0J(X^DFBw>>!MG;C6O?p+lA&hC}I?0{GCy7mh4 zJHYk<=StC`=Lqp{I27@7DLM>oi>K5zVZG7JA7ht^AXX)U^2T<f8 zk?NJuxooO!X3_?M>b=R85mF?ZOx3X*Xn?uSyo-X+ay+CM8Xi!UptbF_le1ALf_5bO z9?I*2_pyxc>KsMbMBl2JaF(nq?JsK9Karw1yx>63vUcPyyV0i6Dg&QbPQp~vznpjf z4qN}9^G=Q@<>#F#K_`5P9R!0Q0owm_@R|OzgU|Jg2j3rO{NEk*ivGb6K*9eVRp<|< zgY^{s|Fr!}a*A=jL=}2p>i#yHh*>ee*@_JV}JXzY2?2YBp6@%hban+zGV5} z_QAYDUqQdr|D|??o`Ndm6jagvpHhXsg8ozeFUc#~2UEF+P08&11Z2rSL;@`S2T7d? ACjbBd diff --git a/src/vai_lab/examples/xml_files/KNN-classification_demo.xml b/src/vai_lab/examples/xml_files/KNN-classification_demo.xml index 5f2ada64..7dc49229 100644 --- a/src/vai_lab/examples/xml_files/KNN-classification_demo.xml +++ b/src/vai_lab/examples/xml_files/KNN-classification_demo.xml @@ -7,10 +7,10 @@ [(350.0, 50), 0, {}] - - - - + + + + diff --git a/src/vai_lab/examples/xml_files/SVR_demo.xml b/src/vai_lab/examples/xml_files/SVR_demo.xml index 66affb74..2e8ceb8e 100644 --- a/src/vai_lab/examples/xml_files/SVR_demo.xml +++ b/src/vai_lab/examples/xml_files/SVR_demo.xml @@ -7,10 +7,10 @@ [(350.0, 50), 0, {}] - - - - + + + + diff --git a/src/vai_lab/examples/xml_files/canvas_demo.xml b/src/vai_lab/examples/xml_files/canvas_demo.xml index 439fbedc..afd2827b 100644 --- a/src/vai_lab/examples/xml_files/canvas_demo.xml +++ b/src/vai_lab/examples/xml_files/canvas_demo.xml @@ -7,7 +7,7 @@ [(350.0, 50), 0, {}] - + diff --git a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml index 778d8f54..997202a7 100644 --- a/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml +++ b/src/vai_lab/examples/xml_files/k-mean_clustering_demo.xml @@ -7,8 +7,8 @@ [(350.0, 50), 0, {}] - - + + diff --git a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml index 64727b3d..f3f6aee6 100644 --- a/src/vai_lab/examples/xml_files/random_forest_class_demo.xml +++ b/src/vai_lab/examples/xml_files/random_forest_class_demo.xml @@ -7,10 +7,10 @@ [(350.0, 50), 0, {}] - - - - + + + + [(350.0,50),0,{}] diff --git a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml index 671d4411..56aa8de1 100644 --- a/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml +++ b/src/vai_lab/examples/xml_files/ridge-scalar-ridge_demo.xml @@ -7,10 +7,10 @@ [(350.0, 50), 0, {}] - - - - + + + + From 0124cfc766a4e2ae22734b52603e07a3cb51a124 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Fri, 29 Sep 2023 12:25:55 +0300 Subject: [PATCH 41/48] Fixed issue with relative paths --- src/vai_lab/_import_helper.py | 23 ++++++++++++++++++++++- src/vai_lab/utils/plugins/MainPage.py | 6 +++--- 2 files changed, 25 insertions(+), 4 deletions(-) diff --git a/src/vai_lab/_import_helper.py b/src/vai_lab/_import_helper.py index e56d2118..3095ddbc 100644 --- a/src/vai_lab/_import_helper.py +++ b/src/vai_lab/_import_helper.py @@ -43,4 +43,25 @@ def rel_to_abs(filename: str) -> str: filename = path.normpath(path.join(get_lib_parent_dir(), filename)) elif filename[0] == "/" or (filename[0].isalpha() and filename[0].isupper()): filename = filename - return filename \ No newline at end of file + return filename + +def abs_to_rel(filename: str) -> str: + """Checks if path is relative or absolute + If absolute, converts path to relative if possible + If relative, returns itself + """ + if filename[0] == ".": + #Relative path + return filename + elif filename[0].isalpha() and filename[0] != get_lib_parent_dir()[0]: + #Different drive -> Absolute path + return filename + else: + #Same drive not relative + _folder = path.relpath(filename, get_lib_parent_dir()) + if _folder[:2] == '..': + # Absolute path + return filename + else: + # Relative path + return path.join('.',_folder) diff --git a/src/vai_lab/utils/plugins/MainPage.py b/src/vai_lab/utils/plugins/MainPage.py index ad30fc60..ca78db04 100644 --- a/src/vai_lab/utils/plugins/MainPage.py +++ b/src/vai_lab/utils/plugins/MainPage.py @@ -6,7 +6,7 @@ import pandas as pd from vai_lab.utils.plugins.dataLoader import dataLoader -from vai_lab._import_helper import get_lib_parent_dir, rel_to_abs +from vai_lab._import_helper import get_lib_parent_dir, abs_to_rel _PLUGIN_READABLE_NAMES = {"main": "default", "main page": "alias", @@ -311,7 +311,7 @@ def start_dataloader(self): # Infers by default, should it be None? data[variable] = pd.read_csv(filename) isVar[i] = 1 - self.controller.xml_handler.append_input_data(variable, rel_to_abs(filename)) + self.controller.xml_handler.append_input_data(variable, abs_to_rel(filename)) if i == 0: self.controller.Data.set(True) if any(isVar[1::2]) and ( @@ -337,7 +337,7 @@ def upload_data_path(self): title='Select a folder', mustexist=True) if folder is not None and len(folder) > 0: - self.controller.xml_handler.append_input_data('X', rel_to_abs(folder)) + self.controller.xml_handler.append_input_data('X', abs_to_rel(folder)) def upload_data_folder(self): """ Stores the directory containing the data that will be later loaded From 614cdf670d43b3d25d2f8178435cec8df99007e3 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Tue, 3 Oct 2023 14:51:52 +0300 Subject: [PATCH 42/48] Solve issue with relative paths --- src/vai_lab/Data/xml_handler.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 79d45292..bbb1bb14 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -9,7 +9,7 @@ root_mod = path.dirname(path.dirname(path.dirname(__file__))) sys.path.append(root_mod) -from vai_lab._import_helper import get_lib_parent_dir +from vai_lab._import_helper import get_lib_parent_dir, rel_to_abs class XML_handler: @@ -567,7 +567,7 @@ def append_input_data(self, if save_dir_as_relative: data_dir = data_dir.replace(self.lib_base_path, "./") data_dir = data_dir.replace("\\", "/") - if path.exists(path.dirname(data_dir)): + if path.exists(path.dirname(rel_to_abs(data_dir))): plugin_elem.set('file', data_dir) else: plugin_elem.set('module', data_dir) From 8b5471f384b29ac0bfcbb650b5a4612e8a1c30f1 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 5 Oct 2023 10:40:13 +0300 Subject: [PATCH 43/48] Restructure conditions --- src/vai_lab/_plugin_templates.py | 53 ++++++++++++++++---------------- 1 file changed, 27 insertions(+), 26 deletions(-) diff --git a/src/vai_lab/_plugin_templates.py b/src/vai_lab/_plugin_templates.py index 2cf0ad72..437ff74c 100644 --- a/src/vai_lab/_plugin_templates.py +++ b/src/vai_lab/_plugin_templates.py @@ -104,32 +104,33 @@ def _reshape(self, data, shape): def _parse_options_dict(self,options_dict:Dict): for key, val in options_dict.items(): - if type(val) == str and val.replace('.', '').replace(',', '').isnumeric(): - cleaned_opts = [] - for el in val.split(","): - val = float(el) - if val.is_integer(): - val = int(val) - cleaned_opts.append(val) - options_dict[key] = cleaned_opts - elif type(val) == str and val.lower() in ('yes', 'true'): - options_dict[key] = True - elif type(val) == str and val.lower() in ('no', 'false'): - options_dict[key] = False - elif type(val) == str and val.lower() in ('none'): - options_dict[key] = None - elif val == 'X': - options_dict[key] = self.X - elif val == 'Y': - options_dict[key] = self.Y - elif val == 'X_test': - options_dict[key] = self.X_test - elif val == 'Y_test': - options_dict[key] = self.Y_test - elif key.lower() == 'x': - options_dict[key] = self.X - elif key.lower() == 'y': - options_dict[key] = self.Y + if type(val) == str: + if val.replace('.', '').replace(',', '').isnumeric(): + cleaned_opts = [] + for el in val.split(","): + val = float(el) + if val.is_integer(): + val = int(val) + cleaned_opts.append(val) + options_dict[key] = cleaned_opts + elif val.lower() in ('yes', 'true'): + options_dict[key] = True + elif val.lower() in ('no', 'false'): + options_dict[key] = False + elif val.lower() in ('none'): + options_dict[key] = None + elif val == 'X': + options_dict[key] = self.X + elif val == 'Y': + options_dict[key] = self.Y + elif val == 'X_test': + options_dict[key] = self.X_test + elif val == 'Y_test': + options_dict[key] = self.Y_test + elif key.lower() == 'x': + options_dict[key] = self.X + elif key.lower() == 'y': + options_dict[key] = self.Y return options_dict def _clean_options(self): From 9eea13222be5225c6bb314a2e9cf30107163b08a Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 5 Oct 2023 11:00:45 +0300 Subject: [PATCH 44/48] Fix options reading from XML and change data reading for looping --- src/vai_lab/Core/vai_lab_core.py | 8 ++++---- src/vai_lab/Data/xml_handler.py | 26 +++++++++++++++++--------- 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/vai_lab/Core/vai_lab_core.py b/src/vai_lab/Core/vai_lab_core.py index 3eb4ad19..ebe9ca5d 100644 --- a/src/vai_lab/Core/vai_lab_core.py +++ b/src/vai_lab/Core/vai_lab_core.py @@ -47,9 +47,9 @@ def load_config_file(self, filename: Union[str,List,Tuple]): self._xml_handler.load_XML(filedir) self._initialised = True - def _load_data(self, module = 'Initialiser') -> None: + def _load_data(self, specs, module = 'Initialiser') -> None: """Loads data from XML file into Data object""" - init_data_fn = self._xml_handler.data_to_load(module) + init_data_fn = self._xml_handler.data_to_load(modules=specs, module=module) if module not in self.data.keys(): self.data[module] = Data() if isinstance(init_data_fn, str): @@ -66,7 +66,7 @@ def _execute_module(self, specs): mod: ModuleInterface = import_module(globals(), specs["module_type"]).__call__() mod._debug = self._debug mod.set_avail_plugins(self._avail_plugins) - self._load_data(specs["name"]) + self._load_data(specs, specs["name"]) mod.set_data_in(self.data[specs["name"]]) mod.set_options(specs) print("\t"*self.loop_level @@ -185,7 +185,7 @@ def run(self): self._initialise_with_gui() print("Running pipeline...") if len(self._xml_handler.loaded_modules) > 0: - self._load_data() + self._load_data(self._xml_handler.loaded_modules) self._init_status(self._xml_handler.loaded_modules) self._execute(self._xml_handler.loaded_modules) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index bbb1bb14..0e7e5697 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -200,9 +200,10 @@ def _load_options(self, element: ET.Element, parent: dict) -> None: """ for child in element: if child.text is not None: - val = self._parse_text_to_list(child) - val = (val[0] if len(val) == 1 else val) - parent["options"][child.tag] = val + try: + parent["options"][child.tag] = literal_eval(child.text.strip()) + except Exception as exc: + parent["options"][child.tag] = child.text.strip() for key in child.attrib: if key == "val": @@ -718,10 +719,15 @@ def append_pipeline_loop(self, xml_parent_element.append(new_loop) - def _get_data_structure(self, module) -> Dict[str, Any]: - data_struct = self._find_dict_with_key_val_pair( - self.loaded_modules[module], - "class", "data") + def _get_data_structure(self, modules, module) -> Dict[str, Any]: + try: + data_struct = self._find_dict_with_key_val_pair( + modules[module], + "class", "data") + except Exception as exc: + data_struct = self._find_dict_with_key_val_pair( + modules, + "class", "data") assert len(data_struct) < 2, \ "Multiple data with same ID, please check XML" @@ -734,8 +740,10 @@ def _get_data_structure(self, module) -> Dict[str, Any]: return out #@property - def data_to_load(self, module='Initialiser') -> Dict[str, str]: - return self._get_data_structure(module)["to_load"] + def data_to_load(self, modules=False, module='Initialiser') -> Dict[str, str]: + if not modules: + modules = self.loaded_modules + return self._get_data_structure(modules, module)["to_load"] # Use case examples: From 19f2052a7dfb0dc3e54d6ddd0fea617c48e9bcc1 Mon Sep 17 00:00:00 2001 From: Carlos Sevilla Salcedo Date: Thu, 5 Oct 2023 11:10:13 +0300 Subject: [PATCH 45/48] Fixed issue with options in multiple lines --- src/vai_lab/Data/xml_handler.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/vai_lab/Data/xml_handler.py b/src/vai_lab/Data/xml_handler.py index 0e7e5697..9c0a62c4 100644 --- a/src/vai_lab/Data/xml_handler.py +++ b/src/vai_lab/Data/xml_handler.py @@ -203,7 +203,8 @@ def _load_options(self, element: ET.Element, parent: dict) -> None: try: parent["options"][child.tag] = literal_eval(child.text.strip()) except Exception as exc: - parent["options"][child.tag] = child.text.strip() + val = self._parse_text_to_list(child) + parent["options"][child.tag] = (val[0] if len(val) == 1 else val) for key in child.attrib: if key == "val": From b1a60aa0d14cf70d7187f1505265575ca3b8b951 Mon Sep 17 00:00:00 2001 From: Teemu Ruokolainen Date: Thu, 5 Oct 2023 11:36:30 +0300 Subject: [PATCH 46/48] MOD: update to Python 3.11 --- .github/workflows/pytest.yml | 2 +- .github/workflows/pythonpackage.yml | 47 ----------------------------- .gitignore | 1 + dev-environment.yml | 2 +- environment.yml | 2 +- pyproject.toml | 2 +- 6 files changed, 5 insertions(+), 51 deletions(-) delete mode 100644 .github/workflows/pythonpackage.yml diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index ea62fcce..826f2aa8 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -22,7 +22,7 @@ jobs: strategy: matrix: os: [ubuntu-latest, macos-latest, windows-latest] - python-version: ["3.10", "3.11"] + python-version: ["3.11"] fail-fast: false steps: diff --git a/.github/workflows/pythonpackage.yml b/.github/workflows/pythonpackage.yml deleted file mode 100644 index e7337f16..00000000 --- a/.github/workflows/pythonpackage.yml +++ /dev/null @@ -1,47 +0,0 @@ -name: CI Tests - -on: - pull_request: - workflow_dispatch: - inputs: - logLevel: - description: 'Log level' - required: true - default: 'warning' - tags: - description: 'Test scenario tags' - -jobs: - build: - runs-on: ${{ matrix.os }} - strategy: - matrix: - os: [ubuntu-latest, macos-latest, windows-latest] - python-version: ["3.10", "3.11"] - defaults: - run: - shell: bash - steps: - - uses: actions/checkout@v3 - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - architecture: x64 - - name: Install dependencies - run: | - python -m pip install --upgrade pip - python -m pip install . - - name: Test with pytest - # uses: GabrielBB/xvfb-action@v1.6 - uses: hankolsen/xvfb-action@dcb076c1c3802845f73bb6fe14a009d8d3377255 - with: - run: | - pip install pytest - pip install pytest-cov - pytest --cov=./src/vai_lab -v --cov-report=xml:./coverage.xml --cov-report term-missing - - - # Note: hankolsen/xvfb-action@dcb076c1c3802845f73bb6fe14a009d8d3377255 is a - # PR waiting to be merged to GabrielBB/xvfb-action@v1.6 to update xvfb-action - # to node.js 16. After the merge, switch back to GabrielBB/xvfb-action@v1.6 \ No newline at end of file diff --git a/.gitignore b/.gitignore index d80f36db..43c14080 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,7 @@ __pycache__/ **/*.pyc *.py[cod] *$py.class +*-env/ # C extensions *.so diff --git a/dev-environment.yml b/dev-environment.yml index 3042da6f..4100690c 100644 --- a/dev-environment.yml +++ b/dev-environment.yml @@ -2,7 +2,7 @@ name: dev-vai-lab-env channels: - conda-forge dependencies: - - python=3.10 + - python=3.11 - c-compiler - pip - pip: diff --git a/environment.yml b/environment.yml index e65480a8..3b78aaa5 100644 --- a/environment.yml +++ b/environment.yml @@ -2,7 +2,7 @@ name: vai-lab-env channels: - conda-forge dependencies: - - python=3.10 + - python=3.11 - c-compiler - pip - pip: diff --git a/pyproject.toml b/pyproject.toml index f1c9a7c5..cca9659f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -15,7 +15,7 @@ classifiers = [ 'Programming Language :: Python :: 3.10', 'Programming Language :: Python :: 3.11' ] -requires-python = ">=3.8" +requires-python = ">=3.11s" version = "0.0.dev4" description = "AI assisted Virtual Laboratory framework." From 03b6b2a6a1a874f83ea7f310e6c56e64dd1a24a5 Mon Sep 17 00:00:00 2001 From: Teemu Ruokolainen Date: Thu, 5 Oct 2023 11:40:26 +0300 Subject: [PATCH 47/48] FIX: python3.11 for sphinx --- .github/workflows/sphinx.yml | 2 +- pyproject.toml | 3 +-- sphinx-environment.yml | 2 +- sphinx-requirements.txt | 1 - 4 files changed, 3 insertions(+), 5 deletions(-) diff --git a/.github/workflows/sphinx.yml b/.github/workflows/sphinx.yml index 6a227951..d974f1cf 100644 --- a/.github/workflows/sphinx.yml +++ b/.github/workflows/sphinx.yml @@ -8,7 +8,7 @@ jobs: - uses: actions/checkout@v2 - uses: actions/setup-python@v2 with: - python-version: '3.10' + python-version: '3.11' # pip cache - name: pip cache diff --git a/pyproject.toml b/pyproject.toml index cca9659f..21e92a40 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,10 +12,9 @@ authors = [ readme = "README.md" classifiers = [ "License :: OSI Approved :: MIT License", - 'Programming Language :: Python :: 3.10', 'Programming Language :: Python :: 3.11' ] -requires-python = ">=3.11s" +requires-python = ">=3.11" version = "0.0.dev4" description = "AI assisted Virtual Laboratory framework." diff --git a/sphinx-environment.yml b/sphinx-environment.yml index 45f3b5ae..8950f8ab 100644 --- a/sphinx-environment.yml +++ b/sphinx-environment.yml @@ -2,7 +2,7 @@ name: sphinx-env channels: - conda-forge dependencies: - - python=3.10 + - python=3.11 - sphinx - sphinx-rtd-theme - myst-parser diff --git a/sphinx-requirements.txt b/sphinx-requirements.txt index 4cbfb8b7..56c8516d 100644 --- a/sphinx-requirements.txt +++ b/sphinx-requirements.txt @@ -1,4 +1,3 @@ -# requirements.txt for sphinx sphinx sphinx-rtd-theme myst-parser From bfce8f3ec7b72451693553877843bd1c9b8bc58c Mon Sep 17 00:00:00 2001 From: Teemu Ruokolainen Date: Thu, 5 Oct 2023 11:43:22 +0300 Subject: [PATCH 48/48] DEL: delete obsolete workflow --- .github/workflows/documentation.yml | 26 -------------------------- 1 file changed, 26 deletions(-) delete mode 100644 .github/workflows/documentation.yml diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml deleted file mode 100644 index e592d3dd..00000000 --- a/.github/workflows/documentation.yml +++ /dev/null @@ -1,26 +0,0 @@ -name: Docs -on: [push, pull_request, workflow_dispatch] -jobs: - docs: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - with: - python-version: '3.10' - - name: Install dependencies - run: | - python -m pip install . - python -m pip install sphinx sphinx-rtd-theme myst-parser - - name: Sphinx build - run: | - sphinx-apidoc --templatedir docs/templates/apidoc -o docs/source src/vai_lab - sphinx-build -M html docs/source docs/build - - name: Deploy - uses: peaceiris/actions-gh-pages@v3 - if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} - with: - publish_branch: gh-pages - github_token: ${{ secrets.GITHUB_TOKEN }} - publish_dir: docs/build/html - force_orphan: true \ No newline at end of file