Update to HDF5 and PEtab v2

[sebapersson]

This is a large update to the repo. As most test files have changed I provide an overview here.

Changes in neural network tests

The neural networks tests (net tests) are still the same. The difference is that the input, parameters and output are now provided in HDF5 file format. The saved arrays are in row-major, following PyTorch indexing. I have tested and confirmed consistent results between Julia and PyTorch.

Changes in hybrid tests

I have now updated the yaml file to version 2. Basically, now the file looks like:

format_version: 2
problems:
  - model_files:
      location: "lv.xml"
      language: "sbml"
    measurement_files:
      - "measurements.tsv"
    observable_files:
      - "observables.tsv"
    condition_files:
      - "conditions.tsv"
    mapping_files:
      - "mapping_table.tsv"
extensions:
  petab_sciml:
    net1:
      file: "net1.yaml"
      hybridization:
        output: "ode"
        input: "ode"
      parameters: "net1_ps.h5"
parameter_file: "parameters_ude.tsv"

As discussed each network now has its own parameter file, e.g. net1_ps.h5. Additionally, I have:

• Fixed proper PEtab formatting for the observable table
• Changed such that neural network input is hdf5 (see test case 015 and 016)
• I have updated the mapping table to follow the format in PEtab YAML formatting #4.

Note that settings for neural networks (such as whether to estimate the parameters or not) can be specified in the parameters table. The only thing left is the point regarding condition table in #4, but I will open a separate issue on this.

I have run all the test-cases through PEtab.jl.

@FFroehlich and @dilpath if this looks good I will merge this, close #4, and open a new issue with the points that are left to address.

[dilpath]

Looks good to me! Are there read/write model+HDF5 parameter methods for Python yet? If not, can assign that to me.

[sebapersson]

Looks good to me! Are there read/write model+HDF5 parameter methods for Python yet? If not, can assign that to me.

I have functions for reading parameters when testing against Julia, basically in the HDF5 the parameters are already in PyTorch order, but I can to the TODO list to add more formal functions for this (see #11 and testing the roundtrip). Also functions for reading the model are needed.