Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Update legacy json and param conversion function for new json format #772

Merged

Conversation

gtdang
Copy link
Collaborator

@gtdang gtdang commented May 16, 2024

This changes the convert_to_hdf5 function to convert_to_json.

  • The function converts legacy flat json and param files to the new hierarchical json format.
  • It also adds network connectivity. By default it uses the Jones 2009 model for the connectivity but supports the other pre-defined network models. No network connectivity can be added by supplying None.

Question:

  • Should the default be None instead of the Jones 2009 for adding network connectivity?

@gtdang gtdang requested a review from ntolley May 16, 2024 22:06
doc/whats_new.rst Outdated Show resolved Hide resolved
network_connectivity='jones_2009_model',
include_drives=True,
overwrite=True):
"""Converts legacy json or param format to hierarchical json format
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry I missed the discussion. Do you guys have a specification of the hierarchical json format? how will you store connectivity etc? Or is it an intermediate format before going to hdf5?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format is Network object converted to a dictionary format. So the connectivities are just the same hierarchical dictionary format as the Network Object. Pretty much everything is preserved except numpy arrays need to be converted to lists when written to json and converted back upon read-in.

This is not an intermediate format before going to hdf5. All the network connections, drives, cell parameters can be saved and loaded with this format.

What is not included with this format is the ability to save outputs (the optional recordings during simulation saved to the Network). [Is there a better term to use than "outputs"?] We plan to use the hdf5 format for the saving of network outputs. For that hdf5 file, a version of this json will be saved as a single dataset in the hdf5 (A string json binarized), that way the network configurations are paired with the outputs.

@gtdang gtdang marked this pull request as ready for review May 23, 2024 18:24
@jasmainak
Copy link
Collaborator

Do these new json format files work with the GUI?

@gtdang
Copy link
Collaborator Author

gtdang commented Jun 12, 2024

Do these new json format files work with the GUI?

No, not yet. I've been working on it but got side tracked with the CI issues. Should be able to refocus on it next week.

@gtdang gtdang force-pushed the update-param-conversion-to-json branch from 77d2281 to b4d565d Compare June 26, 2024 19:32

Parameters
----------
params_fname : str or Path
Path to file
out_fname: str
Path to output
network_connectivity: str or None, default:' jones_2009_model'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
network_connectivity: str or None, default:' jones_2009_model'
model_template: str or None, default:' jones_2009_model'

?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made this argument name change. Thanks!

assert net_json == net_params

def test_no_network_connectivity(self, tmp_path):
"""Tests conversion with no network connectivity model"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good edge case to check!

Copy link
Contributor

@ntolley ntolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good on my end, thanks @gtdang !!

@ntolley ntolley merged commit 4b7fdd7 into jonescompneurolab:master Jun 27, 2024
12 checks passed
@gtdang gtdang deleted the update-param-conversion-to-json branch August 29, 2024 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

tutorial: convert tutorial json and param files to hierarchical json
3 participants