Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential improvements for _SearchIndexer #741

Open
3 tasks
vyasr opened this issue Apr 12, 2022 · 3 comments
Open
3 tasks

Potential improvements for _SearchIndexer #741

vyasr opened this issue Apr 12, 2022 · 3 comments
Labels
pinned Instructs stale bot to ignore this issue.

Comments

@vyasr
Copy link
Contributor

vyasr commented Apr 12, 2022

#683 replaced the old Collection class with a stripped down _SearchIndexer that does just enough for signac's internal use cases. However, that PR left a few tasks outstanding that we should consider:

  • Replacing the _SearchIndexer with a plain dict and free functions. I don't anticipate that these changes will have a significant effect on the code base, but it's worth considering since it may improve clarity. The _SearchIndexer has essentially no state beyond that of a normal dict, and while its contents are more restricted there is no particular enforcement, so conceptually using a free function might be clearer. Additionally, it is possible to create the object in an invalid state with the current approach. Some additional discussion is here.
  • Remove the potentially unnecessary JSON normalization as discussed in Replace Collection with lightweight _SearchIndexer. #667 (comment) (currently found here). I believe that this function is only invoked in places that have already performed suitable normalization, but we should verify this before making any final changes.
  • Renaming the _SearchIndexer. This is the lowest priority task and would be moot if we reimplemented it as a dict with free functions.
@bdice
Copy link
Member

bdice commented Apr 23, 2022

These are good ideas. Since all this is internal, this work can be done post-2.0.

@stale
Copy link

stale bot commented Jul 31, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 31, 2022
@stale stale bot closed this as completed Sep 21, 2022
@bdice bdice added the pinned Instructs stale bot to ignore this issue. label Sep 21, 2022
@bdice bdice added this to the v2.1.0 milestone Sep 21, 2022
@bdice bdice reopened this Sep 21, 2022
@bdice bdice removed this from the v2.1.0 milestone Sep 21, 2022
@stale stale bot removed the stale label Sep 21, 2022
@vyasr
Copy link
Contributor Author

vyasr commented May 6, 2023

FWIW just tried running the test suite with the normalization removed, and everything passed. I also tried running it throwing an error whenever the normalization changed the input, and the only difference I observed was a conversion from tuples to lists. In terms of performance, the normalization costs a few microseconds for a trivial filter (e.g.{'a': [1]}) rising to 10s of microseconds for filters including more inputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pinned Instructs stale bot to ignore this issue.
Projects
None yet
Development

No branches or pull requests

2 participants