Xppmask #373
Open
Xppmask #373
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
March 26, 2024 17:55
Date: Tue Mar 26 14:22:07 2024 -0700
write interfaces
commit fe9d1e7
Author: O hoidn <[email protected]>
Date: Mon Mar 25 16:29:59 2024 -0700
remove unneeded symlinks
commit 209a91b
Author: O hoidn <[email protected]>
Date: Mon Mar 25 16:28:34 2024 -0700
New modules for the xppmask pipeline code
- analysis_tasks.py: Contains functions for calculating EMD values, p-values, and generating signal/background masks
- histogram_analysis.py: Contains functions for identifying connected clusters and creating background masks
- histogram_utils.py: Contains functions for calculating histograms from pixel data
- pvalues.py: Contains functions for calculating EMD values and p-values from histograms
- completed histogram calculation, emd calculation, p value calculation, and mask generation - Reorganize output files and directories by experiment and run - Add input/output file path checks and error handling - Save PNG images of intermediate results like EMD, p-values, masks - Improve input parameter validation - Refactor tests to use new interfaces and cover more cases - Update configuration file with example parameters
|
This includes the implementation classes for four tasks (histogram, EMD, p values, and masks) and a series of tests that runs them in sequence. If you want to run the tests you'll have to download the test data, data.npz: https://drive.google.com/file/d/189xj8J07qAgzgKgwNi-rwutE0yg1-Nqf/view?usp=sharing The pump probe delay curve calculation part is still in progress. |
Test loading data from XPP experiments Add LoadData class to load and save data from XPP experiments Add utility functions for preprocessing XPP data using smd Update xppmask.py Add LoadData class to load and preprocess data Refactor MakeHistogram to use numpy data files Add save_histogram_image method to save histogram as image Update test configs
- Add test for generating histograms from npz file - Update tests to return signal/background mask paths - Run tests with LoadData output and pre-existing npz file - Refactor energy filtering and add TimeTool filtering in xpploader - Save additional summary info and avg frame in LoadData - Return histogram and mask file paths from save methods Improves test coverage, enables testing with existing data, and enhances data processing with filtering options and more detailed summaries/visualizations.
- Pass I0 data to PumpProbeAnalysis.run - Use provided bg_mask in calculate_signal_and_background - Normalize signals and std_devs using I0 and laser masks - Remove unused extract_stacks_by_delay method - Import scipy.stats.norm for p-value calculation - Autogenerate mask paths and handle missing config keys - Update test_pump_probe_analysis with I0 data and root_dir - Comment out irrelevant assert statements in test
Refactor test_xppmask.py to load YAML config instead of manually defining configs Update test functions to use loaded YAML config and assert on output file paths Add output directory structure based on task names in xppmask.py Use consistent function names (e.g., make_histogram instead of generate_histograms) Simplify initialization of task classes by removing required keys check Save reports and summaries with consistent file names Handle potential errors in generate_background_mask Update xppmask.yaml to use make_histogram and adjust threshold
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is not yet fully integrated. For example, the new implementation code doesn't use package-relative imports so you'll have to run things from the btx/btx/processing/tests directory