Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POSYDON v2.0.0-pre1 #451

Merged
merged 321 commits into from
Dec 12, 2024
Merged

POSYDON v2.0.0-pre1 #451

merged 321 commits into from
Dec 12, 2024

Conversation

maxbriel
Copy link
Collaborator

This PR is the prerelease for the POSYDON v2 code, as described in Andrews et al. (submitted).

It has been used to produce the reprocessed V1 dataset such that they're compatible with the POSYDON v2 code.
POSYDON v1 grids for v2 code
https://zenodo.org/records/14205146

POSYDON v1 super-Eddington accretion grid for v2 code
https://zenodo.org/records/14216817

mkruckow and others added 30 commits February 8, 2023 11:15
* Update README.md

* Update install.rst

* Update install.rst
* raise warning instead of breaking post processing

* spelling mistake

* use warnigns module

* debug

* spelling

* catch other warinings and exception without braking

* prevent type error from breaking code

* fix grammar

* prevent CC2 from breakig whne gamma_center limit from star_2 is reached

* fix syntax error

* prevent the code from breaking in check_state_of_star

* fix
* Adding psygrid option for keeping tracks without profiles.

* Allowing for missing profiles in single grids.

* By default do not include stars without a final profile in a single star grid.
* support for different metallicities

* Adding metallicity to BinaryPopulation. Forcing constant Z across grid steps.

* fix bug

* fix initial X, Y, Z

* Adding metallicity, uncomment previously broken kwargs in the ini

* Added the SyntheticPopulation class to evolve mutliple BinaryPopulations. Added MPI options to inifile.

* Add the common function to check for metallicity.

* add first step of the post processing pipeline

* bugfix

* fix bug

* debug

* bug fix

* fix bug

* fix bug

* bug fix

* fix bug

* rub test

* add step 2 of pipleline

* add logs

* rebmove break

* debug

* add plotting step to pipeline

* debug

* debug

* debug

* debug

* debug

* add post processing and train interpolators steps

* some cleaning

* cleaning

* cleaning

* cleaning

* debug

* cleaning

* clearning

* cleaning

* cleaning

* debug

* debug

* cleaning

* cleaning

* cleaning

* add support for CO-HMS and CO-HMS_RLO grids

* cleaning

* debug

* debug

* bug

* debug

* debug

* debug

* cleanig

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* debug

* add step 7 to export POSYDON pop synth dataset

* debug

* debug

* support reruns creation

* cleaning

* debug

* debug

* remove grid processing pipeline from this PR

---------

Co-authored-by: ka-rocha <[email protected]>
Correct typo: .fale. -> .false.
* correct typo: profile1 -> profile2

* Fix missing composition values on grid creation

A try to check whether better defined types solve the problem.

* Debug output

* more debugging

* debug N_Z, where_to_add

* try fix

where_to_add in self.initial_values[i]

* remove debug outputs

* Fix for issue introduced in PR#36

Add "accept_missing_profile" to PROPERTIES_TO_BE_CONSISTENT
* add post processing piepeline

* header

* copy the ini file togeather with the rerun points

* fix bug rerun

* automatically infer cluster name

* debug

* debug

* debug

* debug

* debug

* debug

* automatically replace zamsfilename

* add option to not start at RLO for the CO-HMS grid in case of reruns

* add all grid slices to post process

* make version an input

* debug step 2 slurm job by setting slurm_array var default to a number

* debug

* debug

* debug

* debug

* debug

* debug

* check quest setup

* debug

* debug

* debug

* debug

* parallelise step 2

* check that file exists before exporting the csv file, if they are missing print a warning and remove them from the csv file

* debug

* drop missing files is customisable for steps 2 to 7 and reruns

* debug

* debug

* debug

* debug

* increase walltime on yggdrasil because some HMS-HMS slices require more than 8 hours to process

* debug

* debug

* correct typo: profile1 -> profile2

* Fix missing composition values on grid creation

A try to check whether better defined types solve the problem.

* Debug output

* more debugging

* debug N_Z, where_to_add

* try fix

where_to_add in self.initial_values[i]

* remove debug outputs

* debug on quest

* cluster name is now an input

* include the random grids in the plotting

* debug

* Fix for issue introduced in PR#36

Add "accept_missing_profile" to PROPERTIES_TO_BE_CONSISTENT

* add ini file configuration of the processing pipeline

* add kyle's authorship

* move pipeline ini file outside bin directories to solve installation issue

* add rerun in pipeline_quest.ini

* debug opacity max rerun

---------

Co-authored-by: mkruckow <[email protected]>
* Update README.md

* Update install.rst

* Update install.rst

* Added three extra magnetic braking prescriptions in deffeq(), along with options to select from them, and added relevant parameters used in their calculations to this function's arguments. All calculated dOmega/dt from magnetic braking are in units of [yr-2].

* Added three extra magnetic braking prescriptions in deffeq(). Updated docstring, and conv_tau is now converted to years from sec in get_star_data

---------

Co-authored-by: Simone Bavera <[email protected]>
* Update run-pipeline
* Add (P)PISN to TF1

Add the new termination flags:
- 'Primary enters pair-instability regime'
- 'Secondary enters pair-instability regime'
- 'Primary enters pulsational pair-instability regime'
- 'Secondary enters pulsational pair-instability regime'

* Add (P)PISN to stable TF1

Add (P)PISN to pool of stable TF1

* Add (P)PISN to debug

Add (P)PISN to debug flag, too

* Add missing L2 flag

Add flag:
- 'overflow from L2 (R_L2) surface for q(=Macc/Mdon)>1, donor is star 2'

* Add missing L2 overflow

Add to unstable pool:
- 'overflow from L2 (R_L2) surface for q(=Macc/Mdon)>1, donor is star 2'
* Add initial_z to be read from directory name

Add initial_z to be read from directory name, which is the case for v2 and newer versions.

* Try to get metallicity form directory name

Try to get the metallicity value from the directory name in case, there is no history file.

* Correct typo

step_7 -> step_6

* Add to reset DISPLAY for plotting 

Add to reset DISPLAY for plotting step on a cluster to avoid problems when logged in with window mode on ssh.

* Check for initial RLO in join_grids

Check for initial RLO in join_grids, too.
Use a joint initial RLO boundary.

* Debug

Flush output for debugging

* Fix bug in adding too many boundary systems

Fix bug in adding too many boundary systems

* Remove debugging

Remove debugging

* Fix list entry replacement (psygrid.py)

Fix list entry replacement in psygrid.py

* Fix list entry replacement (termination_flags.py)

Fix list entry replacement in termination_flags.py

* Add STEPID

Add the number of the step to the slurm job-name

* Update last developement changes (#62)

* Fixed more dict syntax for kwargs in docs (#53)

In SingleStar object docs

* Fixed dict syntax for kwargs1, kwargs2 definitions (#52)

---------

Co-authored-by: Camille Liotine <[email protected]>

* account for suggestions 

- add else (nevertheless it will raise an error message later; I thought I predefined STEPID, but it looks like I put it out/or missed to put it in; using job_name here as content of the STEPID is a bad idea, because it is expected to be a single character and not a string)
- add the check for the length after the split (again, this would cause an error message later nevertheless)

* rename variable

e -> exists_already

* rename

e -> exists_already

---------

Co-authored-by: Camille Liotine <[email protected]>
* Add initial_z to be read from directory name

Add initial_z to be read from directory name, which is the case for v2 and newer versions.

* Try to get metallicity form directory name

Try to get the metallicity value from the directory name in case, there is no history file.

* Correct typo

step_7 -> step_6

* Add to reset DISPLAY for plotting 

Add to reset DISPLAY for plotting step on a cluster to avoid problems when logged in with window mode on ssh.

* Check for initial RLO in join_grids

Check for initial RLO in join_grids, too.
Use a joint initial RLO boundary.

* Debug

Flush output for debugging

* Fix bug in adding too many boundary systems

Fix bug in adding too many boundary systems

* Remove debugging

Remove debugging

* Fix list entry replacement (psygrid.py)

Fix list entry replacement in psygrid.py

* Fix list entry replacement (termination_flags.py)

Fix list entry replacement in termination_flags.py

* Add STEPID

Add the number of the step to the slurm job-name

* Update last developement changes (#62)

* Fixed more dict syntax for kwargs in docs (#53)

In SingleStar object docs

* Fixed dict syntax for kwargs1, kwargs2 definitions (#52)

---------

Co-authored-by: Camille Liotine <[email protected]>

* account for suggestions 

- add else (nevertheless it will raise an error message later; I thought I predefined STEPID, but it looks like I put it out/or missed to put it in; using job_name here as content of the STEPID is a bad idea, because it is expected to be a single character and not a string)
- add the check for the length after the split (again, this would cause an error message later nevertheless)

* rename variable

e -> exists_already

* rename

e -> exists_already

* Add run_pipeline.sh

- create the bash script "run_pipeline.sh", which starts all the setup steps with dependency

* Automatic kill on failure

set option to kill pending jobs, where the depended job failed

* Check all steps, which can run in parallel

- allow step_5 to go in parallel to step_3, too

---------

Co-authored-by: Camille Liotine <[email protected]>
…ed step (#45)

* Update common_functions.py

* Update step_detached.py

* Update step_mesa.py

* Update step_detached.py
* Adding gzip support when reading MESA grids.

* Adding GZIP support to psygrid metadata printing.

* Adding GZIP support to grid points files.

* Adding GZIP support to EEP files.

* Clarifications for GZIP files inside the code.

* Adding more clarification regarding support of GZIP files.

* Adding GZIP support to function converting MESA output to table.

* Adding GZIP support when inferring termination flag from MESA run output.

* Adding GZIP support in EEP class.

* Correcting types in io
move option "--kill-on-invalid-dep=yes" in the brackets
* started the step_isolated files (2 is for superclass with detached) and first changes in detached and in flow

* allowed for different psygrid for isoalted step, selected lists of stellar states, still need to solve absence of logR for mergers

* deleted old step_isolated

* made step_merging

* integrated the step_merged correctly in the flow

* before intorducing the option for not using log_R as a matching criteria, having the option do decide the parameters of the matching

* added most merger cases, still need to solve logR avsence in matching

* finished step_merged combinations

* made the new disrupted_step

* added spin of merged product

* made step_initially_single too and changes of structure after discussion with Konstantinos

* latest changes

* Eirini's change,name of mist0 to match_to_single_star

* making lists for each matching

* changes in lambda minimize function to a more general function that can take an arbitrary list

* typo

* added the option of the different list of attributes in the step_merged

* matching should work the same for all subclasses now

* syntax errors prob fixed

* fixing runtime errors in the new files

* some more fixing

* Some print statements

* change the flowchart to not go to sterp_disurpted if one of the events is detached is called

* prints

* typo in flowchart

* Friday's work

* .

* fix init orbit

* disrupted bianry evolves until CC2

* step SN updated to expect a 2nd CC in an already disrupted system. Matching for postMS and maybe HeStars may still need work

* matching fixing

* slightly improved the verbose text of the matching

* print errors

* changes to match_to_single_star

* generalized alternative matching

* for this PR, step merged does nothing

* matching happening only when needed (once)

* step merged and intially single go to step end for now

* testing cases of ZAMS stars not needed to match

* binary fraction

* minor changes in isolated or initially single

* else statement, self.companion

* typo

* deleted the option of fidning ZAMS through logR

* syntax error fixing

* resolving merging issues

* went back to grid_Hrich and grid_strippedHe

* ini file

* ini

* surf_avg_omega_div_omega_crit is not always stored, so it is no longer called for and passed to solve_ivp. It is calculated in the Matt+2015 magnetic braking calculation from stellar properties now, as it is only needed there.

* del grid from detached params

* ini

* Removed additional instances where interp1d_sec[surf_avg_omega_div_omega_crit](t - t_offset_sec) was called.

* changing the random

* removed superfluous commas in defining detached step properties

* Adding step_isolated,step_disrupted in the STEP_NAMES_LOADING_GRIDS so the metallicity can propagate in ther kwargs

* fix in ini file

* I think I went the POSYDON-MESA-INLIST submodule back

* changed class names according to CamelCase convention

* time profiling

* Corrections to the detached verbose with timestamp.

* no matching for the CO star, using m0,t0 of the previously matched star

* Giving additional option 1 in verbose to print only the matching parameters

* Step 1 in cleaning detached step.

* Step 2 in cleaing detached step.

* Removing duplicate code in square difference function.

* Deciding on which column are going to use relative distance during matching.

* Deleting obsolete transform method in detached step.

* Removing unnecessary transform when scaling columns in detached step.

* Using stored scalers and removing some useless self.grid assignments

* in the 3rd matching step, I added the change of htrack also for postMS stars

* added htrack  = star.htrack

* commenting out the relative differences during matching for center_he4

* in the 3rd alternative matching we also change the parameters used for matching to the ones according to the new htrack

* typo

* htrack is updated outside match_to_single_star to be used in the rest of get_star_data

* removed simulationproperties from CE folder...

---------

Co-authored-by: Eirini Kasdagli <[email protected]>
Co-authored-by: kasdaglie <[email protected]>
Co-authored-by: sg <[email protected]>
Co-authored-by: kkovlakas <[email protected]>
* updating modules to newest versions for python 3.11

* merge developmet and fix sintax warning in step_SN where is not was incorrectly used whencomparig strigs

---------

Co-authored-by: ssbvr <[email protected]>
…ted using grids of pre-calculated detection probabilities living at /projects/b1119/POSYDON_GRIDS/POSYDON_popsynth_data/v2/POSYDON_data/selection_effects/grids/pdet_grid.hdf5. This file contains pdets assuming a 3-detector HLV network operating at different sensitivities. (#82)
celiotine and others added 4 commits December 4, 2024 11:34
* skip redirect steps when appending to binary history and step names

* skip appending step_times for redirect steps

* add "history_verbose" option

* add missing verbose params to default .ini file

* handle history_verbose setting for all step hooks

* remove history_verbose check from pre-step hooks

* update docs and .ini file with history_verbose parameter

* Merge branch 'development' into camille_remove_redirect_from_df

* update binary_star docs
@maxbriel
Copy link
Collaborator Author

maxbriel commented Dec 4, 2024

I've implemented the requested changes to this PR:

@mkruckow

This comment was marked as resolved.

@maxbriel
Copy link
Collaborator Author

maxbriel commented Dec 5, 2024

  • updated to 7GB

@elizabethteng elizabethteng self-requested a review December 5, 2024 15:02
@maxbriel maxbriel changed the title POSYDON v2.0pre1 POSYDON v2.0.0-pre1 Dec 5, 2024
@maxbriel maxbriel self-assigned this Dec 6, 2024
Copy link
Collaborator

@mkruckow mkruckow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the new version name, we said, everywhere in the setups for conda, pip, e.g. in conda/meta.yaml

Additionally, can you update the release page of the documentation: docs/_source/release-notes/major-updates.rst and docs/_source/release-notes/version-history.rst?

Finally the installation instructions would benefit from an update to the new data set on zenodo: docs/_source/getting-started/installation-guide.rst.

elizabethteng
elizabethteng previously approved these changes Dec 6, 2024
@elizabethteng elizabethteng dismissed their stale review December 6, 2024 18:06

failed checks

@elizabethteng
Copy link
Contributor

failed the following unit tests:

posydon/unit_tests/utils/test_data_download.py::TestValues::test_value_data_url FAILED [ 68%]
posydon/unit_tests/utils/test_data_download.py::TestValues::test_value_original_md5 FAILED [ 68%]

@mkruckow
Copy link
Collaborator

mkruckow commented Dec 7, 2024

posydon/unit_tests/utils/test_data_download.py::TestValues::test_value_data_url FAILED [ 68%]
posydon/unit_tests/utils/test_data_download.py::TestValues::test_value_original_md5 FAILED [ 68%]

Those are the unit tests for the data download, which are expected to fail as long as the new Zenodo url and corresponding check sum aren't propagated to the unit tests.
I have prepared the branch matthias_update_data_download (it contains the latest changes in development + the update of the data download including the corresponding unit test) to merge into max_requirements in case you like to include the unit-tests in the v2.0.0-pre1. Otherwise, the pre release should work without the passed unit test. But surely the update of the unit test should be included in PR #456 .
P.S.: Why are the unit tests executed here, strictly speaking their code don't exist in this branch until the current development gets merged in here, see e.g. continuous_integration.yml of this branch.

@maxbriel
Copy link
Collaborator Author

maxbriel commented Dec 9, 2024

I think GitHub is confused due to the presence of two PRs from the same branch:
Here's the actual Github actions report for this run:
https://github.com/POSYDON-code/POSYDON/actions/runs/12202509354
It passed.
Here's the one that it thinks is associated with this PR: https://github.com/POSYDON-code/POSYDON/actions/runs/12202509369

@maxbriel
Copy link
Collaborator Author

maxbriel commented Dec 9, 2024

I've implemented the suggested changes by @mkruckow for the version number and I've changed the documentation to point to the Zenodo community + Github releases page. This should reduce the number of places we have to maintain with accurate project information.

The PR passes the Github action checks: https://github.com/POSYDON-code/POSYDON/actions/runs/12202509354
The failed checks shown here are not for this PR but for #456.

@astroJeff astroJeff merged commit 92b0ae0 into main Dec 12, 2024
3 of 6 checks passed
@astroJeff astroJeff deleted the max_requirements branch December 12, 2024 15:19
@astroJeff astroJeff restored the max_requirements branch December 12, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.