Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement dataset index and add PSTracks v4.2, NT v5.1 #180

Closed
wants to merge 60 commits into from

Conversation

mlincett
Copy link
Collaborator

@mlincett mlincett commented Aug 4, 2022

This PR:

@mlincett mlincett marked this pull request as ready for review September 26, 2022 13:44
@mlincett mlincett changed the title Implement dataset index and add PSTracks v4.2 Implement dataset index and add PSTracks v4.2, NT v5.1 Oct 3, 2022
@mlincett
Copy link
Collaborator Author

mlincett commented Oct 6, 2022

Requesting a review!

Please note that the commit history is a bit chaotic due to some kind of merge/rebase mistake on my part. Will make sure to squash everything on merge.

@robertdstein
Copy link
Member

Thanks for all this @mlincett! One broad comment though: this is a lot of changes which are not really all directly coupled together. As a general rule, I think one PR per issue is optimal, and here at least I'd say this could have been more easy to review if it was several independent PRs. Reviewing smaller pieces is easier than doing it all at once.

@mlincett
Copy link
Collaborator Author

mlincett commented Oct 6, 2022

Thanks for all this @mlincett! One broad comment though: this is a lot of changes which are not really all directly coupled together. As a general rule, I think one PR per issue is optimal, and here at least I'd say this could have been more easy to review if it was several independent PRs. Reviewing smaller pieces is easier than doing it all at once.

You are right, and I was indeed a bit worried about this. Somehow it came natural to implement one feature/fix to help with testing another. But now that everything is there I am fine with splitting this into three/four separate PRs.

flarestack/cluster/make_desy_cluster_script.py Outdated Show resolved Hide resolved
@@ -437,7 +437,9 @@ def make_cluster_submission_script(self):
"eval $(/cvmfs/icecube.opensciencegrid.org/py3-v4.1.0/setup.sh) \n"
"export PYTHONPATH=" + DESYSubmitter.root_dir + "/ \n"
"export FLARESTACK_SCRATCH_DIR=" + flarestack_scratch_dir + " \n"
"python " + fs_dir + "core/multiprocess_wrapper.py -f $1 -n $2 \n"
f"{sys.executable} "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason this is over 3 lines? The rest is one python code line per outputted line

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what black does.

flarestack/data/icecube/northern_tracks/nt_v005_p01.py Outdated Show resolved Hide resolved
flarestack/data/icecube/northern_tracks/nt_v005_p01.py Outdated Show resolved Hide resolved
flarestack/data/icecube/ps_tracks/ps_v004_p02.py Outdated Show resolved Hide resolved
@robertdstein
Copy link
Member

Thanks for all this @mlincett! One broad comment though: this is a lot of changes which are not really all directly coupled together. As a general rule, I think one PR per issue is optimal, and here at least I'd say this could have been more easy to review if it was several independent PRs. Reviewing smaller pieces is easier than doing it all at once.

You are right, and I was indeed a bit worried about this. Somehow it came natural to implement one feature/fix to help with testing another. But now that everything is there I am fine with splitting this into three/four separate PRs.

I think it's too much work to split this now, but just something to bear in mind for next time

@mlincett
Copy link
Collaborator Author

mlincett commented Oct 7, 2022

I think it's too much work to split this now, but just something to bear in mind for next time

Uhm, I still think it will save us a lot of headaches and make the review quicker. Will split this in five different PR!

@mlincett mlincett closed this Oct 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants