Coordination with Scikit-HEP, mplhep and common goals of packages

[jonas-eschle]

Hi all, I just came to learn about the package from the upcoming pyhep workshop, and I am pleasently surprised to see what you have here, great stuff! (and good that you'll present it to a wider audience!)

(I am the main dev of zfit and a member of Scikit-HEP, the org behind boost-histogram and many more, you're most likely familiar with?) I presume you're both Belle II btw?

To coordinate all the efforst a bit better, I was wondering if you're aware of the mplhep package? The goal of the package seems to be somewhat overlapping, but also has distinct features and it's probably best to disentangle a bit and see where things can be improved/better coordinated accross the Python HEP ecosystem.
Just FYI: The way Scikit-HEP works is quite open, we're a bunch of (HEPphysicists/) devs that agree on which packages are part of it and will help to maintain them (take me - I haven't been part of mplhep before but came into it thanks to being a Scikit-HEP dev), which also prospers long term maintenance under a common name. Since the ecosystem is made up of many different, small packages that need to interact with each other (-> UHI was developed for this), we always try to coordinate among ourselves (and with the outside, awkward-array, numpy, xarray etc) to avoid duplication and make packages as interchangeable (and usable) as possible. And actually, in plotting we're rather "understaffed" (given as personpower times percent working on it) and seeing new functionality here that we may thought about but never even started to realize is quite encouraging! Here just a few thoughts

• The main motivation of mplhep was also to separate the creation and plotting of histograms (mplhep.hitsplot(h) with h either an np.histogram output or a UHI compatible object), as plothist does something very similar. What was the main motivation not to just use mplhep.plothist? If there are any drawbacks that you needed to circumvent, glad to learn abouth them (maybe it's only of limited use to have two different histplot?)
• mplhep also has style settings for different experiments that I found personally very useful. Maybe there is some common ground on how to set styles or something?
• mplhep is quite limited in terms of plots to a few specific ones (1Dim and 2Dim histograms) but does (not yet) offer many things out of the box (such as plotting PDFs, residuals, statistical result) that would be very helpful to have. I see that plothist goes more into that direction of "nice plots out-of-the-box"? (that's why we recently added something even to zfit itself, but preferrably, this should reside somewhere else)
• I also really like the gallery and overall impression of the package, that's surely also something where mplhep is a bit lacking.

Overall, how do you see both packages evolve, where can we maybe merge/split etc? We can also gladly have a chat ([email protected] for contact). (Also, I assume this question will be anyway raised at pyhep, so good to be ahead from the dev side ;) )

cc @andrzejnovak as main dev of mplhep

[cyrraz]

Hi Jonas,

Thank you for your message and for raising these interesting points.

We agree that Scikit-HEP is a great platform for coordinating and maintaining Python tools for HEP, and we would be very interested in joining the project in the future.

The reason we did not use mplhep in the first place is that we found it easier to build our code using a simple plothist.plot_hist() function as a base, which only supports boost_histogram.Histogram objects. If you look at the code of the function, you will see that it is a very light wrapper around the matplotlib function ax.hist(). The simplicity of this base function and the flexibility of the boost_histogram.Histogram objects helped us a lot in developing high level functions.

I think the easiest way would be to meet online to discuss how to coordinate our efforts. I will send you an email.

[jonas-eschle]

Discussing this further, I think we have three options to go ahead in terms of packages:

1. Merging both packages, since it's hep specific and mplhep is known, it seems like the better suited name (to me at least after all the discussions)
2. split them, having a general and a more specific package. Disadvantage is the large overhead IMHO of maintaining two packages, having different docs etc.
3. making a single general package with subpackages. As proposed by Giordon, this could be histplotlib, imported as hpl. The hep specific stuff lives in hpl.hep, which can be imported as mplhep even (or whatever). This would mean that it could be extended to the astro community etc and we have everything in one package. The disadvantage is again that we have to get users introduced to it

Any thoughts on 3?

EDIT:
Additional thought on libraries: there is https://github.com/scikit-hep/histoprint, which actually fits a similar scope, especially if we're talking about histogram plotting. This could probably be part of a more general package? Just putting that up here

[cyrraz]

Hi Jonas,

Thanks for the summary.

• After discussion with @0ctagon, we tend to favour option 1 (including keeping the name mplhep), mainly based on the arguments of maintainability, documentation and existing user community of mplhep. If we go this way, the next step would be to meet again to discuss the technicalities of the merge (backward compatibility, feature overlaps, names of functions, ...).
• We think that the non-hep community could still use the new version of mplhep in scenario 1, as long as we make sure that the documentation and the names of the functions and arguments are not hep-jargon.
• Regarding histoprint, we would think about it at a later stage.