From 2dccd1530b4cbd4fecad0ca53aacfd98ae1642f8 Mon Sep 17 00:00:00 2001 From: Juraj Smiesko Date: Thu, 26 Sep 2024 21:15:51 +0200 Subject: [PATCH] Fixing issues before Prague tutorial --- fast-sim-and-analysis/FccFastSimGeneration.md | 33 +++- .../starterkit/FccFastSimDelphes/Readme.md | 9 +- .../Visualization/Visualization.md | 166 +++++++----------- 3 files changed, 95 insertions(+), 113 deletions(-) diff --git a/fast-sim-and-analysis/FccFastSimGeneration.md b/fast-sim-and-analysis/FccFastSimGeneration.md index 5ed297e3..8ecba55c 100644 --- a/fast-sim-and-analysis/FccFastSimGeneration.md +++ b/fast-sim-and-analysis/FccFastSimGeneration.md @@ -22,12 +22,11 @@ source /cvmfs/fcc.cern.ch/sw/latest/setup.sh Note however that not all the `cvmfs` tier-1 centers replicate the `fcc.cern.ch`, so this may lead to slowdowns or even failures. ::: -Builds exist on CernVM-FS for `CentOS7` (this is the Operating System run on `lxplus`) using as compiler `Gcc 11` (currently gcc version `11.2.0`). - :::{admonition} Nota Bene :class: callout -The combination of old `glibc` version available on `CentOS7` with the backward compatibility attributes of glibc makes the provided stack in principle working for newer OSes, such as `CentOS8`, `AlmaLinux9` or `Fedora37`. This in general holds, though less core aspects, such graphics, might still be OS specific. +Currently, the Key4hep stack supports `AlmaLinux9` (compatible with `lxplus` +machines) and `Ubuntu 22.04`. ::: The `gaudimain` steering application here is called `k4run` which should be available at this point: @@ -43,7 +42,8 @@ The application `fccrun` is still provided, fully equivalent to `k4run`. :::{admonition} Nota Bene :class: callout -You will need to source the `/cvmfs/sw.hsf.org/key4hep/setup.sh` script everytime you want to use the software. +You will need to source the `/cvmfs/sw.hsf.org/key4hep/setup.sh` script +everytime you want to use the FCC Software. ::: @@ -114,6 +114,25 @@ It is advised to work in a separate directory for each process. For example, for ```bash mkdir -p test_whizard/Z_mumu; cd test_whizard/Z_mumu wget https://fccsw.web.cern.ch/fccsw/share/gen/whizard/Zpole/Z_mumu.sin +``` + +In order to change output format from LHEf to HepMC3 one needs to comment out +last two lines of the Sindarin configuration file and comment in HepMC3 +configuration line: +```diff +< # simulate (zmumu) { $sample = "z_mumu" sample_format = hepmc } +--- +> simulate (zmumu) { $sample = "z_mumu" sample_format = hepmc } +54,55c54,55 +< $lhef_version = "3.0" +< simulate (zmumu) { $sample = "z_mumu" sample_format = lhef } +--- +> # $lhef_version = "3.0" +> # simulate (zmumu) { $sample = "z_mumu" sample_format = lhef } +``` + +Afterwards run +```bash whizard Z_mumu.sin ``` @@ -172,7 +191,7 @@ Warning: Encountered events with excess weight: 6 events ( 0.600 %) ::: -The file `z_mumu.hepmc` contains 100 e+e- → mu+mu-(gamma) events in HepMC 3 format . +The file `z_mumu.hepmc` contains 100 e+e- → mu+mu-(gamma) events in HepMC 3 format. ### KKMCee @@ -679,7 +698,7 @@ babayaga -c babayaga.input -o bbyg.LHE ### Herwig -`Herwig` is another historical LEP generator providing a difefrent approach to hadronization wrt `Pythia8`. It is available as standalone program: +`Herwig` is another historical LEP generator providing a diferent approach to hadronization wrt. `Pythia8`. It is available as standalone program: ``` $ which Herwig @@ -689,7 +708,7 @@ $ which Herwig ### MadGraph5 -`MadGraph5` was developed for `LHC` but it is reality general purpose and can be used also for `FCC-ee`. It is available as standalone program: +`MadGraph5` was developed for `LHC` but it is in reality general purpose and can be used also for `FCC-ee`. It is available as standalone program: ```bash $ which mg5_aMC diff --git a/fast-sim-and-analysis/k4simdelphes/doc/starterkit/FccFastSimDelphes/Readme.md b/fast-sim-and-analysis/k4simdelphes/doc/starterkit/FccFastSimDelphes/Readme.md index ba2350c3..98c79c3f 100644 --- a/fast-sim-and-analysis/k4simdelphes/doc/starterkit/FccFastSimDelphes/Readme.md +++ b/fast-sim-and-analysis/k4simdelphes/doc/starterkit/FccFastSimDelphes/Readme.md @@ -1,4 +1,3 @@ - # FCC: Getting started with simulating events in Delphes :::{admonition} Learning Objectives @@ -38,17 +37,17 @@ When sourcing the stack, you should see a message like: ``` ... Key4HEP release: key4hep-stack/2023-04-08 ... Use the following command to reproduce the current environment: - ... + ... source /cvmfs/sw.hsf.org/spackages7/key4hep-stack/2023-04-08/x86_64-centos7-gcc11.2.0-opt/urwcv/setup.sh - ... - ... done. + ... + ... done. ``` which means that the version `2023-04-08` of `key4hep-stack` is sourced. (delphesedm4hep)= -## Part I: Generate and simulate Events with DelphesEDM4Hep +## Generate and Simulate Events with DelphesEDM4Hep For this tutorial we will consider the following **physics processes**: diff --git a/full-detector-simulations/Visualization/Visualization.md b/full-detector-simulations/Visualization/Visualization.md index 9a8532db..bf3c726a 100644 --- a/full-detector-simulations/Visualization/Visualization.md +++ b/full-detector-simulations/Visualization/Visualization.md @@ -1,6 +1,6 @@ # Visualization -Visualization is of paramount importance in order to easily understand the behaviour of the detector and simulation. In this section, several tools of visualizing the detector geometry and the particle tracks are provided. +Visualization is of paramount importance in order to easily understand the behavior of the detector and simulation. In this section, several tools of visualizing the detector geometry and the particle tracks are provided. ## Key4hep tool: geoDisplay @@ -16,9 +16,9 @@ To use it, just pass as first argument the `xml` file with the detector descript geoDisplay $DD4hepINSTALL/DDDetectors/compact/SiD.xml ``` -## Geant4 +## Geant4 -The `Geant 4` visualization capabilities can be accesed as +The `Geant 4` visualization capabilities can be accessed as ```shell source /cvmfs/sft.cern.ch/lcg/views/dev4/latest/x86_64-centos7-gcc11-opt/setup.sh @@ -33,92 +33,99 @@ ddsim --compactFile $DD4hepINSTALL/DDDetectors/compact/SiD.xml --runType qt As in the previous step, the `LCG` version of `Key4hep` was sourced. + ## Phoenix@FCC [Phoenix](https://github.com/HSF/phoenix) is a web based event display for High Energy Physics. To visualize FCC events one needs to provide detector geometry and generated events --- event data. -In this tutorial we will be working with files/programs stored on two computers. First -computer will be the one which can source FCCSW stack, e.g. `lxplus` and the -second one will be yours with the recent web browser. We will call the first +In this tutorial we will be working with files/programs stored on two computers. +First computer will be the one which can source Key4hep stack, e.g. `lxplus` and +the second one will be yours with the recent web browser. We will call the first one the _remote machine_ and the second one the _local machine_. ### Event Data from CLD Reconstructed Events Let's start with the visualization of the event data in the established detector -design CLD, which started its life as the detector designed for the CLIC linear -collider concept. The detectors in FCCSW are described in +design -- CLD, which started its life as the detector designed for the linear +collider concept CLIC. The detectors in Key4hep are described using [DD4hep](https://dd4hep.web.cern.ch/dd4hep/) compact files. The compact files are written in XML and expose configuration options for the detector. -Example CLD compact file can be examined after sourcing of the FCCSW stack on +Example CLD compact file can be examined after sourcing of the Key4hep stack on the remote machine ```sh -source /cvmfs/fcc.cern.ch/sw/latest/setup.sh +source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh ``` like so ```sh -less ${FCCDETECTORS}/Detector/DetFCCeeCLD/compact/FCCee_o2_v02/FCCee_o2_v02.xml +less ${K4GEO}/FCCee/CLD/compact/CLD_o2_v07/CLD_o2_v07.xml ``` -This is a copy of the -[FCCDetectors](https://github.com/HEP-FCC/FCCDetectors/blob/main/doc/DD4hepInFCCSW.md#visualisation) -repository, which stores FCC detector designs. +This file is part part of the [k4geo](https://github.com/key4hep/k4geo/) +repository, which stores among other detector concepts integrated in Key4hep +also FCC-ee detector designs. -The Phoenix web application already showcases the CLD detector -[here](https://fccsw.web.cern.ch/fccsw/phoenix/#/fccee-cld/) and we will +The Phoenix web application already showcases the CLD detector in different +options [here](https://fccsw.web.cern.ch/fccsw/phoenix/) and we will simulate few events in order to visualize them there. On the remote machine we will need to clone the -[CLICPerformance](https://github.com/iLCSoft/CLICPerformance) repository +[CLDConfig](https://github.com/iLCSoft/CLICPerformance) repository ```sh -git clone https://github.com/iLCSoft/CLICPerformance.git -cd CLICPerformance/fcceeConfig +git clone https://github.com/key4hep/CLDConfig.git +cd CLDConfig/CLDConfig ``` run the simulation ```sh -ddsim --compactFile $LCGEO/FCCee/compact/FCCee_o1_v04/FCCee_o1_v04.xml \ - --outputFile tops_edm4hep.root \ - --steeringFile fcc_steer.py \ - --inputFiles ../Tests/yyxyev_000.stdhep \ - --numberOfEvents 7 +ddsim --compactFile ${K4GEO}/FCCee/CLD/compact/CLD_o2_v07/CLD_o2_v07.xml \ + --steeringFile cld_steer.py \ + --inputFiles ../test/yyxyev_000.stdhep \ + --outputFile tops_cld_SIM_edm4hep.root \ + --numberOfEvents 5 ``` -and after that, run the reconstruction with the help of -[this](https://fccsw.web.cern.ch/fccsw/tutorials/static/python/fccRec_e4h_input.py) -steering file +and after that, run the reconstruction with the help of the +`CLDReconstruction.py` steering file ```sh -wget https://fccsw.web.cern.ch/fccsw/tutorials/static/python/fccRec_e4h_input.py -k4run fccRec_e4h_input.py +k4run CLDReconstruction.py --inputFiles tops_cld_SIM_edm4hep.root \ + --outputBasename tops_cld \ + --num-events -1 ``` -There should be a file called `tops_cld.root` inside of our working directory. +There should now be a file called `tops_cld_REC_edm4hep.root` inside our working +directory. More information about running of the CLD FullSim you can visit +[FCC-ee Detector Full Sim](https://fcc-ee-detector-full-sim.docs.cern.ch/CLD/) +page. -In order to visualize the events from this file we need co convert the -[EDM4hep](https://edm4hep.web.cern.ch/) ROOT file into intermediate JSON format -with the command: +In order to visualize the events inside `tops_cld_REC_edm4hep.root` file we +need to convert the [EDM4hep](https://edm4hep.web.cern.ch/) ROOT file into +intermediate JSON representation with the command: ```sh -edm4hep2json tops_cld.root +edm4hep2json -e 0 tops_cld_REC_edm4hep.root ``` -:::{admonition} EDM4hep Collections +:::{admonition} EDM4hep in JSON :class: callout -One should specify a list collections to be exported with the `-l` flag. The -default ones are: -* `GenParticles` -* `BuildUpVertices` -* `SiTracks` -* `PandoraClusters` -* `VertexJets` -* `EventHeader` +Be careful when converting EDM4hep ROOT files into JSON files as JSON is very +ineffective in storing information leading to very large file size. + +Always check size of your file to be around 200MB at maximum. One can check the +file size using `du` command +```bash +du -h tops_cld_REC.edm4hep.json +``` + +The `edm4hep2json` offers possibility to limit either number of events exported +or which collections. See `edm4hep2json -h` for more details. ::: Now we download the obtained file from the remote machine into our local one. Most easily done with `scp` ```sh -scp lxplus.cern.ch:CLICPerformance/fcceeConfig/tops_cld.edm4hep.json . +scp lxplus.cern.ch:CLDConfig/CLDConfig/tops_cld_REC.edm4hep.json . ``` To upload the EDM4hep JSON file into the @@ -155,76 +162,33 @@ The obtained EDM4hep JSON file should look similar to ### Event Data from Delphes Fast Simulation -Starting the same way as in [](delphesedm4hep) we will first generate few -events on the remote machine. One can reuse the files generated for that -exercise e.g. `p8_ee_ZH_ecm240_edm4hep.root` or quickly generate them with the -following steps. - -First download Pythia card -```sh -wget https://raw.githubusercontent.com/HEP-FCC/FCC-config/spring2021/FCCee/Generator/Pythia8/p8_noBES_ee_ZH_ecm240.cmd -``` - -and then Delphes cards. One for the detector response -```sh -wget https://raw.githubusercontent.com/HEP-FCC/FCC-config/spring2021/FCCee/Delphes/card_IDEA.tcl -``` +Let's start by generating sample files using [](delphesedm4hep) tutorial section. -and the other for the [EDM4hep](https://edm4hep.web.cern.ch/) formated output -```sh -wget https://raw.githubusercontent.com/HEP-FCC/FCC-config/spring2021/FCCee/Delphes/edm4hep_IDEA.tcl -``` +Produced EDM4hep ROOT file(s) needs to be converted into intermediate JSON +format with the `edm4hep2json` converter. Let's look at the file more closely, +so we can decide what to keep/convert. -Finally, run the simulation with +Primary tool for inspecting EDM4hep files is `podio-dump`. This command can show +you list of all collections and also metadata stored in the file. To list the +collections in the default frame use ```sh -DelphesPythia8_EDM4HEP card_IDEA.tcl edm4hep_IDEA.tcl p8_noBES_ee_ZH_ecm240.cmd p8_ee_ZH_ecm240_edm4hep.root +podio-dump p8_ee_ZH_ecm240_edm4hep.root ``` -Produced EDM4hep ROOT file needs to be converted into intermediate JSON format -with the `edm4hep2json` converter. Let's look at the file more closely, so we -can decide what to keep/convert. - -Start root with +Alternatively, one can always use ROOT to inspect the file. Start by running ```sh root p8_ee_ZH_ecm240_edm4hep.root ``` - and execute the `Show()` command on the `events` TTree ``` root [1] events->Show(2); ``` +this will show you all the branches which are by the PODIO reader to recreate +EDM4hep objects and their relations in memory. -this will show you all trees and branches, similar to this example -``` -MCRecoAssociations = (vector*)0x5578924809e0 -MCRecoAssociations.weight = 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000 -MCRecoAssociations#0 = (vector*)0x557890797c00 -MCRecoAssociations#0.index = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 -MCRecoAssociations#0.collectionID = 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14, 14 -MCRecoAssociations#1 = (vector*)0x5578925c1f00 -MCRecoAssociations#1.index = 133, 118, 117, 152, 153, 115, 164, 155, 138, 224, 217, 228, 165, 227, 218, 90, 230, 122, 87, 88 -MCRecoAssociations#1.collectionID = 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11 -EFlowTrack = (vector*)0x557892a7d3a0 -EFlowTrack.type = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -EFlowTrack.chi2 = 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000 -EFlowTrack.ndf = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -EFlowTrack.dEdx = 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000 -EFlowTrack.dEdxError = 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000, 0.000000 -EFlowTrack.radiusOfInnermostHit = 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000, 320.000000, 655.494995, 17.000000, 655.494995, 320.000000, 17.000000, 17.000000, 17.000000, 17.000000, 17.000000 -EFlowTrack.subDetectorHitNumbers_begin = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -EFlowTrack.subDetectorHitNumbers_end = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -EFlowTrack.trackStates_begin = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 -EFlowTrack.trackStates_end = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 -EFlowTrack.dxQuantities_begin = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 -EFlowTrack.dxQuantities_end = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 -EFlowTrack.trackerHits_begin = 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38 -EFlowTrack.trackerHits_end = 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40 -EFlowTrack.tracks_begin = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -EFlowTrack.tracks_end = 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -``` - -We select desired collections with `-l` flag and limit number of converted -events to 10 with `-n` flag +Using `edm4hep2json` for converting the EDM4hep ROOT file into JSON we select +desired collections with `-l` flag and limit number of converted events to 10 +with `-n` flag ```sh edm4hep2json -l ReconstructedParticles,Jet,EFlowTrack,TrackerHits,Particle,MCRecoAssociations,MissingET -n 10 p8_ee_ZH_ecm240_edm4hep.root ```