Replies: 6 comments
-
|
I was able to run this on lxplus using a pocketcoffea apptainer image @valsdav created with coffea 2023 and dask lxplus installed. import json
import socket
import os
import dask
from distributed import Client
from dask_lxplus import CernCluster
from coffea.nanoevents import NanoEventsFactory
files = {
"root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root": "Events",
"root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012C_DoubleMuParked.root": "Events",
}
if __name__ == "__main__":
env_extra = [
"export XRD_RUNFORKHANDLER=1",
f"export X509_USER_PROXY=/tmp/x509up_u154691",
f"export PYTHONPATH=$PYTHONPATH:{os.getcwd()}",
]
cluster = CernCluster(
cores=1,
memory="4GB",
disk="1GB",
image_type="singularity",
worker_image="/cvmfs/unpacked.cern.ch/gitlab-registry.cern.ch/cms-analysis/general/pocketcoffea:lxplus-cc7-coffea2023",
death_timeout="3600",
scheduler_options={"port": 8786, "host": socket.gethostname()},
job_extra={
"log": "dask_job_output.log",
"output": "dask_job_output.out",
"error": "dask_job_output.err",
"should_transfer_files": "Yes",
"when_to_transfer_output": "ON_EXIT",
"+JobFlavour": '"longlunch"',
},
env_extra=env_extra,
)
cluster.adapt(minimum=0, maximum=10)
client = Client(cluster)
events = NanoEventsFactory.from_root(files).events()
out = events.Muon.pt
print(dask.compute(out)) |
Beta Was this translation helpful? Give feedback.
-
|
I'm able to run more complicated stuff on lxplus with the dask-lxplus plugin using @valsdav's apptainer image with coffea 2023 and dask-lxplus. It appears it's a good idea to create coffea images with dask-lxplus installed. |
Beta Was this translation helpful? Give feedback.
-
|
Great that the image improves the workflow! About workers being slow to start I guess that is just the HTCondor queue.. not much to be done. Asking for a shorter queue gives usually workers faster (like avoid "nextweek" queue). |
Beta Was this translation helpful? Give feedback.
-
|
There's nothing stopping you from using the shorter queues (so long as it is not too short)... If you lose critical data dask will (try to) recalculate it. We should figure out if the cluster admins would be ok with a more rapid-acquisition, shortlived queue. Even "longlunch" takes some time to spin up workers. |
Beta Was this translation helpful? Give feedback.
-
|
What about docker/apptainer images by the way?. Should the default coffea image have dask-lxplus installed or should this be a separate image? |
Beta Was this translation helpful? Give feedback.
-
|
We are moving images here https://github.com/CoffeaTeam/af-images (for for old coffea and new coffea) and we can add a separate image for dask-lxplus as well as it will be done for other facilities. |
Beta Was this translation helpful? Give feedback.
-
I'm starting this discussion to ask about the status of running analyses in lxplus using coffea 202x.
One very common thing people within CMS want when running code, is for it to be able to run on lxplus. Not everyone can get an LPC account and I think a lot of people are used to just automatically sshing into lxplus for work.
I've tried using https://github.com/cernops/dask-lxplus in the past with coffea 202x but workers just wouldn't spawn. I could be doing something wrong obviously so my question is: Are we able to run coffea 202x on lxplus?
If not, I believe we should be able to.
@lgray @nsmith- @JaLuka98 @valsdav mentioning a few people who might know the answer.
Beta Was this translation helpful? Give feedback.
All reactions