-
Notifications
You must be signed in to change notification settings - Fork 2
Add Your Own Packages to the desc stack Environment
If you are new to DESC and DESC python environments, please consult our related documentation for NERSC or IN2P3.
You may have additional packages or an updated version of a package that you want to install alongside desc-stack (or desc-stack-weekly or desc-stack-weekly-latest).
- If this is a package of general interest, please consult within your Working Groups and then request the package be added to desc-python and/or desc-stack by opening an issue in desc-help.
Create a directory to store the installation artifacts associated with your new packages, create the DESCSTACKUSERBASE environment variable and then pip install the packages you wish to add.
-
cd $SCRATCH # I just chose SCRATCH.. you could use another area mkdir <yourInstallArea> export DESCSTACKUSERBASE=$PWD/<yourInstallArea> # This env var is recognized by the desc-stack-weekly setup, so --user installs will use this dir # For desc-python, use DESCPYTHONUSERAREA. For desc-python-bleed, use DESCPYTHONBLEEDUSERBASE python /global/common/software/lsst/common/miniconda/start-kernel-cli.py desc-stack-weekly pip install --user --no-build-isolation <NewPackage>
To enable your installed packages the next time you set up desc-stack-weekly in Jupyter or on the command line, you will need to set the following in your $HOME/.bashrc:
export DESCSTACKUSERBASE=<fullPathtoYourInstallArea>
- Create a full clone of the
lsst-scipipe
environment in your own area (this will take a few minutes). You will want to note the environment name after runningstart-kernel-cli.py
and use it when runningconda create --clone
-
Please note that
python /global/common/software/lsst/common/miniconda/start-kernel-cli.py desc-stack-weekly conda create --clone lsst-scipipe-0.6.0 -p <PathToYourDir>/mydescstack
lsst-scipipe-0.6.0
is just an example, the version of the LSST Science Pipelines associated withdesc-stack
changes as we upgrade to more recent weekly builds. - Activate your new environment and add your desired packages
conda activate <PathToYourDir>/mydescstack conda install <Your Package> OR pip install <Your Package> OR mamba install <Your Package>
-
Your current NERSC session now has your cloned conda environment set up and it is ready to use.
- To enable your cloned environment automatically the next time you set up the
desc-stack-weekly
env, including when using Jupyter at NERSC, you need to set up an environment variable in your NERSC$HOME/.bashrc
or$HOME/.cshrc
.
If using bash, you would add this to your$HOME/.bashrc
or$HOME/.bashrc.ext
(create this file if it does not already exist):
export DESCSTACKUSERENV=<FullPathToYourCondaEnv>
Example:
export DESCSTACKUSERENV=/global/common/software/lsst/users/heatherk/mydescstack
With the DESCSTACKUSERENV
environment variable set, the next time you start the desc-stack-weekly
jupyter kernel, or in a new session on Cori, you will be using your own cloned environment, rather than the official desc-stack-weekly
.
Here is a full concrete example at NERSC
python /global/common/software/lsst/common/miniconda/start-kernel-cli.py desc-stack-weekly
conda create --clone desc -p /global/common/software/lsst/users/heatherk/mydescstack
conda activate /global/common/software/lsst/users/heatherk/mydescstack
conda install -c conda-forge astrocats
Now your package(s) are installed in your personal clone of the desc-stack-weekly conda environment. Your cloned conda environment is set up and ready to use in your current NERSC session.
-
In order to enable your cloned environment the next time you set up
desc-stack-weekly
or in jupyter.nersc.gov, you should set up aDESCSTACKUSERENV
environment variable in your$HOME/.bashrc
or $HOME/.bashrc.ext`, as described above. -
If you later decide to ignore or discard your cloned desc conda environment, just unset or remove
DESCSTACKUSERENV
and the next time you enabledesc-stack-weekly
you will be using the default desc conda environment.