Skip to content

Commit

Permalink
Merge pull request #154 from LSSTDESC/u/stuart/update_install_docs
Browse files Browse the repository at this point in the history
Update installation docs
  • Loading branch information
stuartmcalpine authored Sep 30, 2024
2 parents 11d44cb + 7accbaa commit a062735
Show file tree
Hide file tree
Showing 6 changed files with 125 additions and 71 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@

## Version 1.0.2

Update documentation for release, new installation instructions etc.

## Version 1.0.0 (Release)

- Update default NERSC site to
Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ them.

dev_notes_spin
dev_notes_database
installation_locally

.. toctree::
:maxdepth: 2
Expand Down
103 changes: 37 additions & 66 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,76 +3,44 @@
Installation
============

Currently the DESC data registry software can only be used at NERSC (i.e.,
PerlMutter).
Currently the DESC ``dataregistry`` database is only accessible through NERSC
(i.e., PerlMutter).

Main installation steps
-----------------------
Using the ``dataregistry`` at NERSC
------------------------------------

When installing the ``dataregistry`` package, it is recommended to work within
your own Conda or Python virtual environment.
The ``dataregistry`` package is readily available as part of the
``desc-python-bleed`` environment (see `here
<https://confluence.slac.stanford.edu/display/LSSTDESC/Getting+Started+with+Anaconda+Python+at+NERSC>`__
for details about the *Conda* environments available at NERSC). Therefore
before getting started, make sure to activate the ``desc-python-bleed``
environment from the command line, or, when working at the NERSC JupyterHub,
select the ``desc-python-bleed`` kernel.

Creating a Conda environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you wish to install the ``dataregistry`` package yourself, see the
instructions :ref:`here <local-installation>`.

You can make a new Conda environment via
Access accounts
---------------

.. code-block:: bash
module load conda/Mambaforge-22.11.1-4
conda create -p ./datareg_env psycopg2
where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash
conda activate <path to your env>
Creating a Python venv
~~~~~~~~~~~~~~~~~~~~~~

or, you can work within a Python virtual environment via

.. code-block:: bash
module load python/3.10
python3 -m venv ./datareg_env
where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash
Authentication to the DESC ``dataregistry`` database works through two primary
group accounts, ``reg_reader`` and ``reg_writer``. These accounts have
different privileges depending on the database schema you are connected to.
Both ``reg_reader`` and ``reg_writer`` have query (read) access to the primary
working schema (``lsst_desc_working``), but only ``reg_writer`` has write
access to register new entries in the database.

source <path to your env>/bin/activate
Both ``reg_reader`` and ``reg_writer`` accounts have read and write access to
the tutorial schemas (used for the tutorial notebooks, i.e., the
``tutorial_working`` and ``tutorial_production`` schemas).

Note the specific version of Python used above (``3.10``) is only an example,
the ``dataregistry`` package is supported on Python versions ``>3.7``.
Neither ``reg_reader`` or ``reg_writer`` can write to the main production
schema (``lsst_desc_production``), however they both have read access. If you
need to register production entries, please consult one of the data registry
admins.

Installing the ``dataregistry`` package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Now we can install the DESC data registry software. First clone the GitHub
repository

.. code-block:: bash
git clone https://github.com/LSSTDESC/dataregistry.git
then, navigate to the ``dataregistry`` directory and install via *pip* using

.. code-block:: bash
python3 -m pip install .
You can test to see if the ``dataregistry`` package has installed successfully
by typing

.. code-block:: bash
python3 -c "import dataregistry; print(dataregistry.__version__)"
If you see the current package version printed to the console, success!
Depending on which account you have access to, you will need to perform a
one-time-setup to authenticate, detailed below.

.. _one-time-setup:

Expand All @@ -90,19 +58,22 @@ entry

.. code-block:: yaml
sqlalchemy.url : postgresql://reg_writer@data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432/desc_data_registry
sqlalchemy.url : postgresql://<username>@dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432/desc_data_registry
where ``<username>`` should either be ``reg_writer`` or ``reg_reader``,
depending on what account you have access to.

Then (if you don't have one already), create a file named ``~/.pgpass`` in your
``$HOME`` directory, and append the entry

.. code-block:: bash
# data registry db
data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432:desc_data_registry:reg_writer:<password>
dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432:desc_data_registry:<username>:<password>
where ``<password>`` is provided on demand by the DESC data registry admins. As
a final step, the ``.pgpass`` file must only be readable by you, which you
can ensure by doing
a final step, the ``.pgpass`` file must only be readable by you, which you can
ensure by doing

.. code-block:: bash
Expand Down
78 changes: 78 additions & 0 deletions docs/source/installation_locally.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
Local installation
==================

.. _local-installation:

When installing the ``dataregistry`` package locally yourself, it is
recommended to work within your own Conda or Python virtual environment.

Creating a Conda environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can make a new Conda environment via

.. code-block:: bash
module load conda/Mambaforge-22.11.1-4
conda create -p ./datareg_env psycopg2
where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash
conda activate <path to your env>
Creating a Python venv
~~~~~~~~~~~~~~~~~~~~~~

or, you can work within a Python virtual environment via

.. code-block:: bash
module load python/3.10
python3 -m venv ./datareg_env
where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash
source <path to your env>/bin/activate
Note the specific version of Python used above (``3.10``) is only an example,
the ``dataregistry`` package is supported on Python versions ``>=3.9``.

Installing the ``dataregistry`` package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The easiest way to install the ``dataregistry`` package is via `PyPi
<https://pypi.org/project/lsstdesc-dataregistry/>`__, i.e.,

.. code-block:: bash
pip install lsstdesc-dataregistry
Install from source
~~~~~~~~~~~~~~~~~~~

To install from source, first clone the GitHub repository

.. code-block:: bash
git clone https://github.com/LSSTDESC/dataregistry.git
then, navigate to the ``dataregistry`` directory and install via *pip* using

.. code-block:: bash
python3 -m pip install .
You can test to see if the ``dataregistry`` package has installed successfully
by typing

.. code-block:: bash
python3 -c "import dataregistry; print(dataregistry.__version__)"
If you see the current package version printed to the console, success!
7 changes: 3 additions & 4 deletions docs/source/tutorial_setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ This section has some tutorials on how to get started using the

Before we begin, make sure that you have the ``dataregistry`` package installed
and available to you at NERSC (installation instructions :ref:`here
<installation>`). This is generally
simplest within a Conda or Python virtual environment (note that the
``dataregistry`` is not yet part of the `desc-python` environment).
<installation>`). This is generally simplest working within the
``desc-python-bleed`` Conda environment.

Also, make sure you have completed the one-time-setup for the data registry
(more details :ref:`here <one-time-setup>`),
Expand All @@ -30,6 +29,6 @@ reference materials, however they can also be used interactively.
To do this at NERSC, first clone the ``dataregistry`` repository to your home
space, then load the notebooks through the NERSC JupyterHub portal. Note that
you will need to be operating within a kernel that has the ``dataregistry``
package installed (see `here
package installed, i.e., ``desc-python-bleed`` (see `here
<https://docs.nersc.gov/services/jupyter/how-to-guides/>`__ how to register
your own Python environments as Jupyter kernels).
2 changes: 1 addition & 1 deletion src/dataregistry/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.0.1"
__version__ = "1.0.2"

0 comments on commit a062735

Please sign in to comment.