Merge pull request #29 from lukemartinlogan/main

Update jarvis-cd documentation for new spack repo

docs/02-hpc-tutorials/04-cpp-introduction/03-cpp-build-with-cmake.md

@@ -68,7 +68,7 @@ directory. Generally, the root CMake is responsible for the following:
 3. Setting variables global to the build
 4. Setting compiler flags (e.g., optimization)
 
-In this section, we will describe our root [CMakeLists.txt](https://github.com/scs-lab/scs-tutorial/blob/main/3.3.building_cpp_cmake/CMakeLists.txt).
+In this section, we will describe our root [CMakeLists.txt](https://github.com/grc-iit/grc-tutorial/blob/main/cpp/03-cpp-build-with-cmake/CMakeLists.txt).
 
 ### CMake Preamble
 ```cmake
@@ -245,7 +245,7 @@ directory and execute the CMakeLists.txt in that subdirectory.
 
 ## src/CMakeLists.txt
 
-In this section, we will discuss [src/CMakeLists.txt](https://github.com/scs-lab/scs-tutorial/blob/main/3.3.building_cpp_cmake/src/CMakeLists.txt).
+In this section, we will discuss [src/CMakeLists.txt](https://github.com/grc-iit/grc-tutorial/blob/main/cpp/03-cpp-build-with-cmake/src/CMakeLists.txt).
 This CMake file is responsible for defining how to build + install the
 source code in this repo.
 
@@ -372,7 +372,7 @@ they are with no modification.
 
 ## test/CMakeLists.txt
 
-In this section, we will discuss [test/CMakeLists.txt](https://github.com/scs-lab/scs-tutorial/blob/main/3.3.building_cpp_cmake/test/CMakeLists.txt). This CMake is responsible for creating unit tests.
+In this section, we will discuss [test/CMakeLists.txt](https://github.com/grc-iit/grc-tutorial/blob/main/cpp/03-cpp-build-with-cmake/test/CMakeLists.txt). This CMake is responsible for creating unit tests.
 
 ### Creating a CTest
 ```cmake

docs/05-jarvis/02-jarvis-cd/01-index.md

@@ -1,38 +1,45 @@
-# Introduction
-
-`jarvis-cd` is a unified platform for deploying various applications, including
+Jarvis-CD is a unified platform for deploying various applications, including
 storage systems and benchmarks. Many applications have complex configuration
 spaces and are difficult to deploy across different machines.
 
 We provide a builtin repo which contains various applications to deploy.
 We refer to applications as "jarivs pkgs" which can be connected to form
 "deployment pipelines".
 
-## Dependencies
-
-Jarvis-CD depends on jarvis-util. jarvis-util contains functions to execute
-binaries in python and collect their output.
+# Installation
 
+Get the GRC spack repo:
 ```bash
-git clone https://github.com/scs-lab/jarvis-util.git
-cd jarvis-util
-python3 -m pip install -r requirements.txt
-python3 -m pip install -e .
+git clone https://github.com/grc-iit/grc-repo
+spack repo add grc-repo
 ```
 
-## Installation
+Install jarvis-cd:
+```bash
+spack external find python
+spack install py-jarvis-cd
+```
 
+Spack packages must be loaded to use them.
+You'll have to do this for each new terminal.
 ```bash
-cd /path/to/jarvis-cd
-python3 -m pip install -r requirements.txt
-python3 -m pip install -e .
+spack load py-jarvis-cd
 ```
 
-## Configuring Jarvis
+# Building the Jarvis Configuration
+
+## Bootstrapping for a single-node machine
 
-### Bootstrapping from a specific machine
+You may be trying to test things on just a single node. 
 
-Jarivs has been pre-configured on some machines. To bootstrap from
+In this case, run:
+```bash
+jarvis bootstrap from local
+```
+
+## Bootstrapping from a specific machine
+
+Jarvis has been pre-configured on some machines. To bootstrap from
 one of them, run the following:
 
 ```bash
@@ -42,27 +49,75 @@ jarvis bootstrap from ares
 NOTE: Jarvis must be installed from the compute nodes in Ares, NOT the master node. This is because we store configuration data in /mnt/ssd by default, which is only on compute nodes. We do not store data in /tmp since it will be eventually destroyed.
 
 To check the set of available machines to bootstrap from, run:
-
 ```bash
-jarvis boostrap list
+jarvis bootstrap list
 ```
 
-### Creating a new configuration
+## Creating a new configuration
 
 A configuration can be generated as follows:
-
 ```bash
 jarvis init [CONFIG_DIR] [PRIVATE_DIR] [SHARED_DIR (optional)]
 ```
 
-- **CONFIG_DIR:** A directory where jarvis metadata for pkgs and pipelines
-  are stored. This directory can be anywhere that the current user can access.
-- **PRIVATE_DIR:** A directory which is common across all machines, but
-  stores data locally to the machine. Some jarvis pkgs require certain data to
-  be stored per-machine. OrangeFS is an example.
-- **SHARED_DIR:** A directory which is common across all machines, where
-  each machine has the same view of data in the directory. Most jarvis pkgs
-  require this, but on machines without a global filesystem (e.g., Chameleon Cloud),
-  this parameter can be set later.
+* **CONFIG_DIR:** A directory where jarvis metadata for pkgs and pipelines
+are stored. This directory can be anywhere that the current user can access.
+* **PRIVATE_DIR:** A directory which is common across all machines, but
+stores data locally to the machine. Some jarvis pkgs require certain data to
+be stored per-machine. OrangeFS is an example.
+* **SHARED_DIR:** A directory which is common across all machines, where
+each machine has the same view of data in the directory. Most jarvis pkgs
+require this, but on machines without a global filesystem (e.g., Chameleon Cloud),
+this parameter can be set later.
 
 For a personal machine, these directories can be the same directory.
+
+# Building the Resource Graph
+
+Python jarvis:
+```bash
+jarvis rg build
+```
+
+# Manual Installation (Mainly Devs)
+
+## Jarvis-Util
+Jarvis-CD depends on jarvis-util. jarvis-util contains functions to execute
+binaries in python and collect their output.
+
+```bash
+git clone https://github.com/grc-iit/jarvis-util.git
+cd jarvis-util
+python3 -m pip install -r requirements.txt
+python3 -m pip install -e .
+```
+
+## Scspkg
+
+Scspkg is a tool for building modulefiles using a CLI. It's not strictly
+necessary for Jarvis to function, but many of the readmes use it to provide
+structure to manual installations.
+
+```bash
+git clone https://github.com/grc-iit/scspkg.git
+python3 -m pip install -r requirements.txt
+python3 -m pip install -e .
+echo "module use \`scspkg module dir\`" >> ~/.bashrc
+```
+
+The wiki for scspkg is [here](https://github.com/grc-iit/scspkg.git).
+
+## Jarvis-CD
+
+```bash
+cd /path/to/jarvis-cd
+python3 -m pip install -r requirements.txt
+python3 -m pip install -e .
+```
+
+## Net Test
+
+Network test tool for identifying valid networks.
+```bash
+spack install chi-nettest
+```

docs/05-jarvis/02-jarvis-cd/02-getting-started.md

@@ -44,7 +44,7 @@ This information will be stored under `${JARVIS_ROOT}/config/resource_graph.yaml
 The following command will create an empty pipeline "my_pipeline":
 
 ```bash
-jarvis pipeline create ior_test
+jarvis ppl create ior_test
 ```
 
 After creating a pipeline, jarvis will focus on that pipeline.

docs/05-jarvis/02-jarvis-cd/04-design-motivation.md

@@ -57,10 +57,10 @@ files are stored in the `${PRIVATE_DIR}` and `${SHARED_DIR}` directories stored
 in the jarvis configuration.
 
 ```bash
-jarvis pipeline create ofs-global
-jarvis pipeline append orangefs
-jarvis pipeline start &
-jarvis pipeline stop
+jarvis ppl create ofs-global
+jarvis ppl append orangefs
+jarvis ppl start &
+jarvis ppl stop
 ```
 
 ## Application Pkgs
@@ -71,11 +71,11 @@ Applications can include:
 - simulation codes
 
 ```bash
-jarvis pipeline create my-benchmark
+jarvis ppl create my-benchmark
 jarvis cd my-benchmark
-jarvis pipeline append orangefs
-jarvis pipeline append hermes
-jarvis pipeline append ior
+jarvis ppl append orangefs
+jarvis ppl append hermes
+jarvis ppl append ior
 ```
 
 ## Interceptor Pkgs
@@ -84,14 +84,14 @@ A library can be used to intercept or monitor some functionality of an
 application. For example, a library may include
 
 ```bash
-jarvis pipeline create my-app
+jarvis ppl create my-app
 jarvis cd my-app
-jarvis pipeline append orangefs \
+jarvis ppl append orangefs \
   client_path=${HOME}/llogan/pfs
-jarvis pipeline append hermes
-jarvis pipeline append monitor
-jarvis pipeline append qmcpack
-jarvis pipeline run
+jarvis ppl append hermes
+jarvis ppl append monitor
+jarvis ppl append qmcpack
+jarvis ppl run
 ```
 
 ## Pipeline Metadata

docs/05-jarvis/02-jarvis-cd/06-building-package.md

@@ -28,10 +28,10 @@ jarvis repo create gray_scott app
 We can then create an example pipeline as follows:
 
 ```bash
-jarvis pipeline create test
-jarvis pipeline append hermes
-jarvis pipeline append hermes_mpiio
-jarvis pipeline append gray_scott
+jarvis ppl create test
+jarvis ppl append hermes
+jarvis ppl append hermes_mpiio
+jarvis ppl append gray_scott
 ```
 
 This is an example of a pipeline which will deploy Hermes, the Hermes MPI-IO
@@ -313,7 +313,7 @@ will make the kwargs dict be:
 
 ### `start`
 
-The start function is called during `jarvis pipeline run` and `jarvis pipeline start`.
+The start function is called during `jarvis ppl run` and `jarvis ppl start`.
 This function should execute the program itself.
 
 Below is an example for Hermes:
@@ -336,7 +336,7 @@ def start(self):
 
 ### `stop`
 
-The stop function is called during `jarvis pipeline run` and `jarvis pipeline stop`.
+The stop function is called during `jarvis ppl run` and `jarvis ppl stop`.
 This function should terminate the program.
 
 Below is an example for Hermes:
@@ -363,7 +363,7 @@ This is not typically implemented for Applications, but it is for Services.
 
 ### `clean`
 
-The `clean` function is called during `jarvis pipeline clean`.
+The `clean` function is called during `jarvis ppl clean`.
 It clears all intermediate data produced by a pipeline.
 
 Below is the prototype
@@ -381,7 +381,7 @@ def clean(self):
 
 ### `status`
 
-The `status` function is called during `jarvis pipeline status`
+The `status` function is called during `jarvis ppl status`
 It determines whether or not a service is running. This is not typically
 implemented for Applications, but it is for Services.
 

docs/05-jarvis/02-jarvis-cd/09-schedulers.md

@@ -23,7 +23,7 @@ Jarvis-CD integrates with the SLURM scheduler through the pipeline sbatch menu.
 To run a job using SLURM:
 
 ```bash
-jarvis pipeline sbatch job_name=test nnodes=4
+jarvis ppl sbatch job_name=test nnodes=4
 ```
 
 ## PBS Scheduler
@@ -42,5 +42,5 @@ Jarvis-CD also supports the PBS scheduler through the pipeline pbs menu. The fol
 To run a job using PBS:
 
 ```bash
-jarvis pipeline pbs nnodes=2 system=other_system
+jarvis ppl pbs nnodes=2 system=other_system
 ```