Skip to content

Latest commit

 

History

History
219 lines (182 loc) · 10.6 KB

README.md

File metadata and controls

219 lines (182 loc) · 10.6 KB
Raw

Description

This module can be used to install spack on a VM. This includes:

  1. Cloning spack into a predefined directory
  2. Checking out a specific version of spack
  3. Configuring compilers within spack
  4. Installing application licenses that spack packages might depend on
  5. Installing various spack specs.

The output of this module is a startup script that is intended to be attached to either the login or controller node of a scheduler, or a vm-instance. The resulting installation of spack can then be mounted across many other VMs to share a software stack.

NOTE: This module currently is capable of re-running to install additional packages, but cannot be used to uninstall packages from the VM.

NOTE: Currently, license installation is performed by copying a license file from a GCS bucket to a specific directory on the target VM.

NOTE: When populating a buildcache with packages, the VM this spack module is running on requires the following scope: https://www.googleapis.com/auth/devstorage.read_write

Example

As an example, the below is a possible definition of a spack installation. To see this module used in a full blueprint, see the spack-gromacs.yaml example.

  - id: spack
    source: community/modules/scripts/spack-install
    settings:
      install_dir: /sw/spack
      spack_url: https://github.com/spack/spack
      spack_ref: v0.19.0
      spack_cache_url:
      - mirror_name: 'gcs_cache'
        mirror_url: gs://example-buildcache/linux-centos7
      configs:
      - type: single-config
        scope: defaults
        value: "config:build_stage:/sw/spack/spack-stage"
      - type: file
        scope: defaults
        value: |
          modules:
            default:
              tcl:
                hash_length: 0
                all:
                  conflict:
                    - '{name}'
                projections:
                  all: '{name}/{version}-{compiler.name}-{compiler.version}'
      compilers:
      - [email protected] target=x86_64
      packages:
      - cmake%[email protected] target=x86_64
      environments:
      - name: main-env
        packages:
        - intel-mkl%[email protected] target=skylake
        - [email protected]%[email protected] target=skylake
        - fftw%[email protected] target=skylake ^[email protected]%[email protected] target=x86_64
      - name: explicit-env
        content: |
          spack:
            definitions:
            - compilers:
              - [email protected]
            - mpis:
              - [email protected]
            - packages:
              - intel-mkl
            - mpi_packages:
              - fftw
            specs:
            - matrix:
              - - $packages
              - - $%compilers
            - matrix:
              - - $mpis
              - - $%compilers
            - matrix:
              - - $mpi_packages
              - - $%compilers
              - - $^mpis

Following the above description of this module, it can be added to a Slurm deployment via the following:

- id: slurm_controller
  source: community/modules/scheduler/SchedMD-slurm-on-gcp-controller
  use: [spack]
  settings:
    subnetwork_name: ((module.network1.primary_subnetwork.name))
    login_node_count: 1
    partitions:
    - $(compute_partition.partition)

Alternatively, it can be added as a startup script via:

  - id: startup
    source: modules/scripts/startup-script
    settings:
      runners:
      - $(spack.install_spack_deps_runner)
      - $(spack.install_spack_runner)

Environment Setup

Spack installation produces a setup script that adds spack to your PATH as well as some other command-line integration tools. This script can be found at <install path>/share/spack/setup-env.sh. This script will be automatically added to bash startup by the install_spack_runner. In the case that you are using Spack on a different machine than the one where Spack was installed, you can use the setup_spack_runner to make sure Spack is also available on that machine.

Example using setup_spack_runner

The following examples assumes that a different machine is running $(spack.install_spack_runner) and the Slurm login node has access to the Spack instal through a shared file system.

  - id: spack
    source: community/modules/scripts/spack-install
    ...

  - id: spack-setup
    source: modules/scripts/startup-script
    settings:
      runners:
      - $(spack.setup_spack_runner)

  - id: slurm_login
    source: community/modules/scheduler/SchedMD-slurm-on-gcp-login-node
    use: [spack-setup, ...]

License

Copyright 2022 Google LLC

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Requirements

Name Version
terraform >= 0.14.0

Providers

No providers.

Modules

No modules.

Resources

No resources.

Inputs

Name Description Type Default Required
caches_to_populate Defines caches which will be populated with the installed packages.
Each cache must specify a type (either directory, or mirror).
Each cache must also specify a path. For directory caches, this path
must be on a local file system (i.e. file:///path/to/cache). For
mirror paths, this can be any valid URL that spack accepts.

NOTE: GPG Keys should be installed before trying to populate a cache
with packages.

NOTE: The gpg_keys variable can be used to install existing GPG keys
and create new GPG keys, both of which are acceptable for populating a
cache.		 list(map(any)) [] no
compilers Defines compilers for spack to install before installing packages. list(string) [] no
concretize_flags Defines the flags to pass into spack concretize string "" no
configs List of configuration options to set within spack.
Configs can be of type 'single-config' or 'file'.
All configs must specify content, and a
a scope.		 list(map(any)) [] no
environments Defines spack environments to configure, given as a list.
Each environment must define a name.
Additional optional attributes are 'content' and 'packages'.
'content' must be a string, defining the content of the Spack Environment YAML file.
'packages' must be a list of strings, defining the spack specs to install.
If both 'content' and 'packages' are defined, 'content' is processed first.		 any [] no
gpg_keys GPG Keys to trust within spack.
Each key must define a type. Valid types are 'file' and 'new'.
Keys of type 'file' must define a path to the key that
should be trusted.
Keys of type 'new' must define a 'name' and 'email' to create
the key with.		 list(map(any)) [] no
install_dir Directory to install spack into. string "/sw/spack" no
install_flags Defines the flags to pass into spack install string "" no
licenses List of software licenses to install within spack. 
list(object({
    source = string
    dest   = string
  }))
 null no
log_file Defines the logfile that script output will be written to string "/var/log/spack.log" no
packages Defines root packages for spack to install (in order). list(string) [] no
project_id Project in which the HPC deployment will be created. string n/a yes
spack_cache_url List of buildcaches for spack. 
list(object({
    mirror_name = string
    mirror_url  = string
  }))
 null no
spack_ref Git ref to checkout for spack. string "v0.19.0" no
spack_url URL to clone the spack repo from. string "https://github.com/spack/spack" no
zone The GCP zone where the instance is running. string n/a yes

Outputs

Name Description
controller_startup_script Path to the Spack installation script, duplicate for SLURM controller.
install_spack_deps_runner Runner to install dependencies for spack using an ansible playbook. The
startup-script module will automatically handle installation of ansible.
- id: example-startup-script
source: modules/scripts/startup-script
settings:
runners:
- $(your-spack-id.install_spack_deps_runner)
...
install_spack_runner Runner to install Spack using the startup-script module
setup_spack_runner Adds Spack setup-env.sh script to /etc/profile.d so that it is called at shell startup. Among other things this adds Spack binary to user PATH.
startup_script Path to the Spack installation script.