Introduction to using the shell in a High-Performance Computing context

This lesson provides an introduction to the bash shell aimed at researchers who will be using the command line to use remote, high-performance computing (HPC) systems. The material is also suitable for teaching the use of the shell for any remote, advanced computing resources.

Using this material

1. Follow the instructions found in the Software Carpentry example lesson source to create a repository for your lesson.
2. Edit _config.yml to modify the configuration options at the bottom for the remote host you will be using. These options set such things as the address of the host to login to and the ID that is used to incorporate host-specific code snippets (see below).
3. Create the required host-specific code snippets in subdirectories in _includes/snippets. These snippets provide inputs and outputs that are host-specific and that are included automatically based on the configuration in the _config.yml file.
   1. Code snippets are in files named snippet_name.host_id and are included automatically when the lesson is built. For example, if the snippet_name was login_output and the host_id was ComputeCanada_Graham, then the snippet file would be called login_output.ComputeCanada_Graham.
   2. Code snippets are placed in subdirectories that are named according to the episode they appear in. For example, if the snippet is for episode 01, then it will be in a subdirectory called 01.

Lesson writing instructions

This is a fast overview of the Software Carpentry lesson template. This won't cover lesson style or formatting (address that during review?).

For a full guide to the lesson template, see the Software Carpentry example lesson.

Lesson structure

Software Carpentry lessons are generally episodic, with one clear concept for each episode (example).

An episode is a markdown file that lives under the _episodes folder. Here is a link to a markdown cheatsheet with most markdown syntax. Additionally, the Software Carpentry lesson template uses several extra bits of formatting- see here for a full guide. The most significant change is the addition of a YAML header that adds metadata (key questions, lesson teaching times, etc.) and special syntax for code blocks, exercises, and the like.

Episode names should be prefixed with a number of their section plus the number of their episode within that section. This is important because the Software Carpentry lesson template will auto-post our lessons in the order that they would sort in. As long as your lesson sorts into the correct order, it will appear in the correct order on the website.

Publishing changes to Github + the Github pages website

The lesson website is viewable at https://hpc-carpentry.github.io/hpc-shell/

The lesson website itself is auto-generated from the gh-pages branch of this repository. Github pages will rebuild the website as soon as you push to the Github gh-pages branch. Because of this gh-pages is considered the "master" branch.

Previewing changes locally

Obviously having to push to Github every time you want to view your changes to the website isn't very convenient. To preview the lesson locally, run make serve. You can then view the website at localhost:4000 in your browser. Pages will be automatically regenerated every time you write to them.

Note that the autogenerated website lives under the _site directory (and doesn't get pushed to Github).

This process requires Ruby, Make, and Jekyll. You can find setup instructions here.

Example lessons

Some links to example SWC workshop lessons for reference:

• Example Bash lesson
• Example Python lesson
• Example R lesson (uses R markdown files instead of markdown)