-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.Rmd
134 lines (96 loc) · 5.46 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r setup, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%",
eval = FALSE
)
```
```{r sticker, eval=FALSE, include=FALSE}
library(extrafont)
library(hexSticker)
sticker(subplot = "man/figures/slu_icon_church.png",
s_x = 1,
s_y = 1.15,
s_width = 0.4,
s_height = 0.4,
package = "sluhpc",
p_x = 1,
p_y = 0.45,
p_color = "#003da5",
p_family = "Crimson Text",
p_size = 5,
h_fill = "#ffffff",
h_color = "#003da5",
url = "github.com/Saint-Louis-University/sluhpc",
u_color = "#003da5",
u_family = "Crimson Text",
u_size = 1.1,
u_x = 1.005,
u_y = 0.1,
filename = "man/figures/sluhpc.png")
```
# sluhpc <img src="man/figures/sluhpc.png" align="right" width="125px" />
<!-- badges: start -->
[](https://github.com/Saint-Louis-University/sluhpc)
[](https://github.com/Saint-Louis-University/sluhpc/commits/master)
[](https://travis-ci.org/Saint-Louis-University/sluhpc)
[](https://ci.appveyor.com/project/Saint-Louis-University/sluhpc)
[](https://codecov.io/gh/Saint-Louis-University/sluhpc?branch=master)
<!-- badges: end -->
## Overview
The goal of `sluhpc` is to simplify the [parallelization](https://en.wikipedia.org/wiki/Parallel_computing) and execution of [R](https://en.wikipedia.org/wiki/R_(programming_language)) code on the [Saint Louis University](https://www.slu.edu) (SLU) [High Performance Cluster](https://apex.slu.edu) (HPC).
<br />
## Installation
You can install `sluhpc` from [GitHub](https://github.com/Saint-Louis-University/sluhpc) with:
```{r, eval=FALSE}
remotes::install_github("Saint-Louis-University/sluhpc")
```
<br />
## Notes
### R Version
The current version of R on the cluster is [Microsoft R Open](https://mran.microsoft.com/open) 3.3.2. You may avoid many potential errors by working in a local copy of R circa 3.3.2.
### IP Restriction
The cluster only accepts connections from [IP addresses](https://en.wikipedia.org/wiki/IP_address) registered to SLU. If working off campus, you will need to log into the [VPN](https://vpn.slu.edu/+CSCOE+/logon.html) using your SLU Net ID and password.
### Credentials
By default, credentials to connect to the HPC are read from the environment variables `APEX.SLU.EDU_USER` and `APEX.SLU.EDU_PASS` using [`base::Sys.getenv()`](https://www.rdocumentation.org/packages/base/versions/3.6.1/topics/Sys.getenv). These variables are commonly set via a [.Renviron file](https://www.rdocumentation.org/packages/base/versions/3.6.1/topics/Startup).
<br />
## Example
This is a three-step example which shows how to execute R code in parallel on the cluster.
### Step 1
We define a function and construct a corresponding parameter set. Then we create a local folder containing of all the files necessary to run the code in parallel on the cluster.
```{r}
library(sluhpc)
my_function <- function(parameter_mu, parameter_sd) {
sample <- rnorm(10^6, parameter_mu, parameter_sd)
c(sample_mu = mean(sample), sample_sd = sd(sample))
}
my_parameters <- data.frame(parameter_mu = 1:10,
parameter_sd = seq(0.1, 1, length.out = 10))
slurm_job <- slurm_apply(my_function,
my_parameters,
"my_apply")
```
### Step 2
We open a [secure shell](https://en.wikipedia.org/wiki/Secure_Shell) (SSH) connection to the cluster using credentials stored in your [.Renviron file](https://www.rdocumentation.org/packages/base/versions/3.6.1/topics/Startup), upload the previously created local folder, and submit the job to the [Slurm Workload Manager](https://en.wikipedia.org/wiki/Slurm_Workload_Manager).
```{r}
session <- apex_connect()
slurm_upload(session, slurm_job)
slurm_submit(session, slurm_job)
```
### Step 3
The `slurm_download()` function will block until the job has completed running on the cluster and then download the results via [SCP](https://en.wikipedia.org/wiki/Secure_copy). We can then bind the results from each node together into a data frame object, and disconnect our SSH session.
```{r}
slurm_download(session, slurm_job)
results <- slurm_output_dfr(slurm_job)
apex_disconnect(session)
```
<br />
## About
### Saint Louis University <img src="man/figures/edu.slu.marcom-logowithyear_rgb.png" align="right" width="125px" />
Founded in 1818, [Saint Louis University](https://www.slu.edu) is one of the nation’s oldest and most prestigious Catholic institutions. Rooted in Jesuit values and its pioneering history as the first university west of the Mississippi River, SLU offers nearly 13,000 students a rigorous, transformative education of the whole person. At the core of the University’s diverse community of scholars is SLU’s service-focused mission, which challenges and prepares students to make the world a better, more just place.