diff --git a/docs/rtd/source/user_guide/Private-Clusters.md b/docs/rtd/source/user_guide/Private-Clusters.md index a15514e51..2a7bfe204 100644 --- a/docs/rtd/source/user_guide/Private-Clusters.md +++ b/docs/rtd/source/user_guide/Private-Clusters.md @@ -18,51 +18,70 @@ For users of SlideRule that have access to their own funding, private clusters p ## Getting Started with Private Clusters -#### 1. Creating an Account +### 1. Creating an Account Create an account on the [SlideRule Provisioning System](https://ps.slideruleearth.io). Both local and GitHub social accounts are supported. -#### 2. Joining a Private Cluster +### 2. Joining a Private Cluster Log into the provisioning system. If you are already affiliated with an organization that has a private cluster on slideruleearth.io, then request membership to that organization using the "Request Membership" button under the organization's dropdown view. If you want to create your own organization, please reach out to the SlideRule science team via one of the methods described on our [Contact Us](https://slideruleearth.io/contact/) page. -#### 3. Authenticating +### 3. Authenticating to a Private Cluster -For local accounts, there are three ways to provide your credentials to the Python client: -1. Set up a __.netrc__ file in your home directory with the following entry: +For **local** accounts, there are three ways to provide your credentials to the Python client: + +Option 1. Set up a __.netrc__ file in your home directory with the following entry: ``` machine ps.slideruleearth.io login password ``` -2. Set environment variables with your username and password: + +Option 2. Set environment variables with your username and password: ```bash export PS_USERNAME= export PS_PASSWORD= ``` -3. Provide your credentials directly to the Python client: + +Option 3. Provide your credentials directly to the Python client: ```Python sliderule.authenticate("", ps_username="", ps_password="") ``` +For **GitHub** social accounts, there are two ways to provide your credentials to the Python client: + +Option 1. Set an environment variable with your access token: +```bash +export PS_GITHUB_TOKEN= +``` + +Option 2. Provide your credentials directly to the Python client: +```Python +sliderule.authenticate("", github_token="") +``` + +### 4. Accessing a Private Cluster + +To use a private cluster from your Python script, you must let the Python client know that you are accessing the private cluster instead of the public cluster. To accomplish that, you must supply the name of the organization to your initialization call: -4. Modify your Python scripts to supply your organization in the call to initialize the client, like so: ```Python -icesat2.init("slideruleearth.io", organization="", desired_nodes=<>) +sliderule.init("slideruleearth.io", organization="", ...) ``` -## Starting and Scaling a Private Cluster +> Note that behind the scenes, the `sliderule.init` call makes a call to`sliderule.authenticate` to automatically authenticate you as a user of the cluster. It will first look in the environment for your credentials, and then if not found there, will look for a .netrc file. If you want to provide credentials directly to the `sliderule.authenticate` call, then that can be done after the `sliderule.init` call. + +### 5. Scaling and Provisioning a Private Cluster -A private cluster is configured by the manager/owner of the cluster with a minimal number of nodes. If the minimal number of nodes is unsufficent for the processing the user wants to accomplish, the user can temporarily increase the number of nodes running for a user-defined amount of time, up to a maximum number of nodes set by the owner. This is called scaling the cluster. In some cases, the cluster is configured with a minimal number of nodes of zero. In this case, anyone wishing to use the cluster must first scale the cluster to at least one node before they are able to use it. +A private cluster is configured by the manager/owner of the cluster with a minimal number of nodes. If the minimal number of nodes is insufficent for the processing the user wants to accomplish, the user can temporarily increase the number of nodes running for a user-defined amount of time, up to a maximum number of nodes set by the owner. This is called scaling the cluster. In most cases, the cluster is configured with the minimal number of nodes set to zero. In this case, anyone wishing to use the cluster must first scale the cluster to at least one node before they are able to use it (we call this provisioning the cluster). -Scaling a cluster can be accomplished a few different ways depending on the needs of the user and whether or not the cluster is at zero nodes (fully shutdown) or not. +Scaling / provisioning a cluster can be accomplished a few different ways depending on the needs of the user and whether or not the cluster is at zero nodes (fully shutdown) or not. * If you want to guarantee that the cluster is up and that a minimal number of nodes is running before you do any data processing in your script, you can specify the desired number of nodes and a time to live in the `sliderule.init` call. For example: ```Python sliderule.init("slideruleearth.io", organization="{your_organization}", desired_nodes=5, time_to_live=60) # run 5 nodes for 60 minutes ``` -* If you want to make a non-blocking request to increase the number of nodes in the cluster, then you want to use `sliderule.update_available_servers`. This will allow you to keep using the cluster while more nodes are started. (Of course, if the cluster is not up at all, then subsequent requests to the cluster will fail until it all comes up). For example: +* If you want to make a non-blocking request to increase the number of nodes in the cluster, then use `sliderule.update_available_servers`. This will allow you to keep using the cluster while more nodes are started. (Of course, if the cluster is not up at all, then subsequent requests to the cluster will fail until it all comes up). For example: ```Python sliderule.update_available_servers(desired_nodes=5, time_to_live=60) # kick off starting 5 nodes for 60 minutes ``` -* If you want to make a blocking request to increase the number of nodes in the cluster, then you want to use `sliderule.scaleout`. This is the call that `sliderule.init` makes underneath, which means that using the `init` family of functions with the `desired_nodes` argument set, will also block. +* If you want to make a blocking request to increase the number of nodes in the cluster, then use `sliderule.scaleout`. This is the call that `sliderule.init` makes underneath, which means that using the `init` family of functions with the `desired_nodes` argument set, will also block. ```Python sliderule.scaleout(desired_nodes=5, time_to_live=60) # request and wait for 5 nodes for 60 minutes ```