diff --git a/_quarto.yml b/_quarto.yml index 528f859..f32d11d 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -33,6 +33,18 @@ website: - overview.md - team.md - setup.qmd + - section: Orientation + contents: + - text: RStudio + href: content/02-rstudio.qmd + - text: Earthdata login + href: content/02-earthdata.md + - text: Local set-up + href: content/02-local-setup.qmd + - text: Git-JupyterLab + href: content/02-git-jupyter.md + - text: Git-RStudio + href: content/02-git-rstudio.md - coc.md - section: "Tutorials" contents: diff --git a/content/earthdata.md b/content/02-earthdata.md similarity index 100% rename from content/earthdata.md rename to content/02-earthdata.md diff --git a/content/02-git-jupyter.md b/content/02-git-jupyter.md index 3c10ab4..d1e7229 100644 --- a/content/02-git-jupyter.md +++ b/content/02-git-jupyter.md @@ -117,7 +117,7 @@ https://github.com/nmfs-opensci/Git-Lesson 3. Click "Clone a Repository" -4. Where it says "Enter the URI of the remote Git repository", paste in the URL https://github.com/nmfs-opensci/Git-Lesson +4. Where it says "Enter the URI of the remote Git repository", paste in the URL https://github.com/nmfs-opensci/EDMW-EarthData-Workshop-2024 5. The folder appears and you can enter the folder and edit and create files. diff --git a/content/03-git-rstudio.md b/content/02-git-rstudio.md similarity index 99% rename from content/03-git-rstudio.md rename to content/02-git-rstudio.md index fe85b1f..555e4cd 100644 --- a/content/03-git-rstudio.md +++ b/content/02-git-rstudio.md @@ -8,6 +8,7 @@ title: Basic Git/GitHub Skills Using RStudio - Set up Git in the JupyterHub - Learn how to use the Git GUI in RStudio +- Clone GitHub repositories to RStudio = get the repo into RStudio - Practice 4 basic Git/GitHub skills: cloning, committing, push/pull ::: diff --git a/content/02-github-2.md b/content/02-github-2.md deleted file mode 100644 index 279711c..0000000 --- a/content/02-github-2.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: GitHub - Lecture 2 ---- - -::: {.callout-note icon="false"} -## Goals - -In this lab, you will learn how to - -- import a GitHub repository = make a copy of a repository in your own account -- clone GitHub repositories to RStudio = get the repo into RStudio -- set up authentication so you can push up changes -::: - -## Import a GitHub repository - -1. In your browser, navigate to -2. Copy the URL -3. Click on the "+" in the top right (on GitHub) and select "import repository" -4. Follow instructions to import to your GitHub account -5. Now you have your OWN copy of `Quarto-Website` -6. Copy the URL for your OWN copy. It will look like this -``` -https://github.com/yourgithubusername/Quarto-Website -``` - -## Clone a GitHub repository into RStudio - -1. Login to the JupyterHub. **You might have to wait awhile** for the page to load. -2. Click on the RStudio button when the Launcher appears -![Jupyterhub Launcher](./img/jhub-launcher.png) -3. Click on the tab that opens with the R in a circle. You should see this -![RStudio](./img/rstudio.png) -3. Click File > New Project > Version Control > Git -4. Paste the URL into the box -5. Click "Create Project" -6. The project is created and the Git tab appears in the upper right panel. - -Congratulations! You **pulled** changes from GitHub into the JupyterHub! - -## Let's explore the files - -1. This happens to be a website. Let's **build** it using the Build tab in the upper right panel. -2. Click "Render Website". The website should build and appear. - - If it doesn't open automatically, go to docs > index.html, right click and open that in a browser. -3. Open the `about.qmd` file. Let's edit that! You can use the "Source" or "Visual" editor. -4. When you have made some edits, click "Render" to see how it looks. -5. You can also click Build > Render Website to re-create the whole website. - -## Set up authentication - -You need to tell GitHub who you are so you can **push** your local changes up to GitHub. There are a few ways to do this. I am going to show you a way that works on any computer, including a virtual computer like the JupyterHub. - -### Step 1: Generate a Personal Access Token - -We are going to generate a **classic** token. - -1. Go to https://github.com/settings/tokens -2. Click Generate new token > Generate new token (classic) -3. When the pop-up shows up, fill in a description, click the "repo" checkbox, and then scroll to bottom to click "Generate". -4. SAVE the token. You need it for the next step. - -### Step 2: Tell Git who your are - -1. Return to RStudio -2. Click the terminal tab in the bottom right panel -3. Paste these 3 lines of code into the terminal -``` -git config --global user.email "" -git config --global user.name "" -git config --global pull.rebase false -git config --global credential.helper store -``` - -### Step 3: Push changes up to GitHub - -1. Click the Git tab (upper right panel) -2. Click the checkboxes next to what changes you want to push -3. Click "Commit" -4. Fill out a comment and click "Commit" -5. Close the pop-ups -6. Click **Push** - -[Full instructions with ways to do this from R](https://rverse-tutorials.github.io/RWorkflow-NWFSC-2022/set-up.html#Git_from_RStudio_Desktop) - -## Optional: Make your website live! - -*For those who get through all the steps and want to play around more.* - -1. Go to your Quarto Website repository on Github -2. Go to Settings > Pages (in left navbar) -3. You need to select the branch and the folder where the webpage files are. In this case, they are in `docs` -![GitHub Pages](./img/github-pages.png) -4. The URL where your website will appear is shown. - -*Have you noticed that people have personal websites at username.github.io? How did they do that? If you create a repo named username.github.io in GitHub, it will automatically appear with that URL. Look for a repo with that name in someone's GitHub account if you see that they use that as their personal website.* - diff --git a/content/02-local-setup.qmd b/content/02-local-setup.qmd new file mode 100644 index 0000000..d900047 --- /dev/null +++ b/content/02-local-setup.qmd @@ -0,0 +1,41 @@ +--- +title: Setting up on your computer +--- + + +Here are instructions for installing on your own computer. + +Install the development version of earthdatalogin and update terra. + +```{r eval=FALSE} +devtools::install_github("boettiger-lab/earthdatalogin") +install.packages("terra") +install.packages("rstac") +install.packages("gdalcubes") +install.packages("here") + +library("earthdatalogin") +library("terra") +library("rstac") +library("gdalcubes") +library("here") +``` + +You will need GDAL installed. See these instructions if you do not have it installed: https://developers.planet.com/docs/integrations/qgis/install-qgis-gdal/ + +You may need to install `terra` and `sf` from source to get them to use the latest GDAL installation. + +```{r eval=FALSE} +install.packages("terra", type = "source") +install.packages("sf", type = "source") +sf_extSoftVersion() +``` + +The environment we are using today is the `py-rocket-geospatial` image. This is part of work on a [Data Science Docker Stack](https://github.com/nmfs-opensci/container-images) for NOAA Fisheries. + + +# R Version Metadata + +```{r} +sessionInfo() +``` diff --git a/content/git.md b/content/git.md deleted file mode 100644 index 6b84ee7..0000000 --- a/content/git.md +++ /dev/null @@ -1,299 +0,0 @@ ---- -title: Git - Jupyter Lab ---- - -## Summary - -In this tutorial, we will provide a brief introduction to: - -1. Command line (**terminal**/shell) -2. Navigating around folders in Jupyter Lab -3. Version Control (code management using **git**) -4. Setting up Git in Jupyter Lab -5. The Git GUI in Jupyter Lab -6. Basic Git commands - -## Introduction :: Jupyter Lab - -When you start the JupyterHub, you will be in Jupyter Lab. From there you can click on the RStudio box and open RStudio. However for this tutorial, we will stay in Juptyer Lab. - -## Introduction :: Terminal/Shell - -Log into the JupyterHub. If you do not see this - -![](img/jhub-launcher.png) - -Then go to File \> New Launcher - -Click on the "Terminal" box to open a new terminal window. - -### Shell or Terminal Basics - -1. [What is Terminal or Shell?](https://swcarpentry.github.io/shell-novice/01-intro/index.html) -2. [Navigating Files and Directories](https://swcarpentry.github.io/shell-novice/02-filedir/index.html) -3. [Working with Files and Directories](https://swcarpentry.github.io/shell-novice/03-create/index.html) -4. Optional: Detailed self-paced lesson on running scripts from the shell: [Shell Lesson from Software Carpentry](http://swcarpentry.github.io/shell-novice/) - -You will need only basic navigation skills for this course: `cd`, `ls` and `cat` - -- `pwd` where am I -- `cd nameofdir` move into a directory -- `cd ..` move up a directory -- `ls` list the files in the current directory -- `ls -a` list the files including hidden files -- `ls -l` list the files with more info -- `cat filename` print out the contents of a file - -### Let's try - -``` -ls -ls -a -cd shared -ls -cd shell-tutorial -cat lesson1.sh -cd .. -cd .. -``` - -### Close the terminal - -Just click on the X in the terminal tab - -## Introduction :: File Navigation - -In the far left, you will see a line of icons. The top one is a folder and allows us to move around our file system. - -1. Click on `shared`. Now you can see the files in the shared directory. - -2. Click on `shell-tutorial`. Then click on `lesson1.sh`. The file opens. You won't be able to save changes here because you don't have write permission on this drive. - -3. Click on the folder icon that looks like this. Click on the actual folder image. ![](img/folder-icon.png) - - Now it should look like this folder / - - This shows me doing this - - ![](img/folder-select.gif) - -4. Create a new folder. - - * Next to the blue rectange with a +, is a grey folder with a +. Click that to create a new folder, called `lesson-scripts`. - * Then click on `lesson-scripts` to enter the folder - - ![](img/folder-create-folder.gif) - -5. Create a new file - - * Create with File > New > Text file - * The file will open and you can edit it. - * Save with File > Save Text - * Delete the file by right-clicking on it and clicking "Delete" - -## Introduction :: Version Control (Git) - -### What is version control, git, github, and how to set it up? - -Version control is managing and tracking changes to your documents (program source code, images, websites, data files, etc.). `git` is a popular tool used for version control of software code. [github.com](https://github.com/) is popular platform that provides remote server hosting for git repositories. A repository is a collection of various files that you are tracking for changes and versions. Currently GitHub is the most popular platform for file sharing code and code packages. - -This section is a step-by-step guide to set up `git` on our [JupyterHub](https://dhub.opensci.live/). We will also configure `git` to use your [github.com](https://github.com/) account for managing your repositories hosted on [github.com](https://github.com/). There are 5 main steps. - -### Step 1: Create a GitHub account - -To complete the setup, you will need an account on [github.com](https://github.com/). If you don't have an account, please visit [github.com](https://github.com/), create an account (free) and come back to this guide for setting up git. - -### Step 2: Clone a repository - -We have created a demo repository for you to clone: - -https://github.com/nmfs-opensci/Git-Lesson - -1. Start your [JupyterHub](https://dhub.opensci.live/) - -2. Click on the Git icon - -![](img/gitextension.png) - -3. Click "Clone a Repository" - -4. Where is says "Enter the URI of the remote Git repository", paste in the URL https://github.com/nmfs-opensci/Git-Lesson - -5. The folder appears and you can enter the folder and edit and create files. - -> Your task: Create a file with your name and save to the Git-Lesson folder - -## Step 3: - -Configure git with your name and email address. - -```` -``` bash -git config --global user.name "Makhan Virdi" -git config --global user.email "Makhan.Virdi@gmail.com" -``` - -**Note:** This name and email could be different from your github.com credentials. Remember `git` is a program that keeps track of your changes locally (on 2i2c JupyterHub or your own computer) and github.com is a platform to host your repositories. However, since your changes are tracked by `git`, the email/name used in git configuration will show up next to your contributions on github.com when you `push` your repository to github.com (`git push` is discussed in a later step). -```` - -4. Configure `git` to store your github credentials to avoid having to enter your github username and token each time you push changes to your repository(in [Step 5](#step-5.-create-access-token-on-github.com), we will describe how to use github token instead of a password) - - ``` bash - git config --global credential.helper store - ``` - -5. Copy link for the demo repository from your github account. Click the green "Code" button and copy the link as shown. - - ![](img/github-clone.png) - -6. Clone the repository using `git clone` command in the terminal - - To clone a repository from github, copy the link for the repository (previous step) and use `git clone`: - - ``` bash - git clone https://github.com/YOUR-GITHUB-USERNAME/check_github_setup - ``` - - **Note:** Replace `YOUR-GITHUB-USERNAME` here with your github.com username. For example, it is `virdi` for my github.com account as seen in this image. - - ![](img/terminal-clone.png){width="520"} - - Use `ls` (list files) to verify the existence of the repository that you just cloned - - ![](img/terminal-clone-confirm-ls.png){width="520"} - -7. Change directory to the cloned repository using `cd check_github_setup` and check the current directory using `pwd` command (present working directory) - - ![](img/terminal-cd-repo.png){width="520"} - -8. Check status of your git repository to confirm git set up using `git status` - - ![](img/terminal-git-status.png) - - You are all set with using git on your 2i2c JupyterHub! But the collaborative power of git through github needs some additional setup. - - In the next step, we will create a new file in this repository, track changes to this file, and link it with your github.com account. - -### Step 4. Creating new file and tracking changes - -1. In the left panel on your 2i2c JupyterHub, click on the "directory" icon and then double click on "check_github_setup" directory. - - ![](img/notebook-cd.png) - - ![](img/notebook-cd-result.png) - -2. Once you are in the **check_github_setup** directory, create a new file using the text editor in your 2i2c JupyterHub (`File >> New >> Text File`). - - ![](img/%20notebook-new-file.png) - - Name the file lastname.txt. For example, **virdi.txt** for me (use your last name). Add some content to this file (for example, I added this to my **virdi.txt** file: `my last name is virdi`). - - ![](img/notebook-rename-file.png) - -3. Now you should have a new file (lastname.txt) in the git repository directory **check_github_setup** - -4. Check if `git` can see that you have added a new file using `git status`. Git reports that you have a new file that is not tracked by git yet, and suggests adding that file to the git tracking system. - - ![](img/git-status.png) - -5. As seen in this image, `git` suggests adding that file so it can be tracked for changes. You can add file to `git` for tracking changes using `git add`. Then, you can commit changes to this file's content using `git commit` as shown in the image. - - ``` bash - git add virdi.txt - git status - git commit -m "adding a new file" - git status - ``` - - ![](img/git-add-commit.png) - -6. As seen in the image above, `git` is suggesting to push the change that you just committed to the remote server at github.com (so that your collaborators can also see what changes you made). - - **Note: DO NOT** execute `push` yet. Before we push to github.com, let's configure `git` further and store our github.com credentials to avoid entering the credentials every time we invoke `git push`. For doing so, we need to create a **token** on github.com to be used in place of your github.com password. - -### Step 5. Create access token on github.com {#step-5.-create-access-token-on-github.com} - -1. Go to your github account and create a new "personal access token": - - [![Generate Personal Access Token on github.com](img/github-token.png)](https://github.com/settings/tokens/new) - -2. Enter a description in "Note" field as seen above, select "repo" checkbox, and scroll to the bottom and click the green button "Generate Token". Once generated, copy the token (or save it in a text file for reference). - - **IMPORTANT:** You will see this token only once, so be sure to copy this. If you do not copy your token at this stage, you will need to generate a new token. - - ![](img/github-token-generated.png) - -3. To push (transfer) your changes to github, use `git push` in terminal. It requires you to enter your github credentials. You will be prompted to enter your github username and "password". **When prompted for your "password", DO NOT use your github password, use the github token** that was copied in the previous step. - - ``` bash - git push - ``` - - ![](img/terminal-github-credentials.png) - - **Note:** When you paste your token in the terminal window, windows users will press Ctrl+V and mac os users will press Cmd+V. If it does not work, try generating another token and use the copy icon next to the token to copy the token. Then, paste using your computer's keyboard shortcut for paste. - -4. Now your password is stored in `~/.git-credentials` and you will not be prompted again unless the Github token expires. You can check the presence of this git-credentials file using Terminal. Here the `~` character represents your home directory (`/home/jovyan/`). - - ``` bash - ls -la ~ - ``` - - The output looks like this: - - ``` bash - drwxr-xr-x 13 jovyan jovyan 6144 Oct 22 17:35 . - drwxr-xr-x 1 root root 4096 Oct 4 16:21 .. - -rw------- 1 jovyan jovyan 1754 Oct 29 18:30 .bash_history - drwxr-xr-x 4 jovyan jovyan 6144 Oct 29 16:38 .config - -rw------- 1 jovyan jovyan 66 Oct 22 17:35 .git-credentials - -rw-r--r-- 1 jovyan jovyan 84 Oct 22 17:14 .gitconfig - drwxr-xr-x 10 jovyan jovyan 6144 Oct 21 16:19 2021-Cloud-Hackathon - ``` - - You can also verify your git configuration - - ``` bash - (notebook) jovyan@jupyter-virdi:~$ git config -l - ``` - - The output should have `credential.helper = store`: - - ``` bash - user.email = Makhan.Virdi@gmail.com - user.name = Makhan Virdi - credential.helper = store - ``` - -Now we are all set to collaborate with github on the JupyterHub during the Cloud Hackathon! - -### Summary: Git Commands - -| Git Command | Description | -|-------------|----------------------------------------------------------| -| `git status` | Shows the current state of the repository: the current working branch, files in the staging area, *etc.* | -| `git add` | Adds a new, previously untracked file to version control and marks already tracked files to be committed with the next `commit` | -| `git commit` | Saves the current state of the repository and creates an entry in the log | -| `git log` | Shows the history for the repository | -| `git diff` | Shows content differences between commits, branches, individual files and more | -| `git clone` | Copies a repository to your local environment, including all the history | -| `git pull` | Gets the latest changes of a previously cloned repository | -| `git push` | Pushes your local changes to the remote repository, sharing them with others | - -: Commonly used git commands (modified from [source](https://uwhackweek.github.io/jupyterbook-template/tutorials/jupyter.html)) - -### Git: More Details - -**Lesson**: For a more detailed self-paced lesson on git, visit [Git Lesson from Software Carpentry](http://swcarpentry.github.io/git-novice/) - -**Cheatsheet**: [Frequently used git commands](https://training.github.com/downloads/github-git-cheat-sheet.pdf) - -**Dangit, Git!?!**: If you are stuck after a git mishap, there are ready-made solutions to common problems at [Dangit, Git!?!](https://dangitgit.com/en) - -### Cloning our repository using the [**git Jupyter lab extension**](https://github.com/jupyterlab/jupyterlab-git). - -If we're already familiar with git commands and feel more confortable using a GUI our Jupyterhub deployment comes with a git extension. This plugin allows us to operate with git using a simple user interface. - -For example we can clone our repository using the extension. - -![git extension](img/gitextension.png) diff --git a/content/jamboard.png b/content/jamboard.png deleted file mode 100644 index 11ae7cd..0000000 Binary files a/content/jamboard.png and /dev/null differ diff --git a/content/jupyter-notebooks.md b/content/jupyter-notebooks.md deleted file mode 100644 index e4300a4..0000000 --- a/content/jupyter-notebooks.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -title: Jupyter Notebooks - Python ---- - -## Summary - -In this tutorial, we will provide a brief introduction to: - -1. Jupyter Notebooks in the JupyterHub -2. Programming in **Python** (using Jupyter Notebook) -3. Geospatial Fundamentals (optional, self-study) -4. Command line (**terminal**/shell) -5. Version Control (code management using **git**) - - -## Introduction :: Programming in Python - -Switch to Jupyter Notebook for an introduction to programming in Python - -- Variables (and mathematical operations) -- [Data Structures](https://swcarpentry.github.io/python-novice-inflammation/04-lists/index.html) (list, tuple, dict) -- [Flow Control](https://swcarpentry.github.io/python-novice-inflammation/05-loop/index.html) using loops (for, while) -- [Conditionals](https://swcarpentry.github.io/python-novice-inflammation/07-cond/index.html) (if, else, elif) -- [Functions](https://swcarpentry.github.io/python-novice-inflammation/08-func/index.html) -- [Errors and Exceptions](https://swcarpentry.github.io/python-novice-inflammation/09-errors/index.html) (understanding and handling errors) -- Using modules (libraries, packages) - - [pandas](https://pandas.pydata.org/docs/): high-performance, easy-to-use data structures and data analysis tools - - [rioxarray](https://corteva.github.io/rioxarray/stable/): based on the rasterio package for working with rasters and [`xarray`](http://xarray.pydata.org/en/stable/) - -### Python Learning Resources - -Self-paced lesson on [Programming with Python](https://swcarpentry.github.io/python-novice-inflammation/) from Software Carpentry - -## Introduction :: Geospatial Fundamentals (Optional) - -Detailed self-paced lesson on [Fundamentals of Geospatial Raster and Vector Data with Python](https://carpentries-incubator.github.io/geospatial-python/) from Data Carpentry - -## Jupyter Notebooks in Earth Sciences - -There are many Jupyter Notebooks that you can copy and run in our JupyterHub. Note our JupyterHub is set up for geospatial research and connection to NASA's data in the cloud. It is also set up for parallel processing with Dask. - -- [NASA Cloud Hackweek 2022](https://nasa-openscapes.github.io/2021-Cloud-Hackathon/) -- [OceanHackWeek](https://oceanhackweek.org) -- [ICESat-2 Hackweeks](https://icesat-2-2022.hackweek.io/intro.html) - -## Python/Conda environments - -Python users can create conda environments if they need to install modules. You create a `environments.yml` file like this: - -```yaml -name: nsidc -channels: - - conda-forge -dependencies: - - ipykernel - - awscli~=1.21.4 - - requests - - pip -``` - -And then in a terminal do this to activate the environment. -``` -conda env create -f environment.yml --name myenv -conda activate myenv -conda list -``` - - - diff --git a/content/notebooks.md b/content/notebooks.md deleted file mode 100644 index 7f2dbbd..0000000 --- a/content/notebooks.md +++ /dev/null @@ -1,270 +0,0 @@ ---- -title: RMarkdown, R, Git ---- - -## Summary - -In this session, we will provide a brief introduction to: - -1. Command line (**terminal**/shell) -2. Version Control (code management using **git**) -3. Programming in **Python** (using Jupyter Notebook) -4. Geospatial Fundamentals (optional, self-study) - -You will need a working knowledge of git and terminal for this hackathon. We will provide an overview of these topics and also share resources for self-paced learning. - -## Introduction :: Command Line (Terminal/Shell) - -### Shell Basics - -1. [What is Terminal or Shell?](https://swcarpentry.github.io/shell-novice/01-intro/index.html) -2. [Navigating Files and Directories](https://swcarpentry.github.io/shell-novice/02-filedir/index.html) -3. [Working with Files and Directories](https://swcarpentry.github.io/shell-novice/03-create/index.html) - -### Shell: More Details - -Detailed self-paced lesson on shell: [Shell Lesson from Software Carpentry](http://swcarpentry.github.io/shell-novice/) - -## Introduction :: Version Control (Git and Github) - - -### What is version control, git, github, and how to set it up? - -Version control is managing and tracking changes to your documents (program source code, images, websites, data files, etc.). `git` is a popular tool used for version control of software code. [github.com](https://github.com/) is popular platform that provides remote server hosting for git repositories. A repository is a collection of various files that you are tracking for changes and versions (think of it as a directory with files that are being tracked for changes, using `git` for taking snapshots of versions as you are developing). - - - - - -This section is a step-by-step guide to set up `git` on your [2i2c JupyterHub instance](openscapes.2i2c.cloud) (referred to as `2i2c JupyterHub` in these instruction). We will also configure `git` to use your [github.com](https://github.com/) account for managing your repositories hosted on [github.com](https://github.com/). There are 5 main steps with substeps, includes instruction for addressing github's new approach for [token authentication](https://github.blog/changelog/2021-08-12-git-password-authentication-is-shutting-down/). - -### Step 1: Create a github account - -To complete the setup, you will need an account on [github.com](https://github.com/). If you don't have an account, please visit [github.com](https://github.com/), create an account (free) and come back to this guide for setting up git. - -### Step 2: Fork a repository - -A **fork** is a copy of a repository from another github account (for example **NASA-Openscapes** account) to your github account (for example, my account **virdi**) that then you have permission to edit. To help you finish this setup correctly, we have created a demo repository on Openscapes github account named **check_github_setup**. You can **fork** this repository into your github account following these steps: - -1. Log in to your [github.com](https://github.com/) account - -2. Go to the demo repository at [NASA-Openscapes github](https://github.com/NASA-Openscapes/check_github_setup) - - ![Demo repository on NASA-Openscapes github](img/github-fork-1.png){width="520"} - -3. Click on the fork icon in the top right corner, as shown in the image below and click your user name if prompted to do so - - ![](img/github-fork-2.png){width="520"} - -### Step 3: Clone the repository that you just forked - -Now you have a fork of the demo repository in your github account that we can `clone` it in your 2i2c instance. In the code below, commands beginning with `git` is a git command for version control and synching; commands that don't start with `git` are bash/linux/command line commands. - -1. Start your 2i2c JupyterHub and open a terminal - - `File >> New >> Terminal` - -2. Make sure you are in your **home directory** by using`pwd` command and verifying the output as below - - `/home/jovyan` - - ![](img/terminal-pwd.png){width="520"} - -3. Configure git with your name and email address. - - ``` bash - git config --global user.name "Makhan Virdi" - git config --global user.email "Makhan.Virdi@gmail.com" - ``` - - **Note:** This name and email could be different from your github.com credentials. Remember `git` is a program that keeps track of your changes locally (on 2i2c JupyterHub or your own computer) and github.com is a platform to host your repositories. However, since your changes are tracked by `git`, the email/name used in git configuration will show up next to your contributions on github.com when you `push` your repository to github.com (`git push` is discussed in a later step). - -4. Configure `git` to store your github credentials to avoid having to enter your github username and token each time you push changes to your repository(in [Step 5](#step-5.-create-access-token-on-github.com), we will describe how to use github token instead of a password) - - ``` bash - git config --global credential.helper store - ``` - -5. Copy link for the demo repository from your github account. Click the green "Code" button and copy the link as shown. - - ![](img/github-clone.png) - -6. Clone the repository using `git clone` command in the terminal - - To clone a repository from github, copy the link for the repository (previous step) and use `git clone`: - - ```bash - git clone https://github.com/YOUR-GITHUB-USERNAME/check_github_setup - ``` - - **Note:** Replace `YOUR-GITHUB-USERNAME` here with your github.com username. For example, it is `virdi` for my github.com account as seen in this image. - - ![](img/terminal-clone.png){width="520"} - - Use `ls` (list files) to verify the existence of the repository that you just cloned - - ![](img/terminal-clone-confirm-ls.png){width="520"} - -7. Change directory to the cloned repository using `cd check_github_setup` and check the current directory using `pwd` command (present working directory) - - ![](img/terminal-cd-repo.png){width="520"} - -8. Check status of your git repository to confirm git set up using `git status` - - ![](img/terminal-git-status.png) - - You are all set with using git on your 2i2c JupyterHub! But the collaborative power of git through github needs some additional setup. - - In the next step, we will create a new file in this repository, track changes to this file, and link it with your github.com account. - -### Step 4. Creating new file and tracking changes - -1. In the left panel on your 2i2c JupyterHub, click on the "directory" icon and then double click on "check_github_setup" directory. - - ![](img/notebook-cd.png) - - ![](img/notebook-cd-result.png) - -2. Once you are in the **check_github_setup** directory, create a new file using the text editor in your 2i2c JupyterHub (`File >> New >> Text File`). - - ![](img/%20notebook-new-file.png) - - Name the file lastname.txt. For example, **virdi.txt** for me (use your last name). Add some content to this file (for example, I added this to my **virdi.txt** file: `my last name is virdi`). - - ![](img/notebook-rename-file.png) - -3. Now you should have a new file (lastname.txt) in the git repository directory **check_github_setup** - -4. Check if `git` can see that you have added a new file using `git status`. Git reports that you have a new file that is not tracked by git yet, and suggests adding that file to the git tracking system. - - ![](img/git-status.png) - -5. As seen in this image, `git` suggests adding that file so it can be tracked for changes. You can add file to `git` for tracking changes using `git add`. Then, you can commit changes to this file's content using `git commit` as shown in the image. - - ``` bash - git add virdi.txt - git status - git commit -m "adding a new file" - git status - ``` - - ![](img/git-add-commit.png) - -6. As seen in the image above, `git` is suggesting to push the change that you just committed to the remote server at github.com (so that your collaborators can also see what changes you made). - - **Note: DO NOT** execute `push` yet. Before we push to github.com, let's configure `git` further and store our github.com credentials to avoid entering the credentials every time we invoke `git push`. For doing so, we need to create a **token** on github.com to be used in place of your github.com password. - -### Step 5. Create access token on github.com {#step-5.-create-access-token-on-github.com} - -1. Go to your github account and create a new "personal access token": - - [![Generate Personal Access Token on github.com](img/github-token.png)](https://github.com/settings/tokens/new) - -2. Enter a description in "Note" field as seen above, select "repo" checkbox, and scroll to the bottom and click the green button "Generate Token". Once generated, copy the token (or save it in a text file for reference). - - **IMPORTANT:** You will see this token only once, so be sure to copy this. If you do not copy your token at this stage, you will need to generate a new token. - - ![](img/github-token-generated.png) - -3. To push (transfer) your changes to github, use `git push` in terminal. It requires you to enter your github credentials. You will be prompted to enter your github username and "password". **When prompted for your "password", DO NOT use your github password, use the github token** that was copied in the previous step. - - ``` bash - git push - ``` - - ![](img/terminal-github-credentials.png) - - **Note:** When you paste your token in the terminal window, windows users will press Ctrl+V and mac os users will press Cmd+V. If it does not work, try generating another token and use the copy icon next to the token to copy the token. Then, paste using your computer's keyboard shortcut for paste. - -4. Now your password is stored in `~/.git-credentials` and you will not be prompted again unless the Github token expires. You can check the presence of this git-credentials file using Terminal. Here the `~` character represents your home directory (`/home/jovyan/`). - - ``` bash - ls -la ~ - ``` - - The output looks like this: - - ``` bash - drwxr-xr-x 13 jovyan jovyan 6144 Oct 22 17:35 . - drwxr-xr-x 1 root root 4096 Oct 4 16:21 .. - -rw------- 1 jovyan jovyan 1754 Oct 29 18:30 .bash_history - drwxr-xr-x 4 jovyan jovyan 6144 Oct 29 16:38 .config - -rw------- 1 jovyan jovyan 66 Oct 22 17:35 .git-credentials - -rw-r--r-- 1 jovyan jovyan 84 Oct 22 17:14 .gitconfig - drwxr-xr-x 10 jovyan jovyan 6144 Oct 21 16:19 2021-Cloud-Hackathon - ``` - - You can also verify your git configuration - - ``` bash - (notebook) jovyan@jupyter-virdi:~$ git config -l - ``` - - The output should have `credential.helper = store`: - - ``` bash - user.email = Makhan.Virdi@gmail.com - user.name = Makhan Virdi - credential.helper = store - ``` - -Now we are all set to collaborate with github on the JupyterHub during the Cloud Hackathon! - -### Summary: Git Commands - -| Git Command | Description | -|--------------|---------------------------------------------------------------------------------------------------------------------------------| -| `git status` | Shows the current state of the repository: the current working branch, files in the staging area, *etc.* | -| `git add` | Adds a new, previously untracked file to version control and marks already tracked files to be committed with the next `commit` | -| `git commit` | Saves the current state of the repository and creates an entry in the log | -| `git log` | Shows the history for the repository | -| `git diff` | Shows content differences between commits, branches, individual files and more | -| `git clone` | Copies a repository to your local environment, including all the history | -| `git pull` | Gets the latest changes of a previously cloned repository | -| `git push` | Pushes your local changes to the remote repository, sharing them with others | - -: Commonly used git commands (modified from [source](https://uwhackweek.github.io/jupyterbook-template/tutorials/jupyter.html)) - -### Git: More Details - -**Lesson**: For a more detailed self-paced lesson on git, visit [Git Lesson from Software Carpentry](http://swcarpentry.github.io/git-novice/) - -**Cheatsheet**: [Frequently used git commands](https://training.github.com/downloads/github-git-cheat-sheet.pdf) - -**Dangit, Git!?!**: If you are stuck after a git mishap, there are ready-made solutions to common problems at [Dangit, Git!?!](https://dangitgit.com/en) - - -### Cloning our repository using the [**git Jupyter lab extension**](https://github.com/jupyterlab/jupyterlab-git). - -If we're already familiar with git commands and feel more confortable using a GUI our Jupyterhub deployment comes with a git extension. This plugin allows us to operate with git using a simple user interface. - -For example we can clone our repository using the extension. - -![git extension](img/gitextension.png) - -## Introduction :: Programming in Python - - -## Introduction :: Programming in Python - -Switch to Jupyter Notebook for an introduction to programming in Python - -- Variables (and mathematical operations) -- [Data Structures](https://swcarpentry.github.io/python-novice-inflammation/04-lists/index.html) (list, tuple, dict) -- [Flow Control](https://swcarpentry.github.io/python-novice-inflammation/05-loop/index.html) using loops (for, while) -- [Conditionals](https://swcarpentry.github.io/python-novice-inflammation/07-cond/index.html) (if, else, elif) -- [Functions](https://swcarpentry.github.io/python-novice-inflammation/08-func/index.html) -- [Errors and Exceptions](https://swcarpentry.github.io/python-novice-inflammation/09-errors/index.html) (understanding and handling errors) -- Using modules (libraries, packages) - - [pandas](https://pandas.pydata.org/docs/): high-performance, easy-to-use data structures and data analysis tools - - [rioxarray](https://corteva.github.io/rioxarray/stable/): based on the rasterio package for working with rasters and [`xarray`](http://xarray.pydata.org/en/stable/) - -### Python Learning Resources - -Self-paced lesson on [Programming with Python](https://swcarpentry.github.io/python-novice-inflammation/) from Software Carpentry - -## Introduction :: Geospatial Fundamentals (Optional) - -Detailed self-paced lesson on [Fundamentals of Geospatial Raster and Vector Data with Python](https://carpentries-incubator.github.io/geospatial-python/) from Data Carpentry - -The end! diff --git a/content/shell.md b/content/shell.md deleted file mode 100644 index 9e2d2f7..0000000 --- a/content/shell.md +++ /dev/null @@ -1,311 +0,0 @@ ---- -title: Terminal or Shell ---- - -## Summary - -In this tutorial, we will provide a brief introduction to - -1. What is the **terminal**/shell -2. How to get to the terminal in Jupyter Lab and RStudio -2. Navigating around folders in the terminal -3. Closing the terminal window - -## What is the terminal or shell? - -A way to interact with your computer from text commands instead of a graphical user interface (GUI). There are a few different types of shells and there are slight differences in syntax. However, we will be using very basic commands so the syntax difference won't affect us. - -When in a terminal window you can type `echo $0` to find out what shell type you are in. - -## Introduction :: Terminal/Shell - -### RStudio - -Log into the JupyterHub. If you do not see this - -![](img/jhub-launcher.png) - -Then go to File \> New Launcher - -Click on the RStudio box to open RStudio. - -### Jupyter Lab - -Log into the JupyterHub. If you do not see this - -![](img/jhub-launcher.png) - -Then go to File \> New Launcher - -Click on the "Terminal" box to open a new terminal window. - -### Shell or Terminal Basics - -1. [What is Terminal or Shell?](https://swcarpentry.github.io/shell-novice/01-intro/index.html) -2. [Navigating Files and Directories](https://swcarpentry.github.io/shell-novice/02-filedir/index.html) -3. [Working with Files and Directories](https://swcarpentry.github.io/shell-novice/03-create/index.html) -4. Optional: Detailed self-paced lesson on running scripts from the shell: [Shell Lesson from Software Carpentry](http://swcarpentry.github.io/shell-novice/) - -You will need only basic navigation skills for this course: `cd`, `ls` and `cat` - -- `pwd` where am I -- `cd nameofdir` move into a directory -- `cd ..` move up a directory -- `ls` list the files in the current directory -- `ls -a` list the files including hidden files -- `ls -l` list the files with more info -- `cat filename` print out the contents of a file - -### Let's try - -``` -ls -ls -a -cd shared -ls -cd shell-tutorial -cat lesson1.sh -cd .. -cd .. -``` - -### Close the terminal - -Just click on the X in the terminal tab - -## Introduction :: File Navigation - -In the far left, you will see a line of icons. The top one is a folder and allows us to move around our file system. - -1. Click on `shared`. Now you can see the files in the shared directory. - -2. Click on `shell-tutorial`. Then click on `lesson1.sh`. The file opens. You won't be able to save changes here because you don't have write permission on this drive. - -3. Click on the folder icon that looks like this. Click on the actual folder image. ![](img/folder-icon.png) - - Now it should look like this folder / - - This shows me doing this - - ![](img/folder-select.gif) - -4. Create a new folder. - - * Next to the blue rectange with a +, is a grey folder with a +. Click that to create a new folder, called `lesson-scripts`. - * Then click on `lesson-scripts` to enter the folder - - ![](img/folder-create-folder.gif) - -5. Create a new file - - * Create with File > New > Text file - * The file will open and you can edit it. - * Save with File > Save Text - * Delete the file by right-clicking on it and clicking "Delete" - -## Introduction :: Version Control (Git) - -### What is version control, git, github, and how to set it up? - -Version control is managing and tracking changes to your documents (program source code, images, websites, data files, etc.). `git` is a popular tool used for version control of software code. [github.com](https://github.com/) is popular platform that provides remote server hosting for git repositories. A repository is a collection of various files that you are tracking for changes and versions. Currently GitHub is the most popular platform for file sharing code and code packages. - -This section is a step-by-step guide to set up `git` on our [JupyterHub](https://dhub.opensci.live/). We will also configure `git` to use your [github.com](https://github.com/) account for managing your repositories hosted on [github.com](https://github.com/). There are 5 main steps. - -### Step 1: Create a GitHub account - -To complete the setup, you will need an account on [github.com](https://github.com/). If you don't have an account, please visit [github.com](https://github.com/), create an account (free) and come back to this guide for setting up git. - -### Step 2: Clone a repository - -We have created a demo repository for you to clone: - -https://github.com/nmfs-opensci/Git-Lesson - -1. Start your [JupyterHub](https://dhub.opensci.live/) - -2. Click on the Git icon - -![](img/gitextension.png) - -3. Click "Clone a Repository" - -4. Where is says "Enter the URI of the remote Git repository", paste in the URL https://github.com/nmfs-opensci/Git-Lesson - -5. The folder appears and you can enter the folder and edit and create files. - -> Your task: Create a file with your name and save to the Git-Lesson folder - -## Step 3: - -Configure git with your name and email address. - -```` -``` bash -git config --global user.name "Makhan Virdi" -git config --global user.email "Makhan.Virdi@gmail.com" -``` - -**Note:** This name and email could be different from your github.com credentials. Remember `git` is a program that keeps track of your changes locally (on 2i2c JupyterHub or your own computer) and github.com is a platform to host your repositories. However, since your changes are tracked by `git`, the email/name used in git configuration will show up next to your contributions on github.com when you `push` your repository to github.com (`git push` is discussed in a later step). -```` - -4. Configure `git` to store your github credentials to avoid having to enter your github username and token each time you push changes to your repository(in [Step 5](#step-5.-create-access-token-on-github.com), we will describe how to use github token instead of a password) - - ``` bash - git config --global credential.helper store - ``` - -5. Copy link for the demo repository from your github account. Click the green "Code" button and copy the link as shown. - - ![](img/github-clone.png) - -6. Clone the repository using `git clone` command in the terminal - - To clone a repository from github, copy the link for the repository (previous step) and use `git clone`: - - ``` bash - git clone https://github.com/YOUR-GITHUB-USERNAME/check_github_setup - ``` - - **Note:** Replace `YOUR-GITHUB-USERNAME` here with your github.com username. For example, it is `virdi` for my github.com account as seen in this image. - - ![](img/terminal-clone.png){width="520"} - - Use `ls` (list files) to verify the existence of the repository that you just cloned - - ![](img/terminal-clone-confirm-ls.png){width="520"} - -7. Change directory to the cloned repository using `cd check_github_setup` and check the current directory using `pwd` command (present working directory) - - ![](img/terminal-cd-repo.png){width="520"} - -8. Check status of your git repository to confirm git set up using `git status` - - ![](img/terminal-git-status.png) - - You are all set with using git on your 2i2c JupyterHub! But the collaborative power of git through github needs some additional setup. - - In the next step, we will create a new file in this repository, track changes to this file, and link it with your github.com account. - -### Step 4. Creating new file and tracking changes - -1. In the left panel on your 2i2c JupyterHub, click on the "directory" icon and then double click on "check_github_setup" directory. - - ![](img/notebook-cd.png) - - ![](img/notebook-cd-result.png) - -2. Once you are in the **check_github_setup** directory, create a new file using the text editor in your 2i2c JupyterHub (`File >> New >> Text File`). - - ![](img/%20notebook-new-file.png) - - Name the file lastname.txt. For example, **virdi.txt** for me (use your last name). Add some content to this file (for example, I added this to my **virdi.txt** file: `my last name is virdi`). - - ![](img/notebook-rename-file.png) - -3. Now you should have a new file (lastname.txt) in the git repository directory **check_github_setup** - -4. Check if `git` can see that you have added a new file using `git status`. Git reports that you have a new file that is not tracked by git yet, and suggests adding that file to the git tracking system. - - ![](img/git-status.png) - -5. As seen in this image, `git` suggests adding that file so it can be tracked for changes. You can add file to `git` for tracking changes using `git add`. Then, you can commit changes to this file's content using `git commit` as shown in the image. - - ``` bash - git add virdi.txt - git status - git commit -m "adding a new file" - git status - ``` - - ![](img/git-add-commit.png) - -6. As seen in the image above, `git` is suggesting to push the change that you just committed to the remote server at github.com (so that your collaborators can also see what changes you made). - - **Note: DO NOT** execute `push` yet. Before we push to github.com, let's configure `git` further and store our github.com credentials to avoid entering the credentials every time we invoke `git push`. For doing so, we need to create a **token** on github.com to be used in place of your github.com password. - -### Step 5. Create access token on github.com {#step-5.-create-access-token-on-github.com} - -1. Go to your github account and create a new "personal access token": - - [![Generate Personal Access Token on github.com](img/github-token.png)](https://github.com/settings/tokens/new) - -2. Enter a description in "Note" field as seen above, select "repo" checkbox, and scroll to the bottom and click the green button "Generate Token". Once generated, copy the token (or save it in a text file for reference). - - **IMPORTANT:** You will see this token only once, so be sure to copy this. If you do not copy your token at this stage, you will need to generate a new token. - - ![](img/github-token-generated.png) - -3. To push (transfer) your changes to github, use `git push` in terminal. It requires you to enter your github credentials. You will be prompted to enter your github username and "password". **When prompted for your "password", DO NOT use your github password, use the github token** that was copied in the previous step. - - ``` bash - git push - ``` - - ![](img/terminal-github-credentials.png) - - **Note:** When you paste your token in the terminal window, windows users will press Ctrl+V and mac os users will press Cmd+V. If it does not work, try generating another token and use the copy icon next to the token to copy the token. Then, paste using your computer's keyboard shortcut for paste. - -4. Now your password is stored in `~/.git-credentials` and you will not be prompted again unless the Github token expires. You can check the presence of this git-credentials file using Terminal. Here the `~` character represents your home directory (`/home/jovyan/`). - - ``` bash - ls -la ~ - ``` - - The output looks like this: - - ``` bash - drwxr-xr-x 13 jovyan jovyan 6144 Oct 22 17:35 . - drwxr-xr-x 1 root root 4096 Oct 4 16:21 .. - -rw------- 1 jovyan jovyan 1754 Oct 29 18:30 .bash_history - drwxr-xr-x 4 jovyan jovyan 6144 Oct 29 16:38 .config - -rw------- 1 jovyan jovyan 66 Oct 22 17:35 .git-credentials - -rw-r--r-- 1 jovyan jovyan 84 Oct 22 17:14 .gitconfig - drwxr-xr-x 10 jovyan jovyan 6144 Oct 21 16:19 2021-Cloud-Hackathon - ``` - - You can also verify your git configuration - - ``` bash - (notebook) jovyan@jupyter-virdi:~$ git config -l - ``` - - The output should have `credential.helper = store`: - - ``` bash - user.email = Makhan.Virdi@gmail.com - user.name = Makhan Virdi - credential.helper = store - ``` - -Now we are all set to collaborate with github on the JupyterHub during the Cloud Hackathon! - -### Summary: Git Commands - -| Git Command | Description | -|-------------|----------------------------------------------------------| -| `git status` | Shows the current state of the repository: the current working branch, files in the staging area, *etc.* | -| `git add` | Adds a new, previously untracked file to version control and marks already tracked files to be committed with the next `commit` | -| `git commit` | Saves the current state of the repository and creates an entry in the log | -| `git log` | Shows the history for the repository | -| `git diff` | Shows content differences between commits, branches, individual files and more | -| `git clone` | Copies a repository to your local environment, including all the history | -| `git pull` | Gets the latest changes of a previously cloned repository | -| `git push` | Pushes your local changes to the remote repository, sharing them with others | - -: Commonly used git commands (modified from [source](https://uwhackweek.github.io/jupyterbook-template/tutorials/jupyter.html)) - -### Git: More Details - -**Lesson**: For a more detailed self-paced lesson on git, visit [Git Lesson from Software Carpentry](http://swcarpentry.github.io/git-novice/) - -**Cheatsheet**: [Frequently used git commands](https://training.github.com/downloads/github-git-cheat-sheet.pdf) - -**Dangit, Git!?!**: If you are stuck after a git mishap, there are ready-made solutions to common problems at [Dangit, Git!?!](https://dangitgit.com/en) - -### Cloning our repository using the [**git Jupyter lab extension**](https://github.com/jupyterlab/jupyterlab-git). - -If we're already familiar with git commands and feel more confortable using a GUI our Jupyterhub deployment comes with a git extension. This plugin allows us to operate with git using a simple user interface. - -For example we can clone our repository using the extension. - -![git extension](img/gitextension.png) diff --git a/docs/cloud-paradigm.html b/docs/cloud-paradigm.html index d1c232f..620f988 100644 --- a/docs/cloud-paradigm.html +++ b/docs/cloud-paradigm.html @@ -165,9 +165,50 @@ Set-up + +
  • +
    + + +
    + +
    • +
  • +
    Code of Conduct
    @@ -176,22 +217,22 @@
  • - -
    -