documentation updates and fixes

SlideRuleEarth · Sep 30, 2023 · 5f64f81 · 5f64f81
1 parent 9af28a7
commit 5f64f81
Show file tree

Hide file tree

Showing 6 changed files with 40 additions and 16 deletions.
diff --git a/docs/jekyll/index.html b/docs/jekyll/index.html
@@ -13,7 +13,7 @@
 opening:
 - title: 'What Is SlideRule?'
 intro:
-- excerpt: '___SlideRule___ is a public web service implemented in C++/Lua that provides REST APIs for processing science data and returning results. This enables researchers and other data systems to have low-latency access to generated data products using processing parameters supplied at the time of the request.  _SlideRule_ runs in AWS us-west-2 and has access to ICESat-2, GEDI, Landsat, and ArcticDEM datasets stored in S3. While its web services can be accessed by any http client (e.g. curl), [Python](https://github.com/ICESat2-SlideRule/sliderule) and [Node.js](https://github.com/ICESat2-SlideRule/sliderule) clients are provided to make it easier to interact with _SlideRule_.'
+- excerpt: '___SlideRule___ is a public web service with REST APIs for processing science data and returning results. It provides researchers and other data systems with low-latency access to generated data products using processing parameters supplied at the time of the request.  _SlideRule_ runs in AWS us-west-2 and has access to ICESat-2, GEDI, Landsat, and a growing list of other datasets stored in S3. While its web services can be accessed by any http client (e.g. curl), [Python](https://github.com/ICESat2-SlideRule/sliderule) and [Node.js](https://github.com/ICESat2-SlideRule/sliderule) clients are provided to make it easier to interact with _SlideRule_.'
 feature_row:
   - image_path: /assets/images/doc_frontpage.jpg
     alt: "Documentation"

diff --git a/docs/rtd/source/api_reference/earthdata.rst b/docs/rtd/source/api_reference/earthdata.rst
@@ -32,3 +32,9 @@ stac
 tnm
 ---
 .. autofunction:: sliderule.earthdata.tnm
+
+""""""""""""""""
+
+search
+------
+.. autofunction:: sliderule.earthdata.search
diff --git a/docs/rtd/source/api_reference/raster.rst b/docs/rtd/source/api_reference/raster.rst
@@ -16,3 +16,8 @@ sample
 .. autofunction:: sliderule.raster.sample
 
 """"""""""""""""
+
+subset
+------
+.. autofunction:: sliderule.raster.subset
+
diff --git a/docs/rtd/source/index.rst b/docs/rtd/source/index.rst
@@ -4,20 +4,28 @@ SlideRule
 
 **Process Earth science datasets in the cloud through REST API calls to SlideRule web services.**
 
-SlideRule is a web service for on-demand science data processing, which provides researchers and other Earth science data systems low-latency access to customized data products using processing parameters supplied at the time of the request. SlideRule runs in AWS us-west-2 and has access to ICESat-2, GEDI, Landsat, and ArcticDEM datasets stored in S3.
+SlideRule is a web service for on-demand science data processing, which provides researchers and other Earth science data systems low-latency access to customized data products using processing parameters supplied at the time of the request. SlideRule runs in AWS us-west-2 and has access to ICESat-2, GEDI, Landsat, ArcticDEM, REMA, and other datasets stored in S3 (see `Assets <user_guide/SlideRule.html#assets>`_ for a full list).
 
+"Using SlideRule" typically means running a Python script you've developed to analyze Earth science data, and in that script calling functions in the **sliderule** Python package to make processing requests to SlideRule web services to perform some of the data intensive parts of your analysis.  Most of the documentation and examples we provide are focused on this use-case.  We do provide other means of interacting with SlideRule (most notably the current demo and future web client), but those features are still under development and documentation for them is sparse.
 
 What It Is
 ----------
 
 The SlideRule project consists of mulitple resources available to the public:
+
 - **Public Cluster**: The public cluster is the heart of the sliderule system, and can be reached at https://sliderule.slideruleearth.io.  It consists of a bunch of servers in AWS that provide the web services to process science data.  When someone "makes a request to SlideRule", they are, in affect, sending an HTTP request to the public cluster to process data and return the results.
+
 - **Documentation Website**:  This is what you are reading right now, and is located at https://slideruleearth.io/web/rtd/index.html.  The documentation is provided via the Read-The-Docs framework and includes information on how to install and use SlideRule.
+
 - **Web Interface**: The current web interface for sliderule is a demo built using Voila and a Jupyter Notebook; it can be reached at https://demo.slideruleearth.io. We are currently working on proper web interface for sliderule which will supercede the demo.
+
 - **Python Client**: The primary method of interacting with SlideRule is via the Python client, installable via conda or pip (both under the name sliderule).  
+
 - **Node.js Client (beta)**: In support of the future web client, a Node.js client has been written for sliderule.  It is installable via npm under the name @sliderule/sliderule.
+
 - **Github Repositories**: All code for SlideRule is open source and openly developed at https://github.com/ICESat2-SlideRule/sliderule.  Anyone can download/fork/browse the code, open issues, and post discussions.
-- **Provisioning System**: Behind the scense, all assets in AWS needed to run a SlideRule cluster is deployed and cost-monitored by the SlideRule Provisioning System.  The provisioning system can be used to run private clusters as well.  Please `contact us </web/contact/>`_ if you are interested in a private cluster. 
+
+- **Provisioning System**: Behind the scense, all assets in AWS needed to run a SlideRule cluster are deployed and cost-monitored by the SlideRule Provisioning System.  The provisioning system can be used to run private clusters as well.  Please `contact us </web/contact/>`_ if you are interested in a private cluster. 
 
 
 Where To Begin
@@ -85,7 +93,7 @@ The SlideRule project is funded by NASA's ICESat-2 program and is led by the Uni
    :maxdepth: 1
    :caption: Getting Started
 
-   getting_started/index.rst
+   index.rst
    getting_started/Install.rst
    getting_started/Getting-Started.rst
    getting_started/Examples.rst

diff --git a/docs/rtd/source/release_notes/release-v4.0.0.md b/docs/rtd/source/release_notes/release-v4.0.0.md
@@ -8,20 +8,20 @@ Version description of the v4.0.0 release of ICESat-2 SlideRule.  This document
 
 This version contains a number of backward-incompatible changes, specifically to the names of the fields being returned by the `atl03s` and `atl06` APIs, and the Python client function APIs.  These changes were made to standardize the downstream processing of the photon and elevation data, and also to bring the names of the fields being returned by SlideRule closer to the ICESat-2 Standard Data Products.
 
-1. For `atl03s` and `atl03sp` APIs:
-* `distance` is now `x_atc`
-* Across track distance is now `y_atc`
+* For `atl03s` and `atl03sp` APIs:
+  * `distance` is now `x_atc`
+  * Across track distance is now `y_atc`
 
-2. For `atl06` and `atl06p` APIs:
-* `distance` is now `x_atc`
-* Across track distance is now `y_atc`
-* `dh_fit_dy` has been removed
-* `lon` is now `longitude`
-* `lat` is now `latitude`
+* For `atl06` and `atl06p` APIs:
+  * `distance` is now `x_atc`
+  * Across track distance is now `y_atc`
+  * `dh_fit_dy` has been removed
+  * `lon` is now `longitude`
+  * `lat` is now `latitude`
 
-3. The compact ATL06 record `atl06rec-compact` has been removed entirely and is no longer supported.
+* The compact ATL06 record `atl06rec-compact` has been removed entirely and is no longer supported.
 
-4. The `asset` parameter has been removed from the Python client functions and is now managed internally by the client.  Users can still override the asset by supplying an `asset` key in the request parameters; but going forward users should not generally need to worry about which assets the servers are using.
+* The `asset` parameter has been removed from the Python client functions and is now managed internally by the client.  Users can still override the asset by supplying an `asset` key in the request parameters; but going forward users should not generally need to worry about which assets the servers are using.
 
 ## Major Changes
 
@@ -62,7 +62,7 @@ parms = { "poly": region['poly'],
 
 * Overhaul of how ancillary fields are handled by the server-side code.  A new system-wide record object type has been added called a "container record".  A container record holds inside of it other records that are grouped and processed together.  Ancillary data associated with an ATL03 extent or ATL06 elevation is now placed in a container record along with the corresponding ATL03 or ATL06 record.
 
-* Parquet file generation has been optimized and the gzip compression has been replaced with snappy compression.
+* Parquet file generation has been optimized and the gzip compression has been replaced with snappy compression.  Also the ability to generate a non-geo parquet file has been fixed, where the latitude and longitude are present as normal columns and there is no geometry column.
 
 * Default ICESat-2 products used now set to version 006.
 
@@ -78,6 +78,8 @@ parms = { "poly": region['poly'],
 
 * Major rewrite of GDAL raster sampling code: new class structure has GdalRaster as a stand-alone class, with RasterObject parenting a GeoRaster and a GeoIndexedRaster for mosaic and strip datasets respectively.
 
+* The `latest` tagged docker images for the sliderule compute node, the intelligent load balancer, the proxy, and the monitor, are now all built automatically on every push to `main` by a "Build and Deploy" GitHub Action workflow.  The new images are used when the automated Pytests are kicked off for the `developers` cluster.
+
 ## Issues Resolved
 
 * [#182](https://github.com/ICESat2-SlideRule/sliderule/issues/182) - Across track slope not calculated for ATL06

diff --git a/targets/slideruleearth-aws/Makefile b/targets/slideruleearth-aws/Makefile
@@ -174,6 +174,9 @@ static-website-docker: ## make the static website docker image; needs VERSION
 	docker tag $(ECR)/static-website:latest $(ECR)/static-website:$(VERSION)
 	docker tag $(ECR)/static-website:latest $(ECR)/static-website:$(MAJOR_VERSION)
 
+static-website-run: ## locally run the static website; needs VERSION
+	docker run -it -p 8080:4000 --rm --name website $(ECR)/static-website:$(VERSION)
+
 static-website-deploy: ## deploy static-website using terraform; needs VERSION, DOMAIN
 	cd terraform/static-website && terraform init
 	cd terraform/static-website && terraform workspace select $(DOMAIN)-static-website || terraform workspace new $(DOMAIN)-static-website