Reorganize and update docs (#35)

* remove tutorials (moved to user-demos repo)

* rename docs overview -> main

* rename docs api -> api-reference

* update docs for-developers

* fix type checking for docs build

docs/source/api-reference/exceptions.rst

@@ -0,0 +1,7 @@
+pittgoogle.exceptions
+=====================
+
+.. automodule:: pittgoogle.exceptions
+   :members:
+   :private-members:
+   :member-order: bysource

docs/source/api-reference/registry.rst

@@ -0,0 +1,7 @@
+pittgoogle.registry
+===================
+
+.. automodule:: pittgoogle.registry
+   :members:
+   :private-members:
+   :member-order: bysource

docs/source/for-developers/manage-dependencies-poetry.md

@@ -0,0 +1,58 @@
+# Managing Dependencies with Poetry
+
+This page contains instructions for managing the `pittgoogle` package dependencies using [Poetry](https://python-poetry.org/).
+Poetry was implemented in this repo in [pull #7](https://github.com/mwvgroup/pittgoogle-client/pull/7).
+
+## Setup your environment
+
+Create a new conda environment for poetry and install it ([Poetry installation](https://python-poetry.org/docs/#installation)).
+
+```bash
+conda create --name poetry-py311 python=3.11
+conda activate poetry-py311
+
+# pipx is recommended, but it requires a brew install on MacOS and I (Raen) avoid brew whenever possible.
+# pip seems to work fine.
+pip install poetry
+```
+
+## Install existing dependencies
+
+This repo already contains a poetry.lock file, so running `poetry install` will give you
+the exact versions specified there ([Poetry install dependencies](https://python-poetry.org/docs/basic-usage/#installing-dependencies)).
+
+If you would rather start over completely, skip ahead to the next section.
+
+```bash
+poetry install
+```
+
+## Update Dependency Versions
+
+To upgrade to the latest versions compatible with the pyproject.toml file, you have two options below
+([Poetry update dependencies](https://python-poetry.org/docs/basic-usage/#updating-dependencies-to-their-latest-versions)):
+
+```bash
+# Option 1: Start over completely by deleting the lock file and re-installing.
+rm poetry.lock
+poetry install
+
+# Option 2: Update dependencies starting from the existing lock file (assumes you've run poetry install).
+poetry update
+```
+
+Now commit the updated poetry.lock file to the repo.
+
+## Add a Dependency
+
+Here are two examples
+([Poetry add dependencies](https://python-poetry.org/docs/managing-dependencies/#adding-a-dependency-to-a-group),
+see also: [Poetry version-constraint syntax](https://python-poetry.org/docs/dependency-specification/)):
+
+```bash
+# This example adds pandas to the main dependencies.
+poetry add pandas
+
+# This example adds sphinx to the docs dependencies.
+poetry add sphinx --group docs.dependencies
+```

docs/source/for-developers/release-new-version.md

@@ -0,0 +1,28 @@
+# Release a New Version of pittgoogle-client
+
+When you are ready to release a new version of `pittgoogle-client`, publish to PyPI using the following steps:
+
+1. Make sure the code in the main branch is ready for release.
+
+2. Make sure the CHANGELOG.md file has been updated to reflect the changes being released.
+
+3. On the repo's GitHub [releases](https://github.com/mwvgroup/pittgoogle-client/releases) page:
+    - Click "Draft a new release".
+    - Under "Choose a tag", enter the version tag as "v" followed by the release version
+      ([semantic versioning](https://semver.org/) MAJOR.MINOR.PATCH).
+    - Enter the same tag for the release title.
+    - Click "Publish release".
+
+Completing step 3 will:
+
+- Execute the test suite.
+- Publish the documentation to GitHub pages.
+- Publish the package to PyPI.org.
+
+You will now be able to install the new version using:
+
+```bash
+pip install --upgrade pittgoogle-client
+```
+
+This release process was implemented and described in [pull #7](https://github.com/mwvgroup/pittgoogle-client/pull/7).

docs/source/for-developers/setup-development-mode.md → docs/source/for-developers/setup-environment.md

@@ -1,12 +1,10 @@
-# Development Mode
+# Set up and Use a Developer Environment
 
-Instructions for setting up development or "editable" mode are given below.
-This is a method of pip-installing pointed at your local repository so you can iterate code and import changes for testing.
+Instructions for setting up an environment with `pittgoogle` installed in development or "editable" mode are given below.
+This is a method of pip-installing the package from your local files so that you have quick access to
+your changes as you develop code.
 
-See also: [Python Packaging User Guide](https://packaging.python.org/en/latest/).
-
-When you are ready to release a new version of `pittgoogle-client`, publish to PyPI using the release
-process described in [issues #7](https://github.com/mwvgroup/pittgoogle-client/pull/7).
+See also: [Working in “development mode”](https://packaging.python.org/guides/distributing-packages-using-setuptools/#working-in-development-mode).
 
 ## Setup
 
@@ -38,5 +36,3 @@ importlib.reload(pittgoogle)
 # if you don't have access to the new changes at this point, try reloading again
 # if that doesn't work, restart your python interpreter
 ```
-
-See also: [Working in “development mode”](https://packaging.python.org/guides/distributing-packages-using-setuptools/#working-in-development-mode).

docs/source/index.rst

@@ -7,38 +7,31 @@
     :maxdepth: 3
     :hidden:
 
-    Install<overview/install>
-    overview/authentication
-    overview/project
-    overview/env-vars
-    overview/cost
-    overview/adv-setup
-
-.. toctree::
-    :caption: Tutorials
-    :maxdepth: 3
-    :hidden:
-
-    tutorials/bigquery
-    tutorials/cloud-storage
-    tutorials/ztf-figures
+    main/listings
+    Install<main/one-time-setup/install>
+    main/one-time-setup/index
+    main/faq/index
 
 .. toctree::
     :caption: For Developers
     :maxdepth: 3
     :hidden:
 
-    for-developers/setup-development-mode
+    for-developers/setup-environment
+    for-developers/manage-dependencies-poetry
+    for-developers/release-new-version
 
 .. toctree::
     :caption: API Reference
     :maxdepth: 3
     :hidden:
 
-    api/auth
-    api/bigquery
-    api/pubsub
-    api/utils
+    api-reference/auth
+    api-reference/bigquery
+    api-reference/exceptions
+    api-reference/pubsub
+    api-reference/registry
+    api-reference/utils
 
 pittgoogle-client
 ==============================================
@@ -54,98 +47,3 @@ It is being developed  the `Pitt-Google alert broker <https://github.com/mwvgrou
 
 If you run into trouble, please
 `open an Issue <https://github.com/mwvgroup/pittgoogle-client/issues>`__.
-
-**Data overview**
-
-.. _data pubsub:
-
-Pub/Sub Message Streams
-=======================
-
-.. list-table:: Streams
-    :class: tight-table
-    :widths: 25 75
-    :header-rows: 1
-
-    * - Topic
-      - Description
-
-    * - ztf-alerts
-      - Full ZTF alert stream
-
-    * - ztf-lite
-      - Lite version of ztf-alerts (every alert, subset of fields)
-
-    * - ztf-tagged
-      - ztf-lite with basic categorizations such as “is pure” and “is likely extragalactic
-        transient” added to the message metadata.
-
-    * - ztf-SuperNNova
-      - ztf-tagged plus SuperNNova classification results (Ia vs non-Ia).
-
-    * - ztf-alert_avros
-      - Notification stream from the ztf-alert_avros Cloud Storage bucket indicating
-        that a new alert packet is in file storage.
-        These messages contain no data, only attributes.
-        The file name is in the attribute "objectId",
-        and the bucket name is in the attribute "bucketId".
-
-    * - ztf-BigQuery
-      - Notification stream indicating that alert data is available in BigQuery tables.
-
-    * - **ztf-loop**
-      - Use this stream for testing. Recent ZTF alerts are published to this topic
-        at a roughly constant rate of 1 per second.
-
-.. _data bigquery:
-
-BigQuery Catalogs
-==================
-
-.. list-table:: Datasets and Tables
-    :class: tight-table
-    :widths: 15 15 70
-    :header-rows: 1
-
-    * - Dataset
-      - Table
-      - Description
-
-    * - ztf_alerts
-      - alerts
-      - Complete alert packet, excluding image cutouts.
-        Same schema as the original alert, including nested and repeated fields.
-
-    * - ztf_alerts
-      - DIASource
-      - Alert packet data for the triggering source only. Including the object ID and a
-        list of source IDs for the previous sources included in the alert,
-        excluding cutouts and data for previous sources.
-        Flat schema.
-
-    * - ztf_alerts
-      - SuperNNova
-      - Results from a SuperNNova (Möller \& de Boissière, 2019)
-        Type Ia supernova classification (binary).
-
-    * - ztf_alerts
-      - metadata
-      - Information recording Pitt-Google processing (e.g., message publish times,
-        bucket name and filename, etc.).
-
-.. _data cloud storage:
-
-Cloud Storage
-====================
-
-.. list-table:: Buckets
-    :class: tight-table
-    :widths: 40 60
-    :header-rows: 1
-
-    * - Bucket Name
-      - Description
-
-    * - ardent-cycling-243415-ztf-alert_avros
-      - Contains the complete, original alert packets as Avro files.
-        Filename syntax is: `<ztf_topic>/<objectId>/<candid>.avro`

docs/source/overview/cost.rst → docs/source/main/faq/cost.rst

@@ -3,13 +3,20 @@
 Costs
 --------------
 
-Pitt-Google Broker makes astronomy data available in Google Cloud services like Pub/Sub, BigQuery, and Cloud Storage.
-Google provides a baseline level of access for free with no credit card or billing account required.
-The free tier is structured as a usage quota that renews monthly.
-Access beyond the free tier is "pay-as-you-go".
-Some examples are given in the table below.
-If you exceed the free-tier quota and have not set up billing, your access will be restricted until
-the quota renews.
+The Pitt-Google Alert Broker makes data available in Google Cloud repositories.
+The data are public and user-pays, meaning that anyone can access as much or little as they want, and everyone pays for what *they* use.
+Making the data available in this way can allow us to support a very large number of users.
+Payment goes to Google (not Pitt-Google Broker).
+All authentication and billing is managed through Google Cloud projects.
+
+Compared to more traditional computing costs, cloud charges are much smaller but more frequent.
+Some example charges are given in the table below.
+Small projects can run for free.
+Google provides a baseline level of "free tier" access, structured as a usage quota that renews monthly.
+No credit card or billing account is required.
+Other cost-offset options include $300 in free credits available to everyone, and $5000 in research credits available to many academics (see links below).
+Large projects can use as much as they want to pay for.
+Google's structure is "pay-as-you-go" with a monthly billing cycle, cancel at any time.
 
 .. list-table:: Pricing Examples (as of Aug. 2021)
     :class: tight-table

docs/source/main/faq/find-project-id.rst

@@ -0,0 +1,10 @@
+.. _find-project-id:
+
+Find the Project ID
+===================
+
+If you've created a :ref:`project <projects>` and need to know the ID:
+
+-   Go to the `Google Cloud Console <https://console.cloud.google.com/home/dashboard>`__.
+-   Click on the name of the project in the menu bar at the top of the page.
+-   From there you can see the names and IDs of all the projects you are connected to.

docs/source/main/faq/index.rst

@@ -0,0 +1,8 @@
+Frequently Asked Questions
+==============================================
+
+.. toctree::
+    :maxdepth: 3
+
+    cost
+    find-project-id