Skip to content

Commit

Permalink
Add docs (#30)
Browse files Browse the repository at this point in the history
* update docstrings
* add docs for developers
  • Loading branch information
troyraen authored Jun 9, 2024
1 parent 80a82c6 commit 20db773
Show file tree
Hide file tree
Showing 5 changed files with 143 additions and 20 deletions.
42 changes: 42 additions & 0 deletions docs/source/for-developers/setup-development-mode.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Development Mode

Instructions for setting up development or "editable" mode are given below.
This is a method of pip-installing pointed at your local repository so you can iterate code and import changes for testing.

See also: [Python Packaging User Guide](https://packaging.python.org/en/latest/).

When you are ready to release a new version of `pittgoogle-client`, publish to PyPI using the release
process described in [issues #7](https://github.com/mwvgroup/pittgoogle-client/pull/7).

## Setup

```bash
# clone the repo and cd in
git clone https://github.com/mwvgroup/pittgoogle-client.git
cd pittgoogle-client

# recommended to create a new conda env
# use the latest python version unless you have a specific reason not to
conda create --name pittgoogle python=3.12
conda activate pittgoogle

# install pittgoogle-client in editable mode. use pwd so that the absolute path is registered.
pip install -e $(pwd)
```

## Work

Now you can work with the code in your local pittgoogle-client repo in python:

```python
import pittgoogle

# make new changes in your local pittgoogle-client repo code
# then use importlib to reload the package with the new changes
import importlib
importlib.reload(pittgoogle)
# if you don't have access to the new changes at this point, try reloading again
# if that doesn't work, restart your python interpreter
```

See also: [Working in “development mode”](https://packaging.python.org/guides/distributing-packages-using-setuptools/#working-in-development-mode).
7 changes: 7 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@
tutorials/cloud-storage
tutorials/ztf-figures

.. toctree::
:caption: For Developers
:maxdepth: 3
:hidden:

for-developers/setup-development-mode

.. toctree::
:caption: API Reference
:maxdepth: 3
Expand Down
92 changes: 76 additions & 16 deletions pittgoogle/alert.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,20 +47,21 @@
class Alert:
"""Pitt-Google container for an astronomical alert.
Recommended to instantiate using one of the `from_*` methods.
Don't call this directly.
Use one of the `from_*` methods instead.
All parameters are keyword only.
Parameters
------------
bytes : `bytes`, optional
----------
bytes : `bytes` (optional)
The message payload, as returned by Pub/Sub. It may be Avro or JSON serialized depending
on the topic.
dict : `dict`, optional
dict : `dict` (optional)
The message payload as a dictionary.
metadata : `dict`, optional
metadata : `dict` (optional)
The message metadata.
msg : `google.cloud.pubsub_v1.types.PubsubMessage`, optional
msg : `google.cloud.pubsub_v1.types.PubsubMessage` (optional)
The Pub/Sub message object, documented at
`<https://cloud.google.com/python/docs/reference/pubsub/latest/google.cloud.pubsub_v1.types.PubsubMessage>`__.
schema_name : `str`
Expand Down Expand Up @@ -91,7 +92,24 @@ class Alert:
def from_cloud_run(cls, envelope: Dict, schema_name: Optional[str] = None) -> "Alert":
"""Create an `Alert` from an HTTP request envelope containing a Pub/Sub message, as received by a Cloud Run module.
Example code for a Cloud Run module that uses this method to open a ZTF alert:
Parameters
----------
envelope : dict
The HTTP request envelope containing the Pub/Sub message.
schema_name : str (optional)
The name of the schema to use. Defaults to None.
Returns
-------
Alert : An instance of the `Alert` class.
Raises
------
BadRequest : If the Pub/Sub message is invalid or missing.
Example
-------
Code for a Cloud Run module that uses this method to open a ZTF alert:
.. code-block:: python
Expand All @@ -111,7 +129,7 @@ def index():
# if the request does not contain a valid message, this raises a `BadRequest`
alert = pittgoogle.Alert.from_cloud_run(envelope=flask.request.get_json(), schema_name="ztf")
except pg.exceptions.BadRequest as exc:
except pittgoogle.exceptions.BadRequest as exc:
# return the error text and an HTTP 400 Bad Request code
return str(exc), 400
Expand Down Expand Up @@ -151,20 +169,60 @@ def from_dict(
payload: Dict,
attributes: Optional[Union[Dict, "google._upb._message.ScalarMapContainer"]] = None,
schema_name: Optional[str] = None,
) -> "Alert": # [TODO] update tom_desc to use this
"""Create an `Alert` from a dictionary (`payload`)."""
) -> "Alert":
"""Create an `Alert` object from the given `payload` dictionary.
Parameters
----------
payload : dict
The dictionary containing the data for the `Alert` object.
attributes : dict or google._upb._message.ScalarMapContainer (optional)
Additional attributes for the `Alert` object. Defaults to None.
schema_name : str (optional)
The name of the schema. Defaults to None.
Returns
-------
Alert: An instance of the `Alert` class.
"""
return cls(dict=payload, attributes=attributes, schema_name=schema_name)

@classmethod
def from_msg(
cls, msg: "google.cloud.pubsub_v1.types.PubsubMessage", schema_name: Optional[str] = None
) -> "Alert": # [TODO] update tom_desc to use this
"""Create an `Alert` from a `google.cloud.pubsub_v1.types.PubsubMessage`."""
) -> "Alert":
"""
Create an `Alert` object from a `google.cloud.pubsub_v1.types.PubsubMessage`.
Parameters
----------
msg : `google.cloud.pubsub_v1.types.PubsubMessage`
The PubsubMessage object to create the Alert from.
schema_name : str (optional)
The name of the schema to use for the Alert. Defaults to None.
Returns
-------
Alert : The created `Alert` object.
"""
return cls(msg=msg, schema_name=schema_name)

@classmethod
def from_path(cls, path: Union[str, Path], schema_name: Optional[str] = None) -> "Alert":
"""Create an `Alert` from the file at `path`."""
"""Create an `Alert` object from the file at `path`.
Parameters
----------
path : str or Path
The path to the file containing the alert data.
schema_name : str, optional
The name of the schema to use for the alert, by default None.
Returns
-------
Alert
An instance of the `Alert` class.
"""
with open(path, "rb") as f:
bytes_ = f.read()
return cls(
Expand All @@ -178,7 +236,8 @@ def attributes(self) -> Dict:
If this was not set when the `Alert` was instantiated, a new dictionary will be created using
the `attributes` field in :attr:`pittgoogle.Alert.msg` the first time it is requested.
Update this dictionary as desired (it will not affect the original `msg`).
Update this dictionary as desired.
Updates will not affect the original `msg`.
When publishing the alert using :attr:`pittgoogle.Topic.publish`, this dictionary will be
sent as the Pub/Sub message attributes.
"""
Expand All @@ -193,7 +252,7 @@ def dict(self) -> Dict:
Raises
------
:class:`pittgoogle.exceptions.OpenAlertError`
if unable to deserialize the alert bytes.
If unable to deserialize the alert bytes.
"""
if self._dict is not None:
return self._dict
Expand Down Expand Up @@ -237,6 +296,7 @@ def dict(self) -> Dict:

@property
def dataframe(self) -> "pd.DataFrame":
"""Return a pandas DataFrame containing the source detections."""
if self._dataframe is not None:
return self._dataframe

Expand Down Expand Up @@ -307,7 +367,7 @@ def schema(self) -> types_.Schema:

# ---- methods ---- #
def add_id_attributes(self) -> None:
"""Add the IDs to the attributes."""
"""Add the IDs to the attributes. ("alertid", "objectid", "sourceid")"""
ids = ["alertid", "objectid", "sourceid"]
values = [self.get(id) for id in ids]

Expand Down
18 changes: 16 additions & 2 deletions pittgoogle/bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,10 @@ def from_cloud(
survey: Optional[str] = None,
testid: Optional[str] = None,
):
"""Create a `Table` with a `client` using implicit credentials (no explicit `auth`).
"""Create a `Table` object using a `client` with implicit credentials.
Useful when creating a `Table` object from within a Cloud Run module or similar.
The table in Google BigQuery is expected to exist already.
The `projectid` will be retrieved from the `client`.
Parameters
Expand Down Expand Up @@ -129,7 +131,7 @@ def auth(self) -> Auth:

@property
def id(self) -> str:
"""Fully qualified table ID."""
"""Fully qualified table ID with syntax "projectid.dataset_name.table_name"."""
return f"{self.projectid}.{self.dataset}.{self.name}"

@property
Expand Down Expand Up @@ -157,6 +159,18 @@ def client(self) -> bigquery.Client:
return self._client

def insert_rows(self, rows: Union[list[dict], list[Alert]]) -> list[dict]:
"""Inserts rows into the BigQuery table.
Parameters
----------
rows : list[dict] or list[Alert]
The rows to be inserted. Can be a list of dictionaries or a list of Alert objects.
Returns
-------
list[dict]
A list of errors encountered.
"""
# if elements of rows are Alerts, need to extract the dicts
myrows = [row.dict if isinstance(row, Alert) else row for row in rows]
errors = self.client.insert_rows(self.table, myrows)
Expand Down
4 changes: 2 additions & 2 deletions pittgoogle/types_.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
class Schema:
"""Class for an individual schema.
This class is not intended to be used directly. Instead, get a schema from the registry:
`pittgoogle.registry.Schemas`.
This class is not intended to be used directly.
Use `pittgoogle.registry.Schemas` instead.
"""

name: str = field()
Expand Down

0 comments on commit 20db773

Please sign in to comment.