Skip to content

Commit

Permalink
Merge branch 'dev'
Browse files Browse the repository at this point in the history
  • Loading branch information
jchate6 committed Mar 28, 2024
2 parents 58c0f22 + 32dba3d commit cbb1d19
Show file tree
Hide file tree
Showing 23 changed files with 1,186 additions and 347 deletions.
1 change: 1 addition & 0 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.autosectionlabel',
'sphinx.ext.autosummary',
'sphinx.ext.intersphinx',
'sphinx_panels'
Expand Down
47 changes: 32 additions & 15 deletions docs/managing_data/forced_photometry.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
Integrating Forced Photometry Service Queries
---------------------------------------------

The base TOM Toolkit comes with Atlas, panSTARRS, and ZTF query services. More services
can be added by extending the base ForcedPhotometryService implementation.
The base TOM Toolkit comes with `ATLAS <https://fallingstar-data.com/forcedphot/>`__,
`PanSTARRS <https://outerspace.stsci.edu/display/PANSTARRS>`__,
and (coming soon) ZTF query services. These services are optional and require additional configuration
integrate into your TOM.

Additional services can be added by extending the base ``ForcedPhotometryService`` implementation
(:ref:`see below<Adding a new Forced Photometry Service>`).


Integrating existing Forced Photometry Services
Expand All @@ -20,32 +25,41 @@ shown below:
'url': "https://fallingstar-data.com/forcedphot",
'api_key': os.getenv('ATLAS_FORCED_PHOTOMETRY_API_KEY', 'your atlas account api token')
},
# TODO: these services are coming soon...
# 'PANSTARSS': {
# },
# 'ZTF': {
'PANSTARRS': {
'class': 'tom_dataproducts.forced_photometry.panstarrs_service.panstarrs.PanstarrsForcedPhotometryService',
'url': 'https://catalogs.mast.stsci.edu/api/v0.1/panstarrs', # MAST Base URL
# MAST_API_TOKEN is not required for public data
'api_key': os.getenv('MAST_API_TOKEN', 'MAST_API_TOKEN not set')
},
# TODO: coming soon...
# # 'ZTF': {
# }
}
DATA_PRODUCT_TYPES = {
...
'atlas_photometry': ('atlas_photometry', 'Atlas Photometry'),
'panstarrs_photometry': ('panstarrs_photometry', 'PanSTARRS Photometry'),
...
}
DATA_PROCESSORS = {
...
'atlas_photometry': 'tom_dataproducts.processors.atlas_processor.AtlasProcessor',
'panstarrs_photometry': 'tom_dataproducts.processors.panstarrs_processor.PanstarrsProcessor',
...
}
As you can see in the ``FORCED_PHOTOMETRY_SERVICES`` configuration dictionary above, some services require an API key.
Information on how to obtain an API key is available for both for `ATLAS <https://fallingstar-data.com/forcedphot/apiguide/>`_
and for `PanSTARRS <https://auth.mast.stsci.edu/info>`_. (PanSTARRS Forced Photometry is accessed via `Catalogs.MAST <https://catalogs.mast.stsci.edu/>`_).

Configuring your TOM to serve tasks asynchronously:
***************************************************

Several of the services are best suited to be queried asynchronously, especially if you plan to make large
queries that would take a long time. The TOM Toolkit is setup to use `dramatiq <https://dramatiq.io/index.html>`_
as an asynchronous task manager, but doing so requires you to run either a `redis <https://github.com/redis/redis>`_
queries that would take a long time. The TOM Toolkit can be setup to use `dramatiq <https://dramatiq.io/index.html>`_
as an asynchronous task manager, and doing so requires you to run either a `redis <https://github.com/redis/redis>`_
or `rabbitmq <https://github.com/rabbitmq/rabbitmq-server>`_ server to act as the task queue. To use dramatiq with
a redis server, you would add the following to your ``settings.py``:

Expand Down Expand Up @@ -83,11 +97,13 @@ Adding a new Forced Photometry Service

The Forced Photometry services fulfill an interface defined in
`BaseForcedPhotometryService <https://github.com/TOMToolkit/tom_base/blob/dev/tom_dataproducts/forced_photometry/forced_photometry_service.py>`_.
To implement your own Forced Photometry service, you need to do 3 things:
1. Subclass BaseForcedPhotometryService
2. Subclass BaseForcedPhotometryQueryForm
3. Subclass DataProcessor
Once those are implemented, don't forget to update your settings for ``FORCED_PHOTOMETRY_SERVICES``,
To implement your own Forced Photometry service, you need to do three things:

#. Subclass ``BaseForcedPhotometryService``
#. Subclass ``BaseForcedPhotometryQueryForm``
#. Subclass ``DataProcessor``

Once those subclasses are implemented, don't forget to update your settings for ``FORCED_PHOTOMETRY_SERVICES``,
``DATA_PRODUCT_TYPES``, and ``DATA_PROCESSORS`` for your new service and its associated data product type.


Expand All @@ -104,7 +120,8 @@ your code should check to see if ``django_dramatiq`` is in the settings ``INSTAL

The ``get_data_product_type`` method should return the name of your new data product type you are going to define a
DataProcessor for. This must match the name you add to ``DATA_PROCESSORS`` and ``DATA_PRODUCT_TYPES`` in your ``settings.py``.
You will also need to define a `DataProcessor <https://github.com/TOMToolkit/tom_base/blob/dev/tom_dataproducts/data_processor.py#L46>`
You will also need to define a
`DataProcessor <https://github.com/TOMToolkit/tom_base/blob/dev/tom_dataproducts/data_processor.py#L46>`_
for this data type.


Expand All @@ -127,4 +144,4 @@ You must create a custom DataProcessor that knows how to convert data returned f
a series of either photometry or spectroscopy datums. Without defining this step, your queries will still
result in a DataProduct file being stored from the service's ``query_service`` method, but those files will
not be parsed into photometry or spectroscopy datums. You can read more about how to implement a custom
DataProcessor `here <../customizing_data_processing>`_.
DataProcessor `here <./customizing_data_processing.html>`_.
Loading

0 comments on commit cbb1d19

Please sign in to comment.