diff --git a/CHANGELOG.md b/CHANGELOG.md index 7fa808f..da69a19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,30 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) (none) +## \[v0.3.7\] - 2024-07-02 + +### Added + +- A default schema to be used when no schema is provided. +- Child classes for `schema.Schema` that are specific to different serialization formats. + +### Fixed + +- Support for the latest LSST schema version (lsst.v7_1.alert). Note that this is the only LSST schema + version currently supported. + +### Changed + +- Renamed `exceptions.SchemaNotFoundError` -> `exceptions.SchemaError`, repurposed for more general use. +- Updates to documentation. + +### Removed + +- Removed `exceptions.OpenAlertError`. Use `exceptions.SchemaError` instead. +- Removed dependency on `lsst-alert-packet` package. We cannot install this from a git repo and also + publish our package to PyPI. Need to figure out how to fix this. Without it, + 'schema.SchemaHelper.lsst_auto_schema_helper' will not work. + ## \[v0.3.6\] - 2024-07-01 ### Changed diff --git a/docs/source/api-reference/index.rst b/docs/source/api-reference/index.rst index 372ecaf..9b786d7 100644 --- a/docs/source/api-reference/index.rst +++ b/docs/source/api-reference/index.rst @@ -1,15 +1,14 @@ pittgoogle ========== -.. These are from the ___init__.py file. Would be nice to find a way to pull them in automatically -.. and also to make them hyperlinks. +.. Listing the subset of class the user will interact with most. +.. Is this what we want the package index page to look like? # [TODO] .. autosummary:: .. autosummary:: pittgoogle.alert.Alert - pittgoogle.auth.Auth pittgoogle.bigquery.Table pittgoogle.pubsub.Consumer pittgoogle.pubsub.Subscription diff --git a/docs/source/api-reference/schema.rst b/docs/source/api-reference/schema.rst new file mode 100644 index 0000000..802f2a8 --- /dev/null +++ b/docs/source/api-reference/schema.rst @@ -0,0 +1,5 @@ +pittgoogle.schema +================= + +.. automodule:: pittgoogle.schema + :members: diff --git a/docs/source/for-developers/add-new-schema-map.md b/docs/source/for-developers/add-new-schema-map.rst similarity index 87% rename from docs/source/for-developers/add-new-schema-map.md rename to docs/source/for-developers/add-new-schema-map.rst index 8851136..f530e41 100644 --- a/docs/source/for-developers/add-new-schema-map.md +++ b/docs/source/for-developers/add-new-schema-map.rst @@ -1,4 +1,5 @@ -# Add a schema map +Add a schema map +================ This page contains instructions for adding a new schema map. @@ -7,18 +8,20 @@ Pitt-Google defines this set of generic field names so that we can write code th different surveys without having to worry that one survey might call the (e.g.,) time field "MJD" while another calls it "midPointTai". This map is what makes the :meth:`pittgoogle.alert.Alert.get` method work. -Currently, we define schema maps on a per-survey basis. :meth:`pittgoogle.types_.Schema.map` will load -the yaml file for the survey :meth:`pittgoogle.types_.Schema.survey` and return it as a dictionary. +Currently, we define schema maps on a per-survey basis. :meth:`pittgoogle.schema.Schema.map` will load +the yaml file for the survey :meth:`pittgoogle.schema.Schema.survey` and return it as a dictionary. If you need something different, a more significant refactor will be required (left as an exercise for the reader). -## Add a schema map for a new survey +Add a schema map for a new survey +--------------------------------- *pittgoogle/schemas/maps* is the directory containing the schema maps as yaml files. To add a schema map, make a copy of the file *pittgoogle/schemas/maps/TEMPLATE.yml* and alter it. -### How to use the TEMPLATE.yml file +How to use the TEMPLATE.yml file +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *pittgoogle/schemas/maps/TEMPLATE.yml* : Make a copy of this file and name it using the syntax *.yml*. @@ -39,7 +42,8 @@ Alter the new file, keeping these important things to keep in mind: to help guide your decisions. - Try to make decisions that result in schema maps that are conceptually consistent across surveys. -### Excluding or adding field names (keys) from the set of Pitt-Google generics +Excluding or adding field names (keys) from the set of Pitt-Google generics +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Excluding fields : If your survey/schema does not need a particular key, that key does does not need to be included in your new schema map's yaml file. There's not much code in ``pittgoogle-client`` itself diff --git a/docs/source/for-developers/add-new-schema.md b/docs/source/for-developers/add-new-schema.rst similarity index 75% rename from docs/source/for-developers/add-new-schema.md rename to docs/source/for-developers/add-new-schema.rst index 0d83ad1..8adaa48 100644 --- a/docs/source/for-developers/add-new-schema.md +++ b/docs/source/for-developers/add-new-schema.rst @@ -1,7 +1,9 @@ -# Add a new schema to the registry +Add a new schema to the registry +================================ This page contains instructions for adding a new schema to the registry so that it can be loaded -using :meth:`pittgoogle.registry.Schemas.get` and used to serialize and deserialize the alert bytes. +using :meth:`pittgoogle.Schemas.get` and used to serialize and deserialize the alert bytes. + You will need to update at least the "Required" files listed below, and potentially one or more of the others. The schema format is expected to be either Avro or Json. @@ -10,18 +12,20 @@ First, a naming guideline: - Schema names are expected to start with the name of the survey. If the survey has more than one schema, the survey name should be followed by a "." and then schema-specific specifier(s). -## Required +Required +-------- -### pittgoogle/registry_manifests/schemas.yml +pittgoogle/registry_manifests/schemas.yml +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *pittgoogle/registry_manifests/schemas.yml* is the manifest of registered schemas. Add a new section to the manifest following the template provided there. The fields are the same as -those of a :class:`pittgoogle.types_.Schema`. The ``helper`` field must point to code that can find and load +those of a :class:`pittgoogle.schema.Schema`. The ``helper`` field must point to code that can find and load the new schema definition; more information below. -Case 1: The schema definition is not *needed* in order to deserialize the alert bytes. This is true for -all Json and the Avro streams which attach the schema in the data header. Set +Case 1: The schema definition is not needed in order to deserialize the alert bytes. This is true for +all Json, and the Avro streams which attach the schema in the data header. Set ``schemaless_alert_bytes='false'``. Leave ``helper`` and ``path`` as defaults. The rest of the cases assume the schema definition is required. This is true for "schemaless" Avro streams @@ -40,23 +44,27 @@ usually only need to point to the main one. (3) If you've followed the recommend ``helper`` should work, but you should check (more below). If you need to implement your own helper or update the existing, do it. -## Potentially Required +Potentially Required +-------------------- -### pittgoogle/types_.py +pittgoogle/schema.py +^^^^^^^^^^^^^^^^^^^^ -*pittgoogle/types_.py* is the file containing the :class:`pittgoogle.types_.Schema` class. +# [FIXME] +*pittgoogle/schema.py* is the file containing the :class:`pittgoogle.schema.Schema` class. If ``schemaless_alert_bytes='false'``, the defaults (mostly null/None) should work and you can ignore this file (skip to the next section). -A "helper" method must exist in :class:`pittgoogle.types_.Schema` that can find and load your new schema +A "helper" method must exist in :class:`pittgoogle.schema.Schema` that can find and load your new schema definition. The ``helper`` field in the yaml manifest (above) must be set to the name of this method. If a suitable helper method does not already already exist for your schema, add one to this file by following -existing helpers like :meth:`pittgoogle.types_.Schema._local_schema_helper` as examples. **If your helper +existing helpers like :meth:`pittgoogle.schema.Schema.default_schema_helper` as examples. **If your helper method requires a new dependency, be sure to add it following :doc:`/main/for-developers/manage-dependencies-poetry`.** -### pittgoogle/schemas/maps/ +pittgoogle/schemas/maps/ +^^^^^^^^^^^^^^^^^^^^^^^^ *pittgoogle/schemas/maps/* is the directory containing our schema maps. diff --git a/docs/source/for-developers/get-alerts-for-testing.rst b/docs/source/for-developers/get-alerts-for-testing.rst new file mode 100644 index 0000000..79ea885 --- /dev/null +++ b/docs/source/for-developers/get-alerts-for-testing.rst @@ -0,0 +1,48 @@ +Get alerts for testing +====================== + +Setup +----- + +.. code-block:: python + + import pittgoogle + +Get alerts from a Pub/Sub subscription +-------------------------------------- + +If you need to create the subscription, follow the example in :class:`pittgoogle.Subscription`. + +Here are examples that get an alert from each of our "loop" streams: + +.. code-block:: python + + # Choose one of the following + loop_sub = pittgoogle.Subscription("rubin-loop", schema_name="lsst.v7_1.alert") + loop_sub = pittgoogle.Subscription("elasticc-loop", schema_name="elasticc.v0_9_1.alert") + loop_sub = pittgoogle.Subscription("ztf-loop", schema_name="ztf") + + loop_sub.touch() + + alert = loop.pull_batch(max_messages=1)[0] + +Get alerts from a file on disk +------------------------------- + +.. code-block:: python + + # [TODO] Add code snippet + + +Get alerts from Cloud Storage +----------------------------- + +.. code-block:: python + + # [TODO] Add code snippet + +Get alerts from BigQuery +------------------------- +.. code-block:: python + + # [TODO] Add code snippet diff --git a/docs/source/for-developers/index.rst b/docs/source/for-developers/index.rst index eaf383e..74aba33 100644 --- a/docs/source/for-developers/index.rst +++ b/docs/source/for-developers/index.rst @@ -5,6 +5,7 @@ For Developers :maxdepth: 1 setup-environment + get-alerts-for-testing add-new-schema add-new-schema-map manage-dependencies-poetry diff --git a/docs/source/for-developers/manage-dependencies-poetry.md b/docs/source/for-developers/manage-dependencies-poetry.md index 95050e2..caec3e4 100644 --- a/docs/source/for-developers/manage-dependencies-poetry.md +++ b/docs/source/for-developers/manage-dependencies-poetry.md @@ -1,4 +1,4 @@ -# Managing dependencies with Poetry +# Manage dependencies with Poetry This page contains instructions for managing the `pittgoogle` package dependencies using [Poetry](https://python-poetry.org/). Poetry was implemented in this repo in [pull #7](https://github.com/mwvgroup/pittgoogle-client/pull/7). diff --git a/docs/source/index.rst b/docs/source/index.rst index 7894714..735b1c6 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -33,5 +33,6 @@ If you run into trouble, please api-reference/exceptions api-reference/pubsub api-reference/registry + api-reference/schema api-reference/types_ api-reference/utils diff --git a/pittgoogle/__init__.py b/pittgoogle/__init__.py index 2da4e88..a1348ca 100644 --- a/pittgoogle/__init__.py +++ b/pittgoogle/__init__.py @@ -9,7 +9,7 @@ except ImportError: # for Python<3.8 import importlib_metadata as metadata -from . import alert, auth, bigquery, exceptions, pubsub, registry, types_, utils +from . import alert, auth, bigquery, exceptions, pubsub, registry, schema, types_, utils from .alert import Alert from .auth import Auth from .bigquery import Table diff --git a/pittgoogle/alert.py b/pittgoogle/alert.py index c82c773..6fa4af9 100644 --- a/pittgoogle/alert.py +++ b/pittgoogle/alert.py @@ -1,19 +1,17 @@ # -*- coding: UTF-8 -*- """Classes for working with astronomical alerts.""" import base64 +import datetime import importlib.resources -import io import logging -from datetime import datetime from pathlib import Path from typing import TYPE_CHECKING, Any, Mapping, Union -import fastavro +import attrs import google.cloud.pubsub_v1 -from attrs import define, field -from . import registry, types_, utils -from .exceptions import BadRequest, OpenAlertError, SchemaNotFoundError +from . import registry, types_, exceptions +from .schema import Schema # so 'schema' module doesn't clobber 'Alert.schema' attribute if TYPE_CHECKING: import pandas as pd # always lazy-load pandas. it hogs memory on cloud functions and run @@ -22,7 +20,7 @@ PACKAGE_DIR = importlib.resources.files(__package__) -@define(kw_only=True) +@attrs.define(kw_only=True) class Alert: """Container for an astronomical alert. @@ -47,16 +45,16 @@ class Alert: ---- """ - _dict: Mapping | None = field(default=None) - _attributes: Mapping[str, str] | None = field(default=None) - schema_name: str | None = field(default=None) - msg: google.cloud.pubsub_v1.types.PubsubMessage | types_.PubsubMessageLike | None = field( - default=None + _dict: Mapping | None = attrs.field(default=None) + _attributes: Mapping[str, str] | None = attrs.field(default=None) + schema_name: str | None = attrs.field(default=None) + msg: google.cloud.pubsub_v1.types.PubsubMessage | types_.PubsubMessageLike | None = ( + attrs.field(default=None) ) - path: Path | None = field(default=None) + path: Path | None = attrs.field(default=None) # Use "Union" because " | " is throwing an error when combined with forward references. - _dataframe: Union["pd.DataFrame", None] = field(default=None) - _schema: types_.Schema | None = field(default=None, init=False) + _dataframe: Union["pd.DataFrame", None] = attrs.field(default=None) + _schema: Schema | None = attrs.field(default=None, init=False) # ---- class methods ---- # @classmethod @@ -109,17 +107,17 @@ def index(): """ # check whether received message is valid, as suggested by Cloud Run docs if not envelope: - raise BadRequest("Bad Request: no Pub/Sub message received") + raise exceptions.BadRequest("Bad Request: no Pub/Sub message received") if not isinstance(envelope, dict) or "message" not in envelope: - raise BadRequest("Bad Request: invalid Pub/Sub message format") + raise exceptions.BadRequest("Bad Request: invalid Pub/Sub message format") # convert the message publish_time string -> datetime # occasionally the string doesn't include microseconds so we need a try/except publish_time = envelope["message"]["publish_time"].replace("Z", "+00:00") try: - publish_time = datetime.strptime(publish_time, "%Y-%m-%dT%H:%M:%S.%f%z") + publish_time = datetime.datetime.strptime(publish_time, "%Y-%m-%dT%H:%M:%S.%f%z") except ValueError: - publish_time = datetime.strptime(publish_time, "%Y-%m-%dT%H:%M:%S%z") + publish_time = datetime.datetime.strptime(publish_time, "%Y-%m-%dT%H:%M:%S%z") return cls( msg=types_.PubsubMessageLike( @@ -232,26 +230,11 @@ def dict(self) -> Mapping: The alert data as a dictionary. Raises: - OpenAlertError: + SchemaError: If unable to deserialize the alert bytes. """ - if self._dict is not None: - return self._dict - - if self.schema.schemaless_alert_bytes: - bytes_io = io.BytesIO(self.msg.data) - self._dict = fastavro.schemaless_reader(bytes_io, self.schema.definition) - return self._dict - - # [TODO] this should be rewritten to catch specific errors - # for now, just try avro then json, catching basically all errors in the process - try: - self._dict = utils.Cast.avro_to_dict(self.msg.data) - except Exception: - try: - self._dict = utils.Cast.json_to_dict(self.msg.data) - except Exception: - raise OpenAlertError("failed to deserialize the alert bytes") + if self._dict is None: + self._dict = self.schema.deserialize(self.msg.data) return self._dict @property @@ -306,22 +289,15 @@ def sourceid(self) -> str | int: return self.get("sourceid") @property - def schema(self) -> types_.Schema: - """Return the schema from the :class:`pittgoogle.Schemas` registry. + def schema(self) -> Schema: + """Return the schema from the :class:`pittgoogle.registry.Schemas` registry. Raises: - pittgoogle.exceptions.SchemaNotFoundError: + SchemaError: If the `schema_name` is not supplied or a schema with this name is not found. """ - if self._schema is not None: - return self._schema - - # need to load the schema. raise an error if no schema_name given - if self.schema_name is None: - raise SchemaNotFoundError("a schema_name is required") - - # this also may raise SchemaNotFoundError - self._schema = registry.Schemas.get(self.schema_name) + if self._schema is None: + self._schema = registry.Schemas.get(self.schema_name) return self._schema # ---- methods ---- # diff --git a/pittgoogle/exceptions.py b/pittgoogle/exceptions.py index e8a427a..fe3f909 100644 --- a/pittgoogle/exceptions.py +++ b/pittgoogle/exceptions.py @@ -7,9 +7,5 @@ class CloudConnectionError(Exception): """Raised when a problem is encountered while trying to a Google Cloud resource.""" -class OpenAlertError(Exception): - """Raised when unable to deserialize a Pub/Sub message payload.""" - - -class SchemaNotFoundError(Exception): +class SchemaError(Exception): """Raised when a schema with a given name is not found in the registry.""" diff --git a/pittgoogle/pubsub.py b/pittgoogle/pubsub.py index 48c4638..8c8e2fc 100644 --- a/pittgoogle/pubsub.py +++ b/pittgoogle/pubsub.py @@ -1,5 +1,5 @@ # -*- coding: UTF-8 -*- -"""Classes to facilitate connections to Pub/Sub streams. +"""Classes to facilitate connections to Google Cloud Pub/Sub streams. .. note:: @@ -7,23 +7,19 @@ The examples given below assume the use of a :ref:`service account ` and :ref:`environment variables `. """ +import concurrent.futures import datetime import importlib.resources -import io -import json import logging import queue -from concurrent.futures import ThreadPoolExecutor -from time import sleep +import time from typing import Any, Callable, List, Optional, Union -import fastavro -import google.cloud.pubsub_v1 as pubsub_v1 -from attrs import define, field -from attrs.validators import gt, instance_of, is_callable, optional -from google.api_core.exceptions import NotFound +import attrs +import attrs.validators +import google.api_core.exceptions +import google.cloud.pubsub_v1 -# [FIXME] clean up these imports from . import exceptions from .alert import Alert from .auth import Auth @@ -49,21 +45,23 @@ def pull_batch( schema_name: str = str(), **subscription_kwargs, ) -> List["Alert"]: - """Pull a single batch of messages from the `subscription`. - - Parameters - ---------- - subscription : `str` or :class:`pittgoogle.pubsub.Subscription` - Subscription to be pulled. If `str`, the name of the subscription. - max_messages : `int` - Maximum number of messages to be pulled. - schema_name : `str` - One of "ztf", "ztf.lite", "elasticc.v0_9_1.alert", "elasticc.v0_9_1.brokerClassification". - Schema name of the alerts in the subscription. Passed to :class:`pittgoogle.pubsub.Alert` - for unpacking. If not provided, some properties of the `Alert` may not be available. - subscription_kwargs - Keyword arguments sent to :class:`pittgoogle.pubsub.Subscription`. - Ignored if `subscription` is a :class:`pittgoogle.pubsub.Subscription`. + """Pulls a single batch of messages from the specified subscription. + + Args: + subscription (str or Subscription): + The subscription to be pulled. If str, the name of the subscription. + max_messages (int): + The maximum number of messages to be pulled. + schema_name (str): + The schema name of the alerts in the subscription. See :meth:`pittgoogle.registry.Schemas.names` + for the list of options. Passed to Alert for unpacking. If not provided, some properties of + the Alert may not be available. + **subscription_kwargs: + Keyword arguments used to create the :class:`pittgoogle.pubsub.Subscription` object, if needed. + + Returns: + list[Alert]: + A list of Alert objects representing the pulled messages. """ if isinstance(subscription, str): subscription = Subscription(subscription, **subscription_kwargs) @@ -72,7 +70,7 @@ def pull_batch( response = subscription.client.pull( {"subscription": subscription.path, "max_messages": max_messages} ) - except NotFound: + except google.api_core.exceptions.NotFound: msg = "Subscription not found. You may need to create one using `pittgoogle.Subscription.touch`." raise exceptions.CloudConnectionError(msg) @@ -87,31 +85,58 @@ def pull_batch( return alerts -@define +@attrs.define class Topic: - """Basic attributes of a Pub/Sub topic. - - Parameters - ------------ - name : `str` - Name of the Pub/Sub topic. - projectid : `str`, optional - The topic owner's Google Cloud project ID. Either this or `auth` is required. Use this - if you are connecting to a subscription owned by a different project than this topic. Note: - :attr:`pittgoogle.utils.ProjectIds` is a registry containing Pitt-Google's project IDs. - auth : :class:`pittgoogle.auth.Auth`, optional - Credentials for the Google Cloud project that owns this topic. If not provided, - it will be created from environment variables when needed. - client : `pubsub_v1.PublisherClient`, optional - Pub/Sub client that will be used to access the topic. If not provided, a new client will - be created (using `auth`) the first time it is requested. + """Class to manage a Google Cloud Pub/Sub topic. + + Args: + name (str): + Name of the Pub/Sub topic. + projectid (str, optional): + The topic owner's Google Cloud project ID. Either this or `auth` is required. Use this + if you are connecting to a subscription owned by a different project than this topic. Note: + :class:`pittgoogle.registry.ProjectIds` is a registry containing Pitt-Google's project IDs. + auth (Auth, optional): + Credentials for the Google Cloud project that owns this topic. If not provided, + it will be created from environment variables when needed. + client (google.cloud.pubsub_v1.PublisherClient, optional): + Pub/Sub client that will be used to access the topic. If not provided, + a new client will be created the first time it is requested. + + Example: + + .. code-block:: python + + import pittgoogle + + # Create a new topic in your project + my_topic = pittgoogle.Topic(name="my-new-topic") + my_topic.touch() + + # Create a dummy message to publish + my_alert = pittgoogle.Alert( + dict={"message": "Hello, World!"}, # the message payload + attributes={"custom_key": "custom_value"} # custom attributes for the message + ) + + # Publish the message to the topic + my_topic.publish(my_alert) # returns the ID of the published message + + To pull the message back from the topic, use a :class:`Subscription`. + + ---- """ - name: str = field() - _projectid: str = field(default=None) - _auth: Auth = field(default=None, validator=optional(instance_of(Auth))) - _client: Optional[pubsub_v1.PublisherClient] = field( - default=None, validator=optional(instance_of(pubsub_v1.PublisherClient)) + name: str = attrs.field() + _projectid: str = attrs.field(default=None) + _auth: Auth = attrs.field( + default=None, validator=attrs.validators.optional(attrs.validators.instance_of(Auth)) + ) + _client: Optional[google.cloud.pubsub_v1.PublisherClient] = attrs.field( + default=None, + validator=attrs.validators.optional( + attrs.validators.instance_of(google.cloud.pubsub_v1.PublisherClient) + ), ) @classmethod @@ -123,25 +148,24 @@ def from_cloud( survey: Optional[str] = None, testid: Optional[str] = None, ): - """Create a `Topic` with a `client` using implicit credentials (no explicit `auth`). - - Parameters - ---------- - name : `str` - Name of the topic. If `survey` and/or `testid` are provided, they will be added to this - name following the Pitt-Google naming syntax. - projectid : `str` - Project ID of the Goodle Cloud project that owns this resource. Project IDs used by - Pitt-Google are listed in the registry for convenience (:class:`pittgoogle.registry.ProjectIds`). - Required because it cannot be retrieved from the `client` and there is no explicit `auth`. - survey : `str`, optional - Name of the survey. If provided, it will be prepended to `name` following the - Pitt-Google naming syntax. - testid : `str`, optional - Pipeline identifier. If this is not `None`, `False`, or `"False"` it will be appended to - the `name` following the Pitt-Google naming syntax. This used to allow pipeline modules - to find the correct resources without interfering with other pipelines that may have - deployed resources with the same base names (e.g., for development and testing purposes). + """Creates a `Topic` with a `client` using implicit credentials (no explicit `auth`). + + Args: + name (str): + Name of the topic. If `survey` and/or `testid` are provided, they will be added to this + name following the Pitt-Google naming syntax. + projectid (str): + Project ID of the Google Cloud project that owns this resource. Project IDs used by + Pitt-Google are listed in the registry for convenience (:class:`pittgoogle.registry.ProjectIds`). + Required because it cannot be retrieved from the `client` and there is no explicit `auth`. + survey (str, optional): + Name of the survey. If provided, it will be prepended to `name` following the + Pitt-Google naming syntax. + testid (str, optional): + Pipeline identifier. If this is not `None`, `False`, or `"False"`, it will be appended to + the `name` following the Pitt-Google naming syntax. This is used to allow pipeline modules + to find the correct resources without interfering with other pipelines that may have + deployed resources with the same base names (e.g., for development and testing purposes). """ # if survey and/or testid passed in, use them to construct full name using the pitt-google naming syntax if survey is not None: @@ -149,7 +173,7 @@ def from_cloud( # must accommodate False and "False" for consistency with the broker pipeline if testid and testid != "False": name = f"{name}-{testid}" - return cls(name, projectid=projectid, client=pubsub_v1.PublisherClient()) + return cls(name, projectid=projectid, client=google.cloud.pubsub_v1.PublisherClient()) @classmethod def from_path(cls, path) -> "Topic": @@ -161,7 +185,7 @@ def from_path(cls, path) -> "Topic": def auth(self) -> Auth: """Credentials for the Google Cloud project that owns this topic. - This will be created from environment variables if `self._auth` is None. + This will be created from environment variables if needed. """ if self._auth is None: self._auth = Auth() @@ -185,13 +209,15 @@ def projectid(self) -> str: return self._projectid @property - def client(self) -> pubsub_v1.PublisherClient: + def client(self) -> google.cloud.pubsub_v1.PublisherClient: """Pub/Sub client for topic access. - Will be created using `self.auth.credentials` if necessary. + Will be created using :attr:`Topic.auth.credentials` if necessary. """ if self._client is None: - self._client = pubsub_v1.PublisherClient(credentials=self.auth.credentials) + self._client = google.cloud.pubsub_v1.PublisherClient( + credentials=self.auth.credentials + ) return self._client def touch(self) -> None: @@ -200,7 +226,7 @@ def touch(self) -> None: self.client.get_topic(topic=self.path) LOGGER.info(f"topic exists: {self.path}") - except NotFound: + except google.api_core.exceptions.NotFound: self.client.create_topic(name=self.path) LOGGER.info(f"topic created: {self.path}") @@ -208,82 +234,40 @@ def delete(self) -> None: """Delete the topic.""" try: self.client.delete_topic(topic=self.path) - except NotFound: + except google.api_core.exceptions.NotFound: LOGGER.info(f"nothing to delete. topic not found: {self.path}") else: LOGGER.info(f"deleted topic: {self.path}") def publish(self, alert: "Alert") -> int: - """Publish a message with `alert.dict` as the payload and `alert.attributes` as the attributes. - - If the `alert` has an elasticc schema, the payload will be serialized as schemaless Avro. - Otherwise, json will be used. - """ - # we need to decide which format to use: json, avro with schema, or avro without schema - # the format that pitt-google currently (2023-09-23) uses to publish messages depends on the stream: - # - consumer modules pass on the original alert data packet, as produced by the survey. - # they do not need to use this method (in fact, the consumers do not even use python), - # so we can ignore this case. - # - all other broker pipeline modules (Pitt-Google-Broker repo) use json. - # - modules in the pittgoogle-user repo publish classifications for elasticc, and thus - # use schemaless avro. - # at some point, we should re-evaluate the broker pipeline in particular. - # - # for now, we will get close enough to the current behavior if we assume that: - # - elasticc messages should be published as schemaless avro - # - else, we should publish a json message - # this will match the current behavior in all cases except the elasticc broker pipeline modules. - # neither broker pipeline uses pittgoogle-client at this time (they use pgb-broker-utils), - # so we don't need to update or accommodate them yet. - # - # we'll get the survey name from self.schema.survey, but first we should check whether the - # schema exists so we can be lenient and just fall back to json instead of raising an error. - try: - alert.schema - except exceptions.SchemaNotFoundError: - avro_schema = None - else: - if alert.schema.survey in ["elasticc"]: - avro_schema = alert.schema.definition - else: - avro_schema = None - - if not avro_schema: - # serialize using json - message = json.dumps(alert.dict).encode("utf-8") - else: - # serialize as schemaless avro - fout = io.BytesIO() - fastavro.schemaless_writer(fout, avro_schema, alert.dict) - fout.seek(0) - message = fout.getvalue() - - # attribute keys and values must be strings. let's sort the keys while we're at it + """Publish a message with ``alert.dict`` as the payload and ``alert.attributes`` as the attributes.""" + # Pub/Sub requires attribute keys and values to be strings. Sort the keys while we're at it. attributes = {str(key): str(alert.attributes[key]) for key in sorted(alert.attributes)} + message = alert.schema.serialize(alert.dict) future = self.client.publish(self.path, data=message, **attributes) return future.result() -@define +@attrs.define class Subscription: - """Creates a Pub/Sub subscription and provides methods to manage it. + """Class to manage a Google Cloud Pub/Sub subscription. Args: name (str): Name of the Pub/Sub subscription. - auth (pittgoogle.auth.Auth, optional): + auth (Auth, optional): Credentials for the Google Cloud project that owns this subscription. If not provided, it will be created from environment variables. - topic (pittgoogle.pubsub.Topic, optional): + topic (Topic, optional): Topic this subscription should be attached to. Required only when the subscription needs to be created. client (google.cloud.pubsub_v1.SubscriberClient, optional): Pub/Sub client that will be used to access the subscription. This kwarg is useful if you want to reuse a client. If None, a new client will be created. schema_name (str): - Schema name of the alerts in the subscription. Passed to :class:`pittgoogle.Alert` for unpacking. + Schema name of the alerts in the subscription. Passed to :class:`pittgoogle.alert.Alert` for unpacking. If not provided, some properties of the Alert may not be available. For a list of schema names, see - :meth:`pittgoogle.Schemas.names`. + :meth:`pittgoogle.registry.Schemas.names`. Example: @@ -291,14 +275,18 @@ class Subscription: .. code-block:: python - # topic the subscription will be connected to - # only required if the subscription does not yet exist in Google Cloud + # We must provide the topic that the subscription should be connected to. + # If the topic is in your own project, you can just provide the name of the topic. + # Otherwise, you must also provide the topic's project ID. topic = pittgoogle.Topic(name="ztf-loop", projectid=pittgoogle.ProjectIds.pittgoogle) + topic.touch() # make sure the topic exists and we can connect to it - # choose your own name for the subscription + # You can choose your own name for the subscription. + # It is common to name it the same as the topic, but here we'll be more verbose. + # You may also need to provide the schema name (see :attr:`pittgoogle.registry.Schemas.names`). subscription = pittgoogle.Subscription(name="my-ztf-loop-subscription", topic=topic, schema_name="ztf") - # make sure the subscription exists and we can connect to it. create it if necessary + # Create the subscription if it doesn't already exist. subscription.touch() Pull a small batch of alerts. Helpful for testing. (Not recommended for long-running listeners; @@ -306,16 +294,21 @@ class Subscription: .. code-block:: python - alerts = subscription.pull_batch(subscription, max_messages=4) + alerts = subscription.pull_batch(subscription, max_messages=4) # list of pittgoogle.Alert objects """ - name: str = field() - auth: Auth = field(factory=Auth, validator=instance_of(Auth)) - topic: Optional[Topic] = field(default=None, validator=optional(instance_of(Topic))) - _client: Optional[pubsub_v1.SubscriberClient] = field( - default=None, validator=optional(instance_of(pubsub_v1.SubscriberClient)) + name: str = attrs.field() + auth: Auth = attrs.field(factory=Auth, validator=attrs.validators.instance_of(Auth)) + topic: Optional[Topic] = attrs.field( + default=None, validator=attrs.validators.optional(attrs.validators.instance_of(Topic)) ) - schema_name: str = field(factory=str) + _client: Optional[google.cloud.pubsub_v1.SubscriberClient] = attrs.field( + default=None, + validator=attrs.validators.optional( + attrs.validators.instance_of(google.cloud.pubsub_v1.SubscriberClient) + ), + ) + schema_name: str | None = attrs.field(default=None) @property def projectid(self) -> str: @@ -328,12 +321,15 @@ def path(self) -> str: return f"projects/{self.projectid}/subscriptions/{self.name}" @property - def client(self) -> pubsub_v1.SubscriberClient: - """Pub/Sub client that will be used to access the subscription. If not provided, a new - client will be created using `self.auth.credentials`. + def client(self) -> google.cloud.pubsub_v1.SubscriberClient: + """Pub/Sub client that will be used to access the subscription. + + If not provided, a new client will be created using :attr:`Subscription.auth`. """ if self._client is None: - self._client = pubsub_v1.SubscriberClient(credentials=self.auth.credentials) + self._client = google.cloud.pubsub_v1.SubscriberClient( + credentials=self.auth.credentials + ) return self._client def touch(self) -> None: @@ -342,28 +338,27 @@ def touch(self) -> None: Note that messages published to the topic before the subscription was created are not available to the subscription. - Raises - ------ - `TypeError` - if the subscription needs to be created but no topic was provided. + Raises: + TypeError: + if the subscription needs to be created but no topic was provided. - `google.api_core.exceptions.NotFound` - if the subscription needs to be created but the topic does not exist in Google Cloud. + google.api_core.exceptions.NotFound: + if the subscription needs to be created but the topic does not exist in Google Cloud. - `pittgoogle.exceptions.CloudConnectionError` - if the subscription exists but it is not attached to self.topic and self.topic is not None. + CloudConnectionError: + if the subscription exists but it is not attached to self.topic and self.topic is not None. """ try: subscrip = self.client.get_subscription(subscription=self.path) LOGGER.info(f"subscription exists: {self.path}") - except NotFound: + except google.api_core.exceptions.NotFound: subscrip = self._create() # may raise TypeError or (topic) NotFound LOGGER.info(f"subscription created: {self.path}") self._set_topic(subscrip.topic) # may raise CloudConnectionError - def _create(self) -> pubsub_v1.types.Subscription: + def _create(self) -> google.cloud.pubsub_v1.types.Subscription: if self.topic is None: raise TypeError("The subscription needs to be created but no topic was provided.") @@ -371,9 +366,9 @@ def _create(self) -> pubsub_v1.types.Subscription: return self.client.create_subscription(name=self.path, topic=self.topic.path) # this error message is not very clear. let's help. - except NotFound as excep: + except google.api_core.exceptions.NotFound as excep: msg = f"The subscription cannot be created because the topic does not exist: {self.topic.path}" - raise NotFound(msg) from excep + raise google.api_core.exceptions.NotFound(msg) from excep def _set_topic(self, connected_topic_path) -> None: # if the topic is invalid, raise an error @@ -395,7 +390,7 @@ def delete(self) -> None: """Delete the subscription.""" try: self.client.delete_subscription(subscription=self.path) - except NotFound: + except google.api_core.exceptions.NotFound: LOGGER.info(f"nothing to delete. subscription not found: {self.path}") else: LOGGER.info(f"deleted subscription: {self.path}") @@ -409,10 +404,13 @@ def pull_batch(self, max_messages: int = 1) -> List["Alert"]: This method is *not* recommended for long-running listeners as it is likely to be unstable -- use :meth:`~Consumer.stream` instead. - Parameters - ---------- - max_messages : `int` - Maximum number of messages to be pulled. + Args: + max_messages (int): + Maximum number of messages to be pulled. + + Returns: + list[Alert]: + A list of Alert objects representing the pulled messages. """ # Wrapping the module-level function return pull_batch(self, max_messages=max_messages, schema_name=self.schema_name) @@ -432,7 +430,7 @@ def purge(self): ) -@define() +@attrs.define class Consumer: """Consumer class to pull a Pub/Sub subscription and process messages. @@ -489,21 +487,30 @@ def my_batch_callback(results): consumer.stream() """ - _subscription: Union[str, Subscription] = field(validator=instance_of((str, Subscription))) - msg_callback: Callable[["Alert"], "Response"] = field(validator=is_callable()) - batch_callback: Optional[Callable[[list], None]] = field( - default=None, validator=optional(is_callable()) + _subscription: Union[str, Subscription] = attrs.field( + validator=attrs.validators.instance_of((str, Subscription)) + ) + msg_callback: Callable[["Alert"], "Response"] = attrs.field( + validator=attrs.validators.is_callable() ) - batch_maxn: int = field(default=100, converter=int) - batch_max_wait_between_messages: int = field(default=30, converter=int) - max_backlog: int = field(default=1000, validator=gt(0)) - max_workers: Optional[int] = field(default=None, validator=optional(instance_of(int))) - _executor: ThreadPoolExecutor = field( - default=None, validator=optional(instance_of(ThreadPoolExecutor)) + batch_callback: Optional[Callable[[list], None]] = attrs.field( + default=None, validator=attrs.validators.optional(attrs.validators.is_callable()) ) - _queue: queue.Queue = field(factory=queue.Queue, init=False) - streaming_pull_future: pubsub_v1.subscriber.futures.StreamingPullFuture = field( - default=None, init=False + batch_maxn: int = attrs.field(default=100, converter=int) + batch_max_wait_between_messages: int = attrs.field(default=30, converter=int) + max_backlog: int = attrs.field(default=1000, validator=attrs.validators.gt(0)) + max_workers: Optional[int] = attrs.field( + default=None, validator=attrs.validators.optional(attrs.validators.instance_of(int)) + ) + _executor: concurrent.futures.ThreadPoolExecutor = attrs.field( + default=None, + validator=attrs.validators.optional( + attrs.validators.instance_of(concurrent.futures.ThreadPoolExecutor) + ), + ) + _queue: queue.Queue = attrs.field(factory=queue.Queue, init=False) + streaming_pull_future: google.cloud.pubsub_v1.subscriber.futures.StreamingPullFuture = ( + attrs.field(default=None, init=False) ) @property @@ -515,10 +522,10 @@ def subscription(self) -> Subscription: return self._subscription @property - def executor(self) -> ThreadPoolExecutor: + def executor(self) -> concurrent.futures.ThreadPoolExecutor: """Executor to be used by the Google API for a streaming pull.""" if self._executor is None: - self._executor = ThreadPoolExecutor(self.max_workers) + self._executor = concurrent.futures.ThreadPoolExecutor(self.max_workers) return self._executor def stream(self, block: bool = True) -> None: @@ -526,13 +533,12 @@ def stream(self, block: bool = True) -> None: Recommended for long-running listeners. - Parameters - ---------- - block : `bool` - Whether to block the main thread while the stream is open. If `True`, block - indefinitely (use `Ctrl-C` to close the stream and unblock). If `False`, open the - stream and then return (use :meth:`~Consumer.stop()` to close the stream). - This must be `True` in order to use a `batch_callback`. + Args: + block (bool): + Whether to block the main thread while the stream is open. If `True`, block + indefinitely (use `Ctrl-C` to close the stream and unblock). If `False`, open the + stream and then return (use :meth:`~Consumer.stop()` to close the stream). + This must be `True` in order to use a `batch_callback`. """ # open a streaming-pull and process messages through the callback, in the background self._open_stream() @@ -557,12 +563,14 @@ def _open_stream(self) -> None: self.streaming_pull_future = self.subscription.client.subscribe( self.subscription.path, self._callback, - flow_control=pubsub_v1.types.FlowControl(max_messages=self.max_backlog), - scheduler=pubsub_v1.subscriber.scheduler.ThreadScheduler(executor=self.executor), + flow_control=google.cloud.pubsub_v1.types.FlowControl(max_messages=self.max_backlog), + scheduler=google.cloud.pubsub_v1.subscriber.scheduler.ThreadScheduler( + executor=self.executor + ), await_callbacks_on_shutdown=True, ) - def _callback(self, message: pubsub_v1.types.PubsubMessage) -> None: + def _callback(self, message: google.cloud.pubsub_v1.types.PubsubMessage) -> None: """Unpack the message, run the :attr:`~Consumer.msg_callback` and handle the response.""" # LOGGER.info("callback started") response = self.msg_callback(Alert(msg=message)) # Response @@ -584,7 +592,7 @@ def _process_batches(self): # if there's no batch_callback there's nothing to do except wait until the process is killed if self.batch_callback is None: while True: - sleep(60) + time.sleep(60) batch, count = [], 0 while True: @@ -624,31 +632,33 @@ def pull_batch(self, max_messages: int = 1) -> List["Alert"]: Recommended for testing. Not recommended for long-running listeners (use the :meth:`~Consumer.stream` method instead). - Parameters - ---------- - max_messages : `int` - Maximum number of messages to be pulled. + Args: + max_messages (int): + Maximum number of messages to be pulled. + + Returns: + list[Alert]: + A list of Alert objects representing the pulled messages. """ return self.subscription.pull_batch(max_messages=max_messages) -@define(kw_only=True, frozen=True) +@attrs.define(kw_only=True, frozen=True) class Response: """Container for a response, to be returned by a :meth:`pittgoogle.pubsub.Consumer.msg_callback`. - Parameters - ------------ - ack : `bool` - Whether to acknowledge the message. Use `True` if the message was processed successfully, - `False` if an error was encountered and you would like Pub/Sub to redeliver the message at - a later time. Note that once a message is acknowledged to Pub/Sub it is permanently deleted - (unless the subscription has been explicitly configured to retain acknowledged messages). - - result : `Any` - Anything the user wishes to return. If not `None`, the Consumer will collect the results - in a list and pass the list to the user's batch callback for further processing. - If there is no batch callback the results will be lost. + Args: + ack (bool): + Whether to acknowledge the message. Use `True` if the message was processed successfully, + `False` if an error was encountered and you would like Pub/Sub to redeliver the message at + a later time. Note that once a message is acknowledged to Pub/Sub it is permanently deleted + (unless the subscription has been explicitly configured to retain acknowledged messages). + + result (Any): + Anything the user wishes to return. If not `None`, the Consumer will collect the results + in a list and pass the list to the user's batch callback for further processing. + If there is no batch callback the results will be lost. """ - ack: bool = field(default=True, converter=bool) - result: Any = field(default=None) + ack: bool = attrs.field(default=True, converter=bool) + result: Any = attrs.field(default=None) diff --git a/pittgoogle/registry.py b/pittgoogle/registry.py index 7e678ca..54f10d2 100644 --- a/pittgoogle/registry.py +++ b/pittgoogle/registry.py @@ -4,18 +4,17 @@ import logging from typing import Final +import attrs import yaml -from attrs import define -from . import types_ -from .exceptions import SchemaNotFoundError +from . import exceptions, schema LOGGER = logging.getLogger(__name__) PACKAGE_DIR = importlib.resources.files(__package__) SCHEMA_MANIFEST = yaml.safe_load((PACKAGE_DIR / "registry_manifests/schemas.yml").read_text()) -@define(frozen=True) +@attrs.define(frozen=True) class ProjectIds: """Registry of Google Cloud Project IDs.""" @@ -32,7 +31,7 @@ class ProjectIds: """Project running classifiers for ELAsTiCC alerts and reporting to DESC.""" -@define(frozen=True) +@attrs.define(frozen=True) class Schemas: """Registry of schemas used by Pitt-Google. @@ -55,7 +54,7 @@ class Schemas: """ @staticmethod - def get(schema_name: str) -> types_.Schema: + def get(schema_name: str | None) -> schema.Schema: """Return the schema with name matching `schema_name`. Returns: @@ -63,15 +62,23 @@ def get(schema_name: str) -> types_.Schema: Schema from the registry with name matching `schema_name`. Raises: - SchemaNotFoundError: + SchemaError: If a schema with name matching `schema_name` is not found in the registry. - SchemaNotFoundError: + SchemaError: If a schema definition cannot be loaded but one will be required to read the alert bytes. """ + # If no schema_name provided, return the default. + if schema_name is None: + LOGGER.warning("No schema name provided. Returning a default schema.") + mft_schema = [ + schema for schema in SCHEMA_MANIFEST if schema["name"] == "default_schema" + ][0] + return schema.Schema._from_yaml(schema_dict=mft_schema) + # Return the schema with name == schema_name, if one exists. for mft_schema in SCHEMA_MANIFEST: if mft_schema["name"] == schema_name: - return types_.Schema._from_yaml(schema_dict=mft_schema) + return schema.Schema._from_yaml(schema_dict=mft_schema) # Return the schema with name ~= schema_name, if one exists. for mft_schema in SCHEMA_MANIFEST: @@ -79,10 +86,10 @@ def get(schema_name: str) -> types_.Schema: # Catches names like 'lsst.v_.alert' where users replace '<..>' with custom values. split_name, split_mft_name = schema_name.split("."), mft_schema["name"].split(".") if all([split_mft_name[i] == split_name[i] for i in [0, -1]]): - return types_.Schema._from_yaml(schema_dict=mft_schema, name=schema_name) + return schema.Schema._from_yaml(schema_dict=mft_schema, name=schema_name) # That's all we know how to check so far. - raise SchemaNotFoundError( + raise exceptions.SchemaError( f"{schema_name} not found. For valid names, see `pittgoogle.Schemas.names`." ) diff --git a/pittgoogle/registry_manifests/schemas.yml b/pittgoogle/registry_manifests/schemas.yml index 4d1da99..0534132 100644 --- a/pittgoogle/registry_manifests/schemas.yml +++ b/pittgoogle/registry_manifests/schemas.yml @@ -5,36 +5,42 @@ # # TEMPLATE # - name: '' # description: '' -# helper: '_local_schema_helper' +# origin: 'https://...' +# helper: 'default_schema_helper' # path: '' -# schemaless_alert_bytes: false # filter_map: # 1: '' # 2: '' # +# DEFAULT +- name: 'default_schema' + description: 'Default schema used when no other schema is specified.' + origin: '(null)' + helper: 'default_schema_helper' +# # ELASTICC alerts - name: 'elasticc.v0_9_1.alert' description: 'Avro schema of alerts published by ELAsTiCC.' origin: 'https://github.com/LSSTDESC/elasticc/tree/main/alert_schema' - helper: '_local_schema_helper' + helper: 'elasticc_schema_helper' path: 'schemas/elasticc/elasticc.v0_9_1.alert.avsc' - schemaless_alert_bytes: true # # ELASTICC classifications - name: 'elasticc.v0_9_1.brokerClassification' description: 'Avro schema of alerts to be sent to DESC containing classifications of ELAsTiCC alerts.' origin: 'https://github.com/LSSTDESC/elasticc/tree/main/alert_schema' - helper: '_local_schema_helper' + helper: 'elasticc_schema_helper' path: 'schemas/elasticc/elasticc.v0_9_1.brokerClassification.avsc' - schemaless_alert_bytes: true # # LSST alerts -# - name: 'lsst.v7_1.alert' -- name: 'lsst.v_.alert' - description: '' # [TODO] +# [FIXME] Hacking in v7.1 for now. Need to figure out install lsst-alert-packet to use lsst_auto_schema_helper +# - name: 'lsst.v_.alert' + # helper: 'lsst_auto_schema_helper' +- name: 'lsst.v7_1.alert' + path: 'schemas/lsst/7/1/lsst.v7_1.alert.avsc' + description: 'Schema for LSST alerts.' origin: 'https://github.com/lsst/alert_packet/tree/main/python/lsst/alert/packet/schema' - helper: '_lsst_schema_helper' - schemaless_alert_bytes: true + helper: 'lsst_schema_helper' # [FIXME] filter_map is probably int -> {u, g, r, i, z, y}. Check for sure and fill in below filter_map: 1: '' @@ -48,8 +54,7 @@ - name: 'ztf' description: 'ZTF schema. The ZTF survey publishes alerts in Avro format with the schema attached in the header. Pitt-Google publishes ZTF alerts in json format. This schema covers both cases.' origin: 'https://zwickytransientfacility.github.io/ztf-avro-alert/schema.html' - helper: '_local_schema_helper' - schemaless_alert_bytes: false + helper: 'default_schema_helper' path: null filter_map: 1: g diff --git a/pittgoogle/schema.py b/pittgoogle/schema.py new file mode 100644 index 0000000..6ef701c --- /dev/null +++ b/pittgoogle/schema.py @@ -0,0 +1,263 @@ +# -*- coding: UTF-8 -*- +"""Classes to manage alert schemas.""" +import importlib.resources +import io +import json +import logging +import re +from pathlib import Path + +import attrs +import fastavro +import yaml + +from . import exceptions, utils + +LOGGER = logging.getLogger(__name__) +PACKAGE_DIR = importlib.resources.files(__package__) + + +@attrs.define(kw_only=True) +class SchemaHelpers: + """Class to organize helper functions. + + This class is not intended to be used directly, except by developers adding support for a new schema. + + For Developers: + + When a user requests a schema from the registry, the class method :meth:`Schema._from_yaml` is called. + The method will pass ``schema_name``'s dict entry from the registry's `schemas.yml` file to + one of these helper methods, which will then construct the :class:`Schema` object. + + If you are adding support for a new schema, you will need to point to the appropriate helper + method for your schema using the ``helper`` field in the registry's `schemas.yml` file. + If an appropriate method does not exist in this class, you will need to add one. + + ---- + """ + + @staticmethod + def default_schema_helper(schema_dict: dict) -> "Schema": + """Resolve `schema.path`. If it points to a valid ".avsc" file, load it into `schema.avsc`.""" + schema = _DefaultSchema(**schema_dict) + + # Resolve the path. If it is not None, this helper expects it to be the path to + # a ".avsc" file relative to the pittgoogle package directory. + schema.path = PACKAGE_DIR / schema.path if schema.path is not None else None + + # Load the avro schema, if the file exists. Fallback to None. + invalid_path = ( + (schema.path is None) or (schema.path.suffix != ".avsc") or (not schema.path.is_file()) + ) + if invalid_path: + schema.definition = None + else: + schema.definition = fastavro.schema.load_schema(schema.path) + + return schema + + @staticmethod + def elasticc_schema_helper(schema_dict: dict) -> "Schema": + schema = _SchemalessAvroSchema(**schema_dict) + + # Resolve the path and load the schema + schema.path = PACKAGE_DIR / schema.path + schema.definition = fastavro.schema.load_schema(schema.path) + + return schema + + @staticmethod + def lsst_schema_helper(schema_dict: dict) -> "Schema": + """Load the Avro schema definition for lsst.v7_1.alert. + + [FIXME] This is hack to get the latest schema version into pittgoogle-client + until we can get :meth:`SchemaHelpers.lsst_auto_schema_helper` working. + """ + if not schema_dict["name"] == "lsst.v7_1.alert": + raise NotImplementedError("Only 'lsst.v7_1.alert' is supported for LSST.") + + schema = _ConfluentWireAvroSchema(**schema_dict) + + # Resolve the path and load the schema + schema.path = PACKAGE_DIR / schema.path + schema.definition = fastavro.schema.load_schema(schema.path) + + return schema + + @staticmethod + def lsst_auto_schema_helper(schema_dict: dict) -> "Schema": + """Load the Avro schema definition using the ``lsst.alert.packet`` package. + + Raises: + SchemaError: + If an LSST schema called ``schema.name`` cannot be loaded. An error is raised + because the LSST alert bytes are schemaless, so ``schema.definition`` will be + required in order to deserialize the alert. + """ + import lsst.alert.packet.schema + + schema = _ConfluentWireAvroSchema(**schema_dict) + + version_msg = f"For valid versions, see {schema.origin}." + + # Parse major and minor versions out of schema.name. Expecting syntax "lsst.v_.alert". + try: + major, minor = map(int, re.findall(r"\d+", schema.name)) + except ValueError: + msg = ( + f"Unable to identify major and minor version. Please use the syntax " + "'lsst.v_.alert', replacing '' and '' with integers. " + f"{version_msg}" + ) + raise exceptions.SchemaError(msg) + + schema_dir = Path(lsst.alert.packet.schema.get_schema_path(major, minor)) + schema.path = schema_dir / f"{schema.name}.avsc" + + try: + schema.definition = lsst.alert.packet.schema.Schema.from_file(schema.path).definition + except fastavro.repository.SchemaRepositoryError: + msg = f"Unable to load the schema. {version_msg}" + raise exceptions.SchemaError(msg) + + return schema + + +@attrs.define(kw_only=True) +class Schema: + """Class for an individual schema. + + Do not call this class's constructor directly. Instead, load a schema using the registry + :class:`pittgoogle.registry.Schemas`. + + ---- + """ + + # String _under_ field definition will cause field to appear as a property in rendered docs. + name: str = attrs.field() + """Name of the schema.""" + description: str = attrs.field() + """A description of the schema.""" + origin: str = attrs.field() + """Pointer to the schema's origin. Typically this is a URL to a repo maintained by the survey.""" + definition: dict | None = attrs.field(default=None) + """The schema definition used to serialize and deserialize the alert bytes, if one is required.""" + _helper: str = attrs.field(default="default_schema_helper") + """Name of the method in :class:`SchemaHelpers` used to load this schema.""" + path: Path | None = attrs.field(default=None) + """Path where the helper can find the schema, if needed.""" + filter_map: dict = attrs.field(factory=dict) + """Mapping of the filter name as stored in the alert (often an int) to the common name (often a string).""" + # The rest don't need string descriptions because we will define them as explicit properties. + _survey: str | None = attrs.field(default=None) + # _map is important, but don't accept it as an init arg. We'll load it from a yaml file later. + _map: dict | None = attrs.field(default=None, init=False) + + @classmethod + def _from_yaml(cls, schema_dict: dict, **schema_dict_replacements) -> "Schema": + """Create a :class:`Schema` object from an entry in the registry's `schemas.yml` file. + + This method calls a helper method in :class:`SchemaHelpers` to finish the initialization. + + Args: + schema_dict (dict): + A dictionary containing the schema information. + **schema_dict_replacements: + Additional keyword arguments that will override entries in ``schema_dict``. + + Returns: + Schema: + The created `Schema` object. + """ + # Combine the args and kwargs then let the helper finish up the initialization. + my_schema_dict = schema_dict.copy() + my_schema_dict.update(schema_dict_replacements) + helper = getattr(SchemaHelpers, my_schema_dict["helper"]) + return helper(my_schema_dict) + + @property + def survey(self) -> str: + """Name of the survey. This is usually the first part of the schema's name.""" + if self._survey is None: + self._survey = self.name.split(".")[0] + return self._survey + + @property + def map(self) -> dict: + """Mapping of Pitt-Google's generic field names to survey-specific field names.""" + if self._map is None: + yml = PACKAGE_DIR / f"schemas/maps/{self.survey}.yml" + try: + self._map = yaml.safe_load(yml.read_text()) + except FileNotFoundError: + raise ValueError(f"no schema map found for schema name '{self.name}'") + return self._map + + +class _DefaultSchema(Schema): + """Default schema to serialize and deserialize alert bytes.""" + + def serialize(self, alert_dict: dict) -> bytes: + """Serialize `alert_dict` using the JSON format. + + Args: + alert_dict (dict): + The dictionary to be serialized. + + Returns: + bytes: + The serialized data in bytes. + """ + return json.dumps(alert_dict).encode("utf-8") + + def deserialize(self, alert_bytes: bytes) -> dict: + """Deserialize `alert_bytes`. + + Args: + alert_bytes (bytes): + The bytes to be deserialized. This is expected to be serialized as either + Avro with the schema attached in the header or JSON. + + Returns: + A dictionary representing the deserialized ``alert_bytes``. + + Raises: + SchemaError: + If the deserialization fails after trying both JSON and Avro. + """ + # [FIXME] This should be redesigned. + # For now, just try avro then json, catching basically all errors in the process. + try: + return utils.Cast.avro_to_dict(alert_bytes) + except Exception: + try: + return utils.Cast.json_to_dict(alert_bytes) + except Exception: + raise exceptions.SchemaError("Failed to deserialize the alert bytes") + + +class _SchemalessAvroSchema(Schema): + """Schema to serialize and deserialize alert bytes in the schemaless Avro format.""" + + def serialize(self, alert_dict: dict) -> bytes: + """Serialize `alert_dict` using the schemaless Avro format.""" + fout = io.BytesIO() + fastavro.schemaless_writer(fout, self.definition, alert_dict) + fout.seek(0) + message = fout.getvalue() + return message + + def deserialize(self, alert_bytes: bytes) -> dict: + bytes_io = io.BytesIO(alert_bytes) + return fastavro.schemaless_reader(bytes_io, self.definition) # [FIXME] + + +class _ConfluentWireAvroSchema(Schema): + """Schema to serialize and deserialize alert bytes in the Avro Confluent Wire Format.""" + + def serialize(self, alert_dict: dict) -> bytes: + raise NotImplementedError("Confluent Wire Format not yet supported.") + + def deserialize(self, alert_bytes: bytes) -> dict: + bytes_io = io.BytesIO(alert_bytes[5:]) + return fastavro.schemaless_reader(bytes_io, self.definition) diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.alert.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.alert.avsc new file mode 100644 index 0000000..dd05aa6 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.alert.avsc @@ -0,0 +1,24 @@ +{ + "namespace": "lsst.v7_0", + "type": "record", + "name": "alert", + "doc": "Rubin Avro alert schema v7.0", + "fields": [ + {"name": "alertId", "type": "long", "doc": "unique alert identifer"}, + {"name": "diaSource", "type": "lsst.v7_0.diaSource"}, + {"name": "prvDiaSources", "type": ["null", { + "type": "array", + "items": "lsst.v7_0.diaSource"}], "default": null}, + {"name": "prvDiaForcedSources", "type": ["null", { + "type": "array", + "items": "lsst.v7_0.diaForcedSource"}], "default": null}, + {"name": "prvDiaNondetectionLimits", "type": ["null", { + "type": "array", + "items": "lsst.v7_0.diaNondetectionLimit"}], "default": null}, + {"name": "diaObject", "type": ["null", "lsst.v7_0.diaObject"], "default": null}, + {"name": "ssObject", "type": ["null", "lsst.v7_0.ssObject"], "default": null}, + {"name": "cutoutDifference", "type": ["null", "bytes"], "default": null}, + {"name": "cutoutScience", "type": ["null", "bytes"], "default": null}, + {"name": "cutoutTemplate", "type": ["null", "bytes"], "default": null} + ] +} diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaForcedSource.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaForcedSource.avsc new file mode 100644 index 0000000..16af774 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaForcedSource.avsc @@ -0,0 +1,77 @@ +{ + "type": "record", + "namespace": "lsst.v7_0", + "name": "diaForcedSource", + "fields": [ + { + "doc": "Unique id.", + "name": "diaForcedSourceId", + "type": "long" + }, + { + "doc": "Id of the DiaObject that this DiaForcedSource was associated with.", + "name": "diaObjectId", + "type": "long" + }, + { + "doc": "Id of the visit where this forcedSource was measured.", + "name": "visit", + "type": "long" + }, + { + "doc": "Id of the detector where this forcedSource was measured.", + "name": "detector", + "type": "int" + }, + { + "default": null, + "doc": "Point Source model flux.", + "name": "psfFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfFlux.", + "name": "psfFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "x position at which psfFlux has been measured.", + "name": "x", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "y position at which psfFlux has been measured.", + "name": "y", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.", + "name": "midpointMjdTai", + "type": "double" + }, + { + "default": null, + "doc": "Filter band this source was observed with.", + "name": "band", + "type": [ + "null", + "string" + ] + } + ] +} diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaNondetectionLimit.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaNondetectionLimit.avsc new file mode 100644 index 0000000..3967f60 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaNondetectionLimit.avsc @@ -0,0 +1,11 @@ +{ + "namespace": "lsst.v7_0", + "name": "diaNondetectionLimit", + "type": "record", + "fields": [ + {"name": "ccdVisitId", "type": "long"}, + {"name": "midpointMjdTai", "type": "double"}, + {"name": "band", "type": "string"}, + {"name": "diaNoise", "type": "float"} + ] +} diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaObject.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaObject.avsc new file mode 100644 index 0000000..42686d4 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaObject.avsc @@ -0,0 +1,733 @@ +{ + "type": "record", + "namespace": "lsst.v7_0", + "name": "diaObject", + "fields": [ + { + "doc": "Unique identifier of this DiaObject.", + "name": "diaObjectId", + "type": "long" + }, + { + "doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", + "name": "ra", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of ra.", + "name": "raErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Declination coordinate of the position of the object at time radecMjdTai.", + "name": "dec", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of dec.", + "name": "decErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between ra and dec.", + "name": "ra_dec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Time at which the object was at a position ra/dec, expressed as Modified Julian Date, International Atomic Time.", + "name": "radecMjdTai", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Proper motion in right ascension.", + "name": "pmRa", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of pmRa.", + "name": "pmRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Proper motion of declination.", + "name": "pmDec", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of pmDec.", + "name": "pmDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Parallax.", + "name": "parallax", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of parallax.", + "name": "parallaxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmRa and pmDec.", + "name": "pmRa_pmDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmRa and parallax.", + "name": "pmRa_parallax_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmDec and parallax.", + "name": "pmDec_parallax_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the likelihood of the linear proper motion parallax fit.", + "name": "pmParallaxLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 static of the model fit.", + "name": "pmParallaxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the model.", + "name": "pmParallaxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for u filter.", + "name": "u_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of u_psfFluxMean.", + "name": "u_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of u_psfFlux.", + "name": "u_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of u_psfFlux around u_psfFluxMean.", + "name": "u_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute u_psfFluxChi2.", + "name": "u_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for u filter.", + "name": "u_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of u_fpFluxMean.", + "name": "u_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of u_fpFlux.", + "name": "u_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for g filter.", + "name": "g_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of g_psfFluxMean.", + "name": "g_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of g_psfFlux.", + "name": "g_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of g_psfFlux around g_psfFluxMean.", + "name": "g_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute g_psfFluxChi2.", + "name": "g_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for g filter.", + "name": "g_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of g_fpFluxMean.", + "name": "g_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of g_fpFlux.", + "name": "g_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for r filter.", + "name": "r_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of r_psfFluxMean.", + "name": "r_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of r_psfFlux.", + "name": "r_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of r_psfFlux around r_psfFluxMean.", + "name": "r_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute r_psfFluxChi2.", + "name": "r_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for r filter.", + "name": "r_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of r_fpFluxMean.", + "name": "r_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of r_fpFlux.", + "name": "r_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for i filter.", + "name": "i_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of i_psfFluxMean.", + "name": "i_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of i_psfFlux.", + "name": "i_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of i_psfFlux around i_psfFluxMean.", + "name": "i_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute i_psfFluxChi2.", + "name": "i_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for i filter.", + "name": "i_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of i_fpFluxMean.", + "name": "i_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of i_fpFlux.", + "name": "i_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for z filter.", + "name": "z_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of z_psfFluxMean.", + "name": "z_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of z_psfFlux.", + "name": "z_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of z_psfFlux around z_psfFluxMean.", + "name": "z_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute z_psfFluxChi2.", + "name": "z_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for z filter.", + "name": "z_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of z_fpFluxMean.", + "name": "z_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of z_fpFlux.", + "name": "z_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for y filter.", + "name": "y_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of y_psfFluxMean.", + "name": "y_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of y_psfFlux.", + "name": "y_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of y_psfFlux around y_psfFluxMean.", + "name": "y_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute y_psfFluxChi2.", + "name": "y_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for y filter.", + "name": "y_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of y_fpFluxMean.", + "name": "y_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of y_fpFlux.", + "name": "y_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the closest nearby object.", + "name": "nearbyObj1", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj1.", + "name": "nearbyObj1Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj1.", + "name": "nearbyObj1LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the second-closest nearby object.", + "name": "nearbyObj2", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj2.", + "name": "nearbyObj2Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj2.", + "name": "nearbyObj2LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the third-closest nearby object.", + "name": "nearbyObj3", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj3.", + "name": "nearbyObj3Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj3.", + "name": "nearbyObj3LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the u band flux errors.", + "name": "u_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the g band flux errors.", + "name": "g_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the r band flux errors.", + "name": "r_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the i band flux errors.", + "name": "i_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the z band flux errors.", + "name": "z_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the y band flux errors.", + "name": "y_psfFluxErrMean", + "type": [ + "null", + "float" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaSource.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaSource.avsc new file mode 100644 index 0000000..d908c14 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.diaSource.avsc @@ -0,0 +1,1199 @@ +{ + "type": "record", + "namespace": "lsst.v7_0", + "name": "diaSource", + "fields": [ + { + "doc": "Unique identifier of this DiaSource.", + "name": "diaSourceId", + "type": "long" + }, + { + "doc": "Id of the visit where this diaSource was measured.", + "name": "visit", + "type": "long" + }, + { + "doc": "Id of the detector where this diaSource was measured.", + "name": "detector", + "type": "int" + }, + { + "default": null, + "doc": "Id of the diaObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", + "name": "diaObjectId", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Id of the ssObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", + "name": "ssObjectId", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Id of the parent diaSource this diaSource has been deblended from, if any.", + "name": "parentDiaSourceId", + "type": [ + "null", + "long" + ] + }, + { + "doc": "Effective mid-visit time for this diaSource, expressed as Modified Julian Date, International Atomic Time.", + "name": "midpointMjdTai", + "type": "double" + }, + { + "doc": "Right ascension coordinate of the center of this diaSource.", + "name": "ra", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of ra.", + "name": "raErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Declination coordinate of the center of this diaSource.", + "name": "dec", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of dec.", + "name": "decErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between ra and dec.", + "name": "ra_dec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "doc": "x position computed by a centroiding algorithm.", + "name": "x", + "type": "float" + }, + { + "default": null, + "doc": "Uncertainty of x.", + "name": "xErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "y position computed by a centroiding algorithm.", + "name": "y", + "type": "float" + }, + { + "default": null, + "doc": "Uncertainty of y.", + "name": "yErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between x and y.", + "name": "x_y_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General centroid algorithm failure flag; set if anything went wrong when fitting the centroid. Another centroid flag field should also be set to provide more information.", + "name": "centroid_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Flux in a 12 pixel radius aperture on the difference image.", + "name": "apFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of apFlux.", + "name": "apFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General aperture flux algorithm failure flag; set if anything went wrong when measuring aperture fluxes. Another apFlux flag field should also be set to provide more information.", + "name": "apFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Aperture did not fit within measurement image.", + "name": "apFlux_flag_apertureTruncated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "The signal-to-noise ratio at which this source was detected in the difference image.", + "name": "snr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Flux for Point Source model. Note this actually measures the flux difference between the template and the visit image.", + "name": "psfFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfFlux.", + "name": "psfFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for point source model.", + "name": "psfRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfRa.", + "name": "psfRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for point source model.", + "name": "psfDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfDec.", + "name": "psfDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfFlux and psfRa.", + "name": "psfFlux_psfRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfFlux and psfDec.", + "name": "psfFlux_psfDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfRa and psfDec.", + "name": "psfRa_psfDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the point source model.", + "name": "psfLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the point source model fit.", + "name": "psfChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the point source model.", + "name": "psfNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Failure to derive linear least-squares fit of psf model. Another psfFlux flag field should also be set to provide more information.", + "name": "psfFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Object was too close to the edge of the image to use the full PSF model.", + "name": "psfFlux_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Not enough non-rejected pixels in data to attempt the fit.", + "name": "psfFlux_flag_noGoodPixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Flux for a trailed source model. Note this actually measures the flux difference between the template and the visit image.", + "name": "trailFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailFlux.", + "name": "trailFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for trailed source model.", + "name": "trailRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailRa.", + "name": "trailRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for trailed source model.", + "name": "trailDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailDec.", + "name": "trailDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of trail length.", + "name": "trailLength", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailLength.", + "name": "trailLengthErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the trail direction (bearing).", + "name": "trailAngle", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailAngle.", + "name": "trailAngleErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailRa.", + "name": "trailFlux_trailRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailDec.", + "name": "trailFlux_trailDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailLength", + "name": "trailFlux_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailAngle", + "name": "trailFlux_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailDec.", + "name": "trailRa_trailDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailLength.", + "name": "trailRa_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailAngle.", + "name": "trailRa_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailDec and trailLength.", + "name": "trailDec_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailDec and trailAngle.", + "name": "trailDec_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailLength and trailAngle", + "name": "trailLength_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the trailed source model.", + "name": "trailLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the trailed source model fit.", + "name": "trailChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the trailed source model.", + "name": "trailNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "This flag is set if a trailed source extends onto or past edge pixels.", + "name": "trail_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the mean absolute flux of the two lobes for a dipole model.", + "name": "dipoleMeanFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleMeanFlux.", + "name": "dipoleMeanFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the difference of absolute fluxes of the two lobes for a dipole model.", + "name": "dipoleFluxDiff", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleFluxDiff.", + "name": "dipoleFluxDiffErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for dipole model.", + "name": "dipoleRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleRa.", + "name": "dipoleRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for dipole model.", + "name": "dipoleDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleDec.", + "name": "dipoleDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the lobe separation in dipole model.", + "name": "dipoleLength", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleLength.", + "name": "dipoleLengthErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the dipole direction (bearing, from negative to positive lobe).", + "name": "dipoleAngle", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleAngle.", + "name": "dipoleAngleErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleFluxDiff.", + "name": "dipoleMeanFlux_dipoleFluxDiff_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleRa.", + "name": "dipoleMeanFlux_dipoleRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleDec.", + "name": "dipoleMeanFlux_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleLength.", + "name": "dipoleMeanFlux_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleAngle.", + "name": "dipoleMeanFlux_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleRa.", + "name": "dipoleFluxDiff_dipoleRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleDec.", + "name": "dipoleFluxDiff_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleLength.", + "name": "dipoleFluxDiff_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleAngle.", + "name": "dipoleFluxDiff_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleDec.", + "name": "dipoleRa_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleLength.", + "name": "dipoleRa_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleAngle.", + "name": "dipoleRa_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleDec and dipoleLength.", + "name": "dipoleDec_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleDec and dipoleAngle.", + "name": "dipoleDec_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleLength and dipoleAngle.", + "name": "dipoleLength_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the dipole source model.", + "name": "dipoleLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the model fit.", + "name": "dipoleChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the model.", + "name": "dipoleNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Forced PSF photometry on science image failed. Another forced_PsfFlux flag field should also be set to provide more information.", + "name": "forced_PsfFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Forced PSF flux on science image was too close to the edge of the image to use the full PSF model.", + "name": "forced_PsfFlux_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Forced PSF flux not enough non-rejected pixels in data to attempt the fit.", + "name": "forced_PsfFlux_flag_noGoodPixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Calibrated flux for Point Source model centered on radec but measured on the difference of snaps comprising this visit.", + "name": "snapDiffFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of snapDiffFlux.", + "name": "snapDiffFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated sky background at the position (centroid) of the object.", + "name": "fpBkgd", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of fpBkgd.", + "name": "fpBkgdErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "ixx", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of ixx.", + "name": "ixxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "iyy", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of iyy.", + "name": "iyyErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "ixy", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of ixy.", + "name": "ixyErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of ixx and iyy.", + "name": "ixx_iyy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of ixx and ixy.", + "name": "ixx_ixy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of iyy and ixy.", + "name": "iyy_ixy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "ixxPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "iyyPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "ixyPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General source shape algorithm failure flag; set if anything went wrong when measuring the shape. Another shape flag field should also be set to provide more information.", + "name": "shape_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "No pixels to measure shape.", + "name": "shape_flag_no_pixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Center not contained in footprint bounding box.", + "name": "shape_flag_not_contained", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "This source is a parent source; we should only be measuring on deblended children in difference imaging.", + "name": "shape_flag_parent_source", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "A measure of extendedness, Computed using a combination of available moments and model fluxes or from a likelihood ratio of point/trailed source models (exact algorithm TBD). extendedness = 1 implies a high degree of confidence that the source is extended. extendedness = 0 implies a high degree of confidence that the source is point-like.", + "name": "extendedness", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "A measure of reliability, computed using information from the source and image characterization, as well as the information on the Telescope and Camera system (e.g., ghost maps, defect maps, etc.).", + "name": "reliability", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Filter band this source was observed with.", + "name": "band", + "type": [ + "null", + "string" + ] + }, + { + "default": null, + "doc": "General pixel flags failure; set if anything went wrong when setting pixels flags from this footprint's mask. This implies that some pixelFlags for this source may be incorrectly set to False.", + "name": "pixelFlags", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Bad pixel in the DiaSource footprint.", + "name": "pixelFlags_bad", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Cosmic ray in the DiaSource footprint.", + "name": "pixelFlags_cr", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Cosmic ray in the 3x3 region around the centroid.", + "name": "pixelFlags_crCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Some of the source footprint is outside usable exposure region (masked EDGE or NO_DATA, or centroid off image).", + "name": "pixelFlags_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Interpolated pixel in the DiaSource footprint.", + "name": "pixelFlags_interpolated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Interpolated pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_interpolatedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "DiaSource center is off image.", + "name": "pixelFlags_offimage", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Saturated pixel in the DiaSource footprint.", + "name": "pixelFlags_saturated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Saturated pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_saturatedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "DiaSource's footprint includes suspect pixels.", + "name": "pixelFlags_suspect", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Suspect pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_suspectCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Streak in the DiaSource footprint.", + "name": "pixelFlags_streak", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Streak in the 3x3 region around the centroid.", + "name": "pixelFlags_streakCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Injection in the DiaSource footprint.", + "name": "pixelFlags_injected", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Injection in the 3x3 region around the centroid.", + "name": "pixelFlags_injectedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Template injection in the DiaSource footprint.", + "name": "pixelFlags_injected_template", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Template injection in the 3x3 region around the centroid.", + "name": "pixelFlags_injected_templateCenter", + "type": [ + "null", + "boolean" + ] + } + ] +} diff --git a/pittgoogle/schemas/lsst/7/0/lsst.v7_0.ssObject.avsc b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.ssObject.avsc new file mode 100644 index 0000000..d037e1d --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/lsst.v7_0.ssObject.avsc @@ -0,0 +1,471 @@ +{ + "type": "record", + "namespace": "lsst.v7_0", + "name": "ssObject", + "fields": [ + { + "doc": "Unique identifier.", + "name": "ssObjectId", + "type": "long" + }, + { + "default": null, + "doc": "The date the LSST first linked and submitted the discovery observations to the MPC. May be NULL if not an LSST discovery. The date format will follow general LSST conventions (MJD TAI, at the moment).", + "name": "discoverySubmissionDate", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "The time of the first LSST observation of this object (could be precovered) as Modified Julian Date, International Atomic Time.", + "name": "firstObservationDate", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Arc of LSST observations.", + "name": "arc", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Number of LSST observations of this object.", + "name": "numObs", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Minimum orbit intersection distance to Earth.", + "name": "MOID", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "True anomaly of the MOID point.", + "name": "MOIDTrueAnomaly", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Ecliptic longitude of the MOID point.", + "name": "MOIDEclipticLongitude", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "DeltaV at the MOID point.", + "name": "MOIDDeltaV", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (u band).", + "name": "u_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (u band).", + "name": "u_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (u band).", + "name": "u_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (u band).", + "name": "u_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (u band).", + "name": "u_H_u_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (u band).", + "name": "u_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (u band).", + "name": "u_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (g band).", + "name": "g_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (g band).", + "name": "g_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (g band).", + "name": "g_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (g band).", + "name": "g_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (g band).", + "name": "g_H_g_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (g band).", + "name": "g_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (g band).", + "name": "g_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (r band).", + "name": "r_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (r band).", + "name": "r_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (r band).", + "name": "r_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (r band).", + "name": "r_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (r band).", + "name": "r_H_r_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (r band).", + "name": "r_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (r band).", + "name": "r_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (i band).", + "name": "i_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (i band).", + "name": "i_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (i band).", + "name": "i_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (i band).", + "name": "i_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (i band).", + "name": "i_H_i_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (i band).", + "name": "i_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (i band).", + "name": "i_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (z band).", + "name": "z_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (z band).", + "name": "z_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (z band).", + "name": "z_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (z band).", + "name": "z_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (z band).", + "name": "z_H_z_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (z band).", + "name": "z_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (z band).", + "name": "z_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (y band).", + "name": "y_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (y band).", + "name": "y_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (y band).", + "name": "y_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (y band).", + "name": "y_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (y band).", + "name": "y_H_y_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (y band).", + "name": "y_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (y band).", + "name": "y_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "median `extendedness` value from the DIASource.", + "name": "medianExtendedness", + "type": [ + "null", + "float" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/0/sample_data/alert.json b/pittgoogle/schemas/lsst/7/0/sample_data/alert.json new file mode 100644 index 0000000..1a1f952 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/sample_data/alert.json @@ -0,0 +1,109 @@ +{ + "alertId": 1231321321, + "l1dbId": 222222222, + "diaSource": + { + "diaSourceId": 281323062375219200, + "time_processed": 0, + "visit": 1, + "detector": 42, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + }, + "prvDiaSources": [ + { + "diaSourceId": 281323062375219198, + "time_processed": 0, + "visit": 2, + "detector": 43, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + }, + { + "diaSourceId": 281323062375219199, + "time_processed": 0, + "visit": 1, + "detector": 42, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + } + ], + "diaObject": { + "diaObjectId": 281323062375219201, + "validityStart": 0, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "radecMjdTai": 1480360995, + "pmRa": 0.00013, + "pmDec": 0.00014, + "parallax": 2.124124, + "pmRaErr": 0.00013, + "pmDecErr": 0.00013, + "pmParallaxNdata": 0, + "parallaxErr": 0.00013, + "pmRa_pmDec_Cov": 0.00013, + "pmRa_parallax_Cov": 0.00013, + "pmDec_parallax_Cov": 0.00013, + "pmParallaxLnL": 0.00013, + "pmParallaxChi2": 0.00013, + "pmParallaxNdata": 1214 + }, + "ssObject": { + "ssObjectId":5364546, + "numObs": 10 + } +} diff --git a/pittgoogle/schemas/lsst/7/0/sample_data/fakeAlert.avro b/pittgoogle/schemas/lsst/7/0/sample_data/fakeAlert.avro new file mode 100644 index 0000000..a222717 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/sample_data/fakeAlert.avro @@ -0,0 +1,4 @@ +Objavro.codecnullavro.schema{"type": "record", "doc": "Rubin Avro alert schema v7.0", "name": "lsst.v7_0.alert", "fields": [{"doc": "unique alert identifer", "name": "alertId", "type": "long"}, {"name": "diaSource", "type": {"type": "record", "name": "lsst.v7_0.diaSource", "fields": [{"doc": "Unique identifier of this DiaSource.", "name": "diaSourceId", "type": "long"}, {"doc": "Id of the visit where this diaSource was measured.", "name": "visit", "type": "long"}, {"doc": "Id of the detector where this diaSource was measured.", "name": "detector", "type": "int"}, {"default": null, "doc": "Id of the diaObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", "name": "diaObjectId", "type": ["null", "long"]}, {"default": null, "doc": "Id of the ssObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", "name": "ssObjectId", "type": ["null", "long"]}, {"default": null, "doc": "Id of the parent diaSource this diaSource has been deblended from, if any.", "name": "parentDiaSourceId", "type": ["null", "long"]}, {"doc": "Effective mid-visit time for this diaSource, expressed as Modified Julian Date, International Atomic Time.", "name": "midpointMjdTai", "type": "double"}, {"doc": "Right ascension coordinate of the center of this diaSource.", "name": "ra", "type": "double"}, {"default": null, "doc": "Uncertainty of ra.", "name": "raErr", "type": ["null", "float"]}, {"doc": "Declination coordinate of the center of this diaSource.", "name": "dec", "type": "double"}, {"default": null, "doc": "Uncertainty of dec.", "name": "decErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between ra and dec.", "name": "ra_dec_Cov", "type": ["null", "float"]}, {"doc": "x position computed by a centroiding algorithm.", "name": "x", "type": "float"}, {"default": null, "doc": "Uncertainty of x.", "name": "xErr", "type": ["null", "float"]}, {"doc": "y position computed by a centroiding algorithm.", "name": "y", "type": "float"}, {"default": null, "doc": "Uncertainty of y.", "name": "yErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between x and y.", "name": "x_y_Cov", "type": ["null", "float"]}, {"default": null, "doc": "General centroid algorithm failure flag; set if anything went wrong when fitting the centroid. Another centroid flag field should also be set to provide more information.", "name": "centroid_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Flux in a 12 pixel radius aperture on the difference image.", "name": "apFlux", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of apFlux.", "name": "apFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "General aperture flux algorithm failure flag; set if anything went wrong when measuring aperture fluxes. Another apFlux flag field should also be set to provide more information.", "name": "apFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Aperture did not fit within measurement image.", "name": "apFlux_flag_apertureTruncated", "type": ["null", "boolean"]}, {"default": null, "doc": "The signal-to-noise ratio at which this source was detected in the difference image.", "name": "snr", "type": ["null", "float"]}, {"default": null, "doc": "Flux for Point Source model. Note this actually measures the flux difference between the template and the visit image.", "name": "psfFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of psfFlux.", "name": "psfFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for point source model.", "name": "psfRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of psfRa.", "name": "psfRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for point source model.", "name": "psfDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of psfDec.", "name": "psfDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfFlux and psfRa.", "name": "psfFlux_psfRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfFlux and psfDec.", "name": "psfFlux_psfDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfRa and psfDec.", "name": "psfRa_psfDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the point source model.", "name": "psfLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the point source model fit.", "name": "psfChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the point source model.", "name": "psfNdata", "type": ["null", "int"]}, {"default": null, "doc": "Failure to derive linear least-squares fit of psf model. Another psfFlux flag field should also be set to provide more information.", "name": "psfFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Object was too close to the edge of the image to use the full PSF model.", "name": "psfFlux_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Not enough non-rejected pixels in data to attempt the fit.", "name": "psfFlux_flag_noGoodPixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Flux for a trailed source model. Note this actually measures the flux difference between the template and the visit image.", "name": "trailFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailFlux.", "name": "trailFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for trailed source model.", "name": "trailRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of trailRa.", "name": "trailRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for trailed source model.", "name": "trailDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of trailDec.", "name": "trailDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of trail length.", "name": "trailLength", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailLength.", "name": "trailLengthErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the trail direction (bearing).", "name": "trailAngle", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailAngle.", "name": "trailAngleErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailRa.", "name": "trailFlux_trailRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailDec.", "name": "trailFlux_trailDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailLength", "name": "trailFlux_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailAngle", "name": "trailFlux_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailDec.", "name": "trailRa_trailDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailLength.", "name": "trailRa_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailAngle.", "name": "trailRa_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailDec and trailLength.", "name": "trailDec_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailDec and trailAngle.", "name": "trailDec_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailLength and trailAngle", "name": "trailLength_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the trailed source model.", "name": "trailLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the trailed source model fit.", "name": "trailChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the trailed source model.", "name": "trailNdata", "type": ["null", "int"]}, {"default": null, "doc": "This flag is set if a trailed source extends onto or past edge pixels.", "name": "trail_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Maximum likelihood value for the mean absolute flux of the two lobes for a dipole model.", "name": "dipoleMeanFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleMeanFlux.", "name": "dipoleMeanFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood value for the difference of absolute fluxes of the two lobes for a dipole model.", "name": "dipoleFluxDiff", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleFluxDiff.", "name": "dipoleFluxDiffErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for dipole model.", "name": "dipoleRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of dipoleRa.", "name": "dipoleRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for dipole model.", "name": "dipoleDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of dipoleDec.", "name": "dipoleDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood value for the lobe separation in dipole model.", "name": "dipoleLength", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleLength.", "name": "dipoleLengthErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the dipole direction (bearing, from negative to positive lobe).", "name": "dipoleAngle", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleAngle.", "name": "dipoleAngleErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleFluxDiff.", "name": "dipoleMeanFlux_dipoleFluxDiff_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleRa.", "name": "dipoleMeanFlux_dipoleRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleDec.", "name": "dipoleMeanFlux_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleLength.", "name": "dipoleMeanFlux_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleAngle.", "name": "dipoleMeanFlux_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleRa.", "name": "dipoleFluxDiff_dipoleRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleDec.", "name": "dipoleFluxDiff_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleLength.", "name": "dipoleFluxDiff_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleAngle.", "name": "dipoleFluxDiff_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleDec.", "name": "dipoleRa_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleLength.", "name": "dipoleRa_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleAngle.", "name": "dipoleRa_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleDec and dipoleLength.", "name": "dipoleDec_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleDec and dipoleAngle.", "name": "dipoleDec_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleLength and dipoleAngle.", "name": "dipoleLength_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the dipole source model.", "name": "dipoleLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the model fit.", "name": "dipoleChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the model.", "name": "dipoleNdata", "type": ["null", "int"]}, {"default": null, "doc": "Forced PSF photometry on science image failed. Another forced_PsfFlux flag field should also be set to provide more information.", "name": "forced_PsfFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Forced PSF flux on science image was too close to the edge of the image to use the full PSF model.", "name": "forced_PsfFlux_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Forced PSF flux not enough non-rejected pixels in data to attempt the fit.", "name": "forced_PsfFlux_flag_noGoodPixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Calibrated flux for Point Source model centered on radec but measured on the difference of snaps comprising this visit.", "name": "snapDiffFlux", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of snapDiffFlux.", "name": "snapDiffFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Estimated sky background at the position (centroid) of the object.", "name": "fpBkgd", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of fpBkgd.", "name": "fpBkgdErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "ixx", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of ixx.", "name": "ixxErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "iyy", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of iyy.", "name": "iyyErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "ixy", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of ixy.", "name": "ixyErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of ixx and iyy.", "name": "ixx_iyy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of ixx and ixy.", "name": "ixx_ixy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of iyy and ixy.", "name": "iyy_ixy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "ixxPSF", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "iyyPSF", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "ixyPSF", "type": ["null", "float"]}, {"default": null, "doc": "General source shape algorithm failure flag; set if anything went wrong when measuring the shape. Another shape flag field should also be set to provide more information.", "name": "shape_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "No pixels to measure shape.", "name": "shape_flag_no_pixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Center not contained in footprint bounding box.", "name": "shape_flag_not_contained", "type": ["null", "boolean"]}, {"default": null, "doc": "This source is a parent source; we should only be measuring on deblended children in difference imaging.", "name": "shape_flag_parent_source", "type": ["null", "boolean"]}, {"default": null, "doc": "A measure of extendedness, Computed using a combination of available moments and model fluxes or from a likelihood ratio of point/trailed source models (exact algorithm TBD). extendedness = 1 implies a high degree of confidence that the source is extended. extendedness = 0 implies a high degree of confidence that the source is point-like.", "name": "extendedness", "type": ["null", "float"]}, {"default": null, "doc": "A measure of reliability, computed using information from the source and image characterization, as well as the information on the Telescope and Camera system (e.g., ghost maps, defect maps, etc.).", "name": "reliability", "type": ["null", "float"]}, {"default": null, "doc": "Filter band this source was observed with.", "name": "band", "type": ["null", "string"]}, {"default": null, "doc": "General pixel flags failure; set if anything went wrong when setting pixels flags from this footprint's mask. This implies that some pixelFlags for this source may be incorrectly set to False.", "name": "pixelFlags", "type": ["null", "boolean"]}, {"default": null, "doc": "Bad pixel in the DiaSource footprint.", "name": "pixelFlags_bad", "type": ["null", "boolean"]}, {"default": null, "doc": "Cosmic ray in the DiaSource footprint.", "name": "pixelFlags_cr", "type": ["null", "boolean"]}, {"default": null, "doc": "Cosmic ray in the 3x3 region around the centroid.", "name": "pixelFlags_crCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Some of the source footprint is outside usable exposure region (masked EDGE or NO_DATA, or centroid off image).", "name": "pixelFlags_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Interpolated pixel in the DiaSource footprint.", "name": "pixelFlags_interpolated", "type": ["null", "boolean"]}, {"default": null, "doc": "Interpolated pixel in the 3x3 region around the centroid.", "name": "pixelFlags_interpolatedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "DiaSource center is off image.", "name": "pixelFlags_offimage", "type": ["null", "boolean"]}, {"default": null, "doc": "Saturated pixel in the DiaSource footprint.", "name": "pixelFlags_saturated", "type": ["null", "boolean"]}, {"default": null, "doc": "Saturated pixel in the 3x3 region around the centroid.", "name": "pixelFlags_saturatedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "DiaSource's footprint includes suspect pixels.", "name": "pixelFlags_suspect", "type": ["null", "boolean"]}, {"default": null, "doc": "Suspect pixel in the 3x3 region around the centroid.", "name": "pixelFlags_suspectCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Streak in the DiaSource footprint.", "name": "pixelFlags_streak", "type": ["null", "boolean"]}, {"default": null, "doc": "Streak in the 3x3 region around the centroid.", "name": "pixelFlags_streakCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Injection in the DiaSource footprint.", "name": "pixelFlags_injected", "type": ["null", "boolean"]}, {"default": null, "doc": "Injection in the 3x3 region around the centroid.", "name": "pixelFlags_injectedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Template injection in the DiaSource footprint.", "name": "pixelFlags_injected_template", "type": ["null", "boolean"]}, {"default": null, "doc": "Template injection in the 3x3 region around the centroid.", "name": "pixelFlags_injected_templateCenter", "type": ["null", "boolean"]}]}}, {"default": null, "name": "prvDiaSources", "type": ["null", {"type": "array", "items": "lsst.v7_0.diaSource"}]}, {"default": null, "name": "prvDiaForcedSources", "type": ["null", {"type": "array", "items": {"type": "record", "name": "lsst.v7_0.diaForcedSource", "fields": [{"doc": "Unique id.", "name": "diaForcedSourceId", "type": "long"}, {"doc": "Id of the DiaObject that this DiaForcedSource was associated with.", "name": "diaObjectId", "type": "long"}, {"doc": "Id of the visit where this forcedSource was measured.", "name": "visit", "type": "long"}, {"doc": "Id of the detector where this forcedSource was measured.", "name": "detector", "type": "int"}, {"default": null, "doc": "Point Source model flux.", "name": "psfFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of psfFlux.", "name": "psfFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "x position at which psfFlux has been measured.", "name": "x", "type": ["null", "float"]}, {"default": null, "doc": "y position at which psfFlux has been measured.", "name": "y", "type": ["null", "float"]}, {"doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.", "name": "midpointMjdTai", "type": "double"}, {"default": null, "doc": "Filter band this source was observed with.", "name": "band", "type": ["null", "string"]}]}}]}, {"default": null, "name": "prvDiaNondetectionLimits", "type": ["null", {"type": "array", "items": {"type": "record", "name": "lsst.v7_0.diaNondetectionLimit", "fields": [{"name": "ccdVisitId", "type": "long"}, {"name": "midpointMjdTai", "type": "double"}, {"name": "band", "type": "string"}, {"name": "diaNoise", "type": "float"}]}}]}, {"default": null, "name": "diaObject", "type": ["null", {"type": "record", "name": "lsst.v7_0.diaObject", "fields": [{"doc": "Unique identifier of this DiaObject.", "name": "diaObjectId", "type": "long"}, {"doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", "name": "ra", "type": "double"}, {"default": null, "doc": "Uncertainty of ra.", "name": "raErr", "type": ["null", "float"]}, {"doc": "Declination coordinate of the position of the object at time radecMjdTai.", "name": "dec", "type": "double"}, {"default": null, "doc": "Uncertainty of dec.", "name": "decErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between ra and dec.", "name": "ra_dec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Time at which the object was at a position ra/dec, expressed as Modified Julian Date, International Atomic Time.", "name": "radecMjdTai", "type": ["null", "double"]}, {"default": null, "doc": "Proper motion in right ascension.", "name": "pmRa", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of pmRa.", "name": "pmRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Proper motion of declination.", "name": "pmDec", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of pmDec.", "name": "pmDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Parallax.", "name": "parallax", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of parallax.", "name": "parallaxErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmRa and pmDec.", "name": "pmRa_pmDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmRa and parallax.", "name": "pmRa_parallax_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmDec and parallax.", "name": "pmDec_parallax_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the likelihood of the linear proper motion parallax fit.", "name": "pmParallaxLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 static of the model fit.", "name": "pmParallaxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the model.", "name": "pmParallaxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for u filter.", "name": "u_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of u_psfFluxMean.", "name": "u_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of u_psfFlux.", "name": "u_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of u_psfFlux around u_psfFluxMean.", "name": "u_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute u_psfFluxChi2.", "name": "u_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for u filter.", "name": "u_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of u_fpFluxMean.", "name": "u_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of u_fpFlux.", "name": "u_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for g filter.", "name": "g_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of g_psfFluxMean.", "name": "g_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of g_psfFlux.", "name": "g_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of g_psfFlux around g_psfFluxMean.", "name": "g_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute g_psfFluxChi2.", "name": "g_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for g filter.", "name": "g_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of g_fpFluxMean.", "name": "g_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of g_fpFlux.", "name": "g_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for r filter.", "name": "r_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of r_psfFluxMean.", "name": "r_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of r_psfFlux.", "name": "r_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of r_psfFlux around r_psfFluxMean.", "name": "r_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute r_psfFluxChi2.", "name": "r_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for r filter.", "name": "r_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of r_fpFluxMean.", "name": "r_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of r_fpFlux.", "name": "r_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for i filter.", "name": "i_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of i_psfFluxMean.", "name": "i_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of i_psfFlux.", "name": "i_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of i_psfFlux around i_psfFluxMean.", "name": "i_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute i_psfFluxChi2.", "name": "i_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for i filter.", "name": "i_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of i_fpFluxMean.", "name": "i_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of i_fpFlux.", "name": "i_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for z filter.", "name": "z_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of z_psfFluxMean.", "name": "z_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of z_psfFlux.", "name": "z_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of z_psfFlux around z_psfFluxMean.", "name": "z_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute z_psfFluxChi2.", "name": "z_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for z filter.", "name": "z_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of z_fpFluxMean.", "name": "z_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of z_fpFlux.", "name": "z_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for y filter.", "name": "y_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of y_psfFluxMean.", "name": "y_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of y_psfFlux.", "name": "y_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of y_psfFlux around y_psfFluxMean.", "name": "y_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute y_psfFluxChi2.", "name": "y_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for y filter.", "name": "y_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of y_fpFluxMean.", "name": "y_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of y_fpFlux.", "name": "y_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Id of the closest nearby object.", "name": "nearbyObj1", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj1.", "name": "nearbyObj1Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj1.", "name": "nearbyObj1LnP", "type": ["null", "float"]}, {"default": null, "doc": "Id of the second-closest nearby object.", "name": "nearbyObj2", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj2.", "name": "nearbyObj2Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj2.", "name": "nearbyObj2LnP", "type": ["null", "float"]}, {"default": null, "doc": "Id of the third-closest nearby object.", "name": "nearbyObj3", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj3.", "name": "nearbyObj3Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj3.", "name": "nearbyObj3LnP", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the u band flux errors.", "name": "u_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the g band flux errors.", "name": "g_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the r band flux errors.", "name": "r_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the i band flux errors.", "name": "i_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the z band flux errors.", "name": "z_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the y band flux errors.", "name": "y_psfFluxErrMean", "type": ["null", "float"]}]}]}, {"default": null, "name": "ssObject", "type": ["null", {"type": "record", "name": "lsst.v7_0.ssObject", "fields": [{"doc": "Unique identifier.", "name": "ssObjectId", "type": "long"}, {"default": null, "doc": "The date the LSST first linked and submitted the discovery observations to the MPC. May be NULL if not an LSST discovery. The date format will follow general LSST conventions (MJD TAI, at the moment).", "name": "discoverySubmissionDate", "type": ["null", "double"]}, {"default": null, "doc": "The time of the first LSST observation of this object (could be precovered) as Modified Julian Date, International Atomic Time.", "name": "firstObservationDate", "type": ["null", "double"]}, {"default": null, "doc": "Arc of LSST observations.", "name": "arc", "type": ["null", "float"]}, {"default": null, "doc": "Number of LSST observations of this object.", "name": "numObs", "type": ["null", "int"]}, {"default": null, "doc": "Minimum orbit intersection distance to Earth.", "name": "MOID", "type": ["null", "float"]}, {"default": null, "doc": "True anomaly of the MOID point.", "name": "MOIDTrueAnomaly", "type": ["null", "float"]}, {"default": null, "doc": "Ecliptic longitude of the MOID point.", "name": "MOIDEclipticLongitude", "type": ["null", "float"]}, {"default": null, "doc": "DeltaV at the MOID point.", "name": "MOIDDeltaV", "type": ["null", "float"]}, {"default": null, "doc": "Best fit absolute magnitude (u band).", "name": "u_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (u band).", "name": "u_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (u band).", "name": "u_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (u band).", "name": "u_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (u band).", "name": "u_H_u_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (u band).", "name": "u_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (u band).", "name": "u_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (g band).", "name": "g_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (g band).", "name": "g_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (g band).", "name": "g_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (g band).", "name": "g_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (g band).", "name": "g_H_g_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (g band).", "name": "g_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (g band).", "name": "g_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (r band).", "name": "r_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (r band).", "name": "r_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (r band).", "name": "r_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (r band).", "name": "r_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (r band).", "name": "r_H_r_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (r band).", "name": "r_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (r band).", "name": "r_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (i band).", "name": "i_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (i band).", "name": "i_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (i band).", "name": "i_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (i band).", "name": "i_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (i band).", "name": "i_H_i_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (i band).", "name": "i_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (i band).", "name": "i_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (z band).", "name": "z_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (z band).", "name": "z_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (z band).", "name": "z_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (z band).", "name": "z_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (z band).", "name": "z_H_z_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (z band).", "name": "z_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (z band).", "name": "z_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (y band).", "name": "y_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (y band).", "name": "y_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (y band).", "name": "y_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (y band).", "name": "y_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (y band).", "name": "y_H_y_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (y band).", "name": "y_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (y band).", "name": "y_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "median `extendedness` value from the DIASource.", "name": "medianExtendedness", "type": ["null", "float"]}]}]}, {"default": null, "name": "cutoutDifference", "type": ["null", "bytes"]}, {"default": null, "name": "cutoutScience", "type": ["null", "bytes"]}, {"default": null, "name": "cutoutTemplate", "type": ["null", "bytes"]}]}1nBr$ޠ 㣖 T!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rV!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rT!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rw5 u@̒9eVt(?̒9$ 9!AP9P99P9@P9P9P9P9P9P91nBr$ޠ \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/0/sample_data/generate.py b/pittgoogle/schemas/lsst/7/0/sample_data/generate.py new file mode 100644 index 0000000..3fddbe2 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/0/sample_data/generate.py @@ -0,0 +1,17 @@ +#!/usr/bin/env python +import json +import lsst.alert.packet as packet +from datetime import datetime +import pandas as pd + +with open("alert.json", "r") as f: + data = json.load(f) + + data_time = datetime.now() + + data['diaSource']['time_processed'] = pd.Timestamp(data_time) + data['diaObject']['validityStart'] = pd.Timestamp(data_time) + +schema = packet.SchemaRegistry.from_filesystem(schema_root="lsst.v7_0.alert").get_by_version("7.0") +with open("fakeAlert.avro", "wb") as f: + schema.store_alerts(f, [data]) diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.alert.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.alert.avsc new file mode 100644 index 0000000..0a70068 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.alert.avsc @@ -0,0 +1,24 @@ +{ + "namespace": "lsst.v7_1", + "type": "record", + "name": "alert", + "doc": "Rubin Avro alert schema v7.1", + "fields": [ + {"name": "alertId", "type": "long", "doc": "unique alert identifer"}, + {"name": "diaSource", "type": "lsst.v7_1.diaSource"}, + {"name": "prvDiaSources", "type": ["null", { + "type": "array", + "items": "lsst.v7_1.diaSource"}], "default": null}, + {"name": "prvDiaForcedSources", "type": ["null", { + "type": "array", + "items": "lsst.v7_1.diaForcedSource"}], "default": null}, + {"name": "prvDiaNondetectionLimits", "type": ["null", { + "type": "array", + "items": "lsst.v7_1.diaNondetectionLimit"}], "default": null}, + {"name": "diaObject", "type": ["null", "lsst.v7_1.diaObject"], "default": null}, + {"name": "ssObject", "type": ["null", "lsst.v7_1.ssObject"], "default": null}, + {"name": "cutoutDifference", "type": ["null", "bytes"], "default": null}, + {"name": "cutoutScience", "type": ["null", "bytes"], "default": null}, + {"name": "cutoutTemplate", "type": ["null", "bytes"], "default": null} + ] +} diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaForcedSource.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaForcedSource.avsc new file mode 100644 index 0000000..db7ffca --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaForcedSource.avsc @@ -0,0 +1,69 @@ +{ + "type": "record", + "namespace": "lsst.v7_1", + "name": "diaForcedSource", + "fields": [ + { + "doc": "Unique id.", + "name": "diaForcedSourceId", + "type": "long" + }, + { + "doc": "Id of the DiaObject that this DiaForcedSource was associated with.", + "name": "diaObjectId", + "type": "long" + }, + { + "doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", + "name": "ra", + "type": "double" + }, + { + "doc": "Declination coordinate of the position of the object at time radecMjdTai.", + "name": "dec", + "type": "double" + }, + { + "doc": "Id of the visit where this forcedSource was measured.", + "name": "visit", + "type": "long" + }, + { + "doc": "Id of the detector where this forcedSource was measured. Datatype short instead of byte because of DB concerns about unsigned bytes.", + "name": "detector", + "type": "int" + }, + { + "default": null, + "doc": "Point Source model flux.", + "name": "psfFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfFlux.", + "name": "psfFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.", + "name": "midpointMjdTai", + "type": "double" + }, + { + "default": null, + "doc": "Filter band this source was observed with.", + "name": "band", + "type": [ + "null", + "string" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaNondetectionLimit.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaNondetectionLimit.avsc new file mode 100644 index 0000000..d1d9b77 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaNondetectionLimit.avsc @@ -0,0 +1,11 @@ +{ + "namespace": "lsst.v7_1", + "name": "diaNondetectionLimit", + "type": "record", + "fields": [ + {"name": "ccdVisitId", "type": "long"}, + {"name": "midpointMjdTai", "type": "double"}, + {"name": "band", "type": "string"}, + {"name": "diaNoise", "type": "float"} + ] +} diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaObject.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaObject.avsc new file mode 100644 index 0000000..f153875 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaObject.avsc @@ -0,0 +1,733 @@ +{ + "type": "record", + "namespace": "lsst.v7_1", + "name": "diaObject", + "fields": [ + { + "doc": "Unique identifier of this DiaObject.", + "name": "diaObjectId", + "type": "long" + }, + { + "doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", + "name": "ra", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of ra.", + "name": "raErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Declination coordinate of the position of the object at time radecMjdTai.", + "name": "dec", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of dec.", + "name": "decErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between ra and dec.", + "name": "ra_dec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Time at which the object was at a position ra/dec, expressed as Modified Julian Date, International Atomic Time.", + "name": "radecMjdTai", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Proper motion in right ascension.", + "name": "pmRa", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of pmRa.", + "name": "pmRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Proper motion of declination.", + "name": "pmDec", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of pmDec.", + "name": "pmDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Parallax.", + "name": "parallax", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of parallax.", + "name": "parallaxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmRa and pmDec.", + "name": "pmRa_pmDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmRa and parallax.", + "name": "pmRa_parallax_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of pmDec and parallax.", + "name": "pmDec_parallax_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the likelihood of the linear proper motion parallax fit.", + "name": "pmParallaxLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 static of the model fit.", + "name": "pmParallaxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the model.", + "name": "pmParallaxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for u filter.", + "name": "u_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of u_psfFluxMean.", + "name": "u_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of u_psfFlux.", + "name": "u_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of u_psfFlux around u_psfFluxMean.", + "name": "u_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute u_psfFluxChi2.", + "name": "u_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for u filter.", + "name": "u_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of u_fpFluxMean.", + "name": "u_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of u_fpFlux.", + "name": "u_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for g filter.", + "name": "g_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of g_psfFluxMean.", + "name": "g_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of g_psfFlux.", + "name": "g_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of g_psfFlux around g_psfFluxMean.", + "name": "g_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute g_psfFluxChi2.", + "name": "g_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for g filter.", + "name": "g_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of g_fpFluxMean.", + "name": "g_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of g_fpFlux.", + "name": "g_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for r filter.", + "name": "r_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of r_psfFluxMean.", + "name": "r_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of r_psfFlux.", + "name": "r_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of r_psfFlux around r_psfFluxMean.", + "name": "r_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute r_psfFluxChi2.", + "name": "r_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for r filter.", + "name": "r_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of r_fpFluxMean.", + "name": "r_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of r_fpFlux.", + "name": "r_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for i filter.", + "name": "i_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of i_psfFluxMean.", + "name": "i_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of i_psfFlux.", + "name": "i_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of i_psfFlux around i_psfFluxMean.", + "name": "i_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute i_psfFluxChi2.", + "name": "i_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for i filter.", + "name": "i_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of i_fpFluxMean.", + "name": "i_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of i_fpFlux.", + "name": "i_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for z filter.", + "name": "z_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of z_psfFluxMean.", + "name": "z_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of z_psfFlux.", + "name": "z_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of z_psfFlux around z_psfFluxMean.", + "name": "z_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute z_psfFluxChi2.", + "name": "z_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for z filter.", + "name": "z_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of z_fpFluxMean.", + "name": "z_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of z_fpFlux.", + "name": "z_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Weighted mean point-source model magnitude for y filter.", + "name": "y_psfFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of y_psfFluxMean.", + "name": "y_psfFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of y_psfFlux.", + "name": "y_psfFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic for the scatter of y_psfFlux around y_psfFluxMean.", + "name": "y_psfFluxChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to compute y_psfFluxChi2.", + "name": "y_psfFluxNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Weighted mean forced photometry flux for y filter.", + "name": "y_fpFluxMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard error of y_fpFluxMean.", + "name": "y_fpFluxMeanErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Standard deviation of the distribution of y_fpFlux.", + "name": "y_fpFluxSigma", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the closest nearby object.", + "name": "nearbyObj1", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj1.", + "name": "nearbyObj1Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj1.", + "name": "nearbyObj1LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the second-closest nearby object.", + "name": "nearbyObj2", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj2.", + "name": "nearbyObj2Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj2.", + "name": "nearbyObj2LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Id of the third-closest nearby object.", + "name": "nearbyObj3", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Distance to nearbyObj3.", + "name": "nearbyObj3Dist", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj3.", + "name": "nearbyObj3LnP", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the u band flux errors.", + "name": "u_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the g band flux errors.", + "name": "g_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the r band flux errors.", + "name": "r_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the i band flux errors.", + "name": "i_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the z band flux errors.", + "name": "z_psfFluxErrMean", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Mean of the y band flux errors.", + "name": "y_psfFluxErrMean", + "type": [ + "null", + "float" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaSource.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaSource.avsc new file mode 100644 index 0000000..36cb640 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.diaSource.avsc @@ -0,0 +1,1199 @@ +{ + "type": "record", + "namespace": "lsst.v7_1", + "name": "diaSource", + "fields": [ + { + "doc": "Unique identifier of this DiaSource.", + "name": "diaSourceId", + "type": "long" + }, + { + "doc": "Id of the visit where this diaSource was measured.", + "name": "visit", + "type": "long" + }, + { + "doc": "Id of the detector where this diaSource was measured. Datatype short instead of byte because of DB concerns about unsigned bytes.", + "name": "detector", + "type": "int" + }, + { + "default": null, + "doc": "Id of the diaObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", + "name": "diaObjectId", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Id of the ssObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", + "name": "ssObjectId", + "type": [ + "null", + "long" + ] + }, + { + "default": null, + "doc": "Id of the parent diaSource this diaSource has been deblended from, if any.", + "name": "parentDiaSourceId", + "type": [ + "null", + "long" + ] + }, + { + "doc": "Effective mid-visit time for this diaSource, expressed as Modified Julian Date, International Atomic Time.", + "name": "midpointMjdTai", + "type": "double" + }, + { + "doc": "Right ascension coordinate of the center of this diaSource.", + "name": "ra", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of ra.", + "name": "raErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "Declination coordinate of the center of this diaSource.", + "name": "dec", + "type": "double" + }, + { + "default": null, + "doc": "Uncertainty of dec.", + "name": "decErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between ra and dec.", + "name": "ra_dec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "doc": "x position computed by a centroiding algorithm.", + "name": "x", + "type": "float" + }, + { + "default": null, + "doc": "Uncertainty of x.", + "name": "xErr", + "type": [ + "null", + "float" + ] + }, + { + "doc": "y position computed by a centroiding algorithm.", + "name": "y", + "type": "float" + }, + { + "default": null, + "doc": "Uncertainty of y.", + "name": "yErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between x and y.", + "name": "x_y_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General centroid algorithm failure flag; set if anything went wrong when fitting the centroid. Another centroid flag field should also be set to provide more information.", + "name": "centroid_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Flux in a 12 pixel radius aperture on the difference image.", + "name": "apFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of apFlux.", + "name": "apFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General aperture flux algorithm failure flag; set if anything went wrong when measuring aperture fluxes. Another apFlux flag field should also be set to provide more information.", + "name": "apFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Aperture did not fit within measurement image.", + "name": "apFlux_flag_apertureTruncated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "The signal-to-noise ratio at which this source was detected in the difference image.", + "name": "snr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Flux for Point Source model. Note this actually measures the flux difference between the template and the visit image.", + "name": "psfFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfFlux.", + "name": "psfFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for point source model.", + "name": "psfRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfRa.", + "name": "psfRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for point source model.", + "name": "psfDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of psfDec.", + "name": "psfDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfFlux and psfRa.", + "name": "psfFlux_psfRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfFlux and psfDec.", + "name": "psfFlux_psfDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance between psfRa and psfDec.", + "name": "psfRa_psfDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the point source model.", + "name": "psfLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the point source model fit.", + "name": "psfChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the point source model.", + "name": "psfNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Failure to derive linear least-squares fit of psf model. Another psfFlux flag field should also be set to provide more information.", + "name": "psfFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Object was too close to the edge of the image to use the full PSF model.", + "name": "psfFlux_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Not enough non-rejected pixels in data to attempt the fit.", + "name": "psfFlux_flag_noGoodPixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Flux for a trailed source model. Note this actually measures the flux difference between the template and the visit image.", + "name": "trailFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailFlux.", + "name": "trailFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for trailed source model.", + "name": "trailRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailRa.", + "name": "trailRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for trailed source model.", + "name": "trailDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailDec.", + "name": "trailDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of trail length.", + "name": "trailLength", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailLength.", + "name": "trailLengthErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the trail direction (bearing).", + "name": "trailAngle", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of trailAngle.", + "name": "trailAngleErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailRa.", + "name": "trailFlux_trailRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailDec.", + "name": "trailFlux_trailDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailLength", + "name": "trailFlux_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailFlux and trailAngle", + "name": "trailFlux_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailDec.", + "name": "trailRa_trailDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailLength.", + "name": "trailRa_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailRa and trailAngle.", + "name": "trailRa_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailDec and trailLength.", + "name": "trailDec_trailLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailDec and trailAngle.", + "name": "trailDec_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of trailLength and trailAngle", + "name": "trailLength_trailAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the trailed source model.", + "name": "trailLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the trailed source model fit.", + "name": "trailChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the trailed source model.", + "name": "trailNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "This flag is set if a trailed source extends onto or past edge pixels.", + "name": "trail_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the mean absolute flux of the two lobes for a dipole model.", + "name": "dipoleMeanFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleMeanFlux.", + "name": "dipoleMeanFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the difference of absolute fluxes of the two lobes for a dipole model.", + "name": "dipoleFluxDiff", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleFluxDiff.", + "name": "dipoleFluxDiffErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Right ascension coordinate of centroid for dipole model.", + "name": "dipoleRa", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleRa.", + "name": "dipoleRaErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Declination coordinate of centroid for dipole model.", + "name": "dipoleDec", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleDec.", + "name": "dipoleDecErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood value for the lobe separation in dipole model.", + "name": "dipoleLength", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleLength.", + "name": "dipoleLengthErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the dipole direction (bearing, from negative to positive lobe).", + "name": "dipoleAngle", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of dipoleAngle.", + "name": "dipoleAngleErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleFluxDiff.", + "name": "dipoleMeanFlux_dipoleFluxDiff_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleRa.", + "name": "dipoleMeanFlux_dipoleRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleDec.", + "name": "dipoleMeanFlux_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleLength.", + "name": "dipoleMeanFlux_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleMeanFlux and dipoleAngle.", + "name": "dipoleMeanFlux_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleRa.", + "name": "dipoleFluxDiff_dipoleRa_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleDec.", + "name": "dipoleFluxDiff_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleLength.", + "name": "dipoleFluxDiff_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleFluxDiff and dipoleAngle.", + "name": "dipoleFluxDiff_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleDec.", + "name": "dipoleRa_dipoleDec_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleLength.", + "name": "dipoleRa_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleRa and dipoleAngle.", + "name": "dipoleRa_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleDec and dipoleLength.", + "name": "dipoleDec_dipoleLength_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleDec and dipoleAngle.", + "name": "dipoleDec_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of dipoleLength and dipoleAngle.", + "name": "dipoleLength_dipoleAngle_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Natural log likelihood of the observed data given the dipole source model.", + "name": "dipoleLnL", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the model fit.", + "name": "dipoleChi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points (pixels) used to fit the model.", + "name": "dipoleNdata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Forced PSF photometry on science image failed. Another forced_PsfFlux flag field should also be set to provide more information.", + "name": "forced_PsfFlux_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Forced PSF flux on science image was too close to the edge of the image to use the full PSF model.", + "name": "forced_PsfFlux_flag_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Forced PSF flux not enough non-rejected pixels in data to attempt the fit.", + "name": "forced_PsfFlux_flag_noGoodPixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Calibrated flux for Point Source model centered on radec but measured on the difference of snaps comprising this visit.", + "name": "snapDiffFlux", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of snapDiffFlux.", + "name": "snapDiffFluxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated sky background at the position (centroid) of the object.", + "name": "fpBkgd", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Estimated uncertainty of fpBkgd.", + "name": "fpBkgdErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "ixx", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of ixx.", + "name": "ixxErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "iyy", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of iyy.", + "name": "iyyErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment of the source intensity.", + "name": "ixy", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of ixy.", + "name": "ixyErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of ixx and iyy.", + "name": "ixx_iyy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of ixx and ixy.", + "name": "ixx_ixy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Covariance of iyy and ixy.", + "name": "iyy_ixy_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "ixxPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "iyyPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Adaptive second moment for the PSF.", + "name": "ixyPSF", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "General source shape algorithm failure flag; set if anything went wrong when measuring the shape. Another shape flag field should also be set to provide more information.", + "name": "shape_flag", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "No pixels to measure shape.", + "name": "shape_flag_no_pixels", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Center not contained in footprint bounding box.", + "name": "shape_flag_not_contained", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "This source is a parent source; we should only be measuring on deblended children in difference imaging.", + "name": "shape_flag_parent_source", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "A measure of extendedness, Computed using a combination of available moments and model fluxes or from a likelihood ratio of point/trailed source models (exact algorithm TBD). extendedness = 1 implies a high degree of confidence that the source is extended. extendedness = 0 implies a high degree of confidence that the source is point-like.", + "name": "extendedness", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "A measure of reliability, computed using information from the source and image characterization, as well as the information on the Telescope and Camera system (e.g., ghost maps, defect maps, etc.).", + "name": "reliability", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Filter band this source was observed with.", + "name": "band", + "type": [ + "null", + "string" + ] + }, + { + "default": null, + "doc": "General pixel flags failure; set if anything went wrong when setting pixels flags from this footprint's mask. This implies that some pixelFlags for this source may be incorrectly set to False.", + "name": "pixelFlags", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Bad pixel in the DiaSource footprint.", + "name": "pixelFlags_bad", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Cosmic ray in the DiaSource footprint.", + "name": "pixelFlags_cr", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Cosmic ray in the 3x3 region around the centroid.", + "name": "pixelFlags_crCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Some of the source footprint is outside usable exposure region (masked EDGE or NO_DATA, or centroid off image).", + "name": "pixelFlags_edge", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Interpolated pixel in the DiaSource footprint.", + "name": "pixelFlags_interpolated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Interpolated pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_interpolatedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "DiaSource center is off image.", + "name": "pixelFlags_offimage", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Saturated pixel in the DiaSource footprint.", + "name": "pixelFlags_saturated", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Saturated pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_saturatedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "DiaSource's footprint includes suspect pixels.", + "name": "pixelFlags_suspect", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Suspect pixel in the 3x3 region around the centroid.", + "name": "pixelFlags_suspectCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Streak in the DiaSource footprint.", + "name": "pixelFlags_streak", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Streak in the 3x3 region around the centroid.", + "name": "pixelFlags_streakCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Injection in the DiaSource footprint.", + "name": "pixelFlags_injected", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Injection in the 3x3 region around the centroid.", + "name": "pixelFlags_injectedCenter", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Template injection in the DiaSource footprint.", + "name": "pixelFlags_injected_template", + "type": [ + "null", + "boolean" + ] + }, + { + "default": null, + "doc": "Template injection in the 3x3 region around the centroid.", + "name": "pixelFlags_injected_templateCenter", + "type": [ + "null", + "boolean" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/1/lsst.v7_1.ssObject.avsc b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.ssObject.avsc new file mode 100644 index 0000000..5865bed --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/lsst.v7_1.ssObject.avsc @@ -0,0 +1,471 @@ +{ + "type": "record", + "namespace": "lsst.v7_1", + "name": "ssObject", + "fields": [ + { + "doc": "Unique identifier.", + "name": "ssObjectId", + "type": "long" + }, + { + "default": null, + "doc": "The date the LSST first linked and submitted the discovery observations to the MPC. May be NULL if not an LSST discovery. The date format will follow general LSST conventions (MJD TAI, at the moment).", + "name": "discoverySubmissionDate", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "The time of the first LSST observation of this object (could be precovered) as Modified Julian Date, International Atomic Time.", + "name": "firstObservationDate", + "type": [ + "null", + "double" + ] + }, + { + "default": null, + "doc": "Arc of LSST observations.", + "name": "arc", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Number of LSST observations of this object.", + "name": "numObs", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Minimum orbit intersection distance to Earth.", + "name": "MOID", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "True anomaly of the MOID point.", + "name": "MOIDTrueAnomaly", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Ecliptic longitude of the MOID point.", + "name": "MOIDEclipticLongitude", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "DeltaV at the MOID point.", + "name": "MOIDDeltaV", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (u band).", + "name": "u_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (u band).", + "name": "u_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (u band).", + "name": "u_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (u band).", + "name": "u_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (u band).", + "name": "u_H_u_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (u band).", + "name": "u_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (u band).", + "name": "u_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (g band).", + "name": "g_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (g band).", + "name": "g_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (g band).", + "name": "g_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (g band).", + "name": "g_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (g band).", + "name": "g_H_g_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (g band).", + "name": "g_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (g band).", + "name": "g_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (r band).", + "name": "r_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (r band).", + "name": "r_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (r band).", + "name": "r_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (r band).", + "name": "r_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (r band).", + "name": "r_H_r_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (r band).", + "name": "r_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (r band).", + "name": "r_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (i band).", + "name": "i_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (i band).", + "name": "i_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (i band).", + "name": "i_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (i band).", + "name": "i_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (i band).", + "name": "i_H_i_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (i band).", + "name": "i_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (i band).", + "name": "i_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (z band).", + "name": "z_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (z band).", + "name": "z_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (z band).", + "name": "z_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (z band).", + "name": "z_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (z band).", + "name": "z_H_z_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (z band).", + "name": "z_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (z band).", + "name": "z_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "Best fit absolute magnitude (y band).", + "name": "y_H", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Best fit G12 slope parameter (y band).", + "name": "y_G12", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of H (y band).", + "name": "y_HErr", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Uncertainty of G12 (y band).", + "name": "y_G12Err", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "H-G12 covariance (y band).", + "name": "y_H_y_G12_Cov", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "Chi^2 statistic of the phase curve fit (y band).", + "name": "y_Chi2", + "type": [ + "null", + "float" + ] + }, + { + "default": null, + "doc": "The number of data points used to fit the phase curve (y band).", + "name": "y_Ndata", + "type": [ + "null", + "int" + ] + }, + { + "default": null, + "doc": "median `extendedness` value from the DIASource.", + "name": "medianExtendedness", + "type": [ + "null", + "float" + ] + } + ] +} \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/1/sample_data/alert.json b/pittgoogle/schemas/lsst/7/1/sample_data/alert.json new file mode 100644 index 0000000..1a1f952 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/sample_data/alert.json @@ -0,0 +1,109 @@ +{ + "alertId": 1231321321, + "l1dbId": 222222222, + "diaSource": + { + "diaSourceId": 281323062375219200, + "time_processed": 0, + "visit": 1, + "detector": 42, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + }, + "prvDiaSources": [ + { + "diaSourceId": 281323062375219198, + "time_processed": 0, + "visit": 2, + "detector": 43, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + }, + { + "diaSourceId": 281323062375219199, + "time_processed": 0, + "visit": 1, + "detector": 42, + "midpointMjdTai": 1480360995, + "band": "r", + "programId": 1, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "x": 112.1, + "y": 121.1, + "xErr": 1.2, + "yErr": 1.1, + "x_y_Cov": 1.2, + "apFlux": 1241.0, + "apFluxErr": 12.0, + "snr": 41.1, + "trailNdata": 5, + "psfFlux": 1241.0, + "psfFluxErr": 12.0 + } + ], + "diaObject": { + "diaObjectId": 281323062375219201, + "validityStart": 0, + "ra": 351.570546978, + "dec": 0.126243049656, + "raErr": 0.00028, + "decErr": 0.00028, + "ra_dec_Cov": 0.00029, + "radecMjdTai": 1480360995, + "pmRa": 0.00013, + "pmDec": 0.00014, + "parallax": 2.124124, + "pmRaErr": 0.00013, + "pmDecErr": 0.00013, + "pmParallaxNdata": 0, + "parallaxErr": 0.00013, + "pmRa_pmDec_Cov": 0.00013, + "pmRa_parallax_Cov": 0.00013, + "pmDec_parallax_Cov": 0.00013, + "pmParallaxLnL": 0.00013, + "pmParallaxChi2": 0.00013, + "pmParallaxNdata": 1214 + }, + "ssObject": { + "ssObjectId":5364546, + "numObs": 10 + } +} diff --git a/pittgoogle/schemas/lsst/7/1/sample_data/fakeAlert.avro b/pittgoogle/schemas/lsst/7/1/sample_data/fakeAlert.avro new file mode 100644 index 0000000..3f4c343 --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/sample_data/fakeAlert.avro @@ -0,0 +1,4 @@ +Objavro.codecnullavro.schema{"type": "record", "doc": "Rubin Avro alert schema v7.1", "name": "lsst.v7_1.alert", "fields": [{"doc": "unique alert identifer", "name": "alertId", "type": "long"}, {"name": "diaSource", "type": {"type": "record", "name": "lsst.v7_1.diaSource", "fields": [{"doc": "Unique identifier of this DiaSource.", "name": "diaSourceId", "type": "long"}, {"doc": "Id of the visit where this diaSource was measured.", "name": "visit", "type": "long"}, {"doc": "Id of the detector where this diaSource was measured. Datatype short instead of byte because of DB concerns about unsigned bytes.", "name": "detector", "type": "int"}, {"default": null, "doc": "Id of the diaObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", "name": "diaObjectId", "type": ["null", "long"]}, {"default": null, "doc": "Id of the ssObject this source was associated with, if any. If not, it is set to NULL (each diaSource will be associated with either a diaObject or ssObject).", "name": "ssObjectId", "type": ["null", "long"]}, {"default": null, "doc": "Id of the parent diaSource this diaSource has been deblended from, if any.", "name": "parentDiaSourceId", "type": ["null", "long"]}, {"doc": "Effective mid-visit time for this diaSource, expressed as Modified Julian Date, International Atomic Time.", "name": "midpointMjdTai", "type": "double"}, {"doc": "Right ascension coordinate of the center of this diaSource.", "name": "ra", "type": "double"}, {"default": null, "doc": "Uncertainty of ra.", "name": "raErr", "type": ["null", "float"]}, {"doc": "Declination coordinate of the center of this diaSource.", "name": "dec", "type": "double"}, {"default": null, "doc": "Uncertainty of dec.", "name": "decErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between ra and dec.", "name": "ra_dec_Cov", "type": ["null", "float"]}, {"doc": "x position computed by a centroiding algorithm.", "name": "x", "type": "float"}, {"default": null, "doc": "Uncertainty of x.", "name": "xErr", "type": ["null", "float"]}, {"doc": "y position computed by a centroiding algorithm.", "name": "y", "type": "float"}, {"default": null, "doc": "Uncertainty of y.", "name": "yErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between x and y.", "name": "x_y_Cov", "type": ["null", "float"]}, {"default": null, "doc": "General centroid algorithm failure flag; set if anything went wrong when fitting the centroid. Another centroid flag field should also be set to provide more information.", "name": "centroid_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Flux in a 12 pixel radius aperture on the difference image.", "name": "apFlux", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of apFlux.", "name": "apFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "General aperture flux algorithm failure flag; set if anything went wrong when measuring aperture fluxes. Another apFlux flag field should also be set to provide more information.", "name": "apFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Aperture did not fit within measurement image.", "name": "apFlux_flag_apertureTruncated", "type": ["null", "boolean"]}, {"default": null, "doc": "The signal-to-noise ratio at which this source was detected in the difference image.", "name": "snr", "type": ["null", "float"]}, {"default": null, "doc": "Flux for Point Source model. Note this actually measures the flux difference between the template and the visit image.", "name": "psfFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of psfFlux.", "name": "psfFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for point source model.", "name": "psfRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of psfRa.", "name": "psfRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for point source model.", "name": "psfDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of psfDec.", "name": "psfDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfFlux and psfRa.", "name": "psfFlux_psfRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfFlux and psfDec.", "name": "psfFlux_psfDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between psfRa and psfDec.", "name": "psfRa_psfDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the point source model.", "name": "psfLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the point source model fit.", "name": "psfChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the point source model.", "name": "psfNdata", "type": ["null", "int"]}, {"default": null, "doc": "Failure to derive linear least-squares fit of psf model. Another psfFlux flag field should also be set to provide more information.", "name": "psfFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Object was too close to the edge of the image to use the full PSF model.", "name": "psfFlux_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Not enough non-rejected pixels in data to attempt the fit.", "name": "psfFlux_flag_noGoodPixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Flux for a trailed source model. Note this actually measures the flux difference between the template and the visit image.", "name": "trailFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailFlux.", "name": "trailFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for trailed source model.", "name": "trailRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of trailRa.", "name": "trailRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for trailed source model.", "name": "trailDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of trailDec.", "name": "trailDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of trail length.", "name": "trailLength", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailLength.", "name": "trailLengthErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the trail direction (bearing).", "name": "trailAngle", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of trailAngle.", "name": "trailAngleErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailRa.", "name": "trailFlux_trailRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailDec.", "name": "trailFlux_trailDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailLength", "name": "trailFlux_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailFlux and trailAngle", "name": "trailFlux_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailDec.", "name": "trailRa_trailDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailLength.", "name": "trailRa_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailRa and trailAngle.", "name": "trailRa_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailDec and trailLength.", "name": "trailDec_trailLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailDec and trailAngle.", "name": "trailDec_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of trailLength and trailAngle", "name": "trailLength_trailAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the trailed source model.", "name": "trailLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the trailed source model fit.", "name": "trailChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the trailed source model.", "name": "trailNdata", "type": ["null", "int"]}, {"default": null, "doc": "This flag is set if a trailed source extends onto or past edge pixels.", "name": "trail_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Maximum likelihood value for the mean absolute flux of the two lobes for a dipole model.", "name": "dipoleMeanFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleMeanFlux.", "name": "dipoleMeanFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood value for the difference of absolute fluxes of the two lobes for a dipole model.", "name": "dipoleFluxDiff", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleFluxDiff.", "name": "dipoleFluxDiffErr", "type": ["null", "float"]}, {"default": null, "doc": "Right ascension coordinate of centroid for dipole model.", "name": "dipoleRa", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of dipoleRa.", "name": "dipoleRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Declination coordinate of centroid for dipole model.", "name": "dipoleDec", "type": ["null", "double"]}, {"default": null, "doc": "Uncertainty of dipoleDec.", "name": "dipoleDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood value for the lobe separation in dipole model.", "name": "dipoleLength", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleLength.", "name": "dipoleLengthErr", "type": ["null", "float"]}, {"default": null, "doc": "Maximum likelihood fit of the angle between the meridian through the centroid and the dipole direction (bearing, from negative to positive lobe).", "name": "dipoleAngle", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of dipoleAngle.", "name": "dipoleAngleErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleFluxDiff.", "name": "dipoleMeanFlux_dipoleFluxDiff_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleRa.", "name": "dipoleMeanFlux_dipoleRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleDec.", "name": "dipoleMeanFlux_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleLength.", "name": "dipoleMeanFlux_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleMeanFlux and dipoleAngle.", "name": "dipoleMeanFlux_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleRa.", "name": "dipoleFluxDiff_dipoleRa_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleDec.", "name": "dipoleFluxDiff_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleLength.", "name": "dipoleFluxDiff_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleFluxDiff and dipoleAngle.", "name": "dipoleFluxDiff_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleDec.", "name": "dipoleRa_dipoleDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleLength.", "name": "dipoleRa_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleRa and dipoleAngle.", "name": "dipoleRa_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleDec and dipoleLength.", "name": "dipoleDec_dipoleLength_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleDec and dipoleAngle.", "name": "dipoleDec_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of dipoleLength and dipoleAngle.", "name": "dipoleLength_dipoleAngle_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log likelihood of the observed data given the dipole source model.", "name": "dipoleLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the model fit.", "name": "dipoleChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points (pixels) used to fit the model.", "name": "dipoleNdata", "type": ["null", "int"]}, {"default": null, "doc": "Forced PSF photometry on science image failed. Another forced_PsfFlux flag field should also be set to provide more information.", "name": "forced_PsfFlux_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "Forced PSF flux on science image was too close to the edge of the image to use the full PSF model.", "name": "forced_PsfFlux_flag_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Forced PSF flux not enough non-rejected pixels in data to attempt the fit.", "name": "forced_PsfFlux_flag_noGoodPixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Calibrated flux for Point Source model centered on radec but measured on the difference of snaps comprising this visit.", "name": "snapDiffFlux", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of snapDiffFlux.", "name": "snapDiffFluxErr", "type": ["null", "float"]}, {"default": null, "doc": "Estimated sky background at the position (centroid) of the object.", "name": "fpBkgd", "type": ["null", "float"]}, {"default": null, "doc": "Estimated uncertainty of fpBkgd.", "name": "fpBkgdErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "ixx", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of ixx.", "name": "ixxErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "iyy", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of iyy.", "name": "iyyErr", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment of the source intensity.", "name": "ixy", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of ixy.", "name": "ixyErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of ixx and iyy.", "name": "ixx_iyy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of ixx and ixy.", "name": "ixx_ixy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of iyy and ixy.", "name": "iyy_ixy_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "ixxPSF", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "iyyPSF", "type": ["null", "float"]}, {"default": null, "doc": "Adaptive second moment for the PSF.", "name": "ixyPSF", "type": ["null", "float"]}, {"default": null, "doc": "General source shape algorithm failure flag; set if anything went wrong when measuring the shape. Another shape flag field should also be set to provide more information.", "name": "shape_flag", "type": ["null", "boolean"]}, {"default": null, "doc": "No pixels to measure shape.", "name": "shape_flag_no_pixels", "type": ["null", "boolean"]}, {"default": null, "doc": "Center not contained in footprint bounding box.", "name": "shape_flag_not_contained", "type": ["null", "boolean"]}, {"default": null, "doc": "This source is a parent source; we should only be measuring on deblended children in difference imaging.", "name": "shape_flag_parent_source", "type": ["null", "boolean"]}, {"default": null, "doc": "A measure of extendedness, Computed using a combination of available moments and model fluxes or from a likelihood ratio of point/trailed source models (exact algorithm TBD). extendedness = 1 implies a high degree of confidence that the source is extended. extendedness = 0 implies a high degree of confidence that the source is point-like.", "name": "extendedness", "type": ["null", "float"]}, {"default": null, "doc": "A measure of reliability, computed using information from the source and image characterization, as well as the information on the Telescope and Camera system (e.g., ghost maps, defect maps, etc.).", "name": "reliability", "type": ["null", "float"]}, {"default": null, "doc": "Filter band this source was observed with.", "name": "band", "type": ["null", "string"]}, {"default": null, "doc": "General pixel flags failure; set if anything went wrong when setting pixels flags from this footprint's mask. This implies that some pixelFlags for this source may be incorrectly set to False.", "name": "pixelFlags", "type": ["null", "boolean"]}, {"default": null, "doc": "Bad pixel in the DiaSource footprint.", "name": "pixelFlags_bad", "type": ["null", "boolean"]}, {"default": null, "doc": "Cosmic ray in the DiaSource footprint.", "name": "pixelFlags_cr", "type": ["null", "boolean"]}, {"default": null, "doc": "Cosmic ray in the 3x3 region around the centroid.", "name": "pixelFlags_crCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Some of the source footprint is outside usable exposure region (masked EDGE or NO_DATA, or centroid off image).", "name": "pixelFlags_edge", "type": ["null", "boolean"]}, {"default": null, "doc": "Interpolated pixel in the DiaSource footprint.", "name": "pixelFlags_interpolated", "type": ["null", "boolean"]}, {"default": null, "doc": "Interpolated pixel in the 3x3 region around the centroid.", "name": "pixelFlags_interpolatedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "DiaSource center is off image.", "name": "pixelFlags_offimage", "type": ["null", "boolean"]}, {"default": null, "doc": "Saturated pixel in the DiaSource footprint.", "name": "pixelFlags_saturated", "type": ["null", "boolean"]}, {"default": null, "doc": "Saturated pixel in the 3x3 region around the centroid.", "name": "pixelFlags_saturatedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "DiaSource's footprint includes suspect pixels.", "name": "pixelFlags_suspect", "type": ["null", "boolean"]}, {"default": null, "doc": "Suspect pixel in the 3x3 region around the centroid.", "name": "pixelFlags_suspectCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Streak in the DiaSource footprint.", "name": "pixelFlags_streak", "type": ["null", "boolean"]}, {"default": null, "doc": "Streak in the 3x3 region around the centroid.", "name": "pixelFlags_streakCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Injection in the DiaSource footprint.", "name": "pixelFlags_injected", "type": ["null", "boolean"]}, {"default": null, "doc": "Injection in the 3x3 region around the centroid.", "name": "pixelFlags_injectedCenter", "type": ["null", "boolean"]}, {"default": null, "doc": "Template injection in the DiaSource footprint.", "name": "pixelFlags_injected_template", "type": ["null", "boolean"]}, {"default": null, "doc": "Template injection in the 3x3 region around the centroid.", "name": "pixelFlags_injected_templateCenter", "type": ["null", "boolean"]}]}}, {"default": null, "name": "prvDiaSources", "type": ["null", {"type": "array", "items": "lsst.v7_1.diaSource"}]}, {"default": null, "name": "prvDiaForcedSources", "type": ["null", {"type": "array", "items": {"type": "record", "name": "lsst.v7_1.diaForcedSource", "fields": [{"doc": "Unique id.", "name": "diaForcedSourceId", "type": "long"}, {"doc": "Id of the DiaObject that this DiaForcedSource was associated with.", "name": "diaObjectId", "type": "long"}, {"doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", "name": "ra", "type": "double"}, {"doc": "Declination coordinate of the position of the object at time radecMjdTai.", "name": "dec", "type": "double"}, {"doc": "Id of the visit where this forcedSource was measured.", "name": "visit", "type": "long"}, {"doc": "Id of the detector where this forcedSource was measured. Datatype short instead of byte because of DB concerns about unsigned bytes.", "name": "detector", "type": "int"}, {"default": null, "doc": "Point Source model flux.", "name": "psfFlux", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of psfFlux.", "name": "psfFluxErr", "type": ["null", "float"]}, {"doc": "Effective mid-visit time for this diaForcedSource, expressed as Modified Julian Date, International Atomic Time.", "name": "midpointMjdTai", "type": "double"}, {"default": null, "doc": "Filter band this source was observed with.", "name": "band", "type": ["null", "string"]}]}}]}, {"default": null, "name": "prvDiaNondetectionLimits", "type": ["null", {"type": "array", "items": {"type": "record", "name": "lsst.v7_1.diaNondetectionLimit", "fields": [{"name": "ccdVisitId", "type": "long"}, {"name": "midpointMjdTai", "type": "double"}, {"name": "band", "type": "string"}, {"name": "diaNoise", "type": "float"}]}}]}, {"default": null, "name": "diaObject", "type": ["null", {"type": "record", "name": "lsst.v7_1.diaObject", "fields": [{"doc": "Unique identifier of this DiaObject.", "name": "diaObjectId", "type": "long"}, {"doc": "Right ascension coordinate of the position of the object at time radecMjdTai.", "name": "ra", "type": "double"}, {"default": null, "doc": "Uncertainty of ra.", "name": "raErr", "type": ["null", "float"]}, {"doc": "Declination coordinate of the position of the object at time radecMjdTai.", "name": "dec", "type": "double"}, {"default": null, "doc": "Uncertainty of dec.", "name": "decErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance between ra and dec.", "name": "ra_dec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Time at which the object was at a position ra/dec, expressed as Modified Julian Date, International Atomic Time.", "name": "radecMjdTai", "type": ["null", "double"]}, {"default": null, "doc": "Proper motion in right ascension.", "name": "pmRa", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of pmRa.", "name": "pmRaErr", "type": ["null", "float"]}, {"default": null, "doc": "Proper motion of declination.", "name": "pmDec", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of pmDec.", "name": "pmDecErr", "type": ["null", "float"]}, {"default": null, "doc": "Parallax.", "name": "parallax", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of parallax.", "name": "parallaxErr", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmRa and pmDec.", "name": "pmRa_pmDec_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmRa and parallax.", "name": "pmRa_parallax_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Covariance of pmDec and parallax.", "name": "pmDec_parallax_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the likelihood of the linear proper motion parallax fit.", "name": "pmParallaxLnL", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 static of the model fit.", "name": "pmParallaxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the model.", "name": "pmParallaxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for u filter.", "name": "u_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of u_psfFluxMean.", "name": "u_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of u_psfFlux.", "name": "u_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of u_psfFlux around u_psfFluxMean.", "name": "u_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute u_psfFluxChi2.", "name": "u_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for u filter.", "name": "u_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of u_fpFluxMean.", "name": "u_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of u_fpFlux.", "name": "u_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for g filter.", "name": "g_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of g_psfFluxMean.", "name": "g_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of g_psfFlux.", "name": "g_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of g_psfFlux around g_psfFluxMean.", "name": "g_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute g_psfFluxChi2.", "name": "g_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for g filter.", "name": "g_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of g_fpFluxMean.", "name": "g_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of g_fpFlux.", "name": "g_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for r filter.", "name": "r_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of r_psfFluxMean.", "name": "r_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of r_psfFlux.", "name": "r_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of r_psfFlux around r_psfFluxMean.", "name": "r_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute r_psfFluxChi2.", "name": "r_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for r filter.", "name": "r_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of r_fpFluxMean.", "name": "r_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of r_fpFlux.", "name": "r_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for i filter.", "name": "i_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of i_psfFluxMean.", "name": "i_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of i_psfFlux.", "name": "i_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of i_psfFlux around i_psfFluxMean.", "name": "i_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute i_psfFluxChi2.", "name": "i_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for i filter.", "name": "i_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of i_fpFluxMean.", "name": "i_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of i_fpFlux.", "name": "i_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for z filter.", "name": "z_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of z_psfFluxMean.", "name": "z_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of z_psfFlux.", "name": "z_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of z_psfFlux around z_psfFluxMean.", "name": "z_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute z_psfFluxChi2.", "name": "z_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for z filter.", "name": "z_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of z_fpFluxMean.", "name": "z_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of z_fpFlux.", "name": "z_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Weighted mean point-source model magnitude for y filter.", "name": "y_psfFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of y_psfFluxMean.", "name": "y_psfFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of y_psfFlux.", "name": "y_psfFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic for the scatter of y_psfFlux around y_psfFluxMean.", "name": "y_psfFluxChi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to compute y_psfFluxChi2.", "name": "y_psfFluxNdata", "type": ["null", "int"]}, {"default": null, "doc": "Weighted mean forced photometry flux for y filter.", "name": "y_fpFluxMean", "type": ["null", "float"]}, {"default": null, "doc": "Standard error of y_fpFluxMean.", "name": "y_fpFluxMeanErr", "type": ["null", "float"]}, {"default": null, "doc": "Standard deviation of the distribution of y_fpFlux.", "name": "y_fpFluxSigma", "type": ["null", "float"]}, {"default": null, "doc": "Id of the closest nearby object.", "name": "nearbyObj1", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj1.", "name": "nearbyObj1Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj1.", "name": "nearbyObj1LnP", "type": ["null", "float"]}, {"default": null, "doc": "Id of the second-closest nearby object.", "name": "nearbyObj2", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj2.", "name": "nearbyObj2Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj2.", "name": "nearbyObj2LnP", "type": ["null", "float"]}, {"default": null, "doc": "Id of the third-closest nearby object.", "name": "nearbyObj3", "type": ["null", "long"]}, {"default": null, "doc": "Distance to nearbyObj3.", "name": "nearbyObj3Dist", "type": ["null", "float"]}, {"default": null, "doc": "Natural log of the probability that the observed diaObject is the same as the nearbyObj3.", "name": "nearbyObj3LnP", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the u band flux errors.", "name": "u_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the g band flux errors.", "name": "g_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the r band flux errors.", "name": "r_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the i band flux errors.", "name": "i_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the z band flux errors.", "name": "z_psfFluxErrMean", "type": ["null", "float"]}, {"default": null, "doc": "Mean of the y band flux errors.", "name": "y_psfFluxErrMean", "type": ["null", "float"]}]}]}, {"default": null, "name": "ssObject", "type": ["null", {"type": "record", "name": "lsst.v7_1.ssObject", "fields": [{"doc": "Unique identifier.", "name": "ssObjectId", "type": "long"}, {"default": null, "doc": "The date the LSST first linked and submitted the discovery observations to the MPC. May be NULL if not an LSST discovery. The date format will follow general LSST conventions (MJD TAI, at the moment).", "name": "discoverySubmissionDate", "type": ["null", "double"]}, {"default": null, "doc": "The time of the first LSST observation of this object (could be precovered) as Modified Julian Date, International Atomic Time.", "name": "firstObservationDate", "type": ["null", "double"]}, {"default": null, "doc": "Arc of LSST observations.", "name": "arc", "type": ["null", "float"]}, {"default": null, "doc": "Number of LSST observations of this object.", "name": "numObs", "type": ["null", "int"]}, {"default": null, "doc": "Minimum orbit intersection distance to Earth.", "name": "MOID", "type": ["null", "float"]}, {"default": null, "doc": "True anomaly of the MOID point.", "name": "MOIDTrueAnomaly", "type": ["null", "float"]}, {"default": null, "doc": "Ecliptic longitude of the MOID point.", "name": "MOIDEclipticLongitude", "type": ["null", "float"]}, {"default": null, "doc": "DeltaV at the MOID point.", "name": "MOIDDeltaV", "type": ["null", "float"]}, {"default": null, "doc": "Best fit absolute magnitude (u band).", "name": "u_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (u band).", "name": "u_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (u band).", "name": "u_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (u band).", "name": "u_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (u band).", "name": "u_H_u_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (u band).", "name": "u_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (u band).", "name": "u_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (g band).", "name": "g_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (g band).", "name": "g_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (g band).", "name": "g_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (g band).", "name": "g_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (g band).", "name": "g_H_g_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (g band).", "name": "g_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (g band).", "name": "g_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (r band).", "name": "r_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (r band).", "name": "r_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (r band).", "name": "r_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (r band).", "name": "r_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (r band).", "name": "r_H_r_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (r band).", "name": "r_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (r band).", "name": "r_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (i band).", "name": "i_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (i band).", "name": "i_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (i band).", "name": "i_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (i band).", "name": "i_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (i band).", "name": "i_H_i_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (i band).", "name": "i_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (i band).", "name": "i_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (z band).", "name": "z_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (z band).", "name": "z_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (z band).", "name": "z_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (z band).", "name": "z_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (z band).", "name": "z_H_z_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (z band).", "name": "z_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (z band).", "name": "z_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "Best fit absolute magnitude (y band).", "name": "y_H", "type": ["null", "float"]}, {"default": null, "doc": "Best fit G12 slope parameter (y band).", "name": "y_G12", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of H (y band).", "name": "y_HErr", "type": ["null", "float"]}, {"default": null, "doc": "Uncertainty of G12 (y band).", "name": "y_G12Err", "type": ["null", "float"]}, {"default": null, "doc": "H-G12 covariance (y band).", "name": "y_H_y_G12_Cov", "type": ["null", "float"]}, {"default": null, "doc": "Chi^2 statistic of the phase curve fit (y band).", "name": "y_Chi2", "type": ["null", "float"]}, {"default": null, "doc": "The number of data points used to fit the phase curve (y band).", "name": "y_Ndata", "type": ["null", "int"]}, {"default": null, "doc": "median `extendedness` value from the DIASource.", "name": "medianExtendedness", "type": ["null", "float"]}]}]}, {"default": null, "name": "cutoutDifference", "type": ["null", "bytes"]}, {"default": null, "name": "cutoutScience", "type": ["null", "bytes"]}, {"default": null, "name": "cutoutTemplate", "type": ["null", "bytes"]}]}Ht_K)G 㣖 T!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rV!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rT!Aw5 u@̒9eVt(?̒9$ 933B?33B̌?? D@Aff$B D@A +rw5 u@̒9eVt(?̒9$ 9!AP9P99P9@P9P9P9P9P9P9Ht_K)G \ No newline at end of file diff --git a/pittgoogle/schemas/lsst/7/1/sample_data/generate.py b/pittgoogle/schemas/lsst/7/1/sample_data/generate.py new file mode 100644 index 0000000..b0fb6cb --- /dev/null +++ b/pittgoogle/schemas/lsst/7/1/sample_data/generate.py @@ -0,0 +1,17 @@ +#!/usr/bin/env python +import json +import lsst.alert.packet as packet +from datetime import datetime +import pandas as pd + +with open("alert.json", "r") as f: + data = json.load(f) + + data_time = datetime.now() + + data['diaSource']['time_processed'] = pd.Timestamp(data_time) + data['diaObject']['validityStart'] = pd.Timestamp(data_time) + +schema = packet.SchemaRegistry.from_filesystem(schema_root="lsst.v7_1.alert").get_by_version("7.1") +with open("fakeAlert.avro", "wb") as f: + schema.store_alerts(f, [data]) diff --git a/pittgoogle/types_.py b/pittgoogle/types_.py index 9d666a5..900ac76 100644 --- a/pittgoogle/types_.py +++ b/pittgoogle/types_.py @@ -3,135 +3,14 @@ import datetime import importlib.resources import logging -import re -from pathlib import Path -import fastavro -import yaml -from attrs import converters, define, field, evolve - -from . import exceptions +import attrs LOGGER = logging.getLogger(__name__) PACKAGE_DIR = importlib.resources.files(__package__) -@define(kw_only=True) -class Schema: - """Class for an individual schema. - - Do not call this class's constructor directly. Instead, load a schema using the registry - :class:`pittgoogle.registry.Schemas`. - - ---- - """ - - # String _under_ field definition will cause field to appear as a property in rendered docs. - name: str = field() - """Name of the schema.""" - origin: str = field() - """Pointer to the schema's origin. Typically this is a URL to a repo maintained by the survey.""" - description: str = field() - """A description of the schema.""" - definition: dict | None = field(default=None) - """The schema definition used to serialize and deserialize the alert bytes, if one is required.""" - schemaless_alert_bytes: bool = field(default=False, converter=converters.to_bool) - """Whether the alert bytes are schemaless. If True, a valid `definition` is required to - serialize or deserialize the alert packet bytes.""" - _helper: str = field(default="_local_schema_helper") - """Name of the helper method that should be used to load the schema definition.""" - path: Path | None = field(default=None) - """Path where the helper can find the schema.""" - filter_map: dict = field(factory=dict) - """Mapping of the filter name as stored in the alert (often an int) to the common name (often a string).""" - # The rest don't need string descriptions because they are explicitly defined as properties below. - _survey: str | None = field(default=None) - # Don't accept _map as an init arg; we'll load it from a yaml file later. - _map: dict | None = field(default=None, init=False) - - @classmethod - def _from_yaml(cls, schema_dict: dict, **evolve_schema_dict) -> "Schema": - """Create a `Schema` from an entry in the registry's schemas.yml file.""" - # initialize the class, then let the helper finish up - schema = evolve(cls(**schema_dict), **evolve_schema_dict) - helper = getattr(cls, schema._helper) - schema = helper(schema) - return schema - - @staticmethod - def _local_schema_helper(schema: "Schema") -> "Schema": - """Resolve `schema.path`. If it points to a valid ".avsc" file, load it into `schema.avsc`.""" - # Resolve the path. If it is not None, this helper expects it to be the path to - # a ".avsc" file relative to the pittgoogle package directory. - schema.path = PACKAGE_DIR / schema.path if schema.path is not None else None - - # Load the avro schema, if the file exists. - invalid_path = ( - (schema.path is None) or (schema.path.suffix != ".avsc") or (not schema.path.is_file()) - ) - if invalid_path: - schema.definition = None - else: - schema.definition = fastavro.schema.load_schema(schema.path) - - return schema - - @staticmethod - def _lsst_schema_helper(schema: "Schema") -> "Schema": - """Load the Avro schema definition using the ``lsst.alert.packet`` package. - - Raises: - SchemaNotFoundError: - If an LSST schema called ``schema.name`` cannot be loaded. An error is raised - because the LSST alert bytes are schemaless, so ``schema.definition`` will be - required in order to deserialize the alert. - """ - import lsst.alert.packet.schema - - version_msg = f"For valid versions, see {schema.origin}." - - # Parse major and minor versions out of schema.name. Expecting syntax "lsst.v_.alert". - try: - major, minor = map(int, re.findall(r"\d+", schema.name)) - except ValueError: - msg = ( - f"Unable to identify major and minor version. Please use the syntax " - "'lsst.v_.alert', replacing '' and '' with integers. " - f"{version_msg}" - ) - raise exceptions.SchemaNotFoundError(msg) - - schema_dir = Path(lsst.alert.packet.schema.get_schema_path(major, minor)) - schema.path = schema_dir / f"{schema.name}.avsc" - - try: - schema.definition = lsst.alert.packet.schema.Schema.from_file(schema.path).definition - except fastavro.repository.SchemaRepositoryError: - msg = f"Unable to load the schema. {version_msg}" - raise exceptions.SchemaNotFoundError(msg) - - return schema - - @property - def survey(self) -> str: - """Name of the survey. This is usually the first part of the schema's name.""" - if self._survey is None: - self._survey = self.name.split(".")[0] - return self._survey - - @property - def map(self) -> dict: - """Mapping of Pitt-Google's generic field names to survey-specific field names.""" - if self._map is None: - yml = PACKAGE_DIR / f"schemas/maps/{self.survey}.yml" - try: - self._map = yaml.safe_load(yml.read_text()) - except FileNotFoundError: - raise ValueError(f"no schema map found for schema name '{self.name}'") - return self._map - - -@define(frozen=True) +@attrs.define(frozen=True) class PubsubMessageLike: """Container for an incoming alert. @@ -149,13 +28,13 @@ class PubsubMessageLike: ---- """ - data: bytes = field() + data: bytes = attrs.field() """Alert data as bytes. This is also known as the message "payload".""" - attributes: dict = field(factory=dict) + attributes: dict = attrs.field(factory=dict) """Alert attributes. This is custom metadata attached to the Pub/Sub message.""" - message_id: str | None = field(default=None) + message_id: str | None = attrs.field(default=None) """Pub/Sub ID of the published message.""" - publish_time: datetime.datetime | None = field(default=None) + publish_time: datetime.datetime | None = attrs.field(default=None) """Timestamp of the published message.""" - ordering_key: str | None = field(default=None) + ordering_key: str | None = attrs.field(default=None) """Pub/Sub ordering key of the published message.""" diff --git a/pittgoogle/utils.py b/pittgoogle/utils.py index 760e575..823eae9 100644 --- a/pittgoogle/utils.py +++ b/pittgoogle/utils.py @@ -1,10 +1,10 @@ # -*- coding: UTF-8 -*- """Classes and functions to support working with alerts and related data.""" import base64 +import collections +import io import json import logging -from collections import OrderedDict -from io import BytesIO import astropy.table import astropy.time @@ -76,7 +76,7 @@ def avro_to_dict(bytes_data): The unpacked dictionary from the ``bytes_data``. """ if bytes_data is not None: - with BytesIO(bytes_data) as fin: + with io.BytesIO(bytes_data) as fin: alert_dicts = list(fastavro.reader(fin)) # list with single dict if len(alert_dicts) != 1: LOGGER.warning(f"Expected 1 Avro record. Found {len(alert_dicts)}.") @@ -111,7 +111,7 @@ def alert_dict_to_table(alert_dict: dict) -> astropy.table.Table: """ # collect rows for the table - candidate = OrderedDict(alert_dict["candidate"]) + candidate = collections.OrderedDict(alert_dict["candidate"]) rows = [candidate] for prv_cand in alert_dict["prv_candidates"]: # astropy 3.2.1 cannot handle dicts with different keys (fixed by 4.1) diff --git a/poetry.lock b/poetry.lock index 662c817..a2c212a 100644 --- a/poetry.lock +++ b/poetry.lock @@ -307,23 +307,6 @@ files = [ [package.extras] toml = ["tomli"] -[[package]] -name = "deprecated" -version = "1.2.14" -description = "Python @deprecated decorator to deprecate old python classes, functions or methods." -optional = false -python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" -files = [ - {file = "Deprecated-1.2.14-py2.py3-none-any.whl", hash = "sha256:6fac8b097794a90302bdbb17b9b815e732d3c4720583ff1b198499d78470466c"}, - {file = "Deprecated-1.2.14.tar.gz", hash = "sha256:e5323eb936458dccc2582dc6f9c322c852a775a27065ff2b0c4970b9d53d01b3"}, -] - -[package.dependencies] -wrapt = ">=1.10,<2" - -[package.extras] -dev = ["PyTest", "PyTest-Cov", "bump2version (<1)", "sphinx (<2)", "tox"] - [[package]] name = "docutils" version = "0.20.1" @@ -414,13 +397,13 @@ grpcio-gcp = ["grpcio-gcp (>=0.2.2,<1.0.dev0)"] [[package]] name = "google-auth" -version = "2.30.0" +version = "2.31.0" description = "Google Authentication Library" optional = false python-versions = ">=3.7" files = [ - {file = "google-auth-2.30.0.tar.gz", hash = "sha256:ab630a1320f6720909ad76a7dbdb6841cdf5c66b328d690027e4867bdfb16688"}, - {file = "google_auth-2.30.0-py2.py3-none-any.whl", hash = "sha256:8df7da660f62757388b8a7f249df13549b3373f24388cb5d2f1dd91cc18180b5"}, + {file = "google-auth-2.31.0.tar.gz", hash = "sha256:87805c36970047247c8afe614d4e3af8eceafc1ebba0c679fe75ddd1d575e871"}, + {file = "google_auth-2.31.0-py2.py3-none-any.whl", hash = "sha256:042c4702efa9f7d3c48d3a69341c209381b125faa6dbf3ebe56bc7e40ae05c23"}, ] [package.dependencies] @@ -775,24 +758,6 @@ doc = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linke perf = ["ipython"] test = ["flufl.flake8", "importlib-resources (>=1.3)", "jaraco.test (>=5.4)", "packaging", "pyfakefs", "pytest (>=6,!=8.1.*)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-mypy", "pytest-perf (>=0.9.2)", "pytest-ruff (>=0.2.1)"] -[[package]] -name = "importlib-resources" -version = "6.4.0" -description = "Read resources from Python packages" -optional = false -python-versions = ">=3.8" -files = [ - {file = "importlib_resources-6.4.0-py3-none-any.whl", hash = "sha256:50d10f043df931902d4194ea07ec57960f66a80449ff867bfe782b4c486ba78c"}, - {file = "importlib_resources-6.4.0.tar.gz", hash = "sha256:cdb2b453b8046ca4e3798eb1d84f3cce1446a0e8e7b5ef4efb600f19fc398145"}, -] - -[package.dependencies] -zipp = {version = ">=3.1.0", markers = "python_version < \"3.10\""} - -[package.extras] -docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (<7.2.5)", "sphinx (>=3.5)", "sphinx-lint"] -testing = ["jaraco.test (>=5.4)", "pytest (>=6)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-mypy", "pytest-ruff (>=0.2.1)", "zipp (>=3.17)"] - [[package]] name = "jinja2" version = "3.1.4" @@ -810,71 +775,6 @@ MarkupSafe = ">=2.0" [package.extras] i18n = ["Babel (>=2.7)"] -[[package]] -name = "lsst-alert-packet" -version = "0.3.0" -description = "Code for interacting with Vera C. Rubin Observatory alert packets" -optional = false -python-versions = "*" -files = [] -develop = false - -[package.dependencies] -fastavro = "*" -lsst-resources = "*" -numpy = "*" -requests = "*" - -[package.source] -type = "git" -url = "https://github.com/lsst/alert_packet.git" -reference = "w.2024.26" -resolved_reference = "bcc3a36e4f666840ba6ad5393cef3ca78d0b3faa" - -[[package]] -name = "lsst-resources" -version = "25.2023.3000" -description = "An abstraction layer for reading and writing from URI file resources." -optional = false -python-versions = "*" -files = [ - {file = "lsst-resources-25.2023.3000.tar.gz", hash = "sha256:870f9b6e6058b369216cb26b9a4cfc7b79853aa5fdba98c0357dfbb7fdd7d555"}, - {file = "lsst_resources-25.2023.3000-py3-none-any.whl", hash = "sha256:7f936c13c20854b78b13ceb70f3edc1f836e34570f6e15e3ff1e036c37a12aa2"}, -] - -[package.dependencies] -importlib-resources = "*" -lsst-utils = "*" - -[package.extras] -gs = ["google-cloud-storage"] -https = ["astropy (>=4.0)", "requests (>=2.26.0)", "responses (>=0.12.0)", "urllib3 (>=1.25.10)"] -s3 = ["backoff (>=1.10)", "boto3 (>=1.13)", "moto (>=1.3)"] -test = ["pytest (>=3.2)", "pytest-openfiles (>=0.5.0)"] - -[[package]] -name = "lsst-utils" -version = "26.2024.1700" -description = "Utility functions from Rubin Observatory Data Management for the Legacy Survey of Space and Time (LSST)." -optional = false -python-versions = ">=3.9.0" -files = [ - {file = "lsst_utils-26.2024.1700-py3-none-any.whl", hash = "sha256:5b24cf201e002718ec967be2025d833d23e2b9e5155622656056fe2b1082d099"}, - {file = "lsst_utils-26.2024.1700.tar.gz", hash = "sha256:7227e350e7f042c55830f425e9692b1241184f3e6679b401dfc051a12428c1b8"}, -] - -[package.dependencies] -astropy = ">=5.0" -deprecated = ">=1.2" -importlib-resources = "*" -numpy = ">=1.17" -psutil = ">=5.7" -pyyaml = ">=5.1" -threadpoolctl = "*" - -[package.extras] -test = ["pytest (>=3.2)", "pytest-openfiles (>=0.5.0)"] - [[package]] name = "markdown-it-py" version = "3.0.0" @@ -1206,35 +1106,6 @@ files = [ {file = "protobuf-5.27.2.tar.gz", hash = "sha256:f3ecdef226b9af856075f28227ff2c90ce3a594d092c39bee5513573f25e2714"}, ] -[[package]] -name = "psutil" -version = "6.0.0" -description = "Cross-platform lib for process and system monitoring in Python." -optional = false -python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,>=2.7" -files = [ - {file = "psutil-6.0.0-cp27-cp27m-macosx_10_9_x86_64.whl", hash = "sha256:a021da3e881cd935e64a3d0a20983bda0bb4cf80e4f74fa9bfcb1bc5785360c6"}, - {file = "psutil-6.0.0-cp27-cp27m-manylinux2010_i686.whl", hash = "sha256:1287c2b95f1c0a364d23bc6f2ea2365a8d4d9b726a3be7294296ff7ba97c17f0"}, - {file = "psutil-6.0.0-cp27-cp27m-manylinux2010_x86_64.whl", hash = "sha256:a9a3dbfb4de4f18174528d87cc352d1f788b7496991cca33c6996f40c9e3c92c"}, - {file = "psutil-6.0.0-cp27-cp27mu-manylinux2010_i686.whl", hash = "sha256:6ec7588fb3ddaec7344a825afe298db83fe01bfaaab39155fa84cf1c0d6b13c3"}, - {file = "psutil-6.0.0-cp27-cp27mu-manylinux2010_x86_64.whl", hash = "sha256:1e7c870afcb7d91fdea2b37c24aeb08f98b6d67257a5cb0a8bc3ac68d0f1a68c"}, - {file = "psutil-6.0.0-cp27-none-win32.whl", hash = "sha256:02b69001f44cc73c1c5279d02b30a817e339ceb258ad75997325e0e6169d8b35"}, - {file = "psutil-6.0.0-cp27-none-win_amd64.whl", hash = "sha256:21f1fb635deccd510f69f485b87433460a603919b45e2a324ad65b0cc74f8fb1"}, - {file = "psutil-6.0.0-cp36-abi3-macosx_10_9_x86_64.whl", hash = "sha256:c588a7e9b1173b6e866756dde596fd4cad94f9399daf99ad8c3258b3cb2b47a0"}, - {file = "psutil-6.0.0-cp36-abi3-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6ed2440ada7ef7d0d608f20ad89a04ec47d2d3ab7190896cd62ca5fc4fe08bf0"}, - {file = "psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5fd9a97c8e94059b0ef54a7d4baf13b405011176c3b6ff257c247cae0d560ecd"}, - {file = "psutil-6.0.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e2e8d0054fc88153ca0544f5c4d554d42e33df2e009c4ff42284ac9ebdef4132"}, - {file = "psutil-6.0.0-cp36-cp36m-win32.whl", hash = "sha256:fc8c9510cde0146432bbdb433322861ee8c3efbf8589865c8bf8d21cb30c4d14"}, - {file = "psutil-6.0.0-cp36-cp36m-win_amd64.whl", hash = "sha256:34859b8d8f423b86e4385ff3665d3f4d94be3cdf48221fbe476e883514fdb71c"}, - {file = "psutil-6.0.0-cp37-abi3-win32.whl", hash = "sha256:a495580d6bae27291324fe60cea0b5a7c23fa36a7cd35035a16d93bdcf076b9d"}, - {file = "psutil-6.0.0-cp37-abi3-win_amd64.whl", hash = "sha256:33ea5e1c975250a720b3a6609c490db40dae5d83a4eb315170c4fe0d8b1f34b3"}, - {file = "psutil-6.0.0-cp38-abi3-macosx_11_0_arm64.whl", hash = "sha256:ffe7fc9b6b36beadc8c322f84e1caff51e8703b88eee1da46d1e3a6ae11b4fd0"}, - {file = "psutil-6.0.0.tar.gz", hash = "sha256:8faae4f310b6d969fa26ca0545338b21f73c6b15db7c4a8d934a5482faa818f2"}, -] - -[package.extras] -test = ["enum34", "ipaddress", "mock", "pywin32", "wmi"] - [[package]] name = "pyasn1" version = "0.6.0" @@ -1655,17 +1526,6 @@ files = [ [package.extras] widechars = ["wcwidth"] -[[package]] -name = "threadpoolctl" -version = "3.5.0" -description = "threadpoolctl" -optional = false -python-versions = ">=3.8" -files = [ - {file = "threadpoolctl-3.5.0-py3-none-any.whl", hash = "sha256:56c1e26c150397e58c4926da8eeee87533b1e32bef131bd4bf6a2f45f3185467"}, - {file = "threadpoolctl-3.5.0.tar.gz", hash = "sha256:082433502dd922bf738de0d8bcc4fdcbf0979ff44c42bd40f5af8a282f6fa107"}, -] - [[package]] name = "tomli" version = "2.0.1" @@ -1705,85 +1565,6 @@ h2 = ["h2 (>=4,<5)"] socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"] zstd = ["zstandard (>=0.18.0)"] -[[package]] -name = "wrapt" -version = "1.16.0" -description = "Module for decorators, wrappers and monkey patching." -optional = false -python-versions = ">=3.6" -files = [ - {file = "wrapt-1.16.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:ffa565331890b90056c01db69c0fe634a776f8019c143a5ae265f9c6bc4bd6d4"}, - {file = "wrapt-1.16.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e4fdb9275308292e880dcbeb12546df7f3e0f96c6b41197e0cf37d2826359020"}, - {file = "wrapt-1.16.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bb2dee3874a500de01c93d5c71415fcaef1d858370d405824783e7a8ef5db440"}, - {file = "wrapt-1.16.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2a88e6010048489cda82b1326889ec075a8c856c2e6a256072b28eaee3ccf487"}, - {file = "wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ac83a914ebaf589b69f7d0a1277602ff494e21f4c2f743313414378f8f50a4cf"}, - {file = "wrapt-1.16.0-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:73aa7d98215d39b8455f103de64391cb79dfcad601701a3aa0dddacf74911d72"}, - {file = "wrapt-1.16.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:807cc8543a477ab7422f1120a217054f958a66ef7314f76dd9e77d3f02cdccd0"}, - {file = "wrapt-1.16.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:bf5703fdeb350e36885f2875d853ce13172ae281c56e509f4e6eca049bdfb136"}, - {file = "wrapt-1.16.0-cp310-cp310-win32.whl", hash = "sha256:f6b2d0c6703c988d334f297aa5df18c45e97b0af3679bb75059e0e0bd8b1069d"}, - {file = "wrapt-1.16.0-cp310-cp310-win_amd64.whl", hash = "sha256:decbfa2f618fa8ed81c95ee18a387ff973143c656ef800c9f24fb7e9c16054e2"}, - {file = "wrapt-1.16.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1a5db485fe2de4403f13fafdc231b0dbae5eca4359232d2efc79025527375b09"}, - {file = "wrapt-1.16.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:75ea7d0ee2a15733684badb16de6794894ed9c55aa5e9903260922f0482e687d"}, - {file = "wrapt-1.16.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a452f9ca3e3267cd4d0fcf2edd0d035b1934ac2bd7e0e57ac91ad6b95c0c6389"}, - {file = "wrapt-1.16.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:43aa59eadec7890d9958748db829df269f0368521ba6dc68cc172d5d03ed8060"}, - {file = "wrapt-1.16.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:72554a23c78a8e7aa02abbd699d129eead8b147a23c56e08d08dfc29cfdddca1"}, - {file = "wrapt-1.16.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:d2efee35b4b0a347e0d99d28e884dfd82797852d62fcd7ebdeee26f3ceb72cf3"}, - {file = "wrapt-1.16.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:6dcfcffe73710be01d90cae08c3e548d90932d37b39ef83969ae135d36ef3956"}, - {file = "wrapt-1.16.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:eb6e651000a19c96f452c85132811d25e9264d836951022d6e81df2fff38337d"}, - {file = "wrapt-1.16.0-cp311-cp311-win32.whl", hash = "sha256:66027d667efe95cc4fa945af59f92c5a02c6f5bb6012bff9e60542c74c75c362"}, - {file = "wrapt-1.16.0-cp311-cp311-win_amd64.whl", hash = "sha256:aefbc4cb0a54f91af643660a0a150ce2c090d3652cf4052a5397fb2de549cd89"}, - {file = "wrapt-1.16.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5eb404d89131ec9b4f748fa5cfb5346802e5ee8836f57d516576e61f304f3b7b"}, - {file = "wrapt-1.16.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:9090c9e676d5236a6948330e83cb89969f433b1943a558968f659ead07cb3b36"}, - {file = "wrapt-1.16.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:94265b00870aa407bd0cbcfd536f17ecde43b94fb8d228560a1e9d3041462d73"}, - {file = "wrapt-1.16.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f2058f813d4f2b5e3a9eb2eb3faf8f1d99b81c3e51aeda4b168406443e8ba809"}, - {file = "wrapt-1.16.0-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:98b5e1f498a8ca1858a1cdbffb023bfd954da4e3fa2c0cb5853d40014557248b"}, - {file = "wrapt-1.16.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:14d7dc606219cdd7405133c713f2c218d4252f2a469003f8c46bb92d5d095d81"}, - {file = "wrapt-1.16.0-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:49aac49dc4782cb04f58986e81ea0b4768e4ff197b57324dcbd7699c5dfb40b9"}, - {file = "wrapt-1.16.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:418abb18146475c310d7a6dc71143d6f7adec5b004ac9ce08dc7a34e2babdc5c"}, - {file = "wrapt-1.16.0-cp312-cp312-win32.whl", hash = "sha256:685f568fa5e627e93f3b52fda002c7ed2fa1800b50ce51f6ed1d572d8ab3e7fc"}, - {file = "wrapt-1.16.0-cp312-cp312-win_amd64.whl", hash = "sha256:dcdba5c86e368442528f7060039eda390cc4091bfd1dca41e8046af7c910dda8"}, - {file = "wrapt-1.16.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:d462f28826f4657968ae51d2181a074dfe03c200d6131690b7d65d55b0f360f8"}, - {file = "wrapt-1.16.0-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a33a747400b94b6d6b8a165e4480264a64a78c8a4c734b62136062e9a248dd39"}, - {file = "wrapt-1.16.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b3646eefa23daeba62643a58aac816945cadc0afaf21800a1421eeba5f6cfb9c"}, - {file = "wrapt-1.16.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3ebf019be5c09d400cf7b024aa52b1f3aeebeff51550d007e92c3c1c4afc2a40"}, - {file = "wrapt-1.16.0-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:0d2691979e93d06a95a26257adb7bfd0c93818e89b1406f5a28f36e0d8c1e1fc"}, - {file = "wrapt-1.16.0-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:1acd723ee2a8826f3d53910255643e33673e1d11db84ce5880675954183ec47e"}, - {file = "wrapt-1.16.0-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:bc57efac2da352a51cc4658878a68d2b1b67dbe9d33c36cb826ca449d80a8465"}, - {file = "wrapt-1.16.0-cp36-cp36m-win32.whl", hash = "sha256:da4813f751142436b075ed7aa012a8778aa43a99f7b36afe9b742d3ed8bdc95e"}, - {file = "wrapt-1.16.0-cp36-cp36m-win_amd64.whl", hash = "sha256:6f6eac2360f2d543cc875a0e5efd413b6cbd483cb3ad7ebf888884a6e0d2e966"}, - {file = "wrapt-1.16.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:a0ea261ce52b5952bf669684a251a66df239ec6d441ccb59ec7afa882265d593"}, - {file = "wrapt-1.16.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7bd2d7ff69a2cac767fbf7a2b206add2e9a210e57947dd7ce03e25d03d2de292"}, - {file = "wrapt-1.16.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9159485323798c8dc530a224bd3ffcf76659319ccc7bbd52e01e73bd0241a0c5"}, - {file = "wrapt-1.16.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a86373cf37cd7764f2201b76496aba58a52e76dedfaa698ef9e9688bfd9e41cf"}, - {file = "wrapt-1.16.0-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:73870c364c11f03ed072dda68ff7aea6d2a3a5c3fe250d917a429c7432e15228"}, - {file = "wrapt-1.16.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:b935ae30c6e7400022b50f8d359c03ed233d45b725cfdd299462f41ee5ffba6f"}, - {file = "wrapt-1.16.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:db98ad84a55eb09b3c32a96c576476777e87c520a34e2519d3e59c44710c002c"}, - {file = "wrapt-1.16.0-cp37-cp37m-win32.whl", hash = "sha256:9153ed35fc5e4fa3b2fe97bddaa7cbec0ed22412b85bcdaf54aeba92ea37428c"}, - {file = "wrapt-1.16.0-cp37-cp37m-win_amd64.whl", hash = "sha256:66dfbaa7cfa3eb707bbfcd46dab2bc6207b005cbc9caa2199bcbc81d95071a00"}, - {file = "wrapt-1.16.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:1dd50a2696ff89f57bd8847647a1c363b687d3d796dc30d4dd4a9d1689a706f0"}, - {file = "wrapt-1.16.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:44a2754372e32ab315734c6c73b24351d06e77ffff6ae27d2ecf14cf3d229202"}, - {file = "wrapt-1.16.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8e9723528b9f787dc59168369e42ae1c3b0d3fadb2f1a71de14531d321ee05b0"}, - {file = "wrapt-1.16.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:dbed418ba5c3dce92619656802cc5355cb679e58d0d89b50f116e4a9d5a9603e"}, - {file = "wrapt-1.16.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:941988b89b4fd6b41c3f0bfb20e92bd23746579736b7343283297c4c8cbae68f"}, - {file = "wrapt-1.16.0-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:6a42cd0cfa8ffc1915aef79cb4284f6383d8a3e9dcca70c445dcfdd639d51267"}, - {file = "wrapt-1.16.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:1ca9b6085e4f866bd584fb135a041bfc32cab916e69f714a7d1d397f8c4891ca"}, - {file = "wrapt-1.16.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:d5e49454f19ef621089e204f862388d29e6e8d8b162efce05208913dde5b9ad6"}, - {file = "wrapt-1.16.0-cp38-cp38-win32.whl", hash = "sha256:c31f72b1b6624c9d863fc095da460802f43a7c6868c5dda140f51da24fd47d7b"}, - {file = "wrapt-1.16.0-cp38-cp38-win_amd64.whl", hash = "sha256:490b0ee15c1a55be9c1bd8609b8cecd60e325f0575fc98f50058eae366e01f41"}, - {file = "wrapt-1.16.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9b201ae332c3637a42f02d1045e1d0cccfdc41f1f2f801dafbaa7e9b4797bfc2"}, - {file = "wrapt-1.16.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:2076fad65c6736184e77d7d4729b63a6d1ae0b70da4868adeec40989858eb3fb"}, - {file = "wrapt-1.16.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c5cd603b575ebceca7da5a3a251e69561bec509e0b46e4993e1cac402b7247b8"}, - {file = "wrapt-1.16.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b47cfad9e9bbbed2339081f4e346c93ecd7ab504299403320bf85f7f85c7d46c"}, - {file = "wrapt-1.16.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f8212564d49c50eb4565e502814f694e240c55551a5f1bc841d4fcaabb0a9b8a"}, - {file = "wrapt-1.16.0-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:5f15814a33e42b04e3de432e573aa557f9f0f56458745c2074952f564c50e664"}, - {file = "wrapt-1.16.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:db2e408d983b0e61e238cf579c09ef7020560441906ca990fe8412153e3b291f"}, - {file = "wrapt-1.16.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:edfad1d29c73f9b863ebe7082ae9321374ccb10879eeabc84ba3b69f2579d537"}, - {file = "wrapt-1.16.0-cp39-cp39-win32.whl", hash = "sha256:ed867c42c268f876097248e05b6117a65bcd1e63b779e916fe2e33cd6fd0d3c3"}, - {file = "wrapt-1.16.0-cp39-cp39-win_amd64.whl", hash = "sha256:eb1b046be06b0fce7249f1d025cd359b4b80fc1c3e24ad9eca33e0dcdb2e4a35"}, - {file = "wrapt-1.16.0-py3-none-any.whl", hash = "sha256:6906c4100a8fcbf2fa735f6059214bb13b97f75b1a61777fcf6432121ef12ef1"}, - {file = "wrapt-1.16.0.tar.gz", hash = "sha256:5f370f952971e7d17c7d1ead40e49f32345a7f7a5373571ef44d800d06b1899d"}, -] - [[package]] name = "zipp" version = "3.19.2" @@ -1802,4 +1583,4 @@ test = ["big-O", "importlib-resources", "jaraco.functools", "jaraco.itertools", [metadata] lock-version = "2.0" python-versions = "^3.9" -content-hash = "e869d348908366f4759924143ad73e5b1081aadea17d2d4df2f8a9ff04e0fa26" +content-hash = "f70bdbe1b9b54ce3070f89ba0206f4f11e099d15195ab353975bfa3e317ed0b4" diff --git a/pyproject.toml b/pyproject.toml index 54b3dca..99a414b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -39,8 +39,9 @@ pandas = ">=1.5" # v1.5.1 required by supernnova v3.0.1 tabulate = ">=0.9" # lsst-alert-packet for the LSST schema helper. # The PyPI version of this looks out of date so install from a git tag instead. -# [FIXME] This is pinned... is there a way to get the latest tag automatically? -lsst-alert-packet = { git = "https://github.com/lsst/alert_packet.git", tag = "w.2024.26" } +# [FIXME] Can't publish to PyPI if we depend on a git repo. +# Without this, `schema.SchemaHelper.lsst_auto_schema_helper` won't work. +# lsst-alert-packet = { git = "https://github.com/lsst/alert_packet.git", tag = "w.2024.26" } [tool.poetry.group.docs] optional = true