- Table of Contents
Fritz is the science data platform for the Zwicky Transient Facility (ZTF) Phase II.
It implements an end-to-end, scalable, API-first system for Time-domain Astronomy featuring:
- A multi-survey data archive and alert broker
- An interactive, mobile-friendly collaborative platform for the transient, variable, and Solar system science cases, with fine-grained access control
- A workhorse for machine learning applications
- Follow-up observation management with robotic and classical facilities
Fritz employs a modular architecture and
integrates and extends two major components: Kowalski
acts as the alert processor and data archive, and SkyPortal,
which handles the rest of the stack.
The schematic overview of our system is shown below:
We note that while Fritz/SkyPortal/Kowalski can be centrally managed for a single-source of truth, it also can have many deployments with different configurations.
The project documentation is hosted at https://docs.fritz.science.
Newly invited Fritz users are first taken to the Profile page.
Here, you can update your user info, preferences, and manage tokens to
interact with Fritz programmatically.
If you have a gravatar associated with your email address,
it will be used as your avatar throughout the portal.
System administrators can invite new users to Fritz and manage their roles/ACLs as well as group membership and
alert stream access.
We use OAuth for user authentication.
On the Dashboard, the landing page, the user can see a collection of configurable widgets displaying the information on the most popular and latest saved sources, the newsfeed, conditions at telescopes and so on.
New Fritz users are added to one or more groups.
Users can also create groups themselves and request admission to groups they are not a member of.
You can find and manage the list of group members, the sources saved to the group, and the group's alert filters (defined on one of the alert streams that the group has access to) on the Group page. Group admins can create new filters (and modify existing ones). System administrators can grant alert stream access to the groups.
Fritz provides rich alert stream filtering capabilities through its Kowalski backend,
which consumes the ZTF Kafka alert stream, persisting the alerts to a database,
and supplementing them with additional data such as the Galactic coordinates,
external catalog cross-matches, machine learning scores etc.
Alert filters are implemented as aggregation pipelines and are executed on the enhanced packets containing, in particular, the full photometry history. For a detailed discussion of this, refer to the alert filters section of the docs.
Fritz performs automated checks of the filter definition such that no code audit is necessary. Valid changes are propagated and applied almost immediately. Users can enable/disable filters, inspect filter version history, opt in for automatically saving passing objects to the filter's group and updating annotations each time an object passes the filter.
Filters can range from very simple that rely on, for example, mostly ML scores, or implement very complicated logic/computations. Watch lists can also be implemented as simple Fritz filters.
Finally, we provide public alert databases for filter design and debugging.
Alerts passing a filter are posted to Fritz's SkyPortal backend as Candidates and appear on the Scanning page (for the corresponding filter groups).
Candidates do not have to originate from Kowalski and could be posted (manually) via the API.
On the Candidates page, the users can filter, scan and inspect the objects that have passed filters of their groups
and save them to one or more groups. Candidates that are not saved to any group within 7 days are removed from Fritz.
Saved Candidates become Sources that are persisted indefinitely.
Candidates/Sources can be supplemented with (structured) annotations, which is particularly useful for scanning:
The Source page aggregates all kinds of information related to an object:
In particular, users can generate finder charts and star lists for the source, check its observability,
add and inspect redshift data:
Annotations provide a way to store structured data related to the source, while comments can be used to enter text/data in free-form. Sources can be classified according to a taxonomy (which users can post themselves provided they have the necessary ACL). Interactive offset plot displays positional information.
Users can send notifications about the source to the group members, provided they opted in for receiving such.
For sources annotated with Gaia color information, an
HR diagram is rendered.
Users can interactively inspect (optionally binned) photometry data in both magnitude and flux spaces.
Tools for data import, export, and granular access management are available.
The photometry plot for sources annotated with period data additionally displays phase-folded light curves:
Tooling for extensive in-browser periodogram analysis for variable sources is also available:
Users can inspect, analyze, and manage (including export/import) spectroscopy data.
We provide interfaces to work with both robotic and classical follow-up facilities, including a framework for dealing with allocations. In particular, we feature automated integration with the SEDM.
Sources can be added to ("classical") observing runs:
Sources can be added to favorites:
Users can query the objects that exist in Fritz's SkyPortal on the Sources page:
Only the objects that have been posted to Fritz's SkyPortal backend are saved in its database.
However, Fritz's users can access the entire archive of ZTF alerts via the Alerts page:
Fritz's interfaces are mobile-friendly, so the app will work as expected on your phone or tablet:
 |
 |
An API enables access to most of the underlying functionality of Fritz/SkyPortal/Kowalski. The workflows described above are all enabled by specific API calls. The complete OpenAPI specification is available at https://docs.fritz.science/api.html.
To use the API, you will need a token that can be generated on the Profile page. Once you have that, you can access Fritz programmatically as follows.
import requests
from typing import Mapping, Optional
import urllib.parse
token = 'ea70a5f0-b321-43c6-96a1-b2de225e0339'
def api(
method: str,
endpoint: str,
data: Optional[Mapping] = None,
base_url: str = "https://fritz.science",
):
headers = {"Authorization": f"token {token}"}
response = requests.request(
method.upper(),
urllib.parse.urljoin(base_url, endpoint),
json=data,
headers=headers,
)
return responseFor example, here is how to retrieve object's annotations:
object_id = "ZTF21aatjavc"
response = api("get", f"api/sources/{object_id}/annotations")
data = response.json().get("data", None)
print(data)Which would yield something similar to:
[{
"author_id": 3,
"created_at": "2021-04-08T06:17:56.980090",
"data": {
"acai_b": 0.00011,
"acai_h": 0.98887,
"acai_n": 0.00361,
"acai_o": 0.00925,
"acai_v": 2e-05,
"age": 0.0,
"candid": "1558254455015015000",
"n_det": 1,
},
"id": 916700,
"modified": "2021-04-08T06:17:56.980090",
"obj_id": "ZTF21aatjavc",
"origin": "au-public:hosted",
}]