Home

PyLOT Design Document

Author               : Abdelhak Marouane (GHRC)
Stack-holders        : DAAC developers/ DAAC operators/ DAAC managers
Product              : PycLoud Operations Tool (PyLOT)
Topic                : Software Development / Scientific / Engineering
Intended Audience    : Developers / Operators
License              : TBD
Programming Language : Python 3.8+ 

Overview

Python cLoud Operations Tool (PyLOT) is a python command line tool designed to help DAAC operators solve the operations edge cases that can't be solved (or which are difficult/time consuming) using Cumulus Dashboard. Since it is powered by Cumulus-API it will allow the operators to interact with Cumulus stack, for example to monitor the granules status, run rules, and create and update Cumulus records (collections and/or granules).
In some cases interacting with Cumulus-API via the dashboard is sufficient, but there are some edge cases that require the operator to have a tool that provides more flexibility than a Web-Based application. PyLOT can run as a command line tool in your local machine and accept options and respond with JSON. It can also be used as a library for AWS lambda and AWS ECS tasks.
PyLOT can overcome the limitation of Cumulus-API by monitoring the status of AWS resources (Cloudwatch, SFN, S3...). This tool will prevent reinventing the wheel, since a solution for a common problem can be easily shared among all the DAACs (sharing is caring).

Problem statement

Based on a list provided by the stack holders and data management we believe that this tool can be used to solve not only GHRC edge cases but those of other DAACs too.
This tool will be developed following a pluggable design, so if a particular edge case arises for a specific DAAC the developers of that DAAC can just "plug" their solution into the tool and potentially share the solution with others

How will it work?

The tool is powered by Cumulus-API and uses AWS libraries to provide DAAC operators with a way to solve some edge cases. With AWS services the tool will only have read-only permission to avoid any risk of directly changing the data without going through Cumulus-API.
It can be installed in a local machine and be used as a command line tool or used as a library in some AWS services such as lambda function or an ECS task.