Skip to content

How to Create a Complete Custom Script

Jump to bottom
Chelsea Troy edited this page Aug 5, 2020 · 3 revisions

Eventually, the idea is for theia to allow researchers to build Landsat (and maybe other satellites, too) data modification pipelines through a UI, which upload to a Zooniverse project.

This is a long-term goal, and in the meantime, theia can take care of image ordering from ESPA and uploading to The Zooniverse, with a custom Python processing script in the middle.

In order for that to work, the research team needs to have the following:

The Script

  1. The script should be in Python.
  2. The script should take in an already untarred version of a Landsat scene. That is, it should take in a scene in the format of a directory named the scene name, containing files inside for each of the bands of the scene. Like so:
LC08_L1TP_023047_20130324_20170505_01_T1
   |__ LC08_L1TP_023047_20130324_20170505_01_T1.xml
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band1.tif
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band2.tif
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band3.tif
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band4.tif
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band5.tif
   |__ LC08_L1TP_023047_20130324_20170505_01_T1_sr_band6.tif

Why: theia already has a task to fetch and untar the data like this. You custom script will go in between this task and task that uploads to The Zooniverse.

  1. The script should create a single directory and fill it with the images you would like uploaded to The Zooniverse—and only those images: no additional files, rejected tiles, or interstitial products should be in this folder. Example:
approved_tiles
   |__ tile_01.png
   |__ tile_02.png
   |__ tile_03.png
  1. If you would also like additional data or details uploaded to be shown in a pop-up for each image, you can do that (Floating Forests uses this to surface information like the latitude and longitude of each image and the date it was taken). Here is how your script needs to work for that to fit into Theia.

As a sibling (next to) your directory of images to upload, create a second folder. The name of this folder should end with "_interstitial_products". Inside of this directory, put a file called manifest.csv. This CSV must have a column called "#filename", and that column must be populated with the names of the files of your images to upload. It can have any other columns and column names you like.

In addition to the manifest.csv, you can put anything you want in this second folder: rejected tiles, interstitial files created by your script, et cetera. It's great for stuff you want to keep, but you don't want uploaded to The Zooniverse. It can also be helpful in case we need to debug something going wrong with your script and those intermediate files can help us figure out the problem. Example (this is what Floating Forest's interstitial products folder looks like):

Floating Forests interstitial product folder structure

Clone this wiki locally