emit-sds-l2a

NOTE - at this time the EMIT repositories are not supporting Pull Requests from members outside of the EMIT-SDS Team. This is expected to change in March, and guidance on branches will be provided at that time. At present, public migration of this repository is a work-in-progress, and not all features are yet fully up to date. See the develop branch - set as default - for the latest code.

Welcome to the EMIT Level 2a science data system repository. To understand how this repository is linked to the rest of the emit-sds repositories, please see the repository guide.

L2A consists of three components:

1. Construction of a surface model (one time)
2. Estimation of surface reflectance and uncertainy using an optimal-estimation based retrieval
3. Generation of masks (cloud / cirrus / water / spacecraft / etc.)

Surface Model Construction

The "surface_v2" subdirectory contains the data files used to generate the EMIT default surface model. The data files mostly come from the USGS spectral library version 7 (Kokaly et al., 2017), with some additional spectra from Tan et al (2016). Some adjustments to smooth out artifacts at the periphery were included, and the finalized, adjusted spectra can be found at these locations: snow / ice, vegetation, water, and other. To construct the surface model, utilize the "surface_model" utility in the isofit/utils subdirectory of the isofit repository. An example call, made from inside surface_v2, may look like:

python -c "from isofit.utils import surface_model; surface_model('./surface.json')"

This will create an output .mat file, at the location specified by "output_model_file" in the json configuration file. This surface model will be passed into subsequent calls to apply_oe (see below) and need only be generated once for general use (though specific investigations may require custom surface models). Changing the wavelengths.txt file facilitates use with different instruments.

Estimating Surface Reflectance and Uncertainty

The core optimal estimation code is best called with the "apply_oe" script in the isofit/utils subdirectory of the isofit repository. The current implementation for the EMIT SDS looks like:

python apply_oe.py,
       --presolve=1, \
       --empirical_line=1, \
       --emulator_base=[sRTMnet_DIRECTORY], \
       --n_cores str(self.n_cores), \
       --surface_path [SURFACE_MODEL .mat FILE], \
       --ray_temp_dir [SCRATCH_PARALLEL_DIRECTORY], \
       --log_file [OUTPUT_LOG_FILE], \
       --logging_level "INFO", \
       [INPUT_RADIANCE_FILE], \
       [INPUT_LOC_FILE], \
       [INPUT_OBS_FILE], \
       [OUTPUT_DIRECTORY], \
       "emit"

The estimated surface reflectance and uncertainty files will appear in [WORKING_DIRECTORY]/output. The isofit repository is the result of extensive scientific inquiry that extends an initial presentation by Thompson et al, and a bibliography can be found here. The sRTMnet dependency comes from Brodrick et al, and is available for download.

Applying Masks

The EMIT mask can be generated using the following call, which is dependent on having completed the above surface reflectance estimation step:

python make_emit_masks.py \
       [INPUT_RADIANCE_FILE]
       [INPUT_LOC_FILE]
       [INPUT_LABEL_FILE]
       [INPUT_STATE_SUBS_FILE]
       [INPUT_IRR_FILE]
       [OUTPUT_MASK_FILE]

The [INPUT_LABEL_FILE] and [INPUT_STATE_SUBS_FILE] can be found in the [WORKING_DIRECTORY]/output file noted about with the extensions _lbl _subs_state, respectively. An [INPUT_IRR/FILE] can be found in data/kurudz_0.1nm.dat. The [OUTPUT_MASK_FILE] is a 7 band output, designating:

1. Cloud flag
2. Cirrus flag
3. Surface water flag
4. Spacecraft flag
5. Dilated cloud flag
6. Aerosol Optical Depth at 550 nm
7. Atmsopheric water vapor [g / cm^2]
8. Aggregate mask flag

Masks are denoted as ones.