Merge pull request #76 from steeleb/main

Adding classification export to ee.ImageCollection() and updating classification values in Methods doc

Methods_Results_Summary.Rmd

@@ -139,14 +139,9 @@ data types were output: a tabular data summary of the area classified and the
 total area of each class for all three AOIs, as well as a .tif raster at the
 resolution the GTB was applied (10m for Sentinel-2 and 30m for Landsat) for
 each classified mission-date image. The .tif rasters were labeled by pixel with
-the following values:
-
-| Pixel Value | Pixel Description |
-|-------------|-----------------------------------------| | 1 | cloud | | 2 |
-open water | | 3 | light, near-shore sediment | | 4 | dark, near-shore sediment
-| | 5 | offshore sediment | | 0 | out of area/masked for saturated pixels |
-
-: Pixel values and description for the GTB tif model output.
+the following values: 0 = out of area/masked for saturated pixels; 1 = cloud; 2
+= open water; 3 = light, near shore sediment; 4 = offshore sediment; 5 = dark,
+near shore sediment.
 
 ## Model evaluation metrics
 
@@ -195,9 +190,9 @@ label_table_join <- full_join(label_table, filtered_label_table)
 ```
 
 The collated crowdsourced label dataset consisted of `r nrow(labels)` labels
-across all classes. There were `r nrow(ml_labels)` labels that were part of
-the classes of interest (cloud, open water, sediment). After filtering for
-outliers from each subset of mission-specific labels, there were `r
+across all classes. There were `r nrow(ml_labels)` labels that were part of the
+classes of interest (cloud, open water, sediment). After filtering for outliers
+from each subset of mission-specific labels, there were `r
 nrow(filtered_labels)` labels with complete band information. Table 1 presents
 a break down of the labels.
 
@@ -417,9 +412,24 @@ or to classify ice. It is important to note that evaluation of the GTB model
 was only done on the available by-pixel labels and that accuracy at
 classification edges may not be precise.
 
-In some cases, hazy dispersed clouds are incorrectly classified as off-shore
-sediment. Caution should be used clouds characterize a large proportion of the
-AOI.
+In many cases, cirrus clouds are incorrectly classified as off-shore sediment.
+Caution should be used when clouds characterize a large proportion of the AOI.
+
+# Image Viewer Links
+
+The following links are Google Earth Engine scripts that allow for manual
+examination of the true color image, the eePlumB classification (version
+2024-01-08), and a measure of atmospheric opacity (Landsat 7) or cirrus cloud
+confidence level (Landsat 8 & 9).
+
+[Landsat
+7](https://code.earthengine.google.com/cd2fc7baeb0dcb2a1d30e065b419bb9e?hideCode=true)
+
+[Landsat
+8](https://code.earthengine.google.com/1c790dcabc46ff9f170a81223928df11?hideCode=true)
+
+[Landsat
+9](https://code.earthengine.google.com/4f8526227eeb88e67ee2e0db84ce77d1?hideCode=true)
 
 # References
 

modeling/04_Landsat_5_GTB.Rmd

@@ -100,12 +100,16 @@ trainedGTB_ls5 = (ee.Classifier.smileGradientTreeBoost(10).train(
   classProperty = 'byte_property',
   inputProperties = ls_input_feat
 ))
-```
 
-Unfortunately, there is no current mechanism to save the GTB object, so all models
-may be *slightly* different if re-created, but will likely have very similar
-outcomes. 
+print(trainedGTB_ls5.getInfo())
+```
 
+Unfortunately, there is no current mechanism to save the GTB object. This is a
+bummer because you can't really set a seed for these either, however! GEE is a bit
+more rudimentary and recognizes the inputs and therefore creates the same output
+objects. I did a quick check of this by running the model here and then again 
+in the browser. Both have identical versions, so I feel confident that GEE is
+making the 'same' model. 
 
 ## Evaluate the models
 
@@ -551,6 +555,30 @@ for d in range(date_length_5):
   #Send next task.
   export_image.start()
 
+for d in range(date_length_5):
+  md = uniqueMissDate_l5.get(d)
+  print(md.getInfo())
+  print(str(d+1) + ' of ' + str(date_length_5))
+  image = (newStack_l5
+    .filter(ee.Filter.eq('missDate', md))
+    .first()
+    .clip(aoi_ee.geometry()))
+  image_new_class = (classifications_to_one_band(image)
+    .select('reclass'))
+  export_image = ee.batch.Export.image.toAsset(
+    image = image_new_class,
+    region = aoi_ee.geometry(),
+    description = 'GTB_v' + v_date + '_' + str(md.getInfo()),
+    assetId = 'projects/ee-ross-superior/assets/LS5/'+'GTB_LS5)'+str(md.getInfo())+'_v'+v_date,
+    scale = 30,
+    crs = img_crs,
+    maxPixels = 1e13)
+  
+  #Check how many existing tasks are running and take a break of 5 mins if it's >10
+  maximum_no_of_tasks(10, 5*60)
+  #Send next task.
+  export_image.start()
+
 ```
 
 

modeling/05_Landsat_7_GTB.Rmd

@@ -99,11 +99,16 @@ trainedGTB_ls7 = (ee.Classifier.smileGradientTreeBoost(10).train(
   classProperty = 'byte_property',
   inputProperties = ls_input_feat
 ))
+
+print(trainedGTB_ls7.getInfo())
 ```
 
-Unfortunately, there is no current mechanism to save the GTB object, so all models
-may be *slightly* different if re-created, but will likely have very similar
-outcomes. 
+Unfortunately, there is no current mechanism to save the GTB object. This is a
+bummer because you can't really set a seed for these either, however! GEE is a bit
+more rudimentary and recognizes the inputs and therefore creates the same output
+objects. I did a quick check of this by running the model here and then again 
+in the browser. Both have identical versions, so I feel confident that GEE is
+making the 'same' model. 
 
 ## Evaluate the models
 
@@ -543,6 +548,30 @@ for d in range(date_length_7):
   maximum_no_of_tasks(10, 5*60)
   #Send next task.
   export_image.start()
+  
+for d in range(date_length_7):
+  md = uniqueMissDate_l7.get(d)
+  print(md.getInfo())
+  print(str(d+1) + ' of ' + str(date_length_7))
+  image = (newStack_l7
+    .filter(ee.Filter.eq('missDate', md))
+    .first()
+    .clip(aoi_ee.geometry()))
+  image_new_class = (classifications_to_one_band(image)
+    .select('reclass'))
+  export_image = ee.batch.Export.image.toAsset(
+    image = image_new_class,
+    region = aoi_ee.geometry(),
+    description = 'GTB_v' + v_date + '_' + str(md.getInfo()),
+    assetId = 'projects/ee-ross-superior/assets/LS7/'+'GTB_LS7_'+str(md.getInfo())+'_v'+v_date,
+    scale = 30,
+    crs = img_crs,
+    maxPixels = 1e13)
+  
+  #Check how many existing tasks are running and take a break of 5 mins if it's >10
+  maximum_no_of_tasks(10, 5*60)
+  #Send next task.
+  export_image.start()
 
 ```
 

modeling/06_Landsat_8_GTB.Rmd

@@ -99,11 +99,16 @@ trainedGTB_ls8 = (ee.Classifier.smileGradientTreeBoost(10).train(
   classProperty = 'byte_property',
   inputProperties = ls_input_feat
 ))
+
+print(trainedGTB_ls8.getInfo())
 ```
 
-Unfortunately, there is no current mechanism to save the GTB object, so all models
-may be *slightly* different if re-created, but will likely have very similar
-outcomes. 
+Unfortunately, there is no current mechanism to save the GTB object. This is a
+bummer because you can't really set a seed for these either, however! GEE is a bit
+more rudimentary and recognizes the inputs and therefore creates the same output
+objects. I did a quick check of this by running the model here and then again 
+in the browser. Both have identical versions, so I feel confident that GEE is
+making the 'same' model. 
 
 ## Evaluate the models
 
@@ -544,6 +549,32 @@ for d in range(date_length_8):
   maximum_no_of_tasks(10, 5*60)
   #Send next task.
   export_image.start()
+  
+
+for d in range(date_length_8):
+  md = uniqueMissDate_l8.get(d)
+  print(md.getInfo())
+  print(str(d+1) + ' of ' + str(date_length_8))
+  image = (newStack_l8
+    .filter(ee.Filter.eq('missDate', md))
+    .first()
+    .clip(aoi_ee.geometry()))
+  image_new_class = (classifications_to_one_band(image)
+    .select('reclass'))
+  export_image = ee.batch.Export.image.toAsset(
+    image = image_new_class,
+    region = aoi_ee.geometry(),
+    description = 'GTB_v' + v_date + '_' + str(md.getInfo()),
+    assetId = 'projects/ee-ross-superior/assets/LS8/'+'GTB_LS8_'+str(md.getInfo())+'_v'+v_date,
+    scale = 30,
+    crs = img_crs,
+    maxPixels = 1e13)
+  
+  #Check how many existing tasks are running and take a break of 5 mins if it's >10
+  maximum_no_of_tasks(10, 5*60)
+  #Send next task.
+  export_image.start()
+
 
 ```
 

modeling/07_Landsat_9_GTB.Rmd

@@ -99,12 +99,16 @@ trainedGTB_ls9 = (ee.Classifier.smileGradientTreeBoost(10).train(
   classProperty = 'byte_property',
   inputProperties = ls_input_feat
 ))
-```
 
-Unfortunately, there is no current mechanism to save the GTB object, so all models
-may be *slightly* different if re-created, but will likely have very similar
-outcomes. 
+print(trainedGTB_ls9.getInfo())
+```
 
+Unfortunately, there is no current mechanism to save the GTB object. This is a
+bummer because you can't really set a seed for these either, however! GEE is a bit
+more rudimentary and recognizes the inputs and therefore creates the same output
+objects. I did a quick check of this by running the model here and then again 
+in the browser. Both have identical versions, so I feel confident that GEE is
+making the 'same' model. 
 
 ## Evaluate the models
 
@@ -546,6 +550,31 @@ for d in range(date_length_9):
   #Send next task.
   export_image.start()
 
+
+for d in range(date_length_9):
+  md = uniqueMissDate_l9.get(d)
+  print(md.getInfo())
+  print(str(d+1) + ' of ' + str(date_length_9))
+  image = (newStack_l9
+    .filter(ee.Filter.eq('missDate', md))
+    .first()
+    .clip(aoi_ee.geometry()))
+  image_new_class = (classifications_to_one_band(image)
+    .select('reclass'))
+  export_image = ee.batch.Export.image.toAsset(
+    image = image_new_class,
+    region = aoi_ee.geometry(),
+    description = 'GTB_v' + v_date + '_' + str(md.getInfo()),
+    assetId = 'projects/ee-ross-superior/assets/LS9/'+'GTB_LS9_'+str(md.getInfo())+'_v'+v_date,
+    scale = 30,
+    crs = img_crs,
+    maxPixels = 1e13)
+  
+  #Check how many existing tasks are running and take a break of 5 mins if it's >10
+  maximum_no_of_tasks(10, 5*60)
+  #Send next task.
+  export_image.start()
+
 ```
 
 

modeling/08_Sentinel2_GTB.Rmd

@@ -100,12 +100,16 @@ trainedGTB_sen = (ee.Classifier.smileGradientTreeBoost(10).train(
   classProperty = 'byte_property',
   inputProperties = sen_input_feat
 ))
-```
 
-Unfortunately, there is no current mechanism to save the GTB object, so all models
-may be *slightly* different if re-created, but will likely have very similar
-outcomes. 
+print(trainedGTB_sen.getInfo())
+```
 
+Unfortunately, there is no current mechanism to save the GTB object. This is a
+bummer because you can't really set a seed for these either, however! GEE is a bit
+more rudimentary and recognizes the inputs and therefore creates the same output
+objects. I did a quick check of this by running the model here and then again 
+in the browser. Both have identical versions, so I feel confident that GEE is
+making the 'same' model. 
 
 ## Evaluate the models
 
@@ -551,4 +555,29 @@ for d in range(date_length_sen):
   #Send next task.
   export_image.start()
 
+
+for d in range(date_length_sen):
+  md = uniqueMissDate_sen.get(d)
+  print(md.getInfo())
+  print(str(d+1) + ' of ' + str(date_length_sen))
+  image = (newStack_sen
+    .filter(ee.Filter.eq('missDate', md))
+    .first()
+    .clip(aoi_ee.geometry()))
+  image_new_class = (classifications_to_one_band(image)
+    .select('reclass'))
+  export_image = ee.batch.Export.image.toAsset(
+    image = image_new_class,
+    region = aoi_ee.geometry(),
+    description = 'GTB_v' + v_date + '_' + str(md.getInfo()),
+    assetId = 'projects/ee-ross-superior/assets/sen/'+'GTB_sen_'+str(md.getInfo())+'_v'+v_date,
+    scale = 30,
+    crs = img_crs,
+    maxPixels = 1e13)
+  
+  #Check how many existing tasks are running and take a break of 5 mins if it's >10
+  maximum_no_of_tasks(10, 5*60)
+  #Send next task.
+  export_image.start()
+
 ```