BAP - dealing with clouds and distance to other clouds?

[Antguz]

Hi community,

I am processing Best Available Pixel (BAP) data from lowland tropical forests, where clouds are always present. Thus, I am looking for guidance on this issue.

First, I have noticed that many scenes ingested in Level 2 ARD do not seem to detect certain types of cumulus clouds. For example, take a look at these OVV images derived from Landsat 8:

How can I adjust the settings to make these types of clouds more detectable? (or can be detectable?) I do not want to modify the Fmaks thresholds without discussing it first. Currently, I am using the default values.

Assuming that cloud detection is not an issue, I have noticed when calculating the BAP for a given year that the distance-to-clouds score does not seem to be applied to all types of clouds. For instance, look at these Level 3 BAP and distance score scenes where all the pixels were selected from the same Level 2 image:

It appears that there are some gaps in the data due to clouds that do not have a distance score. The documentation says distance to clouds and shadows, but similar to #224 the score may need to be applied to all the clouds.

Can I get any guide?

Thanks for you time.

AG.

[davidfrantz]

Hi AG,

this doesn't look very good in this case. Where on Earth are we? Can you send one of those scene IDs?
I am a bit curious as to why this happens...

The cloud distance issue is because the clouds were not detected.
If it cannot be solved at the level of the cloud mask, you could try using the haze score, and/or the correlation score.

Best,
David

[Antguz]

Hi David,

This is the Caribbean lowland forest across Costa Rica.

In this link, you will find some scenes from Landsat at different years.

Even using a lower Fmask threshold (0.125) clouds does not seems to be detected. Also it seems to me this is more frequent in Landsat 8.

Let me know what I can do ...

Regards.

AG.

[davidfrantz]

Hi AG.

so... the issue is FMask's strong reliance on the thermal information. In this area, the clouds appear to be very warm. There is less than 1°C temperature difference between the land and the omitted cumuli. The temperature distribution of the land also overlaps that of those clouds.

This is quite an issue here and not easy to resolve...

I tested with this image: LC08_L1TP_015053_20200204_20200823_02_T1 L2

This is the quicklook of the masks:

This is the mask produced by USGS (downloaded from EarthExplorer). It is probably a bit better, but there is still quite some underdetection:

If you remove the margin of the temperature probability entirely (CLOUD_THRESHOLD = 0), it becomes much better:

This could be an option here, although I would not recommend it as the future default. But in this particular case, it seems to help...

Best,
David

[Antguz]

Thanks David for the analysis.

When you said that CLOUD_THRESHOLD = 0 could be an option here you are recomending that for these images with this underdetection or for all the scenes that comes from this site? No sure how appropriate is to use different thresholds for the same project.

Again thanks for your guide.

AG.

[davidfrantz]

Hi AG,

I would try with one threshold first and see what happens. If you find some season-dependent results (or weather-dependent?), you could also justify different thresholds, I guess.

Best,
David