Added EarthSurfaceWater tutorial

[cordmaur]

Here is the tutorial as demanded by James.

[ashnair1]

Thanks a lot for the tutorial @cordmaur.

Could you change the name of the tutorial to earthsurfacewater.ipynb and add the tutorial to docs/index.rst?

[adamjstewart]

I would actually name it earth_surface_water.ipynb

P.S. I'll try to review this in more detail when I get a chance. We're currently rushing to finish a new TorchGeo release in the next week or two so we have a long list of PRs to merge before then.

[adamjstewart]

@cordmaur is this PR still a work in progress? If so you need to accept the CLA and make a few changes to get the tests working.

[cordmaur]

@microsoft-github-policy-service agree

[cordmaur]

Hi @adamjstewart. I finished updating the tutorial. I could get all tests passing, except for the docs. I added it to the docs/index.rst, but following the errors in my local make html there are issues in demo.rst.

[adamjstewart]

I think if you rename one of the images from [Image] to something else, it should be fine. It's just complaining that the alt text is the same for multiple images.

The time our notebook tests take to run increased from 9 min to 29 min. We'll have to find a way to make the tests run faster (either download less data or monkeypatch to run for fewer epochs/steps).

[cordmaur]

Hi, @adamjstewart, all tests are passing.
Notebook checks are running on 10min...

[adamjstewart]

Thanks, will give this a detailed review as soon as I get a chance!

[adamjstewart]

First round of reviews. Overall, looks great!

FYI: we are presenting a pre-conference workshop (basically a tutorial) on TorchGeo at AGU this weekend. I would love to finish and merge this PR soon so we can include it in the talk. If you don't mind, I can make direct changes to your PR branch tomorrow night after I finish my other sections. But anything you can fix before then would save me a ton of time.

[adamjstewart]

Can we remove the timeout now that the tutorial is fast?

[cordmaur]

Removed the timeout limit and it's still passing.

[adamjstewart]

You can remove the button, we automatically add buttons to all tutorials

[cordmaur]

Removed

[adamjstewart]

I'll probably reformat these to match our other tutorials if you don't mind. I definitely still want to keep your name and links to the original 3-part Medium blog, but would rather not add LinkedIn and Medium accounts if you don't mind.

[cordmaur]

Removed the links.

[cordmaur]

Ok. Removed.

[adamjstewart]

Need an empty newline here, it isn't rendering correctly: https://torchgeo--960.org.readthedocs.build/en/960/tutorials/earth_surface_water.html

[adamjstewart]

Might move this down to the bottom as an Additional Reading section with links to all 3 tutorials.

[cordmaur]

Moved to the Additional Reading secion (at the bottom)

[cordmaur]

Moved to Additional Reading.

[adamjstewart]

Suggested change

	"# Earth Water Surface Tutorial\n",
	"# Earth Surface Water\n",

[adamjstewart]

Let's use a math expression instead, this is very blurry

[cordmaur]

Done.

[adamjstewart]

Can we use kornia.augmentation.Normalize instead? Looks identical

[cordmaur]

Indeed... just had to "workaround" to avoid normalizing the spectral indices...

# normalize = MyNormalize(*calc_statistics(train_imgs))
mean, std = calc_statistics(train_imgs)

# Please, note that we will create spectral indices using the raw (non-normalized) data. Then, when normalizing, the sensors will have more channels (the indices) that should not be normalized.
# To solve this, we will add the indices to the 0's to the mean vector and 1's to the std vectors

mean = np.concat([mean, [0, 0, 0]])
std = np.concat([std, [1, 1, 1]])

norm = K.Normalize(mean=mean, std=std)

tfms = torch.nn.Sequential(
    indices.AppendNDWI(index_green=1, index_nir=3),
    indices.AppendNDWI(index_green=1, index_nir=5),
    indices.AppendNDVI(index_nir=3, index_red=2),
    norm,
)

[adamjstewart]

Could probably replace a lot of this with torchmetrics

[cordmaur]

I can't see where you suggested this....

[cordmaur]

I don't see exactly where you commented this... maybe it's to replace the OA and IOU?

[adamjstewart]

Yes, OA and IoU.

[adamjstewart]

Would be nice if we could end with evaluation (how well does it work on the test set) and inference (can we generate a predicted map for a new region).

[cordmaur]

How can we do this if we don't have enough time to "train" the model during commit?

[adamjstewart]

We'll likely need to use a pre-trained model end transfer learning. We can test for 1 epoch during CI and many epochs for the actual tutorial. See my Introduction to PyTorch tutorial PR for details.

[cordmaur]

Hi @adamjstewart . Solved most of the requests.
Corrected dataset is being hosted on my hugging face because I didnt have the rights to torchgeo repository. You can move it or grant me the rights.

I think it's only missing the scikit dependency, which can be resolved using torchmetrics and the final evaluation... Feel free to change these directly. I will only hava time to work on those tomorrow...

[adamjstewart]

Thanks for the contribution, glad we finally got this in!