Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve checks and tests for preprocess.py #564

Merged
merged 8 commits into from
Dec 12, 2023
Merged

Improve checks and tests for preprocess.py #564

merged 8 commits into from
Dec 12, 2023

Conversation

bw4sz
Copy link
Collaborator

@bw4sz bw4sz commented Dec 5, 2023

This is a follow to PR #552, we needed a test, and to warn users if rasters were in Channel x height x width before trying to coerce them to height x width x channel. I also changed base_dir to save_dir based on the deprecation warning and changed to use tmpdir when we aren't check the output of preprocess.split_raster.

@bw4sz
Copy link
Collaborator Author

bw4sz commented Dec 5, 2023

Not sure why the tag push is this PR?

@bw4sz
Copy link
Collaborator Author

bw4sz commented Dec 5, 2023

This isn't quite right yet, I was testing this on hpg,

(base) [b.weinstein@c39a-s5 MillionTrees]$ conda activate DeepTreeAttention
(DeepTreeAttention) [b.weinstein@c39a-s5 MillionTrees]$  cd /home/b.weinstein/MillionTrees ; /usr/bin/env /orange/ewhite/b.weinstein/miniconda3/envs/DeepTreeAttention/bin/python /home/b.weinstein/.vscode-server/extensions/ms-python.python-2023.20.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 44399 -- /home/b.weinstein/MillionTrees/data_prep/create_train_test.py 
/orange/ewhite/b.weinstein/miniconda3/envs/DeepTreeAttention/lib/python3.11/site-packages/deepforest/preprocess.py:192: UserWarning: Input rasterio had shape (40665, 84370, 4), assuming channels first. Converting to channels last
  warnings.warn(
/orange/ewhite/b.weinstein/miniconda3/envs/DeepTreeAttention/lib/python3.11/site-packages/deepforest/preprocess.py:200: UserWarning: Input rasterio had non-3 band shape of (84370, 4, 40665), ignoring alpha channel
  warnings.warn("Input rasterio had non-3 band shape of {}, ignoring "
/orange/ewhite/b.weinstein/miniconda3/envs/DeepTreeAttention/lib/python3.11/site-packages/deepforest/preprocess.py:203: RuntimeWarning: invalid value encountered in cast
  numpy_image = numpy_image[:, :, :3].astype("uint8")

@bw4sz
Copy link
Collaborator Author

bw4sz commented Dec 5, 2023

Okay, @henrykironde should be ready.

Copy link
Contributor

@henrykironde henrykironde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bw4sz this looks good, I have made a comment on the tests.

The version bump is from the settings we gave the package in setup.cfg

numpy_image = np.zeros((4, 400, 400), dtype=np.uint8)

with pytest.warns(UserWarning):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ Ben Why are we repeating the procedure ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once for when the channel order channels x height x width and once for when the channel order is height x width x channels. The function should work and warn users in either situation. Before it was assuming the correct shape, which is channel x height x width, without actually checking it was true.

@bw4sz
Copy link
Collaborator Author

bw4sz commented Dec 5, 2023

@bw4sz this looks good, I have made a comment on the tests.

The version bump is from the settings we gave the package in setup.cfg

Yes, but we already pushed that. 1.3.1 is already up on pypi,

https://pypi.org/project/deepforest/

image

@bw4sz
Copy link
Collaborator Author

bw4sz commented Dec 8, 2023

@henrykironde I could use this next week, let me if there are concerns here.

* Use pytest.mark.parametrize for multiple test instances
@henrykironde henrykironde merged commit d81c71d into main Dec 12, 2023
6 checks passed
@henrykironde henrykironde deleted the numpy_order branch December 12, 2023 04:49
janjatovic pushed a commit to Treeconomy/DeepForest_new that referenced this pull request Mar 26, 2024
* Bump version: 1.3.0 → 1.3.1

* rasterio reads in a .tif, its channels first, CxHxW
  improve warnings and 4 band checking in preprocess
  pytest.mark.parametrize for multiple test instances

---------

Co-authored-by: henry senyondo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants