Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xarray.combine by coords with Docker #5

Closed
A-Spork opened this issue Jan 20, 2021 · 9 comments
Closed

Xarray.combine by coords with Docker #5

A-Spork opened this issue Jan 20, 2021 · 9 comments

Comments

@A-Spork
Copy link
Member

A-Spork commented Jan 20, 2021

When dockerised the wrappermethod(Collection_Sentinel_SST_Data) throws an error:

Errormessage.txt

The same command on a lokal windowsmachine goes threw without an error

@nuest
Copy link

nuest commented Jan 20, 2021

Can you please provide the exact runtime environments on your Windows machine an in the container? All package versions, Python version, etc.

@A-Spork
Copy link
Member Author

A-Spork commented Jan 20, 2021

The runtime environment of my lokal machine is attatched.

runtime environment lokal machine.txt

The runtime environment for the server will be added shortly by a member of the server-dev team

@Leg-en
Copy link
Collaborator

Leg-en commented Jan 21, 2021

The Docker Specification:
We specified all used packages for this microservice in the req.txt. If no version is specified the most recent is installed. Packages are installed with pip. We use python 3.8.6 as specified in the dockerfile. Docker is running on a windows machine with the wsl2 backend.

@nuest
Copy link

nuest commented Jan 21, 2021

"The most recent version is installed" - the differences that might occur on different systems here might help you to track down the error, so that's why I asked for the exact versions.

@nuest
Copy link

nuest commented Jan 21, 2021

I recommend you compare the two lists and see if anything sticks out.

If you'd like me to take an actual look, please prepare a minimal reproducible example. Thanks!

@Svub1 Svub1 pinned this issue Jan 25, 2021
@Svub1 Svub1 unpinned this issue Jan 25, 2021
@Leg-en
Copy link
Collaborator

Leg-en commented Jan 25, 2021

In the docker-compose branch from our GeoSoft2_Project Repository in the minExample Directory is a Example script which reproduces the error. The job.json file is also required. (https://github.com/GeoSoftII2020-21/GeoSoftII_Projekt/tree/Docker-compose) It needs the requests and docker package installed and also docker and docker compose. Also at least 12gb ram are needed. The whole log from the Server is printed in the Commandline (if exectuted in a commandline). It does not Terminate the the Server after its completion!

@nuest
Copy link

nuest commented Jan 27, 2021

Did you see pydata/xarray#4213 ?

@nuest
Copy link

nuest commented Jan 27, 2021

The problematic function is used a few times in the code: https://github.com/GeoSoftII2020-21/GeoSoftII_DataServer/search?q=xr.combine_by_coords

The problematic line in the code seems to be https://github.com/GeoSoftII2020-21/GeoSoftII_DataServer/blob/main/Wrapper/Collections_Sentinel2_SST_Data.py#L583

You should be able to force the error by writing a test that calls the function merge_datacubes(ds_merge) with the input data that you use.

@A-Spork
Copy link
Member Author

A-Spork commented Feb 3, 2021

Error solved by replacing the xArray.merge_by_coords with the xArray.open_mfdataset, which is based on the merge_by_coord function

@A-Spork A-Spork closed this as completed Feb 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants