Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Model.setup() with global data fails due to high memory usage on own system and CRIB #94

Closed
SarahAlidoost opened this issue Jan 29, 2024 · 1 comment · Fixed by #95
Closed

Model.setup() with global data fails due to high memory usage on own system and CRIB #94

SarahAlidoost opened this issue Jan 29, 2024 · 1 comment · Fixed by #95
Assignees
@SarahAlidoost
Labels
bug Something isn't working

Comments

@SarahAlidoost
Copy link
Member

SarahAlidoost commented Jan 29, 2024 •
edited
Loading

The model run fails on my own system and CRIB using global data. The notebook is either killed due to high memory usage or hangs without a clear error message. Even using a dask client does not fix the issue due to unmanaged memory problems. On snellius it works fine. We have to check which process uses high memory.
@SarahAlidoost SarahAlidoost added the bug Something isn't working label Jan 29, 2024
@SarahAlidoost
Copy link
Member Author

Related to #93

@SarahAlidoost SarahAlidoost self-assigned this Jan 29, 2024
@SarahAlidoost SarahAlidoost mentioned this issue Feb 2, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SarahAlidoost SarahAlidoost

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Setting chunks auto in open_mfdataset EcoExtreML/STEMMUS_SCOPE_Processing
1 participant
@SarahAlidoost