-
Notifications
You must be signed in to change notification settings - Fork 197
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Numba JIT causes up to 15s import times #222
Comments
Hi @fluffbucket, all of the functions use numba's caching option to store the compiled code: Line 11 in 32fb2ef
It should just compile once the first time you use it and then not compile again unless modified. |
Indeed that's what it does, however even loading in the already cached
methods takes that long which is why I reported it. Try measuring it.
…On Wed, 21 Jun 2023 at 18:01, Matt Bartos ***@***.***> wrote:
Hi @fluffbucket <https://github.com/fluffbucket>, all of the functions
use numba's caching option to store the compiled code:
https://github.com/mdbartos/pysheds/blob/32fb2efba94c5da4ee8e2556d484a218d352226b/pysheds/_sgrid.py#L11
It should just compile once the first time you use it and then not compile
again unless modified.
—
Reply to this email directly, view it on GitHub
<#222 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ATH2XB7WO6N4GC4TVLELOKDXMMLFNANCNFSM6AAAAAAZOIZ7HY>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Hi @fluffbucket I'm wondering if you're using Windows? I'm having issues using from pysheds.grid import Grid and it seems to crash in _sgrid.py around def _d8_catchment_iter_numba. I'm thinking it's a Windows thing maybe? |
I've noticed numba sometimes has problems on Windows with the functions that use numba implementations of linked lists and dictionaries. You may want to try using I haven't been able to reproduce the 15+ second loading times though. I suspect that you may have a caching issue that is causing everything to be re-compiled each time you import. |
Compiling sGrid and whatever else uses Numba takes up to 15seconds every time causing long bootup times, consider using AOT instead with Numba.
Measured with cProfile around importing pysheds grid and raster
The text was updated successfully, but these errors were encountered: