Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize Data on GPU #25

Open
sadamov opened this issue May 3, 2024 · 2 comments · May be fixed by #39
Open

Normalize Data on GPU #25

sadamov opened this issue May 3, 2024 · 2 comments · May be fixed by #39
Assignees
Milestone

Comments

@sadamov
Copy link
Collaborator

sadamov commented May 3, 2024

Motivation
Data normalization can be done on the fly on GPU for each batch. It's faster on GPU than CPU and cleans up the dataset init method.

Implementation
Could very nicely use https://lightning.ai/docs/pytorch/stable/common/lightning_module.html#on-after-batch-transfer to normalize once data is on GPU. Makes sure that you never forget about it (all batches on GPU are normalized).

The stats could be provided by a yaml_object handler that can be accessed on the model's init

@sadamov sadamov self-assigned this May 13, 2024
@leifdenby
Copy link
Member

sounds cool @sadamov, are you thinking this for v0.3.0 or a later release? :)

@sadamov sadamov linked a pull request May 25, 2024 that will close this issue
@sadamov
Copy link
Collaborator Author

sadamov commented May 25, 2024

This feature is ready in #39 I don't have a strong opinion about the version it should be published in. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants