Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

All default pretrained weights pushed to HF hub #970

Merged
merged 3 commits into from
Oct 24, 2024
Merged

All default pretrained weights pushed to HF hub #970

merged 3 commits into from
Oct 24, 2024

Conversation

rwightman
Copy link
Collaborator

@rwightman rwightman commented Oct 23, 2024 •
edited
Loading

  • all weights not already on the hub uploaded to timm org for simplicity & safety (metaclip models were incompatible with weights_only=True).
  • OpenAI models no longer use special path that loads from torchscript archive, use same path as other models
  • Handling of QuickGELU consistent btw openai and non, made a bit more safe, warn on mismatch
  • safetensors is default weight load if available

@rwightman
All default pretrained weights pushed to HF hub, stragglers uploaded …
2a61d34 
…to timm org for simplicity.

* OpenAI models no longer use special path that loads from torchscript archive, use same path as other models
* Handling of QuickGELU consistent btw openai and non, made a bit more safe, warn on mismatch
* safetensors is default weight load if available
@rwightman rwightman mentioned this pull request Oct 23, 2024
Closed
@rwightman
Remove safeglobals add, not worth having with all pretrained weights …
c3c3d7c 
…on hub and numpy 1 vs 2 issues
@rwightman
Improve cache_dir behaviour
08991ac 
* make use of cache_dir for HF tokenizer wrapper explicit
* add missing use of cache_dir for an _get_hf_config call
* add cache_dir as argument to train/val script
@rwightman rwightman mentioned this pull request Oct 23, 2024
Closed
@bryant1410
Copy link
Contributor

IIUC, this fixes #771.

bryant1410
bryant1410 approved these changes Oct 23, 2024
View reviewed changes
Copy link
Contributor

@bryant1410 bryant1410 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked at the core changes and it LGTM, though haven't looked at all the details.

This is a great and elegant solution for both the QuickGELU error-prone issue, and the issue with MetaCLIP!!

@aivarasbaranauskas
Copy link

Fixes #966 too.

@rwightman rwightman merged commit d11e54a into main Oct 24, 2024
4 checks passed
@rwightman rwightman deleted the pretrain_cleanup branch October 24, 2024 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bryant1410 bryant1410 bryant1410 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rwightman @bryant1410 @aivarasbaranauskas