Sourced from timm's releases.
Release v1.0.12
Nov 28, 2024
- More optimizers
- Add MARS optimizer (https://arxiv.org/abs/2411.10438, https://github.com/AGI-Arena/MARS)
- Add LaProp optimizer (https://arxiv.org/abs/2002.04839, https://github.com/Z-T-WANG/LaProp-Optimizer)
- Add masking from 'Cautious Optimizers' (https://arxiv.org/abs/2411.16085, https://github.com/kyleliang919/C-Optim) to Adafactor, Adafactor Big Vision, AdamW (legacy), Adopt, Lamb, LaProp, Lion, NadamW, RMSPropTF, SGDW
- Cleanup some docstrings and type annotations re optimizers and factory
- Add MobileNet-V4 Conv Medium models pretrained on in12k and fine-tuned in1k @ 384x384
- Add small cs3darknet, quite good for the speed
Nov 12, 2024
- Optimizer factory refactor
- New factory works by registering optimizers using an OptimInfo dataclass w/ some key traits
- Add
list_optimizers
,get_optimizer_class
,get_optimizer_info
to reworkedcreate_optimizer_v2
fn to explore optimizers, get info or class- deprecate
optim.optim_factory
, move fns tooptim/_optim_factory.py
andoptim/_param_groups.py
and encourage import viatimm.optim
- Add Adopt (https://github.com/iShohei220/adopt) optimizer
- Add 'Big Vision' variant of Adafactor (https://github.com/google-research/big_vision/blob/main/big_vision/optax.py) optimizer
- Fix original Adafactor to pick better factorization dims for convolutions
- Tweak LAMB optimizer with some improvements in torch.where functionality since original, refactor clipping a bit
- dynamic img size support in vit, deit, eva improved to support resize from non-square patch grids, thanks https://github.com/wojtke
Oct 31, 2024
Add a set of new very well trained ResNet & ResNet-V2 18/34 (basic block) weights. See https://huggingface.co/blog/rwightman/resnet-trick-or-treat
Oct 19, 2024
- Cleanup torch amp usage to avoid cuda specific calls, merge support for Ascend (NPU) devices from MengqingCao that should work now in PyTorch 2.5 w/ new device extension autoloading feature. Tested Intel Arc (XPU) in Pytorch 2.5 too and it (mostly) worked.
What's Changed
- mambaout.py: fixed bug by
@NightMachinery
in huggingface/pytorch-image-models#2305- Cleanup some amp related behaviour to better support different (non-cuda) devices by
@rwightman
in huggingface/pytorch-image-models#2308- Add NPU backend support for val and inference by
@MengqingCao
in huggingface/pytorch-image-models#2109- Update some clip pretrained weights to point to new hub locations by
@rwightman
in huggingface/pytorch-image-models#2311- ResNet vs MNV4 v1/v2 18 & 34 weights by
@rwightman
in huggingface/pytorch-image-models#2316- Replace deprecated positional argument with --data-dir by
@JosuaRieder
in huggingface/pytorch-image-models#2322- Fix typo in train.py: bathes > batches by
@JosuaRieder
in huggingface/pytorch-image-models#2321- Fix positional embedding resampling for non-square inputs in ViT by
@wojtke
in huggingface/pytorch-image-models#2317- Add trust_remote_code argument to ReaderHfds by
@grodino
in huggingface/pytorch-image-models#2326- Extend train epoch schedule by warmup_epochs if warmup_prefix enabled by
@rwightman
in huggingface/pytorch-image-models#2325- Extend existing unit tests using Cover-Agent by
@mrT23
in huggingface/pytorch-image-models#2331- An impl of adafactor as per big vision (scaling vit) changes by
@rwightman
in huggingface/pytorch-image-models#2320- Add py.typed file as recommended by PEP 561 by
@antoinebrl
in huggingface/pytorch-image-models#2252- Add CODE_OF_CONDUCT.md and CITATION.cff files by
@AlinaImtiaz018
in huggingface/pytorch-image-models#2333- Add some 384x384 small model weights by
@rwightman
in huggingface/pytorch-image-models#2334- In dist training, update loss running avg every step, sync on log by
@rwightman
in huggingface/pytorch-image-models#2340- Improve WandB logging by
@sinahmr
in huggingface/pytorch-image-models#2341
... (truncated)
553ded5
Version 1.0.12464885e
See if we can avoid some model / layer pickle issues with the aa attr in
Conv...5fe5f9d
Add a different mnv4 conv-small weight303f769
Add cautious mars, improve test reliability by skipping grad diff for
first step82e8677
Make LaProp weight decay match typical PyTorch 'decoupled' behaviour
where it...886eb77
Update README, missed small discrep in adafactor min dim updatee3e434b
To be technically correct, need to check the in-place _ ver of op7c32d3b
Work around _foreach_maximum issue, need scalar other support7cf6836
Cautious optimizer impl plus some typing cleanup.aeb1ed7
Keep basic optim test LR range closer to before w/ updated code