Kaizhao Liang

kz919

AI & ML interests

Multimodal foundational model

Recent Activity

Organizations

SambaNova Systems's profile picture Ontocord's M*DEL's profile picture Sambanova-Gradio-Hackathon's profile picture

kz919's activity

liked a Space 16 days ago
reacted to rwightman's post with ๐Ÿ”ฅ๐Ÿš€ 22 days ago
view post
Post
1309
There's a new timm release, v 1.0.12, with a focus on optimizers. The optimizer factory has been refactored, there's now a timm.optim.list_optimizers() and new way to register optimizers and their attributes. As always you can use an timm optimizer like a torch one, just replace torch.optim with timm.optim

New optimizers include:
* AdafactorBigVision - adfactorbv
* ADOPT - adopt / adoptw (decoupled decay)
* MARS - mars
* LaProp - laprop
* Cautious Optimizers - a modification to all of the above, prefix with c as well as cadamw, cnadamw, csgdw, clamb, crmsproptf

I shared some caution comparisons in this model repo: rwightman/timm-optim-caution

For details, references, see the code: https://github.com/huggingface/pytorch-image-models/tree/main/timm/optim

  • 3 replies
ยท
reacted to qq8933's post with ๐Ÿ”ฅ 22 days ago
view post
Post
3037
  • 3 replies
ยท