Skip to content

4.0.3

Latest
Compare
Choose a tag to compare
@ClashLuke ClashLuke released this 08 Aug 08:47
  • fix(Sign): self-graft correctly; previously, we did update.sign() * update.norm(), omitting the required division by the original norm. Now, it's F.normalize(update.sign()) * update.norm(). This changes the required learning rates for self-grafted tg.optim.Sign.