Code

Support rope scaling#1391

Code

Merged

milocress merged 24 commits intomosaicml:mainmosaicml/llm-foundry:mainfrom milocress:milo/support-llama3-ropemilocress/llm-foundry:milo/support-llama3-ropeCopy head branch name to clipboardJul 24, 2024

+111-35

Commits

Commits on Jul 23, 2024

• support rope scaling

  milocress

  committed
  8fed63a

• Merge branch 'main' into milo/support-llama3-rope

  milocress

  authored
  8c21a1f

• use rope scaling

  milocress

  committed
  aa4736f

Commits on Jul 24, 2024

• Merge branch 'main' into milo/support-llama3-rope

  milocress

  authored
  853bfa6

• update to use rope config

  milocress

  committed
  1e7d8d3

• merged

  milocress

  committed
  2b4f21f

• update config args

  milocress

  committed
  a032676

• use allowlist for config to enforce hygeine

  milocress

  committed
  0d190a2

• Merge branch 'main' into milo/support-llama3-rope

  dakinggg

  authored
  5c24b70

• allow llama3 rope config

  milocress

  committed
  604f0b9

• merged

  milocress

  committed
  77fd401

• add unit test

  milocress

  committed
  0682283

• documented allowed llama config keys

  milocress

  committed
  ff0d3de

• Update llmfoundry/models/mpt/modeling_mpt.py

  dakinggg

  authored
  ef6c8c2

• Address comments 1

  milocress

  authored
  dd1de37

• Apply suggestions from code review

  Show description for 1515708
  

  milocress

  and

  dakinggg

  authored
  1515708

• Apply suggestions from code review

  Show description for 8da6165
  

  milocress

  and

  dakinggg

  authored
  8da6165

• use same codepath for all the hf rotary embeddings

  milocress

  committed
  b0700c9

• fix

  milocress

  committed
  518a3a1

• update

  milocress

  committed
  44ce115

• test WIP but fix get/pop

  milocress

  committed
  d297395

• change the thing being popped

  milocress

  committed
  cf460ec

• give up on testing hf

  milocress

  committed
  d48bcb4

• Merge branch 'main' into milo/support-llama3-rope

  milocress

  authored
  c4051bc