Code

Phi2 multipack#1173

Code

Merged

winglian merged 11 commits intomainaxolotl-ai-cloud/axolotl:mainfrom phi2-multipackaxolotl-ai-cloud/axolotl:phi2-multipackCopy head branch name to clipboardJan 23, 2024

+201-2,269

Commits

Commits on Jan 23, 2024

• phi2 multipack

  winglian

  committed
  494dd38

  Copy full SHA for 494dd38

• update validation and examples for phi

  winglian

  committed
  f4fc1b2

  Copy full SHA for f4fc1b2

• more updates to phi examples

  winglian

  committed
  93b8f8e

  Copy full SHA for 93b8f8e

• make sure to use the correct collator for phi multipack

  winglian

  committed
  ea9ca67

  Copy full SHA for ea9ca67

• phi needs attention mask now for multipack

  winglian

  committed
  c9fa803

  Copy full SHA for c9fa803

• if the special token already exists in the tokenizer, don't require in lora modules to save

  winglian

  committed
  06f1c55

  Copy full SHA for 06f1c55

• fix qlora yml for phi, fix phi test validation

  winglian

  committed
  ee608fe

  Copy full SHA for ee608fe

• test qlora too

  winglian

  committed
  929933f

  Copy full SHA for 929933f

• make sure flash attention is enabled for the test

  winglian

  committed
  97e06f3

  Copy full SHA for 97e06f3

• don't use remote code for phi anymore

  winglian

  committed
  4350b95

  Copy full SHA for 4350b95

• reduce sequence len for sample packing phi

  winglian

  committed
  faad02b

  Copy full SHA for faad02b