Code

Gauntlet v0.1.0 yaml fixes#748

Code

Merged

bmosaicml merged 116 commits intomainmosaicml/llm-foundry:mainfrom execution_predictionmosaicml/llm-foundry:execution_predictionCopy head branch name to clipboardNov 21, 2023

+74-68

Commits

Commits on Aug 11, 2023

• update yaml

  bmosaicml

  committed
  a4c83d3

  Copy full SHA for a4c83d3

Commits on Aug 18, 2023

• adding datasets

  bmosaicml

  committed
  fbf43b0

  Copy full SHA for fbf43b0

Commits on Aug 23, 2023

• adding datasets

  bmosaicml

  committed
  7d9c367

  Copy full SHA for 7d9c367

Commits on Aug 28, 2023

• Merge branch 'model_gauntlet_v0.1' of github.com:mosaicml/llm-foundry into model_gauntlet_v0.1

  bmosaicml

  committed
  5054667

  Copy full SHA for 5054667

Commits on Sep 5, 2023

• added agi eval

  bmosaicml

  committed
  b32224a

  Copy full SHA for b32224a

• test CoT eval

  bmosaicml

  committed
  46b27a3

  Copy full SHA for 46b27a3

• fix broken eval yaml

  bmosaicml

  committed
  4d82180

  Copy full SHA for 4d82180

Commits on Sep 6, 2023

• fix broken eval yaml

  bmosaicml

  committed
  b1bbff1

  Copy full SHA for b1bbff1

Commits on Sep 7, 2023

• Merge branch 'model_gauntlet_v0.1' of github.com:mosaicml/llm-foundry into model_gauntlet_v0.1

  bmosaicml

  committed
  7554b01

  Copy full SHA for 7554b01

Commits on Sep 11, 2023

• merge main

  bmosaicml

  committed
  1845ffa

  Copy full SHA for 1845ffa

• debugging

  bmosaicml

  committed
  1508a9f

  Copy full SHA for 1508a9f

• debugging

  bmosaicml

  committed
  b10c16c

  Copy full SHA for b10c16c

Commits on Sep 28, 2023

• commit

  bmosaicml

  committed
  85f081e

  Copy full SHA for 85f081e

• commit

  bmosaicml

  committed
  de28be4

  Copy full SHA for de28be4

Commits on Sep 29, 2023

• commit

  bmosaicml

  committed
  193b4e0

  Copy full SHA for 193b4e0

• commit

  bmosaicml

  committed
  e896c09

  Copy full SHA for e896c09

• commit

  bmosaicml

  committed
  a16aea1

  Copy full SHA for a16aea1

Commits on Oct 2, 2023

• restore mcli

  bmosaicml

  committed
  6f5fc2c

  Copy full SHA for 6f5fc2c

Commits on Oct 3, 2023

• adding simple tasks

  mcarbin

  committed
  7968ca4

  Copy full SHA for 7968ca4

• Merge branch 'main' into human_eval_simple

  bmosaicml

  committed
  0f8b160

  Copy full SHA for 0f8b160

• add simple human_eval

  bmosaicml

  committed
  a84dda0

  Copy full SHA for a84dda0

• fix yaml

  bmosaicml

  committed
  90a2dca

  Copy full SHA for 90a2dca

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  4c54e3d

  Copy full SHA for 4c54e3d

Commits on Oct 4, 2023

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  bcda9ef

  Copy full SHA for bcda9ef

• fix yaml

  bmosaicml

  committed
  48060e1

  Copy full SHA for 48060e1

• Merge branch 'mike/human-eval-simple' of github.com:mcarbin/llm-foundry into human_eval_simple

  bmosaicml

  committed
  4eee5c1

  Copy full SHA for 4eee5c1

Commits on Oct 5, 2023

• remove breakpoint

  bmosaicml

  committed
  9534d73

  Copy full SHA for 9534d73

• remove breakpoint

  bmosaicml

  committed
  51d1b72

  Copy full SHA for 51d1b72

Commits on Oct 6, 2023

• change bsz

  bmosaicml

  committed
  e11bb34

  Copy full SHA for e11bb34

Commits on Oct 9, 2023

• Merge branch 'main' into human_eval_simple

  bmosaicml

  committed
  fb750f3

  Copy full SHA for fb750f3

• merge main

  bmosaicml

  committed
  b50da7d

  Copy full SHA for b50da7d

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  4b29f92

  Copy full SHA for 4b29f92

• Merge branch 'main' into change_gauntlet_avging

  bmosaicml

  committed
  07c2c07

  Copy full SHA for 07c2c07

• eval gauntlet cb

  bmosaicml

  committed
  841eac0

  Copy full SHA for 841eac0

• Merge branch 'main' into change_gauntlet_avging

  bmosaicml

  authored
  94100fb

  Copy full SHA for 94100fb

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  211def8

  Copy full SHA for 211def8

Commits on Oct 10, 2023

• Merge branch 'main' into change_gauntlet_avging

  dakinggg

  authored
  5cf9b89

  Copy full SHA for 5cf9b89

• Merge branch 'main' into change_gauntlet_avging

  dakinggg

  authored
  9159dc8

  Copy full SHA for 9159dc8

• Merge branch 'main' into mike/human-eval-simple

  dakinggg

  authored
  043f473

  Copy full SHA for 043f473

• Merge branch 'main' into change_gauntlet_avging

  dakinggg

  authored
  d1744fe

  Copy full SHA for d1744fe

• add udpated readme

  bmosaicml

  committed
  cdc2065

  Copy full SHA for cdc2065

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  eb09668

  Copy full SHA for eb09668

• Merge branch 'mike/human-eval-simple' of github.com:mcarbin/llm-foundry into human_eval_simple

  bmosaicml

  committed
  cdbd3d1

  Copy full SHA for cdbd3d1

• Merge branch 'main' into change_gauntlet_avging

  bmosaicml

  authored
  adeb790

  Copy full SHA for adeb790

• merge

  bmosaicml

  committed
  26107e8

  Copy full SHA for 26107e8

• Merge branch 'main' into mike/human-eval-simple

  bmosaicml

  authored
  2fd5c16

  Copy full SHA for 2fd5c16

• merge2

  bmosaicml

  committed
  c8a21c6

  Copy full SHA for c8a21c6

• fix precommit

  bmosaicml

  committed
  71cd420

  Copy full SHA for 71cd420

• Merge branch 'mike/human-eval-simple' of github.com:mcarbin/llm-foundry into human_eval_simple

  bmosaicml

  committed
  ff68b53

  Copy full SHA for ff68b53

• Merge branch 'main' into change_gauntlet_avging

  bmosaicml

  authored
  bd8a9f8

  Copy full SHA for bd8a9f8

Commits on Oct 11, 2023

• add pii

  bmosaicml

  committed
  47ebdfc

  Copy full SHA for 47ebdfc

• restor line

  bmosaicml

  committed
  2d17293

  Copy full SHA for 2d17293

• restor line

  bmosaicml

  committed
  8b7abf1

  Copy full SHA for 8b7abf1

• finish non gen safety

  bmosaicml

  committed
  ed3f52a

  Copy full SHA for ed3f52a

• add execution predicrtion

  bmosaicml

  committed
  f96bbce

  Copy full SHA for f96bbce

• add execution prediction

  bmosaicml

  committed
  6ff9133

  Copy full SHA for 6ff9133

• add execution prediction

  bmosaicml

  committed
  bb7a847

  Copy full SHA for bb7a847

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  6ee0f48

  Copy full SHA for 6ee0f48

• change mosaicml reqs

  bmosaicml

  committed
  f332091

  Copy full SHA for f332091

• change mosaicml reqs

  bmosaicml

  committed
  bb89c97

  Copy full SHA for bb89c97

• fix error

  bmosaicml

  committed
  966b1bb

  Copy full SHA for 966b1bb

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  977b48f

  Copy full SHA for 977b48f

• comment

  bmosaicml

  committed
  e15246e

  Copy full SHA for e15246e

Commits on Oct 12, 2023

• test smaller beams

  bmosaicml

  committed
  e40463b

  Copy full SHA for e40463b

• tesT

  bmosaicml

  committed
  4a4d021

  Copy full SHA for 4a4d021

• tesT

  bmosaicml

  committed
  c5e30ec

  Copy full SHA for c5e30ec

Commits on Oct 13, 2023

• tesT

  bmosaicml

  committed
  19c8b2b

  Copy full SHA for 19c8b2b

• Merge branch 'change_gauntlet_avging' into execution_prediction

  bmosaicml

  committed
  441b004

  Copy full SHA for 441b004

• add coding task

  bmosaicml

  committed
  3025517

  Copy full SHA for 3025517

• tesT

  bmosaicml

  committed
  5e444e3

  Copy full SHA for 5e444e3

Commits on Oct 16, 2023

• finish eval

  bmosaicml

  committed
  b55bdda

  Copy full SHA for b55bdda

Commits on Oct 19, 2023

• finish data

  bmosaicml

  committed
  2d77a0e

  Copy full SHA for 2d77a0e

• fix

  bmosaicml

  committed
  c025ac4

  Copy full SHA for c025ac4

Commits on Oct 24, 2023

• fix

  bmosaicml

  committed
  2702e15

  Copy full SHA for 2702e15

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  d393603

  Copy full SHA for d393603

• remove strategyqa cot

  bmosaicml

  committed
  0350950

  Copy full SHA for 0350950

• merge

  bmosaicml

  committed
  27754e8

  Copy full SHA for 27754e8

Commits on Oct 25, 2023

• remove

  bmosaicml

  committed
  b00a806

  Copy full SHA for b00a806

Commits on Oct 26, 2023

• remove

  bmosaicml

  committed
  7b68b55

  Copy full SHA for 7b68b55

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  57e35cd

  Copy full SHA for 57e35cd

• foo

  bmosaicml

  committed
  814a8c9

  Copy full SHA for 814a8c9

Commits on Oct 27, 2023

• edit

  bmosaicml

  committed
  b142b57

  Copy full SHA for b142b57

Commits on Nov 2, 2023

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  a56b30e

  Copy full SHA for a56b30e

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  368d232

  Copy full SHA for 368d232

• fix

  bmosaicml

  committed
  d86ece1

  Copy full SHA for d86ece1

• rm breakpoint

  bmosaicml

  committed
  0477773

  Copy full SHA for 0477773

Commits on Nov 3, 2023

• rm breakpoint

  bmosaicml

  committed
  7fed6a2

  Copy full SHA for 7fed6a2

Commits on Nov 9, 2023

• remove execution prediction; make coding optional

  bmosaicml

  committed
  48bf32b

  Copy full SHA for 48bf32b

• remove execution prediction; make coding optional

  bmosaicml

  committed
  f2f59df

  Copy full SHA for f2f59df

• merge

  bmosaicml

  committed
  01b3103

  Copy full SHA for 01b3103

• remove import

  bmosaicml

  committed
  20aeafd

  Copy full SHA for 20aeafd

• remove import

  bmosaicml

  committed
  46b7d72

  Copy full SHA for 46b7d72

Commits on Nov 10, 2023

• restore files

  bmosaicml

  committed
  93fa556

  Copy full SHA for 93fa556

• restore

  bmosaicml

  committed
  f17f8c4

  Copy full SHA for f17f8c4

• restore

  bmosaicml

  committed
  a9606b4

  Copy full SHA for a9606b4

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  cd08024

  Copy full SHA for cd08024

Commits on Nov 12, 2023

• update readm; rename gauntlet yamls

  bmosaicml

  committed
  f0c0b9d

  Copy full SHA for f0c0b9d

• edit yamls

  bmosaicml

  committed
  4572056

  Copy full SHA for 4572056

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  297bfa2

  Copy full SHA for 297bfa2

Commits on Nov 13, 2023

• fix yamllint

  bmosaicml

  committed
  e6b696f

  Copy full SHA for e6b696f

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  a1f8ffb

  Copy full SHA for a1f8ffb

• restore mpt eval

  bmosaicml

  committed
  6014dd4

  Copy full SHA for 6014dd4

Commits on Nov 17, 2023

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  90ef388

  Copy full SHA for 90ef388

Commits on Nov 20, 2023

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  1357cb8

  Copy full SHA for 1357cb8

• finish

  bmosaicml

  committed
  65d9604

  Copy full SHA for 65d9604

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  dc525be

  Copy full SHA for dc525be

• follow up

  bmosaicml

  committed
  d5f3139

  Copy full SHA for d5f3139

• fix

  bmosaicml

  committed
  00a0426

  Copy full SHA for 00a0426

Commits on Nov 21, 2023

• Merge branch 'main' into execution_prediction

  dakinggg

  authored
  5e2101d

  Copy full SHA for 5e2101d

• precommit

  bmosaicml

  committed
  4bc2594

  Copy full SHA for 4bc2594

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  6eb14a5

  Copy full SHA for 6eb14a5

• Merge branch 'main' into execution_prediction

  bmosaicml

  authored
  c1e2fc9

  Copy full SHA for c1e2fc9

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  970a4df

  Copy full SHA for 970a4df

• Merge branch 'main' into execution_prediction

  dakinggg

  authored
  0d650c1

  Copy full SHA for 0d650c1

• precommit

  bmosaicml

  committed
  afbd992

  Copy full SHA for afbd992

• Merge branch 'execution_prediction' of github.com:mosaicml/llm-foundry into execution_prediction

  bmosaicml

  committed
  fd890fe

  Copy full SHA for fd890fe