Code

Validation#844

Code

Open

XiaohanZhangCMU wants to merge 126 commits intomosaicml:mainmosaicml/llm-foundry:mainfrom XiaohanZhangCMU:validationXiaohanZhangCMU/llm-foundryX:validationCopy head branch name to clipboard

+2,356-7

Commits

Commits on Dec 23, 2023

• add validation script

  xiaohanzhan-db

  committed
  8cb6522

  Copy full SHA for 8cb6522

Commits on Jan 3, 2024

• update

  xiaohanzhan-db

  committed
  c59c11f

  Copy full SHA for c59c11f

• change token count function

  

  xiaohanzhan-db

  committed
  66f34eb

  Copy full SHA for 66f34eb

Commits on Jan 5, 2024

• reorganize cells

  

  xiaohanzhan-db

  committed
  2cd387b

  Copy full SHA for 2cd387b

• Add unit tests

  xiaohanzhan-db

  committed
  3eac3bf

  Copy full SHA for 3eac3bf

Commits on Jan 6, 2024

• Add a printout for CPT

  xiaohanzhan-db

  committed
  d2d9767

  Copy full SHA for d2d9767

• update question

  xiaohanzhan-db

  committed
  be25591

  Copy full SHA for be25591

Commits on Jan 8, 2024

• Add questions

  

  xiaohanzhan-db

  committed
  4651be7

  Copy full SHA for 4651be7

• Fix lints

  xiaohanzhan-db

  committed
  5cd6a94

  Copy full SHA for 5cd6a94

• Merge branch 'main' into validation

  XiaohanZhangCMU

  authored
  8e2c1f4

  Copy full SHA for 8e2c1f4

• update format

  xiaohanzhan-db

  committed
  e6e4a81

  Copy full SHA for e6e4a81

• Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX into validation

  xiaohanzhan-db

  committed
  34c5690

  Copy full SHA for 34c5690

• update

  xiaohanzhan-db

  committed
  1668b9a

  Copy full SHA for 1668b9a

• nb source

  xiaohanzhan-db

  committed
  2219135

  Copy full SHA for 2219135

• add validation script

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  86c6e87

  Copy full SHA for 86c6e87

• update

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  678b376

  Copy full SHA for 678b376

• change token count function

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  297e057

  Copy full SHA for 297e057

• reorganize cells

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  09d0ebb

  Copy full SHA for 09d0ebb

• Add unit tests

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  460df65

  Copy full SHA for 460df65

• Add a printout for CPT

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  3ffd200

  Copy full SHA for 3ffd200

• update question

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  9362886

  Copy full SHA for 9362886

• Add questions

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  898e5ac

  Copy full SHA for 898e5ac

• Fix lints

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  a4bef71

  Copy full SHA for a4bef71

• update format

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  4ca9cc6

  Copy full SHA for 4ca9cc6

• update

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  d636a0f

  Copy full SHA for d636a0f

• nb source

  

  xiaohanzhan-db

  authored and

  XiaohanZhangCMU

  committed
  827d155

  Copy full SHA for 827d155

• Remove license insert for validation notebook

  xiaohanzhan-db

  committed
  6bbf3fc

  Copy full SHA for 6bbf3fc

• Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX into validation

  xiaohanzhan-db

  committed
  4f6a4fb

  Copy full SHA for 4f6a4fb

Commits on Jan 11, 2024

• Add validation utils

  xiaohanzhan-db

  committed
  5966b68

  Copy full SHA for 5966b68

• Merge branch 'main' into validation

  xiaohanzhan-db

  committed
  da17813

  Copy full SHA for da17813

• Validation (mosaicml#856)

  Show description for 89fb909
  

  XiaohanZhangCMU

  and

  xiaohanzhan-db

  authored
  89fb909

  Copy full SHA for 89fb909

• update utils/__init__.py to include extra validation functions

  xiaohanzhan-db

  committed
  55e4626

  Copy full SHA for 55e4626

• update notebook

  

  xiaohanzhan-db

  committed
  45544a1

  Copy full SHA for 45544a1

• update

  xiaohanzhan-db

  committed
  d2797b3

  Copy full SHA for d2797b3

• Merge branch 'validation' of github.com:XiaohanZhangCMU/llm-foundryX into validation

  xiaohanzhan-db

  committed
  019da77

  Copy full SHA for 019da77

• update

  xiaohanzhan-db

  committed
  756fdae

  Copy full SHA for 756fdae

• Add download remote function to util

  xiaohanzhan-db

  committed
  93b5a9f

  Copy full SHA for 93b5a9f

• update

  xiaohanzhan-db

  committed
  b47c878

  Copy full SHA for b47c878

• update

  xiaohanzhan-db

  committed
  13fd34c

  Copy full SHA for 13fd34c

• update

  xiaohanzhan-db

  committed
  610f669

  Copy full SHA for 610f669

• update

  xiaohanzhan-db

  committed
  9f2e51b

  Copy full SHA for 9f2e51b

• update

  xiaohanzhan-db

  committed
  ec68f10

  Copy full SHA for ec68f10

• update

  xiaohanzhan-db

  committed
  1e76068

  Copy full SHA for 1e76068

• update

  xiaohanzhan-db

  committed
  7a5c164

  Copy full SHA for 7a5c164

• Merge branch 'main' into validation

  xiaohanzhan-db

  committed
  e76038f

  Copy full SHA for e76038f

• update

  xiaohanzhan-db

  committed
  5b413f5

  Copy full SHA for 5b413f5

• update

  xiaohanzhan-db

  committed
  a1aa31f

  Copy full SHA for a1aa31f

• update

  xiaohanzhan-db

  committed
  d24fd5c

  Copy full SHA for d24fd5c

Commits on Jan 12, 2024

• Add dask and dataframe_to_mds

  xiaohanzhan-db

  committed
  55fce37

  Copy full SHA for 55fce37

• update

  xiaohanzhan-db

  committed
  86e2412

  Copy full SHA for 86e2412

• update

  xiaohanzhan-db

  committed
  bbfec65

  Copy full SHA for bbfec65

• update

  xiaohanzhan-db

  committed
  b2e880d

  Copy full SHA for b2e880d

• update

  xiaohanzhan-db

  committed
  596443a

  Copy full SHA for 596443a

• Add notebook

  xiaohanzhan-db

  committed
  ea65187

  Copy full SHA for ea65187

• update

  xiaohanzhan-db

  committed
  378a4e0

  Copy full SHA for 378a4e0

• update

  

  xiaohanzhan-db

  committed
  af6e9aa

  Copy full SHA for af6e9aa

• remove script and tests, keep notebook

  xiaohanzhan-db

  committed
  4e286ec

  Copy full SHA for 4e286ec

• update

  xiaohanzhan-db

  committed
  09c4892

  Copy full SHA for 09c4892

• update

  xiaohanzhan-db

  committed
  c82da6c

  Copy full SHA for c82da6c

• update

  xiaohanzhan-db

  committed
  e5f83cc

  Copy full SHA for e5f83cc

• update

  xiaohanzhan-db

  committed
  17d2b9f

  Copy full SHA for 17d2b9f

• Merge branch 'main' into validation

  xiaohanzhan-db

  committed
  6579d55

  Copy full SHA for 6579d55

• Merge branch 'byod/data_validation' into validation

  XiaohanZhangCMU

  authored
  56308ff

  Copy full SHA for 56308ff

• Validation (mosaicml#862)

  Show description for 00a51b5
  

  3 peopleauthored
  00a51b5

  Copy full SHA for 00a51b5

• updated notebook

  

  xiaohanzhan-db

  committed
  4daa324

  Copy full SHA for 4daa324

• Merge branch 'main' into validation

  xiaohanzhan-db

  committed
  b809691

  Copy full SHA for b809691

• remove scripts keep notebook

  xiaohanzhan-db

  committed
  8b75f94

  Copy full SHA for 8b75f94

• merge with byod/data_validation

  xiaohanzhan-db

  committed
  99bf2cd

  Copy full SHA for 99bf2cd

• Validation (mosaicml#866)

  Show description for 9b37063
  

  3 peopleauthored
  9b37063

  Copy full SHA for 9b37063

• update notebook. rephrase.

  

  xiaohanzhan-db

  committed
  22014d6

  Copy full SHA for 22014d6

• merged

  xiaohanzhan-db

  committed
  d9f28aa

  Copy full SHA for d9f28aa

• Validation (mosaicml#867)

  Show description for f1fa63c
  

  3 peopleauthored
  f1fa63c

  Copy full SHA for f1fa63c

• update

  xiaohanzhan-db

  committed
  43c8ac9

  Copy full SHA for 43c8ac9

Commits on Jan 16, 2024

• Add response tokens

  xiaohanzhan-db

  committed
  b8ac771

  Copy full SHA for b8ac771

• update

  xiaohanzhan-db

  committed
  1b9681c

  Copy full SHA for 1b9681c

• merge

  xiaohanzhan-db

  committed
  16883c2

  Copy full SHA for 16883c2

• Validation (mosaicml#875)

  Show description for a9218d6
  

  3 peopleauthored
  a9218d6

  Copy full SHA for a9218d6

Commits on Jan 20, 2024

• update

  xiaohanzhan-db

  committed
  c7567f1

  Copy full SHA for c7567f1

Commits on Jan 22, 2024

• Disable MDSWrite, return token counts

  xiaohanzhan-db

  committed
  1764b72

  Copy full SHA for 1764b72

Commits on Jan 23, 2024

• Change plot settings

  xiaohanzhan-db

  committed
  808ced5

  Copy full SHA for 808ced5

• Fix conflict

  xiaohanzhan-db

  committed
  26ae516

  Copy full SHA for 26ae516

• update notebook

  

  xiaohanzhan-db

  committed
  a212ee8

  Copy full SHA for a212ee8

• update

  xiaohanzhan-db

  committed
  d279817

  Copy full SHA for d279817

• Validation (mosaicml#898)

  Show description for f1cfe9e
  

  3 peopleauthored
  f1cfe9e

  Copy full SHA for f1cfe9e

• update notebook

  

  xiaohanzhan-db

  committed
  dbe3f4e

  Copy full SHA for dbe3f4e

• update

  xiaohanzhan-db

  committed
  3005718

  Copy full SHA for 3005718

• Validation (mosaicml#900)

  Show description for 8498662
  

  3 peopleauthored
  8498662

  Copy full SHA for 8498662

• update

  

  xiaohanzhan-db

  committed
  f5b900c

  Copy full SHA for f5b900c

• Merge branch 'byod/data_validation' of https://github.com/mosaicml/llm-foundry into validation

  xiaohanzhan-db

  committed
  02d0979

  Copy full SHA for 02d0979

• Validation (mosaicml#901)

  Show description for 205e405
  

  3 peopleauthored
  205e405

  Copy full SHA for 205e405

• update notebook

  

  xiaohanzhan-db

  committed
  2f883a7

  Copy full SHA for 2f883a7

• update

  xiaohanzhan-db

  committed
  0315caf

  Copy full SHA for 0315caf

Commits on Mar 13, 2024

• update pip install link

  xiaohanzhan-db

  committed
  1a510ff

  Copy full SHA for 1a510ff

• Change done file location

  xiaohanzhan-db

  committed
  530a55a

  Copy full SHA for 530a55a

• Validation (mosaicml#902)

  Show description for 5493295
  

  3 peopleauthored
  5493295

  Copy full SHA for 5493295

• Create the dest folder

  xiaohanzhan-db

  committed
  81c3757

  Copy full SHA for 81c3757

• Validation (mosaicml#1025)

  Show description for 5090e13
  

  3 peopleauthored
  5090e13

  Copy full SHA for 5090e13

Commits on Mar 14, 2024

• update notebook

  xiaohanzhan-db

  committed
  f88917d

  Copy full SHA for f88917d

• update

  xiaohanzhan-db

  committed
  4c86f74

  Copy full SHA for 4c86f74

• Merge branch 'byod/data_validation' into validation

  XiaohanZhangCMU

  authored
  962974b

  Copy full SHA for 962974b

• Validation (mosaicml#1027)

  Show description for 9fd91cf
  

  3 peopleauthored
  9fd91cf

  Copy full SHA for 9fd91cf

• Merge pull request #1 from mosaicml/byod/data_validation

  Show description for 67f7b4c
  XiaohanZhangCMU

  authored
  67f7b4c

  Copy full SHA for 67f7b4c

• update notebook

  xiaohanzhan-db

  committed
  28cd2e6

  Copy full SHA for 28cd2e6

• Validation (mosaicml#1028)

  Show description for 944b260
  

  3 peopleauthored
  944b260

  Copy full SHA for 944b260

• fix conflict

  xiaohanzhan-db

  committed
  9a19d8a

  Copy full SHA for 9a19d8a

• Validation (mosaicml#1031)

  Show description for a6b2ae0
  

  28 peopleauthored
  a6b2ae0

  Copy full SHA for a6b2ae0

• update token_counts

  xiaohanzhan-db

  committed
  de90934

  Copy full SHA for de90934

• Validation (mosaicml#1032)

  Show description for 5dfd30c
  

  28 peopleauthored
  5dfd30c

  Copy full SHA for 5dfd30c

• update pip install list

  xiaohanzhan-db

  committed
  61adb43

  Copy full SHA for 61adb43

• Validation (mosaicml#1033)

  Show description for c404dc7
  

  28 peopleauthored
  c404dc7

  Copy full SHA for c404dc7

• fix

  xiaohanzhan-db

  committed
  c77bdf6

  Copy full SHA for c77bdf6

• update

  xiaohanzhan-db

  committed
  ad71cc0

  Copy full SHA for ad71cc0

• fix token counts

  xiaohanzhan-db

  committed
  9bc3a39

  Copy full SHA for 9bc3a39

• Expose validate chat

  xiaohanzhan-db

  committed
  9ec582e

  Copy full SHA for 9ec582e

• Expose more

  xiaohanzhan-db

  committed
  734008e

  Copy full SHA for 734008e

• update

  xiaohanzhan-db

  committed
  51f2eef

  Copy full SHA for 51f2eef

• expose

  xiaohanzhan-db

  committed
  7b6956d

  Copy full SHA for 7b6956d

• add collate

  xiaohanzhan-db

  committed
  60ed7de

  Copy full SHA for 60ed7de

• Fix

  xiaohanzhan-db

  committed
  fba1dcb

  Copy full SHA for fba1dcb

• Fix conflict

  xiaohanzhan-db

  committed
  58185ba

  Copy full SHA for 58185ba

• Validation (mosaicml#1034)

  Show description for 8e8f431
  

  28 peopleauthored
  8e8f431

  Copy full SHA for 8e8f431

• update notebook

  xiaohanzhan-db

  committed
  24f3d9e

  Copy full SHA for 24f3d9e

• Fix conflict

  xiaohanzhan-db

  committed
  714002d

  Copy full SHA for 714002d

• Validation (mosaicml#1035)

  Show description for 1640f30
  

  28 peopleauthored
  1640f30

  Copy full SHA for 1640f30

• Merge branch 'byod/data_validation' of https://github.com/mosaicml/llm-foundry into validation

  xiaohanzhan-db

  committed
  b053363

  Copy full SHA for b053363

Commits on Mar 15, 2024

• update notebook

  xiaohanzhan-db

  committed
  7e1d567

  Copy full SHA for 7e1d567