Code

Evals#269

Code

Merged

MadcowD merged 109 commits intomainMadcowD/ell:mainfrom wguss/metricsMadcowD/ell:wguss/metricsCopy head branch name to clipboardNov 19, 2024

+9,439-725

Commits

Commits on Oct 3, 2024

• more metrics rambling

  MadcowD

  committed
  f471ca2

  Copy full SHA for f471ca2

• basic evaluation class

  MadcowD

  committed
  4ab83de

  Copy full SHA for 4ab83de

• todos

  MadcowD

  committed
  9cf8635

  Copy full SHA for 9cf8635

• fix bedrock

  MadcowD

  committed
  a6100a7

  Copy full SHA for a6100a7

• more ramblings

  MadcowD

  committed
  f622740

  Copy full SHA for f622740

Commits on Oct 4, 2024

• switch to logging

  MadcowD

  committed
  3a699c2

  Copy full SHA for 3a699c2

• more ramblings

  MadcowD

  committed
  b21afd9

  Copy full SHA for b21afd9

• thoughts

  MadcowD

  committed
  1a6ef0d

  Copy full SHA for 1a6ef0d

• working evals

  MadcowD

  committed
  36e81fb

  Copy full SHA for 36e81fb

• eval update

  MadcowD

  committed
  1048e90

  Copy full SHA for 1048e90

• Delete docs/ramblings/test.py

  MadcowD

  authored
  77cb872

  Copy full SHA for 77cb872

• Delete docs/ramblings/typeddictpartial.py

  MadcowD

  authored
  cd64ab9

  Copy full SHA for cd64ab9

• eval update

  MadcowD

  committed
  6afad20

  Copy full SHA for 6afad20

Commits on Oct 5, 2024

• cleaner specification of criteria

  MadcowD

  committed
  3e2a60a

  Copy full SHA for 3e2a60a

• more spec discussions around chat and yielding

  MadcowD

  committed
  17dc98e

  Copy full SHA for 17dc98e

• .

  MadcowD

  committed
  936b5e1

  Copy full SHA for 936b5e1

Commits on Oct 7, 2024

• .

  MadcowD

  committed
  d6d3c05

  Copy full SHA for d6d3c05

Commits on Oct 8, 2024

• interesting human eval spec

  MadcowD

  committed
  866607b

  Copy full SHA for 866607b

Commits on Oct 9, 2024

• refactor types locations

  MadcowD

  committed
  49f550f

  Copy full SHA for 49f550f

• new back relationships for evaluatio nrun summaries

  MadcowD

  committed
  e303f75

  Copy full SHA for e303f75

• some more interesintg thogyuhts on human feedback and data collection

  MadcowD

  committed
  fb86f48

  Copy full SHA for fb86f48

• adding idea of a invocation group but not fully implemented

  MadcowD

  committed
  a6cd5cf

  Copy full SHA for a6cd5cf

• working evals with road to summarizatioin

  MadcowD

  committed
  ed545a1

  Copy full SHA for ed545a1

• .

  MadcowD

  committed
  db22e99

  Copy full SHA for db22e99

• example of a discriminator

  MadcowD

  committed
  ffc770d

  Copy full SHA for ffc770d

• additioanl update

  MadcowD

  committed
  d93d735

  Copy full SHA for d93d735

Commits on Oct 10, 2024

• solve the no dataset problem

  MadcowD

  committed
  a973158

  Copy full SHA for a973158

• working writing to the db

  MadcowD

  committed
  e5df68c

  Copy full SHA for e5df68c

• base models

  MadcowD

  committed
  25443a9

  Copy full SHA for 25443a9

• data model stubs

  MadcowD

  committed
  a992f35

  Copy full SHA for a992f35

• working serverside api

  MadcowD

  committed
  a5c60b5

  Copy full SHA for a5c60b5

• eval cards

  MadcowD

  committed
  8c3d1a3

  Copy full SHA for 8c3d1a3

• .

  MadcowD

  committed
  f1fb2ac

  Copy full SHA for f1fb2ac

Commits on Oct 11, 2024

• working eval cards

  MadcowD

  committed
  26afc57

  Copy full SHA for 26afc57

• shrinking version badges?

  MadcowD

  committed
  5697612

  Copy full SHA for 5697612

• horizontal layout

  MadcowD

  committed
  0a087b7

  Copy full SHA for 0a087b7

• cleaner layout

  MadcowD

  committed
  e5c4476

  Copy full SHA for e5c4476

• trendlines

  MadcowD

  committed
  53fe114

  Copy full SHA for 53fe114

• clean metrics

  MadcowD

  committed
  7577358

  Copy full SHA for 7577358

• latest run summary and invocaiton plots!

  MadcowD

  committed
  6001206

  Copy full SHA for 6001206

• basic eval page

  MadcowD

  committed
  d315026

  Copy full SHA for d315026

Commits on Oct 12, 2024

• generalize version history

  MadcowD

  committed
  99d689c

  Copy full SHA for 99d689c

• diea for the metrics page.

  MadcowD

  committed
  1cd52fa

  Copy full SHA for 1cd52fa

• more goofing

  MadcowD

  committed
  8538de3

  Copy full SHA for 8538de3

• unified icon

  MadcowD

  committed
  4a01ff9

  Copy full SHA for 4a01ff9

• refacroed computation graph a bit more

  MadcowD

  committed
  9d8b194

  Copy full SHA for 9d8b194

• better layouting algorithm

  MadcowD

  committed
  fe6aa91

  Copy full SHA for fe6aa91

• metric displays

  MadcowD

  committed
  fcc8ed2

  Copy full SHA for fcc8ed2

• metrics

  MadcowD

  committed
  aee962e

  Copy full SHA for aee962e

• beautful

  MadcowD

  committed
  e66ebf6

  Copy full SHA for e66ebf6

• .

  MadcowD

  committed
  6fbc927

  Copy full SHA for 6fbc927

Commits on Oct 13, 2024

• added error bars

  MadcowD

  committed
  26abbf6

  Copy full SHA for 26abbf6

• Merge branch 'main' into wguss/metrics

  MadcowD

  committed
  c30410e

  Copy full SHA for c30410e

• .

  MadcowD

  committed
  2270b98

  Copy full SHA for 2270b98

• datanow comes in real time

  MadcowD

  committed
  49379c4

  Copy full SHA for 49379c4

• enable parallel writes

  MadcowD

  committed
  8f4628b

  Copy full SHA for 8f4628b

• fix writing to wrong graph

  MadcowD

  committed
  29a2e05

  Copy full SHA for 29a2e05

Commits on Oct 14, 2024

• .

  MadcowD

  committed
  6d84d6a

  Copy full SHA for 6d84d6a

• .

  MadcowD

  committed
  7dcdd64

  Copy full SHA for 7dcdd64

Commits on Oct 24, 2024

• .

  MadcowD

  committed
  d7ea716

  Copy full SHA for d7ea716

• refactor evalaution location

  MadcowD

  committed
  a606d1c

  Copy full SHA for a606d1c

• evaluations

  MadcowD

  committed
  cf0cda2

  Copy full SHA for cf0cda2

• a bit of cleanup & refactor

  MadcowD

  committed
  2b633fa

  Copy full SHA for 2b633fa

• lcoal evalaution utils

  MadcowD

  committed
  2e3a687

  Copy full SHA for 2e3a687

• local util refactor pt 2

  MadcowD

  committed
  1d67ef6

  Copy full SHA for 1d67ef6

• fix bug where invocation id interface didnt work when the store wasn't set.

  MadcowD

  committed
  5240c5b

  Copy full SHA for 5240c5b

• new tests

  MadcowD

  committed
  245bfa5

  Copy full SHA for 245bfa5

• more tests

  MadcowD

  committed
  a4d8d0f

  Copy full SHA for a4d8d0f

• additioanl refactor todo

  MadcowD

  committed
  cd3b9e7

  Copy full SHA for cd3b9e7

• refactor for serialization module

  MadcowD

  committed
  5cf03bf

  Copy full SHA for 5cf03bf

• a bit cleaner version of the evaluator now

  MadcowD

  committed
  c8bc762

  Copy full SHA for c8bc762

Commits on Oct 28, 2024

• push

  MadcowD

  committed
  e4e8bb3

  Copy full SHA for e4e8bb3

Commits on Oct 29, 2024

• streaming evaluations.

  MadcowD

  committed
  702f24f

  Copy full SHA for 702f24f

• in progress display

  MadcowD

  committed
  fb42417

  Copy full SHA for fb42417

• hierarchical cluster sorting for parallel execution.

  MadcowD

  committed
  6514b69

  Copy full SHA for 6514b69

• partial labelers

  MadcowD

  committed
  38a5c59

  Copy full SHA for 38a5c59

• .

  MadcowD

  committed
  c190b60

  Copy full SHA for c190b60

• convert to flat representation

  MadcowD

  committed
  5d64765

  Copy full SHA for 5d64765

• flat

  MadcowD

  committed
  76fe640

  Copy full SHA for 76fe640

Commits on Oct 31, 2024

• .

  MadcowD

  committed
  cbfea1a

  Copy full SHA for cbfea1a

• new individual evaluation page

  MadcowD

  committed
  e0de211

  Copy full SHA for e0de211

Commits on Nov 1, 2024

• label display

  MadcowD

  committed
  89c4a92

  Copy full SHA for 89c4a92

• is constant subtle bras

  MadcowD

  committed
  1f7bcff

  Copy full SHA for 1f7bcff

• fixing sdiebar bug

  MadcowD

  committed
  033a542

  Copy full SHA for 033a542

• expandable groups

  MadcowD

  committed
  b69a93e

  Copy full SHA for b69a93e

Commits on Nov 2, 2024

• fix bool bug

  MadcowD

  committed
  8e648ef

  Copy full SHA for 8e648ef

• histogram view

  MadcowD

  committed
  85b7f73

  Copy full SHA for 85b7f73

• adding histogram and better sorting

  MadcowD

  committed
  ab86f21

  Copy full SHA for ab86f21

• working search and filteres for the evals

  MadcowD

  committed
  e7929fd

  Copy full SHA for e7929fd

• metrics show up now in the sidebar

  MadcowD

  committed
  8775ce3

  Copy full SHA for 8775ce3

• fix null renderer bug

  MadcowD

  committed
  362565d

  Copy full SHA for 362565d

• some clean asserts

  MadcowD

  committed
  1f62914

  Copy full SHA for 1f62914

• passing tests

  MadcowD

  committed
  bc8097e

  Copy full SHA for bc8097e

• evals

  MadcowD

  committed
  b748679

  Copy full SHA for b748679

Commits on Nov 13, 2024

• sliguhtly improved tables

  MadcowD

  committed
  e98ede6

  Copy full SHA for e98ede6

• all versions of evaluations

  MadcowD

  committed
  7de875c

  Copy full SHA for 7de875c

• evals folder

  MadcowD

  committed
  d8caaff

  Copy full SHA for d8caaff

• dataset hash -> dataset id

  MadcowD

  committed
  09a6dea

  Copy full SHA for 09a6dea

• dataset storage

  MadcowD

  committed
  6c41f19

  Copy full SHA for 6c41f19

• dataset view ish

  MadcowD

  committed
  1836c2b

  Copy full SHA for 1836c2b

• dataset page

  MadcowD

  committed
  b6308aa

  Copy full SHA for b6308aa

Commits on Nov 19, 2024

• merging new main:

  MadcowD

  committed
  08bd13f

  Copy full SHA for 08bd13f

• migration for exisitng ell studio databases

  MadcowD

  committed
  1febcda

  Copy full SHA for 1febcda

• fixed migrations and module seperation

  MadcowD

  committed
  6e866d7

  Copy full SHA for 6e866d7

• preferring main

  MadcowD

  committed
  7f1f162

  Copy full SHA for 7f1f162

• cleanup evals

  MadcowD

  committed
  c1e4d04

  Copy full SHA for c1e4d04

• python 3.9 fix

  MadcowD

  committed
  b8778c5

  Copy full SHA for b8778c5

• clean up eval lsit

  MadcowD

  committed
  a9f1f49

  Copy full SHA for a9f1f49

• 3.9 lru cache fix

  MadcowD

  committed
  18c26eb

  Copy full SHA for 18c26eb