Code

Training on PQ shards#443

Code

Open

knighton wants to merge 12 commits intomainmosaicml/streaming:mainfrom james/pqmosaicml/streaming:james/pqCopy head branch name to clipboard

+1,045-10

Commits

Commits on Sep 22, 2023

• Training on PQ: draft generate.py, index.py, leaviung train.py

  knighton

  committed
  6afa233

Commits on Sep 23, 2023

• Updates.

  knighton

  committed
  9779805

• Rm print.

  knighton

  committed
  bcd5435

Commits on Sep 24, 2023

• Updates.

  knighton

  committed
  2ac4f3c

Commits on Oct 2, 2023

• Add tqdm to generate.py, etc.

  knighton

  committed
  2c43df6

• parquet_to_lance.

  knighton

  committed
  44a2dc2

• Add Lance benchmarking.

  knighton

  committed
  b383b72

• Renames.

  knighton

  committed
  4a5a171

• Split bench and plot.

  knighton

  committed
  68a56b4

Commits on Oct 17, 2023

• Misc.

  knighton

  committed
  c9bb6b9

Commits on Oct 18, 2023

• Update streaming/base/format/pq/reader.py

  Show description for f90a2a0
  

  knighton

  and

  Skylion007

  authored
  f90a2a0

• Merge branch 'main' into james/pq

  knighton

  authored
  4a44b1b