Skip to content

Files

Latest commit

 

History

History
46 lines (34 loc) · 3.05 KB

README.md

File metadata and controls

46 lines (34 loc) · 3.05 KB

about me

i specialize in building big systems to crunch through big data and develop big models.

i have previously developed one of the largest reinforcement learning systems at openai for openai five, along with llm training infrastructure at meta ai / fair that created opt-175b.

opt-175b was the first release in the industry to include:

  • a 175b parameter model for research use
  • a 114-page logbook detailing the challenges encountered during the 56 days it took to train a 175b llm on new hardware for the first time
  • the entire training codebase
  • a full suite of smaller-scale models ranging from 125M to 66B in size for studying scaling laws.

before getting into ai systems, i worked on scaling out data infrastructure and processing pipelines across various cloud providers.

you can refer to my linkedin for more xp info.

talks

in recent years, i have mainly presented talks on openai five and on opt-175b:

openai-five

  • Feb 12, 2022: Harvard CS50 Harvard CS50

  • March 18, 2022: Computer History Museum Computer History Museum

opt-175b

publications