Upstream Main: Fused Ops and Kernels, FSDP and Memory Fixes#35

Merged

fabianlim merged 8 commits intomainfrom dev

Jun 7, 2024

+6,664-289

Commits on May 28, 2024

Linting and Formatting for FMS-Acceleration-Peft package (#24 )
achew010
authored

Commits on May 29, 2024

Workaround Low-Mem-Mode Patch for GPTQ-LoRA (#26 )

achew010
and
fabianlim
authored

Commits on Jun 2, 2024

Add MLP & QLoRA Fused Ops and Kernels, Mixtral (#29 )
fabianlim
authored

Commits on Jun 7, 2024

Address Incorrect Ignoring of Base Layer Modules for FSDP with Kernels (#31 )

fabianlim
and
achew010
authored
Shift GPU Memory Computation to End of Benchmarking Script (#30 )
achew010
authored