Popular repositories Loading

1. cutlass cutlass Public

   Forked from NVIDIA/cutlass

   CUDA Templates for Linear Algebra Subroutines

   C++

2. cute-gemm cute-gemm Public

   Forked from reed-lau/cute-gemm

   C++

3. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

4. qserve qserve Public

   Forked from mit-han-lab/qserve

   QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

   Python

5. lmquant lmquant Public

   Forked from mit-han-lab/deepcompressor

   Python

6. marlin marlin Public

   Forked from IST-DASLab/marlin

   FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

   Python