Repositories list

• buildkite-ci

  Public
  HCL

  •15•6•0•4•Updated Nov 26, 2024Nov 26, 2024

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  amdcudainference+ 15pytorchtransformerllamagptrocmmodel-servingtpu+ 8

  Python

  •

  Apache License 2.0

  •4.7k•31k•1.7k•377•Updated Nov 26, 2024Nov 26, 2024

• llm-compressor

  Public
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

  sparsitycompressionquantization

  Python

  •

  Apache License 2.0

  •61•736•33•28•Updated Nov 26, 2024Nov 26, 2024

• vllm-project.github.io

  Public
  HTML

  •

  MIT License

  •5•4•0•0•Updated Oct 30, 2024Oct 30, 2024

• vllm-blog-source

  Public
  SCSS

  •

  MIT License

  •1•1•0•0•Updated Oct 30, 2024Oct 30, 2024

• flash-attention

  Public
  Fast and memory-efficient exact attention

  C++

  •

  BSD 3-Clause "New" or "Revised" License

  •1.3k•30•0•3•Updated Oct 25, 2024Oct 25, 2024

• vllm-nccl

  Public archive
  Manages vllm-nccl dependency

  Python

  •

  Apache License 2.0

  •2•17•2•0•Updated Jun 3, 2024Jun 3, 2024

• dashboard

  Public
  vLLM performance dashboard

  Python

  •

  Apache License 2.0

  •3•19•0•0•Updated Apr 26, 2024Apr 26, 2024