Code

Enable qwen2vl video#2756

Code

Draft

drbh wants to merge 51 commits intomainhuggingface/text-generation-inference:mainfrom enable-qwen2vl-videohuggingface/text-generation-inference:enable-qwen2vl-videoCopy head branch name to clipboard

+788-65

Commits

Commits on Dec 23, 2024

• WIP video support

  

  mfarre

  authored and

  drbh

  committed
  18c9f06

• router changes

  

  mfarre

  authored and

  drbh

  committed
  7c67939

• adopting video url

  

  mfarre

  authored and

  drbh

  committed
  5ced960

• connecting video to qwen2

  

  mfarre

  authored and

  drbh

  committed
  05464d2

• fix

  

  mfarre

  authored and

  drbh

  committed
  c7c2fda

• downloading videos

  

  mfarre

  authored and

  drbh

  committed
  b9c8152

• fix

  

  mfarre

  authored and

  drbh

  committed
  464609f

• refactoring

  

  mfarre

  authored and

  drbh

  committed
  a25c3ec

• fix

  

  mfarre

  authored and

  drbh

  committed
  3c07391

• feat: support video input chunks and enable qwen2 vl to process video

  drbh

  committed
  b2c5575

• fix: add protobuf update and mp4parse dep

  drbh

  committed
  83a7f18

• fix: remove unused deps and imports

  drbh

  committed
  322165d

• moving video sampling and resize to validation. downstream we receive frames

  

  mfarre

  authored and

  drbh

  committed
  e65ead1

• flatten frames to data block when needed

  

  mfarre

  authored and

  drbh

  committed
  36e095b

• fix: adjust video process, reduce to 1 fps and adjust tensor shape

  drbh

  committed
  bc5e202

• fix: adjust deps after rebase

  drbh

  committed
  1afaa69

• feat: adjust impure shell deps and autodocs workflow

  drbh

  committed
  16007b6

• fix: include more deps for ffmpeg as docs suggest

  drbh

  committed
  39fac7e

• fix: add ffmpeg deps to test build

  drbh

  committed
  b508b10

• fix: debug ffmpeg install in tests workflow

  drbh

  committed
  4a3a724

• fix: debug ffmpeg deps in tests II

  drbh

  committed
  ac7483c

• fix: adjust dependencies and bump pip along with python

  drbh

  committed
  137f3bb

• fix: adjust pkg config in test

  drbh

  committed
  daf83a9

• fix: ensure pip is installed after installing deps in test workflow

  drbh

  committed
  d5cc670

• fix: add libavfilter dep to test

  drbh

  committed
  4a76e8b

• fix: add libavdevice dep to tests workflow

  drbh

  committed
  f0c3841

• fix: add ffmpeg overlay and enable build

  drbh

  committed
  96968a0

• fix: include ffmpeg deps in autodocs workflow

  drbh

  committed
  167c6f0

• Cleanup impure Nix shell

  

  danieldk

  authored and

  drbh

  committed
  98392a7

• Make the pure build work

  

  danieldk

  authored and

  drbh

  committed
  05004a6

• Fix test devshell

  

  danieldk

  authored and

  drbh

  committed
  063104c

• fix: bump deps in other dockerfiles

  drbh

  committed
  2dc078a

• fix: add ffmpeg to final layer of container

  drbh

  committed
  50b5399

• fix: include usr lib in ld path

  drbh

  committed
  b5b2184

• fix: copy shared libraries from builder

  drbh

  committed
  75ab887

• installing ssl requirements prior to rust building stage

  

  mfarre

  authored and

  drbh

  committed
  cbf1d98

• fixing ssl issue

  

  mfarre

  authored and

  drbh

  committed
  af77a0c

• working version

  

  mfarre

  authored and

  drbh

  committed
  19e1c8d

• cleanup prints

  

  mfarre

  authored and

  drbh

  committed
  db97d97

• fix: pre commit and clippy lints

  drbh

  committed
  71ed75a

• fix: resolve rebase issues and add test

  drbh

  committed
  e2b75a5

• fix: remove unnecessary cast

  drbh

  committed
  1d6bf24

• fix: update all vlm forward args, pass shared libraries to final layer in docker and doc bump

  drbh

  committed
  2ae152a

• fix: adjust batch_tokenized_inputs output in mllama

  drbh

  committed
  5c7bc91

• fix: update lints after rebase

  drbh

  committed
  bb00fb3

• fix: update trtllm dockefile after rebase

  drbh

  committed
  91ed362

• fix: adjust whitespace lint

  drbh

  committed
  5322abd

• fix: feature flag video and remove from non cuda dockerfiles

  drbh

  committed
  b4da6ad

• fix: make ffmpeg-next dep optional with feature

  drbh

  committed
  27f758d

• fix: include the video feature in cargo chef command

  drbh

  committed
  4f42d0c

• fix: adjust trtllm looper for video chunk enum

  drbh

  committed
  dcc1194