Update SGLang example to use Qwen2-VL

[advay-modal]

Type of Change

• Example updates (Bug fixes, new features, etc.)

Checklist

• Example is testable in synthetic monitoring system, or lambda-test: false is added to example frontmatter (---)
  • Example is tested by executing with modal run or an alternative cmd is provided in the example frontmatter (e.g. cmd: ["modal", "deploy"])
  • Example is tested by running with no arguments or the args are provided in the example frontmatter (e.g. args: ["--prompt", "Formula for room temperature superconductor:"]
• Example is documented with comments throughout, in a Literate Programming style.
• Example does not require third-party dependencies to be installed locally
• Example pins its dependencies
  • Example pins container images to a stable tag, not a dynamic tag like latest
  • Example specifies a python_version for the base image, if it is used
  • Example pins all dependencies to at least minor version, ~=x.y.z or ==x.y
  • Example dependencies with version < 1 are pinned to patch version, ==0.y.z

Outside contributors

You're great! Thanks for your contribution.

[advay-modal]

@charlesfrye I spent a while trying to get this to work with llama 3.2 11b VL, but the download speed was really slow for some reason

EDIT: I think I needed to set .env({"HF_HUB_ENABLE_HF_TRANSFER": "1"}). Can try with that if we have a strong preference for llama 3.2

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-75112f8.modal.run

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-bafc963.modal.run

[charlesfrye]

Nice! Don't forget to change the title now that the model has changed.

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-05070e3.modal.run

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-97ae146.modal.run

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-9be71ea.modal.run

[charlesfrye]

🚀 The docs preview is ready! Check it out here: https://modal-labs-examples--frontend-preview-487f7b6.modal.run