[CI/Build] improve python-only dev setup

[dtrifiro]

The current process for python-only development and/or builds is kinda painful.

With this PR I'm trying to resurrect the VLLM_USE_PRECOMPILED env var to skip extension compilation and make the overall process easier.

The overall idea is to include the compiled libraries from the nightly wheel in the built package (when the env var is set), so that it can be installed as a regular package (e.g with pip install -e .).
I'm including the possibility of setting the wheel location using the VLLM_PRECOMPILED_WHEEL_LOCATION, which can be:

• an URL (this is the default, and the location is set https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl)
• a path (e.g.VLLM_PRECOMPILED_WHEEL_LOCATION=path/to/vllm.whl)

The wheel is downloaded, the compiled libraries are extracted and added to package_data so they are included in the wheel build and/or install.

Note that by default, the wheel is only downloaded if not present in the current working directory, to avoid to have to do multiple downloads on each install.

This will make it possible to install specific branches quite easily. For example, to test out a branch based on v0.6.3.post1 one could do the following:

export VLLM_USE_PRECOMPILED=1
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://files.pythonhosted.org/packages/4a/4c/ee65ba33467a4c0de350ce29fbae39b9d0e7fcd887cc756fa993654d1228/vllm-0.6.3.post1-cp38-abi3-manylinux1_x86_64.whl

pip install git+https://github.com/dtrifiro/[email protected]

Quickly starting work with the dev branch without having to rebuild from scratch.

As part of this PR:

• modified setup.py to allow for downloading of the prebuilt wheel (only works for the CUDA build)
• updated installation docs
• removed python_only_dev script
• added a sccache section to the install guide right after the ccache section

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[dtrifiro]

I think you might be interested in this @youkaichao

[rafvasq]

This is really helpful @dtrifiro, just offering a few typo catches and suggestions.

[rafvasq]

I think this line makes more sense at the beginning of this section. I offered a suggestion there.

Suggested change

Python code changes, will reflected when you run vLLM thanks to pip's ``--editable`` flag.

[ckadner]

Quick edit on the installation instructions. I still need to take the time to give it try.

[dtrifiro]

Thanks @rafvasq and @ckadner, updated the docs as suggested.

[joerunde]

Nice!

Looking forward to getting rid of our janky dev setups 😉

[youkaichao]

the idea looks good to me

[youkaichao]

I think we also need to copy some python file from vllm flash attention.

[dtrifiro]

Indeed, the previous version was only version because of a previously extracted vllm_flash_attn module.

With this patch, building from scratch results in all the required files being present:

$ unzip -l dist/vllm-0.6.3.post2.dev62+g2acec044e.d20241028.cu126.precompiled-py3-none-any.whl | grep -E flash_attn\|\\\.so
200404640  2024-10-28 15:45   vllm/_C.abi3.so
 82008256  2024-10-28 15:45   vllm/_moe_C.abi3.so
    31950  2024-10-28 09:11   vllm/attention/backends/flash_attn.py
    27715  2024-10-28 09:11   vllm/attention/backends/rocm_flash_attn.py
      347  2024-10-28 15:45   vllm/vllm_flash_attn/__init__.py
    47716  2024-10-28 15:45   vllm/vllm_flash_attn/flash_attn_interface.py
242221240  2024-10-28 15:45   vllm/vllm_flash_attn/vllm_flash_attn_c.abi3.so 

[dtrifiro]

Last push changed a few things:

• handling for PEP517-style builds: pip is not always available, use urllib.request.urlretrieve when pip is not available/fails.
• extract the code into a repackage_wheel() function
• also extract vllm_flash_attn files

[dtrifiro]

@youkaichao anything holding this back?

[youkaichao]

will take a look when i have time this week.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @dtrifiro.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[cermeng]

why this pr is blocked?

[youkaichao]

we create a temp directory to hold the file, because right now all wheels have the same name, while people can try to install different versions.

[dtrifiro]

This will redownload the wheel on every install, as well as filling up /tmp, I think this will cause users issues in the long term.

Also, we should not have to have a workaround due to the choice of using the same wheel name for all releases, which breaks other scenarios (#9244)

[youkaichao]

I don't know why, but i can see the wheel file is downloaded twice:

  Downloading wheel from https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl to /tmp/tmpi_5me_wi/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

  Downloading wheel from https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl to /tmp/tmpj87tpc7q/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

[youkaichao]

@dtrifiro I made several changes and locally tested it. please take a look and let me know if they look good to you.

[dtrifiro]

I don't know why, but i can see the wheel file is downloaded twice:

@youkaichao I fixed the issue in the latest push. It made little sense to use a custom function, using a custom build_ext-based class instead.

The reason for the duplicate calls was that setup.py was being called twice, one for the build dependency resolution, and another to perform the actual build.

@dtrifiro I made several changes and locally tested it. please take a look and let me know if they look good to you.

I have some concerns regarding the usage of temporary directories, but I'm fine with merging it as is, we can revisit/improve this setup when set up the pip index (#9831 (comment))

[cermeng]

I don't know why, but i can see the wheel file is downloaded twice:

  Downloading wheel from https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl to /tmp/tmpi_5me_wi/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

  Downloading wheel from https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl to /tmp/tmpj87tpc7q/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl

pip will call setup.py several times to do something like metadata generation and building wheel(ref). Note repackage_wheel is in the main function of our setup.py, so repackage_wheel will be invoked twice when pip call setup.py multiple times.

I think we can use a fixed tmp dir name and check wheel existence to avoid redownloading. This solution will also resolve #9621 (comment) by @dtrifiro

[dtrifiro]

@cermeng check out 69e6c4e

[youkaichao]

@dtrifiro I removed the super().run() because I find it actually calls some compilation, see 57ee0c1

I will merge the PR as-is right now after ci tests. but we should also add some tests to make sure it works as expected, e.g. removing all compilers and try to run python-only install, and check if it can be installed successfully. that can be a followup PR though.