Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build CUDA packages on CPU agent, test on GPU agent #314

Open
h-vetinari opened this issue Jan 6, 2025 · 1 comment
Open

Build CUDA packages on CPU agent, test on GPU agent #314

h-vetinari opened this issue Jan 6, 2025 · 1 comment

Comments

@h-vetinari
Copy link
Member

We're currently building on

github_actions_labels: # [linux]
- cirun-openstack-gpu-2xlarge # [linux]

There are not many physical GPUs available, so this creates a big bottleneck. It would be nice to build the package on a CPU agent (the vast majority of the time for a given run), and then use a GPU-enabled agent only for the testing part.

This idea has already been on the table for a long time, but needs support (at least) in smithy: conda-forge/conda-smithy#1472.

Opening for visibility, since pytorch is most affected currently by this bottleneck (AFAIU, there's also still a physically broken & offline GPU, which further exacerbates the problem).

@mgorny
Copy link
Contributor

mgorny commented Jan 6, 2025

Hmm, thinking about it, perhaps we could move CPU-only builds to the regular builders?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants