Use torch.xpu.mem_get_info for XPU

[dvrogozh]

torch.xpu.mem_get_info API is available starting from PyTorch 2.6 (and in nightly 2.6.0.dev20241206+xpu or later). To work properly this method requires PyTorch built with the SYCL runtime which supports API to query device memory stats. If not available, exception will be raised.

Requires: pytorch/pytorch#141230
Fixes: #2929
Fixes: huggingface/transformers#31922

CC: @muellerzr @SunMarc

[SunMarc]

LGTM ! Just a few nits

[SunMarc]

is there a link to show to the user how to build it with SYCL runtime ?

[dvrogozh]

I've updated comment a little to clarify the story around SYCL version. It's tight to the version of DPC++ SYCL compiler used to build pytorch with XPU backend support. It's reported with print(torch.version.xpu) and the first version to support required feature is 20250001. I think link on how to build is not necessary here in accelerate comments since that's a generic link to pytorch documentation on how to build it with XPU backend support. I.e. https://github.com/pytorch/pytorch?tab=readme-ov-file#intel-gpu-support. It further links to intel side documentation to install required environment which includes DPC++ SYCL compiler (this one: https://www.intel.com/content/www/us/en/developer/articles/tool/pytorch-prerequisites-for-intel-gpus.html).

Note that I deliberately used try...catch here instead of version checks around torch.version.xpu since I think that's better approach. SYCL support is getting upstream to clang and in the future there might be at least 2 compilers supporting it - intel dpc++ one and clang. I think using try...catch would handle such situation in accelerate counting for the future. We can discuss this further if needed.

[muellerzr]

Thanks for doing this!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[dvrogozh]

Failing ci test:

_____ AcceleratorTester.test_free_memory_dereferences_prepared_components ______
...
        # The less-than comes *specifically* from CUDA CPU things/won't be present on CPU builds
>       assert free_cpu_ram_after <= free_cpu_ram_before
E       assert 14725 <= 14724

Is this a know test flakiness? I guess there is no currently Intel GPUs in the ci. If so this PR should not have an effect on the test run.

[dvrogozh]

Rerun for prev. failing test have passed. New ci failure also seems unrelated:

FAILED tests/test_accelerator.py::AcceleratorTester::test_save_load_model_use_pytorch - assert 3.8623809814453125e-05 > 0.001
 +  where 3.8623809814453125e-05 = abs((3.756920099258423 - 3.7568814754486084))
 +    where 3.7568814754486084 = get_signature(Linear(in_features=2, out_features=4, bias=True))