Does not work on macOS with device="mps": "Can't infer missing attention mask on mps device"
#148
Comments
|
same problem for me. NotImplementedError: Output channels > 65536 not supported at the MPS device. As a temporary fix, you can set the environment variable |
|
On the last commit pushed, I've bumped the transformers version, could you try again, after having installed again from scratch |
|
Same error even with transformers 4.46.1 There's a weird thing. I tried to set torch_dtype = torch.bfloat16 but when I run the code (same code from @ChristianWeyer ) I got the following logs. It seems it uses float32. Flash attention 2 is not installed Config of the audio_encoder: <class 'parler_tts.dac_wrapper.modeling_dac.DACModel'> is overwritten by shared audio_encoder config: DACConfig { Config of the decoder: <class 'parler_tts.modeling_parler_tts.ParlerTTSForCausalLM'> is overwritten by shared decoder config: ParlerTTSDecoderConfig { The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's |
|
It looks like a problem with Pytorch with MPS. :( They just changed the output message to "Output channels > 65536 not supported at the MPS device." removing the message "As a temporary fix, you can set the environment variable // TODO: MPS convolution kernel currently does not support output channels > 2^16 |
|
Feel free to follow pytorch/pytorch#140722 for updates on a fix in PyTorch. Tentative fix in pytorch/pytorch#140726. |
|
The channel size issue has been fixed in PyTorch on macOS 15.1. It should be available in PyTorch nightly in < 24h. While testing the fix I discovered that descript-audiotools, which parler-tts is a transitive dependent of, requires The most straight-forward approach is probably to handle unavailability gracefully in descript-audiotools. I'm quite curious why support was removed for macOS though, since this was definitely supported in the past (ref pytorch/pytorch#20380 (comment)). |
|
Does it work now? Or we still wait for the final fix? |
|
Yes, I think this is fixed, and no further code changes are needed. I think I misread the docs. From what I can tell, Install nightlies with pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cputhen import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
device = "mps"
model_name = "parler-tts/parler-tts-mini-v1"
model = ParlerTTSForConditionalGeneration.from_pretrained(model_name, device_map=device)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "Hey, how are you doing today?"
description = "Jon's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise."
input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)should provide good output. This issue can be closed. cc @ylacombe |
|
Hi @hvaara Thank you for your clarification and support, but it doesn't resolve the issue for me. NotImplementedError: Output channels > 65536 not supported at the MPS device. tnx |
|
@tulas75 you need a pretty recent version of PyTorch nightly. What's the output of |
|
@hvaara 2.6.0.dev20241206 |
|
@tulas75 interesting. And what's the code/command you ran? |
|
I installed everything from scratch. These the steps I followed:
This is the pip list output absl-py 2.1.0 |
|
@tulas75 thanks for the info! My working hypothesis is that the fix from the main branch isn't in nightly for some reason. IIRC I haven't actually tested with nightly (only main). I'm not at my computer right now, so I'll try to repro when I'm back. |
|
@tulas75 I'm not able to repro using nightly versions (same as you) In the script that produces the error, can you do a |
|
Hello, I still get the attention mask error! Followed all the steps and used the code above. Can you tell where I missed?
|
|
I confirm it works. This is what I did from scratch. git clone https://github.com/huggingface/parler-tts
cd parler-tts
python3 -m venv .venv
source .venv/bin/activate
pip install -e .
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu
pip install AccelerateThen try running the testing script that @hvaara provided earlier. |
|
@chigkim it works as in it's producing a good audio file in the end? |
I had to make 2 modifications to the above steps to get it working on my Mac Mini M2.
After this the code worked, I could see GPU usuage and it was very fast (than when from CPU). Thanks to all. |
Will you create a PR? |
|
@ChristianWeyer I didn't think a PR was needed. What and where do you think something needs to be changed? |
|
I mean, @QueryType had to make changes to get it working... |
|
You mean the parler-tts/parler_tts/configuration_parler_tts.py Lines 126 to 128 in d108732 It's just a warning. It's not specific to the current issue, and doesn't prevent you from generating good output with parler-tts. It can also be seen in the fine tuning example notebook. 
I believe the warning is benign and essentially WAI. The original error
would prevent you from generating output with parler-tts, and was likely fixed when @ylacombe bumped the transformers version. The second error
would also prevent you from generating output with parler-tts due to pytorch/pytorch#134416, and was fixed in pytorch/pytorch#140726. I don't think any further changes is needed in order to mark the current issue as fixed, but please let us know if you disagree. |
Me too, I do not think we need a PR. The issue with the compilation, is already under a fix, I think. |
|
Heads up: pytorch/pytorch#140726 introduced a regression affecting at least the |
This is my simple test script:
I get this error:
ValueError: Can't infer missing attention mask on `mps` device. Please provide an `attention_mask` or use a different device.Any idea what could be wrong?
Thanks!
The text was updated successfully, but these errors were encountered: