Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Transformers 4.43 #856

Merged
merged 42 commits into from
Aug 6, 2024
Merged

Support Transformers 4.43 #856

merged 42 commits into from
Aug 6, 2024

Conversation

IlyasMoutawwakil
Copy link
Member

@IlyasMoutawwakil IlyasMoutawwakil commented Jul 31, 2024 •
edited
Loading

What does this PR do?

Adds support of transfromers 4.43 as in huggingface/optimum#1971, mainly:

  • whisper introducing a new forward input argument cache_position.
  • _reorder_cache removed from some causal models to use Cache.reorder_cache instead.
  • missing model.dtype used by transfromers pipelines to cast some features (pixels, audio samples, etc)

The rest is changes in the testing suite to adapt to the modeling changes in the library.
Also fixes the failing OV training tests on main.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@IlyasMoutawwakil
Copy link
Member Author

I think it's ready.
ITREX still needs to fix qbits modeling for WOQ. OV looks good to me (compared to CI on main).

@IlyasMoutawwakil
Copy link
Member Author

IlyasMoutawwakil commented Aug 2, 2024 •
edited
Loading

There's one last qwen test that fails with transfromers 4.37 even tho I did add the transformers_stream_generator package
https://github.com/huggingface/optimum-intel/actions/runs/10213421527/job/28258759384?pr=856#step:5:2163

echarlaix
echarlaix approved these changes Aug 2, 2024
View reviewed changes
Copy link
Collaborator

@echarlaix echarlaix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks a lot @IlyasMoutawwakil !

optimum/intel/openvino/modeling_seq2seq.py Show resolved Hide resolved
helena-intel
helena-intel approved these changes Aug 2, 2024
View reviewed changes
Copy link
Collaborator

@helena-intel helena-intel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Also really appreciate keeping transformers 4.36 support.

echarlaix
echarlaix approved these changes Aug 5, 2024
View reviewed changes
tests/openvino/test_exporters_cli.py Outdated Show resolved Hide resolved
optimum/intel/openvino/configuration.py Outdated Show resolved Hide resolved
optimum/intel/openvino/modeling_base.py Outdated Show resolved Hide resolved
@IlyasMoutawwakil
Copy link
Member Author

ready to merge

@echarlaix echarlaix force-pushed the support-transformers-4.43 branch from cb1714e to 815d0ed Compare August 5, 2024 15:14
@echarlaix echarlaix added the openvino-test Trigger OpenVINO slow tests label Aug 5, 2024
IlyasMoutawwakil
IlyasMoutawwakil commented Aug 6, 2024
View reviewed changes
tests/openvino/test_modeling.py
@@ -943,7 +943,7 @@ def test_beam_search(self, model_arch):
eos_token_id=None,
)

if model_arch == "minicpm":
if model_arch in ["minicpm", "internlm2"]:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this okay ? idk why the generation loop is not deterministic with internlm2

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it is due to random initialization of pt model

@IlyasMoutawwakil
Copy link
Member Author

IlyasMoutawwakil commented Aug 6, 2024 •
edited
Loading

looks good, the failing tests in windows are mostly os errors in the testing logic (use of unsafe rmtree), I can try fixing them in another PR

@IlyasMoutawwakil IlyasMoutawwakil merged commit 12438c4 into main Aug 6, 2024
22 of 24 checks passed
@IlyasMoutawwakil IlyasMoutawwakil deleted the support-transformers-4.43 branch August 6, 2024 11:43
IlyasMoutawwakil added a commit that referenced this pull request Aug 6, 2024
@IlyasMoutawwakil @echarlaix
Support Transformers 4.43 (#856)
49836da 
* install from pr

* updates

* fix

* update TRANSFORMERS_MAX_VERSION

* fix sdpa in training

* fix whisper

* fix

* whisper calibration checks

* fix OVTrainerTextClassificationTrainingTest's expected fake quantize

* fix OVCLIExportTestCase's expected_int4

* update min ci transformers version to 4.37

* fix OVQuantizerTest's expected fake quantize

* reorder_cache

* fix expected compressed matmuls

* fix test_exporters_cli_int4_with_local_model_and_default_config

* fix qwen custom modeling test

* fix failing ipex tests

* fix ipex

* fix the last ipex failing test_compare_with_and_without_past_key_values

* use minimal prepare_inputs_for_generation in OVModelForSpeechSeq2Seq

* keeping compatibility with transformers 4.36

* keep support of whisper using WhisperGenerationMixin.generate a,d dummy model fix

* trigger

* fix

* device property

* standardize .device and ._device attributes/properties

* fix

* fix

* revert

Co-authored-by: Ella Charlaix <[email protected]>

* use falcon

* torch.device property always cpu

* style

* resolve conflicts

* decoder_attention_mask for older versions

* optimum main

* limit inc transformers version

* fix pipeline missing dtype

* add dtype for seq to seq models

* pass phi beam search test and skip internlm2

* fix for internlm2

---------

Co-authored-by: Ella Charlaix <[email protected]>
IlyasMoutawwakil added a commit that referenced this pull request Aug 6, 2024
@IlyasMoutawwakil @echarlaix
Support Transformers 4.43 (#856)
fe864fc 
* install from pr

* updates

* fix

* update TRANSFORMERS_MAX_VERSION

* fix sdpa in training

* fix whisper

* fix

* whisper calibration checks

* fix OVTrainerTextClassificationTrainingTest's expected fake quantize

* fix OVCLIExportTestCase's expected_int4

* update min ci transformers version to 4.37

* fix OVQuantizerTest's expected fake quantize

* reorder_cache

* fix expected compressed matmuls

* fix test_exporters_cli_int4_with_local_model_and_default_config

* fix qwen custom modeling test

* fix failing ipex tests

* fix ipex

* fix the last ipex failing test_compare_with_and_without_past_key_values

* use minimal prepare_inputs_for_generation in OVModelForSpeechSeq2Seq

* keeping compatibility with transformers 4.36

* keep support of whisper using WhisperGenerationMixin.generate a,d dummy model fix

* trigger

* fix

* device property

* standardize .device and ._device attributes/properties

* fix

* fix

* revert

Co-authored-by: Ella Charlaix <[email protected]>

* use falcon

* torch.device property always cpu

* style

* resolve conflicts

* decoder_attention_mask for older versions

* optimum main

* limit inc transformers version

* fix pipeline missing dtype

* add dtype for seq to seq models

* pass phi beam search test and skip internlm2

* fix for internlm2

---------

Co-authored-by: Ella Charlaix <[email protected]>
IlyasMoutawwakil added a commit that referenced this pull request Aug 6, 2024
@IlyasMoutawwakil @echarlaix
Support Transformers 4.43 (#856)
e62d8bd 
* install from pr

* updates

* fix

* update TRANSFORMERS_MAX_VERSION

* fix sdpa in training

* fix whisper

* fix

* whisper calibration checks

* fix OVTrainerTextClassificationTrainingTest's expected fake quantize

* fix OVCLIExportTestCase's expected_int4

* update min ci transformers version to 4.37

* fix OVQuantizerTest's expected fake quantize

* reorder_cache

* fix expected compressed matmuls

* fix test_exporters_cli_int4_with_local_model_and_default_config

* fix qwen custom modeling test

* fix failing ipex tests

* fix ipex

* fix the last ipex failing test_compare_with_and_without_past_key_values

* use minimal prepare_inputs_for_generation in OVModelForSpeechSeq2Seq

* keeping compatibility with transformers 4.36

* keep support of whisper using WhisperGenerationMixin.generate a,d dummy model fix

* trigger

* fix

* device property

* standardize .device and ._device attributes/properties

* fix

* fix

* revert

Co-authored-by: Ella Charlaix <[email protected]>

* use falcon

* torch.device property always cpu

* style

* resolve conflicts

* decoder_attention_mask for older versions

* optimum main

* limit inc transformers version

* fix pipeline missing dtype

* add dtype for seq to seq models

* pass phi beam search test and skip internlm2

* fix for internlm2

---------

Co-authored-by: Ella Charlaix <[email protected]>
IlyasMoutawwakil added a commit that referenced this pull request Aug 6, 2024
@IlyasMoutawwakil @echarlaix
Support Transformers 4.43 (#856)
e2d0638 
* install from pr

* updates

* fix

* update TRANSFORMERS_MAX_VERSION

* fix sdpa in training

* fix whisper

* fix

* whisper calibration checks

* fix OVTrainerTextClassificationTrainingTest's expected fake quantize

* fix OVCLIExportTestCase's expected_int4

* update min ci transformers version to 4.37

* fix OVQuantizerTest's expected fake quantize

* reorder_cache

* fix expected compressed matmuls

* fix test_exporters_cli_int4_with_local_model_and_default_config

* fix qwen custom modeling test

* fix failing ipex tests

* fix ipex

* fix the last ipex failing test_compare_with_and_without_past_key_values

* use minimal prepare_inputs_for_generation in OVModelForSpeechSeq2Seq

* keeping compatibility with transformers 4.36

* keep support of whisper using WhisperGenerationMixin.generate a,d dummy model fix

* trigger

* fix

* device property

* standardize .device and ._device attributes/properties

* fix

* fix

* revert

Co-authored-by: Ella Charlaix <[email protected]>

* use falcon

* torch.device property always cpu

* style

* resolve conflicts

* decoder_attention_mask for older versions

* optimum main

* limit inc transformers version

* fix pipeline missing dtype

* add dtype for seq to seq models

* pass phi beam search test and skip internlm2

* fix for internlm2

---------

Co-authored-by: Ella Charlaix <[email protected]>
IlyasMoutawwakil added a commit that referenced this pull request Aug 6, 2024
@IlyasMoutawwakil @echarlaix
Support Transformers 4.43 (#856)
71ae2da 
* install from pr

* updates

* fix

* update TRANSFORMERS_MAX_VERSION

* fix sdpa in training

* fix whisper

* fix

* whisper calibration checks

* fix OVTrainerTextClassificationTrainingTest's expected fake quantize

* fix OVCLIExportTestCase's expected_int4

* update min ci transformers version to 4.37

* fix OVQuantizerTest's expected fake quantize

* reorder_cache

* fix expected compressed matmuls

* fix test_exporters_cli_int4_with_local_model_and_default_config

* fix qwen custom modeling test

* fix failing ipex tests

* fix ipex

* fix the last ipex failing test_compare_with_and_without_past_key_values

* use minimal prepare_inputs_for_generation in OVModelForSpeechSeq2Seq

* keeping compatibility with transformers 4.36

* keep support of whisper using WhisperGenerationMixin.generate a,d dummy model fix

* trigger

* fix

* device property

* standardize .device and ._device attributes/properties

* fix

* fix

* revert

Co-authored-by: Ella Charlaix <[email protected]>

* use falcon

* torch.device property always cpu

* style

* resolve conflicts

* decoder_attention_mask for older versions

* optimum main

* limit inc transformers version

* fix pipeline missing dtype

* add dtype for seq to seq models

* pass phi beam search test and skip internlm2

* fix for internlm2

---------

Co-authored-by: Ella Charlaix <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@echarlaix echarlaix echarlaix approved these changes

@eaidova eaidova eaidova approved these changes

@helena-intel helena-intel helena-intel approved these changes

Assignees
No one assigned
Labels
openvino-test Trigger OpenVINO slow tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@IlyasMoutawwakil @HuggingFaceDocBuilderDev @eaidova @helena-intel @echarlaix