Support transformers 4.43 #1971
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
3 tasks
|
OwlViT and OwlV2 need an upgrade to their onnx config (opset=11->12 for the einsum operator), I will push it once the onnxruntime tests finish (to see if there's anything left). |
echarlaix
approved these changes
Jul 25, 2024
echarlaix
reviewed
Jul 26, 2024
|
should be all fixed now |
4 tasks
echarlaix
approved these changes
Jul 30, 2024
| if use_torch is True: | ||
| cache_position = cache_position.to(self.device) | ||
|
|
||
| return use_cache_branch_tensor, past_key_values, cache_position |
There was a problem hiding this comment.
this is a breaking change so we should be careful, not sure this method is used by anyone though
There was a problem hiding this comment.
the method is only used by the forward pass, I don't think any sub packages use it
…gface/optimum into support-transformers-4.43
echarlaix
reviewed
Jul 30, 2024
|
@IlyasMoutawwakil Is this PR ready to support transformers 4.43.3 version? |
IlyasMoutawwakil
commented
Aug 1, 2024
3 tasks
|
@sreenivasulureddysura yep it's ready |
4 tasks
echarlaix
added a commit
that referenced
this pull request
Aug 5, 2024
* fix bt bark test * setup * patch clip models for sd * infer ort model dtype property from inputs dtypes * patch all clip variants * device setter * bigger model for now * fix device attribution * onnx opset for owlvit and owlv2 * model dtype * revert * use model part dtype instead * no need for dtype with diffusion pipelines * revert * fix clip text model with projection not outputting hidden states * whisper generation * fix whisper, support cache_position, and using transformers whisper generation loop * style * create cache position for merged decoder and fix test for non whisper speech to text * typo * conditioned cache position argument * update whisper min transformers version * compare whisper ort generation with transformers * fix generation length for speech to text model type * cache position in whisper only with dynamic axis decoder_sequence_length * use minimal prepare_inputs_for_generation in ORTModelForSpeechSeq2Seq * remove version restrictions on whisper * comment * fix * simpler --------- Co-authored-by: Ella Charlaix <[email protected]>
robertknight
added a commit
to robertknight/rten
that referenced
this pull request
Oct 27, 2024
Support the `cache_position` input that was added to Hugging Face Whisper models as part of a revision of how it handles KV-caching. This is like `position_ids`, but there is no batch dimension. See huggingface/optimum#1971 and huggingface/transformers#31166.
robertknight
added a commit
to robertknight/rten
that referenced
this pull request
Oct 27, 2024
Support the `cache_position` input that was added to Hugging Face Whisper models as part of a revision of how it handles KV-caching. This is like `position_ids`, but there is no batch dimension. See huggingface/optimum#1971 and huggingface/transformers#31166.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds support for transformers 4.43 with which the following issues emerge:
clipmodels usingsdpaattention.pipelinecallingmodel.dtypeto convert inputs to the correct floating point precisionbarkmodels can't be saved due to shared tensors (track inBarkModelcan't be saved anymore transformers#32224)cache_position.Before submitting
Who can review?