forked from dottxt-ai/outlines
-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix mamba integration by making it a variant of outlines.models.transformers
#32
Draft
lapp0
wants to merge
12
commits into
transformers-use-logits-processor
Choose a base branch
from
fix-mamba-integration
base: transformers-use-logits-processor
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lapp0
force-pushed
the
fix-mamba-integration
branch
3 times, most recently
from
June 14, 2024 22:07
84ea1eb
to
99813cf
Compare
lapp0
force-pushed
the
transformers-use-logits-processor
branch
17 times, most recently
from
June 16, 2024 21:50
3123f3a
to
9b513af
Compare
lapp0
force-pushed
the
fix-mamba-integration
branch
2 times, most recently
from
June 17, 2024 03:35
783acf6
to
68ea867
Compare
lapp0
force-pushed
the
transformers-use-logits-processor
branch
8 times, most recently
from
June 17, 2024 18:10
872b9c6
to
5ce23f7
Compare
lapp0
force-pushed
the
transformers-use-logits-processor
branch
7 times, most recently
from
July 3, 2024 14:42
32319df
to
7d43bbd
Compare
Co-authored-by: Patrice Bechard <[email protected]>
…ration) (dottxt-ai#1039) As [discussed in our Discord server](https://discord.com/channels/1182316225284554793/1182317446225481788/1261998326077984802) This PR adds support for custom regex parsers. This doesn't change the behavior of Outlines by default. But this allows us to write custom `Guide` classes that uses custom regex parsers for e.g. multimodal generation. Also improves documentation
lapp0
force-pushed
the
fix-mamba-integration
branch
from
July 15, 2024 09:05
f17913b
to
48b6f8f
Compare
rlouf
force-pushed
the
fix-mamba-integration
branch
from
July 15, 2024 13:53
48b6f8f
to
bf3694c
Compare
lapp0
force-pushed
the
fix-mamba-integration
branch
7 times, most recently
from
July 15, 2024 23:14
60449d4
to
75dc370
Compare
lapp0
force-pushed
the
fix-mamba-integration
branch
from
July 16, 2024 00:08
75dc370
to
acb0759
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes dottxt-ai#808
Rendered docs: https://github.com/lapp0/outlines/blob/fix-mamba-integration/docs/reference/models/transformers.md#alternative-model-classes
Problem
SequenceGenerator
. It should useSequenceGeneratorAdapter
and have logits processors manage the automata.models.mamba
doesn't work at all inmain
.Solution
Update
models.transformers
to accept amodel_class
argument allowing for additional model types beyondAutoModelForCausalLM
. Makemodels.mamba
simply a variant ofmodels.transformers
with themodel_class=transformers.MambaForCausalLM
passed. This results in nearly zero maintenance required formodels.mamba
.Unrelated work
Additionally Zach on Discord requested T5-based structured generation. I tested it with
model_class=transformers.AutoModelForSeq2SeqLM
and it works with zero additional changes. The only change I made related to this is adding amodel_t5
fixture to ourtest_generate.py
tested models and documenting it indocs/reference/models/transformers.md
.