Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Attention layer registry #1094

Merged
merged 12 commits into from
Apr 12, 2024
Merged

Attention layer registry #1094

merged 12 commits into from
Apr 12, 2024

Conversation

dakinggg
Copy link
Collaborator

@dakinggg dakinggg commented Apr 4, 2024 •
edited
Loading

Before and after
Screenshot 2024-04-04 at 7 14 24 PM
Screenshot 2024-04-04 at 7 15 23 PM

@dakinggg dakinggg changed the title Attention registry Attention layer registry Apr 4, 2024
@dakinggg dakinggg marked this pull request as ready for review April 5, 2024 02:17
@dakinggg dakinggg requested review from irenedea and mvpatel2000 April 5, 2024 02:20
@dakinggg
Copy link
Collaborator Author

And with the MB integration merged

Screenshot 2024-04-11 at 3 59 54 PM Screenshot 2024-04-11 at 4 00 16 PM

@dakinggg
Copy link
Collaborator Author

@irenedea @mvpatel2000 ready for review post MB merge

mvpatel2000
mvpatel2000 approved these changes Apr 12, 2024
View reviewed changes
tests/models/test_rope_dail_vs_hf.py Outdated Show resolved Hide resolved
@dakinggg dakinggg enabled auto-merge (squash) April 12, 2024 01:46
@dakinggg dakinggg merged commit 560012b into mosaicml:main Apr 12, 2024
9 checks passed
KuuCi pushed a commit that referenced this pull request Apr 18, 2024
@dakinggg @KuuCi
Attention layer registry (#1094)
27ef6ad
@dakinggg dakinggg deleted the attn-registry branch June 22, 2024 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mvpatel2000 mvpatel2000 mvpatel2000 approved these changes

@irenedea irenedea Awaiting requested review from irenedea

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dakinggg @mvpatel2000