Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@divya-kumari32
divya-kumari32 authored Dec 17, 2024
1 parent 0d0daa5 commit 911c031
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ cd flash-attention && pip install . && cd ..

| Model | Params | # Layers | Hidden Dim. | Attention Heads | GQA | KV Heads | Context Length | Tied Embeddings |
|-------------------|--------------|----------|-------------|-----------------|-----|----------|----------------|------------------|
| Bamba | 9B (9.78B) | 32 | 4096 | 32 | Yes | 8 | 4096 | True |
| Bamba | 9B (9.78B) | 32 | 4096 | 32 | Yes | 8 | 4096 | False |

### Checkpoints
We have published our model checkpoints here: TODO: add mamba HF page once public
Expand Down

0 comments on commit 911c031

Please sign in to comment.