From 911c0311dda52e010701e57b8e0ebe5a05a55b81 Mon Sep 17 00:00:00 2001 From: divya-kumari32 <72085811+divya-kumari32@users.noreply.github.com> Date: Tue, 17 Dec 2024 21:06:31 +0300 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b8aefbb..83ae440 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ cd flash-attention && pip install . && cd .. | Model | Params | # Layers | Hidden Dim. | Attention Heads | GQA | KV Heads | Context Length | Tied Embeddings | |-------------------|--------------|----------|-------------|-----------------|-----|----------|----------------|------------------| -| Bamba | 9B (9.78B) | 32 | 4096 | 32 | Yes | 8 | 4096 | True | +| Bamba | 9B (9.78B) | 32 | 4096 | 32 | Yes | 8 | 4096 | False | ### Checkpoints We have published our model checkpoints here: TODO: add mamba HF page once public