Skip to content

Issues: dvlab-research/LongLoRA

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

48 Open 125 Closed
48 Open 125 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

not able to reproduce the passkey retrieval accuracy
#195 opened Sep 16, 2024 by zhuconv
4
LongBench evaluation
#194 opened Aug 27, 2024 by Clement25
是否支持如GPT2这类的supervised fine-tune？
#193 opened Aug 23, 2024 by CharRic
How LongAlpaca Data was constructed?
#192 opened Jul 31, 2024 by S1s-Z
这套代码是否支持qwen/baichuan微调一个中文的长文本模型，代码需要做哪些修改？
#191 opened Jul 22, 2024 by jy-101361-1810897
norm层不是没有参数矩阵吗
#190 opened Jun 3, 2024 by changanxunyi
I am unable to reproduce the results from the paper for llama-7B-32k-longlora ppl.
#188 opened May 28, 2024 by masteryqq
1
embedding 为什么要resize成32001？
#186 opened May 22, 2024 by momandai
Something wrong with the torch version
#185 opened May 19, 2024 by dian1414
What's the trainset is used to obtain “Model with contextg extension via improved LoRA fine-tuning” (LoRA+)？
#184 opened Apr 22, 2024 by ZackZikaiXiao
How did make questions and answers for long context(LongAlpaca)?
#183 opened Mar 4, 2024 by ddoyles
When I set per_device_train_batch_size=2, the S2-Attn would not shift as expected
#182 opened Mar 1, 2024 by linhaojia13
2
HF models missing rope scaling in the config
#181 opened Feb 29, 2024 by hsiehjackson
Machine don't install Flash Attention
#180 opened Feb 27, 2024 by huilong-chen
global_step文件
#179 opened Feb 21, 2024 by xxcoco763
Regarding the results in Table 8 and Table 14
#177 opened Feb 4, 2024 by Statisticss
About the different datasets and corresponding models
#176 opened Feb 2, 2024 by Statisticss
Memory usage "too small" for 7B Llama-2
#174 opened Jan 24, 2024 by Linohong
training a LLM w/ shifted sparse attention from the scratch?
#173 opened Jan 24, 2024 by we1k
merge_lora_weights_and_save_hf_model.py Error while deserializing header: HeaderTooLarge
#172 opened Jan 23, 2024 by Spongeorge
Distributed inference issue
#171 opened Jan 22, 2024 by yixliu1
论文中的evaluate结果，推理时用的attention是shifted sparse attention？还是full attention？
#170 opened Jan 19, 2024 by zhangxiann
Is it possible to increase the context length of phi-2 using LongLora? If yes, what changes need to be done to support it?
#169 opened Jan 18, 2024 by dbanka
1
the value of loss is too unstable when supervised-finetune the 7b-100k-ft model
#168 opened Jan 18, 2024 by seanxuu
1
streaming llm problem
#167 opened Jan 18, 2024 by seanxuu
ProTip! no:milestone will show everything without a milestone.