train 成功了，分享一些细节。 trained successfully! Share some details with guys. #25
Comments
|
Thanks! Here is the English version: The training was successful! Here are the details of the environment:
{"messages":[{"role": "user", "content": "What's the weather like in Beijing?"}, {"role": "assistant", "content": "Today's weather in Beijing is clear with temperatures around 25°C."}]}
{"messages":[{"role": "user", "content": "How can I implement the Singleton pattern in Java?"}, {"role": "assistant", "content": "You can use the eager or lazy initialization method to implement the Singleton pattern. Here's an example of eager initialization:\n```java\npublic class Singleton {\n private static final Singleton instance = new Singleton();\n private Singleton() {}\n public static Singleton getInstance() {\n return instance;\n }\n}\n```"}]}
{"messages":[{"role": "user", "content": "Do you know what the Four Great Inventions of China are?"}, {"role": "assistant", "content": "China's Four Great Inventions are paper making, the compass, gunpowder, and printing."}]}
{"messages":[{"role": "user", "content": "Can you recommend any Chinese books?"}, {"role": "assistant", "content": "If you're interested in historical books, I recommend *Zizhi Tongjian*. If you're into literature, you might enjoy *Dream of the Red Chamber*."}]}
{"messages":[{"role": "user", "content": "What's the weather like in New York?"}, {"role": "assistant", "content": "The weather in New York is cloudy with a temperature around 18°C."}]}
{"messages":[{"role": "user", "content": "How can I implement a stack in Python?"}, {"role": "assistant", "content": "You can implement a stack using a list in Python. Here’s a simple example:\n```python\nstack = []\nstack.append(1)\nstack.append(2)\nprint(stack.pop()) # Outputs 2\n```"}]}
{"messages":[{"role": "user", "content": "Can you explain the concept of machine learning?"}, {"role": "assistant", "content": "Machine learning is a field of artificial intelligence where algorithms learn from data to make predictions or decisions without being explicitly programmed."}]}
{"messages":[{"role": "user", "content": "What's a good book to learn about AI?"}, {"role": "assistant", "content": "A great book to start with is 'Artificial Intelligence: A Modern Approach' by Stuart Russell and Peter Norvig. It covers a wide range of AI topics comprehensively."}]}
If you're unsure whether this step was successful, you can add some comments in these two files yourself. If an error occurs, check if the error code contains those comments. |
|
@LYCnight 大佬 训练的时候预计要多大的显存 |
|
@LYCnight 按照大佬的环境，成功的开启了训练，但是为什么训练完后的文件超级大呢？存储空间不够了…… |
GLM-4-9b 32k训练需要8卡80G。如果显存不够可以试试lora或者qlora。 |
9B的模型存下来就是这么大的，你可以设置只存模型参数不存训练的optimizer state，这样文件能小一些。 |
我的是 8 * 80G 全部跑满 |
确实大，等等官方的压缩大小的设置的文档吧 |
train成功了，分享一下环境：
modeling_chatglm.py和tokenization_chatglm.py都已经替换"stage3_prefetch_bucket_size": 15099494"seq_length": 131072{"messages":[{"role": "user", "content": "北京的天气怎么样？"}, {"role": "assistant", "content": "今天北京的天气晴朗，气温在25度左右。"}]} {"messages":[{"role": "user", "content": "如何在Java中实现单例模式？"}, {"role": "assistant", "content": "你可以使用懒汉式或饿汉式来实现单例模式。以下是饿汉式的示例代码：\n```java\npublic class Singleton {\n private static final Singleton instance = new Singleton();\n private Singleton() {}\n public static Singleton getInstance() {\n return instance;\n }\n}\n```"}]} {"messages":[{"role": "user", "content": "你知道中国的四大发明是什么吗？"}, {"role": "assistant", "content": "中国的四大发明是造纸术、指南针、火药和印刷术。"}]} {"messages":[{"role": "user", "content": "有什么推荐的中文书籍吗？"}, {"role": "assistant", "content": "如果你喜欢历史类书籍，我推荐《资治通鉴》。如果你对文学感兴趣，可以看看《红楼梦》。"}]} {"messages":[{"role": "user", "content": "What's the weather like in New York?"}, {"role": "assistant", "content": "The weather in New York is cloudy with a temperature around 18°C."}]} {"messages":[{"role": "user", "content": "How can I implement a stack in Python?"}, {"role": "assistant", "content": "You can implement a stack using a list in Python. Here’s a simple example:\n```python\nstack = []\nstack.append(1)\nstack.append(2)\nprint(stack.pop()) # Outputs 2\n```"}]} {"messages":[{"role": "user", "content": "Can you explain the concept of machine learning?"}, {"role": "assistant", "content": "Machine learning is a field of artificial intelligence where algorithms learn from data to make predictions or decisions without being explicitly programmed."}]} {"messages":[{"role": "user", "content": "What's a good book to learn about AI?"}, {"role": "assistant", "content": "A great book to start with is 'Artificial Intelligence: A Modern Approach' by Stuart Russell and Peter Norvig. It covers a wide range of AI topics comprehensively."}]}tips:
/glm-4-9b 目录下的
modeling_chatglm.py和tokenization_chatglm.py都已经替换这一步不确定自己是否成功的小伙伴，可以在这两个代码中自己加一些注释，如果报错了就检查一下报错代码中有没有这些注释。
The text was updated successfully, but these errors were encountered: