Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Kedreamix committed Aug 21, 2024
1 parent e936484 commit 2ad0154
Show file tree
Hide file tree
Showing 2 changed files with 104 additions and 50 deletions.
80 changes: 53 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,10 @@
- [Gradio](#gradio)
- [Start WebUI](#start-webui)
- [WebUI](#webui)
- [Old Verison](#old-verison)
- [Old Verison](#old-verison)
- [Folder structure](#folder-structure)
- [Support Us](#support-us)
- [Reference](#reference)
- [License](#license)
- [Star History](#star-history)

<!-- /TOC -->
Expand All @@ -125,6 +125,8 @@ The design philosophy of Linly-Talker is to create a new form of human-computer

![The system architecture of multimodal human–computer interaction.](docs/HOI_en.png)

> [!NOTE]
>
> You can watch the demo video [here](https://www.bilibili.com/video/BV1rN4y1a76x/).
>
> I have recorded a series of videos on Bilibili, which also represent every step of my updates and methods of use. For detailed information, please refer to [Digital Human Dialogue System - Linly-Talker Collection](https://space.bilibili.com/241286257/channel/collectiondetail?sid=2065753).
Expand Down Expand Up @@ -152,8 +154,12 @@ The design philosophy of Linly-Talker is to create a new form of human-computer
- [x] Added CosyVoice, which provides high-quality text-to-speech (TTS) functionality and voice cloning capabilities. Additionally, updated to Wav2Lipv2 to enhance image quality effects.
- [ ] `Real-time` Speech Recognition (Enable conversation and communication between humans and digital entities using voice)

🔆 The Linly-Talker project is ongoing - pull requests are welcome! If you have any suggestions regarding new model approaches, research, techniques, or if you discover any runtime errors, please feel free to edit and submit a pull request. You can also open an issue or contact me directly via email. 📩⭐ If you find this repository useful, please give it a star! 🤩
> [!IMPORTANT]
>
> 🔆 The Linly-Talker project is ongoing - pull requests are welcome! If you have any suggestions regarding new model approaches, research, techniques, or if you discover any runtime errors, please feel free to edit and submit a pull request. You can also open an issue or contact me directly via email. 📩⭐ If you find this repository useful, please give it a star! 🤩
> [!TIP]
>
> If you encounter any issues during deployment, please consult the [Common Issues Summary](https://github.com/Kedreamix/Linly-Talker/blob/main/常见问题汇总.md) section, where I have compiled a list of all potential problems. Additionally, a discussion group is available here, and I will provide regular updates. Thank you for your attention and use of Linly-Talker!
## Example
Expand All @@ -167,15 +173,17 @@ The design philosophy of Linly-Talker is to create a new form of human-computer

## Setup Environment

AutoDL has released an image, which can be used directly at [https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker](https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker). You can also create an environment directly using Docker. I will continue to update the image.

```bash
docker pull registry.cn-beijing.aliyuncs.com/codewithgpu2/kedreamix-linly-talker:cMDvNE4RYl
```

For Windows, I've included an all-in-one Python package. You can run the steps in sequence to install the necessary dependencies and download the corresponding model to get it running. Follow the instructions using `conda` and start installing PyTorch from step 02. If you encounter any issues, please feel free to contact me.

[Windows All-in-One Package](https://pan.quark.cn/s/cc8f19c45a15)
> [!NOTE]
>
> AutoDL has released an image, which can be used directly at [https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker](https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker). You can also create an environment directly using Docker. I will continue to update the image.
>
> ```bash
> docker pull registry.cn-beijing.aliyuncs.com/codewithgpu2/kedreamix-linly-talker:zYGdW3yO3D
> ```
>
> For Windows, I've included an all-in-one Python package. You can run the steps in sequence to install the necessary dependencies and download the corresponding model to get it running. Follow the instructions using `conda` and start installing PyTorch from step 02. If you encounter any issues, please feel free to contact me.
>
> [Windows All-in-One Package](https://pan.quark.cn/s/cc8f19c45a15)
Download the code:
Expand Down Expand Up @@ -253,6 +261,10 @@ sudo apt-get install sox libsox-dev
sudo yum install sox sox-devel
```
> [!NOTE]
>
> The installation process is very slow.
Below are some older installation methods, which might cause dependency conflicts, but they generally don't produce many bugs. For an easier and better installation, I've updated the above version. You can ignore the following versions or refer to them if you encounter issues.
> To install the environment using Anaconda and PyTorch, follow the steps below:
Expand Down Expand Up @@ -329,14 +341,17 @@ Below are some older installation methods, which might cause dependency conflict
> ```
>
Next, you need to install the corresponding models. You can download them using the following methods. Once downloaded, place the files in the specified folder structure (explained at the end of this document).
We recommend downloading from Quark Netdisk for the latest updates.
- [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
- [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
- [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/summary)
- [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
> [!NOTE]
>
> Next, you need to install the corresponding models. You can download them using the following methods. Once downloaded, place the files in the specified folder structure (explained at the end of this document).
>
> We recommend downloading from modelscope for the latest updates.
>
> - [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
> - [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
> - [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/summary)
> - [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
>
I made a script that can download all the models mentioned below without requiring much input from the user. This method is suitable for stable network conditions, especially for Linux users. For Windows users, Git can also be used to download the models. If the network connection is unstable, users can choose to manually download the models or try running a Shell script to complete the download. The script has the following features:
Expand Down Expand Up @@ -797,7 +812,7 @@ This time, we've updated the interface. We can freely select the fine-tuned mode

![](docs/WebUI3.png)

## Old Verison
### Old Verison

There are three modes for the current startup, and you can choose a specific setting based on the scenario.

Expand Down Expand Up @@ -861,12 +876,15 @@ python app_musetalk.py

## Folder structure

The folder structure of the weight files is as follows:

- `Baidu (百度云盘)`: You can download the weights from [here](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`).
- `huggingface`: You can access the weights at [this link](https://huggingface.co/Kedreamix/Linly-Talker).
- `modelscope`: The weights will be available soon at [this link](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/files).
- `Qurak(夸克网盘)`:You can download the weights from [here](https://pan.quark.cn/s/f48f5e35796b)
> [!NOTE]
>
> The folder structure of the weight files is as follows:
>
> - `Baidu (百度云盘)`: You can download the weights from [here](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`).
> - `huggingface`: You can access the weights at [this link](https://huggingface.co/Kedreamix/Linly-Talker).
> - `modelscope`: The weights will be available soon at [this link](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/files).
> - `Qurak(夸克网盘)`:You can download the weights from [here](https://pan.quark.cn/s/f48f5e35796b)
>
```bash
Linly-Talker/
Expand Down Expand Up @@ -1025,6 +1043,14 @@ Linly-Talker/
- [https://github.com/RVC-Boss/GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS)
- [https://github.com/coqui-ai/TTS](https://github.com/coqui-ai/TTS)

## License

> [!CAUTION]
>
> When using this tool, please comply with all applicable laws, including copyright, data protection, and privacy laws. Do not use, modify, distribute, or sublicense this tool without permission from the original authors and/or copyright holders.
`Linly-Talker` follows the MIT License. In addition to adhering to the MIT License, ensure that you comply with all license agreements for any referenced models and components. Unauthorized use may lead to legal consequences.

## Star History

[![Star History Chart](https://api.star-history.com/svg?repos=Kedreamix/Linly-Talker&type=Date)](https://star-history.com/#Kedreamix/Linly-Talker&Date)
Expand Down
74 changes: 51 additions & 23 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,8 +99,8 @@
- [WebUI](#webui)
- [Old Verison](#old-verison)
- [文件夹结构](#文件夹结构)
- [赞助](#赞助)
- [参考](#参考)
- [许可协议](#许可协议)
- [Star History](#star-history)

<!-- /TOC -->
Expand All @@ -122,6 +122,8 @@ Linly-Talker的设计理念是创造一种全新的人机交互方式,不仅

![The system architecture of multimodal human–computer interaction.](docs/HOI.png)

> [!NOTE]
>
> 查看我们的介绍视频 [demo video](https://www.bilibili.com/video/BV1rN4y1a76x/)
>
> 在B站上我录了一系列视频,也代表我更新的每一步与使用方法,详细查看[数字人智能对话系统 - Linly-Talker合集](https://space.bilibili.com/241286257/channel/collectiondetail?sid=2065753)
Expand Down Expand Up @@ -149,8 +151,12 @@ Linly-Talker的设计理念是创造一种全新的人机交互方式,不仅
- [x] 加入了CosyVoice,具备优质的文本转语音(TTS)功能和语音克隆能力。同时,更新了Wav2Lipv2,以提升图片质量效果。
- [ ] `实时`语音识别(人与数字人之间就可以通过语音进行对话交流)

🔆 该项目 Linly-Talker 正在进行中 - 欢迎提出PR请求!如果您有任何关于新的模型方法、研究、技术或发现运行错误的建议,请随时编辑并提交 PR。您也可以打开一个问题或通过电子邮件直接联系我。📩⭐ 如果您发现这个Github Project有用,请给它点个星!🤩
> [!IMPORTANT]
>
> 🔆 该项目 Linly-Talker 正在进行中 - 欢迎提出PR请求!如果您有任何关于新的模型方法、研究、技术或发现运行错误的建议,请随时编辑并提交 PR。您也可以打开一个问题或通过电子邮件直接联系我。📩⭐ 如果您发现这个Github Project有用,请给它点个星!🤩
> [!TIP]
>
> 如果在部署的时候有任何的问题,可以关注[常见问题汇总.md](https://github.com/Kedreamix/Linly-Talker/blob/main/常见问题汇总.md)部分,我已经整理了可能出现的所有问题,另外交流群也在这里,我会定时更新,感谢大家的关注与使用!!!
## 示例
Expand All @@ -164,15 +170,17 @@ Linly-Talker的设计理念是创造一种全新的人机交互方式,不仅

## 创建环境

AutoDL已发布镜像,可以直接使用,[https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker](https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker),也可以使用docker来直接创建环境,我也会持续不断的更新镜像

```bash
docker pull registry.cn-beijing.aliyuncs.com/codewithgpu2/kedreamix-linly-talker:cMDvNE4RYl
```

Windows我加入了一个python一键整合包,可以按顺序进行运行,按照需求按照相应的依赖,并且下载对应的模型,即可运行,主要按照conda以后从02开始安装pytorch进行运行,如果有问题,请随时与我沟通

[Windows一键整合包](https://pan.quark.cn/s/cc8f19c45a15)
> [!NOTE]
>
> AutoDL已发布镜像,可以直接使用,[https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker](https://www.codewithgpu.com/i/Kedreamix/Linly-Talker/Kedreamix-Linly-Talker),也可以使用docker来直接创建环境,我也会持续不断的更新镜像
>
> ```bash
> docker pull registry.cn-beijing.aliyuncs.com/codewithgpu2/kedreamix-linly-talker:zYGdW3yO3D
> ```
>
> Windows我加入了一个python一键整合包,可以按顺序进行运行,按照需求按照相应的依赖,并且下载对应的模型,即可运行,主要按照conda以后从02开始安装pytorch进行运行,如果有问题,请随时与我沟通
>
> [Windows一键整合包](https://pan.quark.cn/s/cc8f19c45a15)
下载代码
Expand Down Expand Up @@ -248,6 +256,10 @@ sudo apt-get install sox libsox-dev
sudo yum install sox sox-devel
```

> [!NOTE]
>
> 安装过程可能耗时很长。
以下是旧版本的一些安装方法,可能存在会一些依赖冲突的问题,但是也不会出现太多bug,但是为了更好更方便的安装,我就更新了上述版本,以下版本可以忽略,或者遇到问题可以参考一下

> 首先使用anaconda安装环境,安装pytorch环境,具体操作如下:
Expand Down Expand Up @@ -327,12 +339,15 @@ sudo yum install sox sox-devel
接下来还需要安装对应的模型,有以下下载方式,下载后安装文件架结构放置,文件夹结构在本文最后有说明,建议从夸克网盘下载,会第一时间更新
- [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
- [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
- [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/summary)
- [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
> [!NOTE]
>
> 接下来还需要安装对应的模型,有以下下载方式,下载后安装文件架结构放置,文件夹结构在本文最后有说明,建议从modelscope下载,会第一时间更新
>
> - [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
> - [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
> - [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/summary)
> - [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
>
我制作一个脚本可以完成下述所有模型的下载,无需用户过多操作。这种方式适合网络稳定的情况,并且特别适合 Linux 用户。对于 Windows 用户,也可以使用 Git 来下载模型。如果网络环境不稳定,用户可以选择使用手动下载方法,或者尝试运行 Shell 脚本来完成下载。脚本具有以下功能。
Expand Down Expand Up @@ -843,12 +858,15 @@ python app_musetalk.py

## 文件夹结构

所有的权重部分可以从这下载,百度网盘可能有时候会更新慢一点,建议从夸克网盘下载,会第一时间更新

- [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
- [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
- [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/files)
- [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
> [!NOTE]
>
> 所有的权重部分可以从这下载,百度网盘可能有时候会更新慢一点,建议从夸克网盘下载,会第一时间更新
>
> - [Baidu (百度云盘)](https://pan.baidu.com/s/1eF13O-8wyw4B3MtesctQyg?pwd=linl) (Password: `linl`)
> - [huggingface](https://huggingface.co/Kedreamix/Linly-Talker)
> - [modelscope](https://www.modelscope.cn/models/Kedreamix/Linly-Talker/files)
> - [Quark(夸克网盘)](https://pan.quark.cn/s/f48f5e35796b)
>
权重文件夹结构如下

Expand Down Expand Up @@ -1010,6 +1028,16 @@ Linly-Talker/
- [https://github.com/RVC-Boss/GPT-SoVITS](https://github.com/RVC-Boss/GPT-SoVITS)
- [https://github.com/coqui-ai/TTS](https://github.com/coqui-ai/TTS)



## 许可协议

> [!CAUTION]
>
> 在使用本工具时,请遵守相关法律,包括版权法、数据保护法和隐私法。未经原作者和/或版权所有者许可,请勿使用本工具。
`Linly-Talker` 遵循 MIT Licence。在使用本工具时,请遵守相关法律,包括版权法、数据保护法和隐私法。未经原作者和/或版权所有者许可,请勿使用本工具。未经原作者和/或版权所有者许可,请勿使用本工具。此外,请确保遵守您参考的模型和组件中的所有许可协议。

## Star History

[![Star History Chart](https://api.star-history.com/svg?repos=Kedreamix/Linly-Talker&type=Date)](https://star-history.com/#Kedreamix/Linly-Talker&Date)
Expand Down

0 comments on commit 2ad0154

Please sign in to comment.