Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TTS component #19

Merged
merged 45 commits into from
Dec 4, 2024
Merged

TTS component #19

merged 45 commits into from
Dec 4, 2024

Conversation

Kostis-S-Z
Copy link
Contributor

@Kostis-S-Z Kostis-S-Z commented Nov 26, 2024 •
edited
Loading

What's changing

  • Add TTS component
  • Add model loader for TTS model (parler)
  • Add text_to_speech module to transform a string to a waveform
  • Add script_to_audio module that parses the whole podcast script that the text_to_text module generates and assigns each Speaker text to the corresponding speaker profile thus generating a waveform of the complete multi-turn podcast discussion.
  • Add config file that stores parameters relevant for the generation of the podcast
  • Add unit and integration tests for above modules
  • Add .idea/, .vscode and *.wav files to .gitignore
  • Update readme installations (use bash script for codespaces, pip for local)
  • Add specific machine specifications for codespaces VM
  • Add Troubleshooting section in README

From #26

  • Add headers and sections to the app.
  • Use personal fork and add setup.sh to work around parler_tts installing issues.
  • Load models using streamlit cache resources.
    • This prevents reloading on each button click
  • Generate and display audio for each chunk.

How to test it

Inside codespaces, run:

bash .github/setup.sh

Locally, run

pip install -e .

And then start the app with:

python -m streamlit run demo/app.py

Additional notes for reviewers

I tried parler-tts-large-v1, but for some reason the audio generated was completely distorted.

@Kostis-S-Z Kostis-S-Z linked an issue Nov 26, 2024 that may be closed by this pull request
Audio Generation Component #6
Closed
Kostis-S-Z
Kostis-S-Z commented Nov 26, 2024
View reviewed changes
demo/app.py Outdated Show resolved Hide resolved
@daavoo daavoo mentioned this pull request Nov 27, 2024
Merged
@Kostis-S-Z Kostis-S-Z marked this pull request as ready for review November 27, 2024 12:33
@Kostis-S-Z Kostis-S-Z requested a review from daavoo November 27, 2024 12:33
Kostis-S-Z and others added 13 commits November 28, 2024 10:00
@Kostis-S-Z
Add pydantic in project requirements
a0eec86
@Kostis-S-Z
Use Python's wave to save audio file instead of scipy
04033fc
@Kostis-S-Z
Update from 6-audio-generation-component
610ac9a
@Kostis-S-Z
Update comment
f213613
@Kostis-S-Z
Improve the TTS model loading process
9d03672 
Improve the TTS model loading process
@Kostis-S-Z
Add parler_tts to project dependencies
c0ef8b8
@Kostis-S-Z
Update TTS part of demo
2e6c8b2
@Kostis-S-Z
Fix wave module saving wav file
b8a5eeb
@Kostis-S-Z
Use soundfile instead of wave for saving .wav file
3bdbf78
@Kostis-S-Z
Fix script format
c35135c
@daavoo
fix(demo/app.py): Drop nested button. (#24)
9ee2c91 
* fix(demo/app.py): Drop nested button.

Background in https://discuss.streamlit.io/t/how-to-do-nested-button-and-print-both-button-outputs/33741

* pre-commit
@daavoo
Merge branch 'main' into 6-audio-generation-component
efa5708
@daavoo
fix(parse_script_to_waveform): Remove extra quote
dae21b3
@Kostis-S-Z Kostis-S-Z mentioned this pull request Dec 2, 2024
Merged
@daavoo daavoo mentioned this pull request Dec 2, 2024
Merged
@daavoo
Updates to demo to include audio part (#26)
be82900 
* Updates to generate audio in chunks

* Update spinner

* wip demo structure

* Use forked parler-tts. Use setup.sh

* Update demo

* Fix input_text

* Use cache_resource

* Drop print

* Add dividers

* Lint
@Kostis-S-Z Kostis-S-Z requested a review from daavoo December 4, 2024 09:28
daavoo
daavoo approved these changes Dec 4, 2024
View reviewed changes
Copy link
Contributor

@daavoo daavoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor comment on the docstring

src/document_to_podcast/inference/text_to_speech.py Outdated Show resolved Hide resolved
@Kostis-S-Z Kostis-S-Z merged commit dccd551 into main Dec 4, 2024
2 checks passed
@Kostis-S-Z Kostis-S-Z deleted the 6-audio-generation-component branch December 4, 2024 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@daavoo daavoo daavoo approved these changes

Assignees

@Kostis-S-Z Kostis-S-Z

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Audio Generation Component
2 participants
@Kostis-S-Z @daavoo