To add OpenAI Whisper, or Fast-Whisper or Faster-Whisper-XXL

[cutegitcat]

Hello everybody!
I have been using Aegisub for a long time to create/make subtitles. It is an essential tool for me. However, the software is missing an important feature: an audio-to-text function such as OpenAI Whisper, Fast-Whisper, or the advanced version, Faster-Whisper-XXL. Another provider of open-source software, Tero Subtitler, already offers such a feature. I would be delighted and very grateful if a similar function could be integrated into Aegisub in the future.
Thank you for helping to improve accessibility for people with hard of hearing (hearing loss) and for those who are deaf.

[AuroraMartell]

Hello everybody! I have been using Aegisub for a long time to create/make subtitles. It is an essential tool for me. However, the software is missing an important feature: an audio-to-text function such as OpenAI Whisper, Fast-Whisper, or the advanced version, Faster-Whisper-XXL. Another provider of open-source software, Tero Subtitler, already offers such a feature. I would be delighted and very grateful if a similar function could be integrated into Aegisub in the future. Thank you for helping to improve accessibility for people with hard of hearing (hearing loss) and for those who are deaf.

Seconded.

[petzku]

While this would no doubt be useful, can users not already use such tools externally and just load those output files into Aegisub? As I understand, Whisper outputs (or at least, can output) SRT, which can be opened normally in Aegisub.

Especially considering how you've already named three different versions of Whisper, integrating this directly into Aegisub would add another dependency that is likely to see significant updates in the near future, so I'm not sure going down that path is the best idea. Perhaps a reasonable compromise is improving the scripting API to allow for e.g. more direct Python interop, in which case such integrations would be handled by more easily-updated scripts, rather than requiring an entirely new Aegisub version. (this sounds especially appealing, considering how long upstream development was stalled out)

[AuroraMartell]

While this would no doubt be useful, can users not already use such tools externally and just load those output files into Aegisub? As I understand, Whisper outputs (or at least, can output) SRT, which can be opened normally in Aegisub.

Especially considering how you've already named three different versions of Whisper, integrating this directly into Aegisub would add another dependency that is likely to see significant updates in the near future, so I'm not sure going down that path is the best idea. Perhaps a reasonable compromise is improving the scripting API to allow for e.g. more direct Python interop, in which case such integrations would be handled by more easily-updated scripts, rather than requiring an entirely new Aegisub version. (this sounds especially appealing, considering how long upstream development was stalled out)

What about a mini app that can directly open the output on Aegisub or any other subtitle software.

[cutegitcat]

Thank you for discussing subtitling! In today's digital age, AI tools such as Whisper offer efficient ways to generate subtitles automatically, saving a significant amount of time. However, final checks are still important to ensure accuracy, correct reading times and appropriate line lengths.

It would be fantastic if technologies like Whisper could be integrated directly into Aegisub, either as a plug-in or as a built-in feature. This would streamline the workflow and eliminate the need to switch between multiple programs such as Tero Subtitler and others, which can often be complex and error-prone. Offline solutions would also be more privacy friendly.

I hope this can be realised in the near future, and I wish the development team every success in implementing this feature! 🙂

[AuroraMartell]

Thank you for discussing subtitling! In today's digital age, AI tools such as Whisper offer efficient ways to generate subtitles automatically, saving a significant amount of time. However, final checks are still important to ensure accuracy, correct reading times and appropriate line lengths.

It would be fantastic if technologies like Whisper could be integrated directly into Aegisub, either as a plug-in or as a built-in feature. This would streamline the workflow and eliminate the need to switch between multiple programs such as Tero Subtitler and others, which can often be complex and error-prone. Offline solutions would also be more privacy friendly.

I hope this can be realised in the near future, and I wish the development team every success in implementing this feature! 🙂

I use Subtitle Edit more often precisely because it has Whisper support.

[cutegitcat]

I use Subtitle Edit more often precisely because it has Whisper support.

I know this software, but it is too complicated and has too many functions that not everyone can use. It also only runs on Windows, not on Mac or Ubuntu. Agisub is more flexible and runs on all operating systems. We are staying with Agisub, but you should also try out other free software to perhaps find ideas for further development. I wish the development team good luck and success.

[AuroraMartell]

I know this software, but it is too complicated and has too many functions that not everyone can use. It also only runs on Windows, not on Mac or Ubuntu. Agisub is more flexible and runs on all operating systems. We are staying with Agisub, but you should also try out other free software to perhaps find ideas for further development. I wish the development team good luck and success.

I use both, so it'd be good to have options for both. And I have Windows and Mac. Tried Linux, too complicated for me.

[arch1t3cht]

I agree with petzku, adding something like Whisper would be a lot of work for something that would also work fairly well as an external tool. I don't want to say it won't ever get added, but it's pretty far down on the list of priorities (and even then I'd much rather find a way to call it via an automation script than integrate it directly). For the moment, my main development focus (aside from fixes and maintenance) is improving the things that can only be done with Aegisub, rather than making it possible to do everything with Aegisub.

[AuroraMartell]

I agree with petzku, adding something like Whisper would be a lot of work for something that would also work fairly well as an external tool. I don't want to say it won't ever get added, but it's pretty far down on the list of priorities (and even then I'd much rather find a way to call it via an automation script than integrate it directly). For the moment, my main development focus (aside from fixes and maintenance) is improving the things that can only be done with Aegisub, rather than making it possible to do everything with Aegisub.

I can understand.