VrChat Sign-Language Transcription (also referred to as VRC_SLT) is an in progress system written in UdonSharp and in HLSL that can do one-to-one word transcriptions from sign to text. I have been working on this behind the scenes for some time, but this still has some ways to go. My hope is that this can be used as a backend for other map makers to build games that use Sign Language, potentially as an avenue to help people practice and learn individual signs.
FirstClip-SL.mp4
TO BE ABUNDANTLY CLEAR: VRC_SLT is not and will NEVER be intended to outright replace knowing sign language! Sign Languages are not 1-1 mappings of any spoken language and have their own complex grammar rules and quirks! I hope for this to one day be a useful tool for practicing sign language vocabulary, such as for creating vocab games. But, to actually learn how to sign properly in conversation, you will need to dig deeper into the grammar rules and practice with fellow human beings, just as you would when learning any other spoken / written language. ANYONE who says this system or systems like this can replace sign language interpreters frankly have no clue what they are talking about.
Once in a releasable state and past testing, VRC_SLT will be made fully open source and free to use! For now Only this documentation shall be visible!
Feel free to read the FAQ HERE
The technology behind this is made possible by Pytorch and my tool for exporting Pytorch models into Unity Torch2VRC