Speech Recognition [resolved]

[93Dennis]

Hey guys, I'm developing an app for illiterates. Because the web API is not widely supported, I decided to do a native app and stumbled over NativeScript (+ Vue) with this plugin: NativeScript Speech Recognition.
I tested it on both iOS and on Android, but on iOS it just crashed and on Android it says the API is not available. The plugin maintainer hasn't answered issues for years, so I want to try here. Are you guys still using that plugin and it works (so it's me?) or is there another way to get the speech recognition working?

Thanks in advance!

My Home.vue:

<template>
  <Page>
    <ActionBar>
      <Label text="Speech API Test"/>
    </ActionBar>

    <StackLayout>
      <Button text="Start" @tap="start" />
      <Button text="Stop" @tap="stop" />
      <TextField :text="recognizedText" :editable="false" />
    </StackLayout>
  </Page>
</template>

<script lang="ts">
import Vue from "nativescript-vue"
import {
  SpeechRecognition,
  SpeechRecognitionOptions,
  SpeechRecognitionTranscription
} from 'nativescript-speech-recognition'

const speechRec = new SpeechRecognition()

export default Vue.extend({
  name: 'Home',
  data() {
    return {
      recognizedText: ''
    }
  },
  methods: {
    start() {
      console.log('tap')
      speechRec.available().then((result: boolean) => {
        console.log('RESULT', result)
        if (!result) {
          alert('no recognition available')
          console.log('no recognition available')
          return
        }

        speechRec.startListening({
          locale: 'de',
          onResult: (transcript: SpeechRecognitionTranscription) => {
            this.recognizedText = transcript.text
          }
        })
      })
    },
    stop() {
      speechRec.stopListening()
    }
  }
})
</script>

[shirakaba]

I don't personally have a hand in that package, but in case it helps, here's how I implemented speech recognition in iOS (not structured as a plugin; just directly calling the APIs) with NativeScript imperative APIs (so it'd work on any flavour): https://github.com/shirakaba/nativescript-grimoire/tree/master/speech%20recognition. All the speech recognition logic is in the Transcriber class; the UI could be set up much more cleanly in a more idiomatic Vue way.

NOTE: on iOS, you'll need to add these four lines to your app's Info.plist file, whether with my code or with nativescript-speech-recognition:

https://github.com/shirakaba/nside/blob/8235d64020a439915dfc8c3c1581db4d27483279/App_Resources/iOS/Info.plist#L57-L60

Be sure to rebuild the app to make sure that the Info.plist file gets copied into the build.

I can't help with the Android side, I'm afraid! Though maybe similarly you need to set up permissions.

[93Dennis]

Thanks! iOS works now, after I added those four keys to the Info.plist:

<key>NSSpeechRecognitionUsageDescription</key>
<string>My custom recognition usage description. Overriding the default empty one in the plugin.</string>
<key>NSMicrophoneUsageDescription</key>
<string>My custom microphone usage description. Overriding the default empty one in the plugin.</string>
<key>NSMicrophoneUsageDescription</key>
<string>I want to record you live</string>
<key>NSSpeechRecognitionUsageDescription</key>
<string>I want to write down everything you say</string>

But Android still doesn't work after adding following permission to the AndroidManifest.xml:
<uses-permission android:name="android.permission.RECORD_AUDIO"/>

Hm.. maybe I did miss something. Havn't even had the permissions added in the first place. One step further, thanks!

[93Dennis]

Alright, got it running. Needed to set the aforementioned permissions for both Android and iOS. After that, it kept not working, because the stopListening method of the plugin I used cancled the recognizer, so there couldn't be any results. Wow.