[DEL-10418] Implement basic langid jobs submitting

CHANGELOG.md

@@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add support for language identification (DEL-10418)
+
+### Fixed
+
+- Fixed an issue where `transcription_config` was not correctly loaded from the JSON config file
+
 ## [1.6.5] - 2023-02-22
 
 ### Fixed

README.md

@@ -168,6 +168,16 @@ A complete list of commands and flags can be found in the SDK docs at https://sp
    # $JOB_ID is from the submit command output
    $ speechmatics batch get-results --job-id $JOB_ID
     ```
+
+- Submit a job with automatic language identification
+
+    ```shell
+   $ speechmatics batch transcribe --language auto --langid-langs en,es example_audio.wav
+    ```
+    If Speechmatics is not able to identify a language with high enough confidence,  the job will be rejected. This is to reduce the risk of transcribing incorrectly.
+
+    `--langid-langs` is optional and specifies what language(s) you expect to be detected in the source files.
+
 
 - Submit a job with translation (translation output only available as JSON)
 
@@ -176,6 +186,9 @@ A complete list of commands and flags can be found in the SDK docs at https://sp
     ```
   `--translation-langs` is supported in asynchronous mode as well, and translation output can be retrieved using `get-results` with `--output-format json-v2` set.
 
+  When combining language identification with translation, we can't know if the identified language can be translated
+  to your translation targets. If the translation pair is not supported, the error will be recorded in the metadata of the transcript.
+
   ### Custom Transcription Config File
 - Instead of passing all the transcription options via the command line you can also pass a transcription config file.
   The config file is a JSON file that contains the transcription options.

docs/_modules/index.html

@@ -112,7 +112,7 @@ <h3 id="searchlabel">Quick search</h3>
 
       |
       Powered by <a href="http://sphinx-doc.org/">Sphinx 4.4.0</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.13</a>
+      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
 
     </div>
 

docs/_modules/speechmatics/batch_client.html

@@ -62,7 +62,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
 
 
 <div class="viewcode-block" id="BatchClient"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient">[docs]</a><span class="k">class</span> <span class="nc">BatchClient</span><span class="p">:</span>
-<span class="w">    </span><span class="sd">&quot;&quot;&quot;Client class for Speechmatics Batch ASR REST API.</span>
+    <span class="sd">&quot;&quot;&quot;Client class for Speechmatics Batch ASR REST API.</span>
 
 <span class="sd">    This client may be used directly but must be closed afterwards, e.g.::</span>
 
@@ -82,7 +82,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
 <span class="sd">    &quot;&quot;&quot;</span>
 
     <span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">connection_settings</span><span class="p">:</span> <span class="n">ConnectionSettings</span><span class="p">):</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;Constructor method.</span>
+        <span class="sd">&quot;&quot;&quot;Constructor method.</span>
 
 <span class="sd">        :param connection_settings: Connection settings for API</span>
 <span class="sd">        :type connection_settings: speechmatics.models.ConnectionSettings.</span>
@@ -103,7 +103,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
         <span class="bp">self</span><span class="o">.</span><span class="n">api_client</span> <span class="o">=</span> <span class="kc">None</span>
 
 <div class="viewcode-block" id="BatchClient.connect"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.connect">[docs]</a>    <span class="k">def</span> <span class="nf">connect</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;Create a connection to a Speechmatics Transcription REST endpoint&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;Create a connection to a Speechmatics Transcription REST endpoint&quot;&quot;&quot;</span>
         <span class="bp">self</span><span class="o">.</span><span class="n">api_client</span> <span class="o">=</span> <span class="n">httpx</span><span class="o">.</span><span class="n">Client</span><span class="p">(</span>
             <span class="n">base_url</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">connection_settings</span><span class="o">.</span><span class="n">url</span><span class="p">,</span>
             <span class="n">timeout</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
@@ -121,7 +121,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
         <span class="bp">self</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
 
 <div class="viewcode-block" id="BatchClient.close"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.close">[docs]</a>    <span class="k">def</span> <span class="nf">close</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="kc">None</span><span class="p">:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Clean up/close client connection pool.</span>
 
 <span class="sd">        This is required when using the client directly, but not required when</span>
@@ -132,7 +132,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
         <span class="bp">self</span><span class="o">.</span><span class="n">api_client</span><span class="o">.</span><span class="n">close</span><span class="p">()</span></div>
 
 <div class="viewcode-block" id="BatchClient.send_request"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.send_request">[docs]</a>    <span class="k">def</span> <span class="nf">send_request</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">method</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">path</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">httpx</span><span class="o">.</span><span class="n">Response</span><span class="p">:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Send a request using httpx.Client()</span>
 
 <span class="sd">        :param method: HTTP request method</span>
@@ -155,7 +155,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
             <span class="k">return</span> <span class="n">response</span></div>
 
 <div class="viewcode-block" id="BatchClient.list_jobs"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.list_jobs">[docs]</a>    <span class="k">def</span> <span class="nf">list_jobs</span><span class="p">(</span><span class="bp">self</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">List</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]]:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Lists last 100 jobs within 7 days associated with auth_token for the SaaS</span>
 <span class="sd">        or all of the jobs for the batch appliance.</span>
 
@@ -171,7 +171,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
             <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">],</span> <span class="n">BatchTranscriptionConfig</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">PathLike</span>
         <span class="p">],</span>
     <span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Submits audio and config for transcription.</span>
 
 <span class="sd">        :param audio: Audio file path or tuple of filename and bytes</span>
@@ -194,7 +194,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
             <span class="n">config_json</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">transcription_config</span><span class="p">)</span>
         <span class="k">else</span><span class="p">:</span>
             <span class="k">raise</span> <span class="ne">ValueError</span><span class="p">(</span>
-<span class="w">                </span><span class="sd">&quot;&quot;&quot;Job configuration must be a BatchTranscriptionConfig object,</span>
+                <span class="sd">&quot;&quot;&quot;Job configuration must be a BatchTranscriptionConfig object,</span>
 <span class="sd">                a filepath as a string or Path object, or a dict&quot;&quot;&quot;</span>
             <span class="p">)</span>
         <span class="n">config_data</span> <span class="o">=</span> <span class="p">{</span><span class="s2">&quot;config&quot;</span><span class="p">:</span> <span class="n">config_json</span><span class="o">.</span><span class="n">encode</span><span class="p">(</span><span class="s2">&quot;utf-8&quot;</span><span class="p">)}</span>
@@ -217,7 +217,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
         <span class="n">job_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span>
         <span class="n">transcription_format</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;json-v2&quot;</span><span class="p">,</span>
     <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Union</span><span class="p">[</span><span class="nb">bool</span><span class="p">,</span> <span class="nb">str</span><span class="p">,</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]]:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Request results of a transcription job.</span>
 
 <span class="sd">        :param job_id: ID of previously submitted job.</span>
@@ -261,7 +261,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
         <span class="k">return</span> <span class="n">response</span><span class="o">.</span><span class="n">text</span></div>
 
 <div class="viewcode-block" id="BatchClient.delete_job"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.delete_job">[docs]</a>    <span class="k">def</span> <span class="nf">delete_job</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">job_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">force</span><span class="p">:</span> <span class="nb">bool</span> <span class="o">=</span> <span class="kc">False</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Delete a job. Must pass force=True to cancel a running job.</span>
 
 <span class="sd">        :param job_id: ID of previously submitted job.</span>
@@ -295,7 +295,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
             <span class="k">return</span> <span class="kc">False</span></div>
 
 <div class="viewcode-block" id="BatchClient.check_job_status"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.check_job_status">[docs]</a>    <span class="k">def</span> <span class="nf">check_job_status</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">job_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Check the status of a job.</span>
 
 <span class="sd">        :param job_id: ID of previously submitted job.</span>
@@ -318,7 +318,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
 <div class="viewcode-block" id="BatchClient.wait_for_completion"><a class="viewcode-back" href="../../batch_client.html#speechmatics.batch_client.BatchClient.wait_for_completion">[docs]</a>    <span class="k">def</span> <span class="nf">wait_for_completion</span><span class="p">(</span>
         <span class="bp">self</span><span class="p">,</span> <span class="n">job_id</span><span class="p">:</span> <span class="nb">str</span><span class="p">,</span> <span class="n">transcription_format</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;txt&quot;</span>
     <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Dict</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">Any</span><span class="p">]]:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;</span>
+        <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Blocks until job is complete, returning a transcript in</span>
 <span class="sd">        the requested format.</span>
 
@@ -373,7 +373,7 @@ <h1>Source code for speechmatics.batch_client</h1><div class="highlight"><pre>
     <span class="k">def</span> <span class="nf">_from_file</span><span class="p">(</span>
         <span class="bp">self</span><span class="p">,</span> <span class="n">path</span><span class="p">:</span> <span class="n">Union</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="n">os</span><span class="o">.</span><span class="n">PathLike</span><span class="p">],</span> <span class="n">filetype</span><span class="p">:</span> <span class="nb">str</span>
     <span class="p">)</span> <span class="o">-&gt;</span> <span class="n">Union</span><span class="p">[</span><span class="n">Dict</span><span class="p">[</span><span class="n">Any</span><span class="p">,</span> <span class="n">Any</span><span class="p">],</span> <span class="n">Tuple</span><span class="p">[</span><span class="nb">str</span><span class="p">,</span> <span class="nb">bytes</span><span class="p">]]:</span>
-<span class="w">        </span><span class="sd">&quot;&quot;&quot;Retrieve data from a file.</span>
+        <span class="sd">&quot;&quot;&quot;Retrieve data from a file.</span>
 <span class="sd">        For filetype==&quot;json&quot;, returns a dict</span>
 <span class="sd">        For filetype==&quot;binary&quot;, returns a tuple of (filename, data)</span>
 <span class="sd">        &quot;&quot;&quot;</span>
@@ -461,7 +461,7 @@ <h3 id="searchlabel">Quick search</h3>
 
       |
       Powered by <a href="http://sphinx-doc.org/">Sphinx 4.4.0</a>
-      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.13</a>
+      &amp; <a href="https://github.com/bitprophet/alabaster">Alabaster 0.7.12</a>
 
     </div>