Increase concurrency capability

[daiDai-study]

I want to increase concurrency capability, but i see source code like:

    with model_lock:
        segments = []
        text = ""
        segment_generator, info = model.transcribe(audio, beam_size=5, **options_dict)
        for segment in segment_generator:
            segments.append(segment)
            text = text + segment.text
        result = {"language": options_dict.get("language", info.language), "segments": segments, "text": text}

can i remove the lock?

[aidancrowther]

From my work with the code, the lock prevents multiple accesses to the same model instance. If you wanted to run multiple in parallel, you would need to initialize multiple models. As the model, when running, will more or less saturate the processing capability of the GPU/CPU it is running on, this likely wouldn't be as useful as you hope. It is much easier/configurable to spawn multiple instances of the docker container instead. For example, using --gpus device=n where n is the id of a compatible GPU allows you to run the model on a specific GPU. I have used this to run testing on two Nvidia Tesla GPUs as well as the system CPU