Segmenting a long audio file

[dgoryeo]

I have a use case to breal a long audio file into smaller chunks for STT processing. Am I right to think that if I set the threshold parametr to very low (something like 0.2) then VAD would act as a basic segmenting tool in my use case? Thanks.

[snakers4]

You need to plot the probabilities with the in-built tool, and you will be able to eye-ball the probabilities easily.

[snakers4]

silero-vad/utils_vad.py

Lines 164 to 175 in 4f57fae

	def get_speech_timestamps(audio: torch.Tensor,
	model,
	threshold: float = 0.5,
	sampling_rate: int = 16000,
	min_speech_duration_ms: int = 250,
	max_speech_duration_s: float = float('inf'),
	min_silence_duration_ms: int = 100,
	window_size_samples: int = 512,
	speech_pad_ms: int = 30,
	return_seconds: bool = False,
	visualize_probs: bool = False,
	progress_tracking_callback: Callable[[float], None] = None):

[EtienneAb3d]

@dgoryeo, I would perhaps rather use a standard threshold, with a high min_silence_duration_ms, perhaps 1 or 2 seconds, to cut on long parts without voice. I think it's more pertinent for your STT purpose: you will get more chance to cut between sentences rather than in a middle of them.

[snakers4]

It is worth tweaking all of these params for your domain, if you have the time.

[dgoryeo]

So far I have tried various permutations of threshold, min_speech, and min_silence. I then compare the results with 2 other results that I have previously generated from split_on_silence and ffmpeg silencedetect. In my test audio, I noticed that there is a spot where any thershold above 0.35 misses --it is a sudden background music behind the dialogue.

I probably am misusing VAD for my usecase. My intended usecase is to identify and split audio into "scenes" based on their characteristics/content. Then preprocess each scene with suitable parameters for feeding to STT (Whisper in my case). Example if in a scene there is highway noise, the scene gets denoised before feeding to Whisper. Or if quiet talk, it can be preprocessed for loudeness, or if baby crying it can supress that.