v0.10.0

[sammlapp]

OpenSoundscape v0.10.0

This release contains substantial updates to the OpenSoundscape code base, including some breaking changes, new features, and bug fixes.

Highlights:

• Use pre-trained bird identification models including BirdNET and Google Perch through the Bioacoustics Model Zoo. Examples here with each supported model. A quick example with Google Perch:

import torch
model=torch.hub.load('kitzeslab/bioacoustics_model_zoo', 'Perch')
predictions = model.predict(['test.wav']) #predict on the model's classes
embeddings = model.generate_embeddings(['test.wav']) #generate embeddings on each 5 sec of audio

• The acoustic localization module, still in Beta, has been updated with improved tools. We've also added a tutorial notebook in the documentation

• Enhanced automatic Weights and Biases (wandb) logging from the CNN class now includes GradCAM visualizations of samples during CNN.predict() and per-class area-under-the-curve metrics during CNN.train()

• Apple Silicon GPU (mps) is automatically used by CNN class if available

• Overlay (mixup) augmentation provides the ability to only perform overlay on a subset of training samples

• Audio class has new filtering methods .lowpass() and .highpass()

• The documentation at opensoundscape.org has been updated including refreshed tutorial notebooks, which can now be run interactively on Google CoLab without requiring any installation of Python or packages on the user's computer.

• We've added utilities (opensoundscape.ml.utils.collate_audio_samples_to_tensors()) and documentation (see "Orientation for PyTorch users") for PyTorch users using OpenSoundscape only for preprocessing

• The behavior of Spectrogram and MelSpectrogram has been improved, resolving issues that created -inf values and/or clipped the values of the spectrogram to a finite range.

The full API is documented at opensoundscape.org. You'll also find detailed tutorials on the most common use-cases for OpenSoundscape: automated sound identification with machine learning and signal processing; training machine learning algorithms to detect sounds; localizing acoustic sound sources; manipulating audio, spectrograms and annotations.

Also check out the Bioacoustics Cookbook for examples of common workflows, including the use of a config file for CNN training experiments.

Breaking changes

• can no longer pass decibel_limits to to_spec action of SpectrogramPreprocessor, see discussion in thread below for alternative approach
• SpectrogramPreprocessor class previously took a "shape" argument now takes separate height and width arguments to clarify which dimension corresponds to h/w.

What's Changed

• resolve MelSpectrogram has all -inf values MelSpectrogram has all -inf values #794 by @sammlapp in resolve MelSpectrogram has all -inf values #794 #802
• resolve change default sample shape of SpectrogramPreprocessor to None #792 no default shape by @sammlapp in resolve #792 no default shape #804
• Issue 789 per class metrics by @sammlapp in Issue 789 per class metrics #806
• Issue 779 load model helpful error message by @sammlapp in Issue 779 load model helpful error message #805
• automatically use mps if available by @sammlapp in automatically use mps if available #809
• dont log_graph in wandb.watch by @sammlapp in dont log_graph in wandb.watch #808
• add Audio.highpass and Audio.lowpass by @sammlapp in add Audio.highpass and Audio.lowpass #815
• Issue 784 boxed ann by @sammlapp in Issue 784 boxed ann #813
• Raven file not retained in BoxedAnnotations if keep_extra_columns=False #769: Raven file not retained in BoxedAnnotations if keep_extra_column not set to True by @syunkova in #769: Raven file not retained in BoxedAnnotations if keep_extra_column not set to True #812
• Add links to source code in docs by @sammlapp in Add links to source code in docs #836
• Issue 840 posix paths by @louisfh in Issue 840 posix paths #846
• fix Audio.from_url raising AxisError Audio.from_url raising AxisError #837 by @sammlapp in fix Audio.from_url raising AxisError #837 #843
• use local version of ScoreCAM with bug fix by @sammlapp in use local version of ScoreCAM with bug fix #842
• add criterion_fn to overlay (selective overlay) by @sammlapp in add criterion_fn to overlay (selective overlay) #839
• Issue 767 tqdm by @sammlapp in Issue 767 tqdm #834
• Support tensorflow model hub inference by @sammlapp in Support tensorflow model hub inference #835
• Issue 768 cam to cpu by @sammlapp in Issue 768 cam to cpu #810
• MPS memory issue on github macos runner workarouns by @louisfh in MPS memory issue on github macos runner workarouns #851
• Docs refresh by @rhine3 in Docs refresh #848
• Feat localize parallelize by @louisfh in Feat localize parallelize #783
• Add notebook for localization tutorial by @louisfh in Add notebook for localization tutorial #853
• Issue 790 pytorch integration by @sammlapp in Issue 790 pytorch integration #849
• make copy in one_hot_labels_on_time_interval by @sammlapp in make copy in one_hot_labels_on_time_interval #858
• Add GradCAM visualization to wandb during CNN.predict by @sammlapp in Add GradCAM visualization to wandb during CNN.predict #862
• Issue_772_bandpass_warning by @sammlapp in Issue_772_bandpass_warning #860
• Tutorial updates by @sammlapp in Tutorial updates #866
• Issue 793 spectrogram limits by @sammlapp in Issue 793 spectrogram limits #868
• refactor safe_dataset resolve "No safe samples" exception in SafeDataset should include traceback of (last?) error #672 by @sammlapp in refactor safe_dataset resolve #672 #869
• 0.10.0 release by @sammlapp in 0.10.0 release #867

Full Changelog: v0.9.1...v0.10.0

---

This discussion was created from the release v0.10.0.

[canihaveabravo]

Hello,

Since updating to this version of OpSo the Spectrogram Pre-processor no longer executes a decibel_limits parameter change:

adjust spectrogram parameters

pre.pipeline.to_spec.params.window_samples = 1024
pre.pipeline.to_spec.params.overlap_fraction = 0.5
pre.pipeline.to_spec.params.decibel_limits = (-120, -30)
pre.pipeline.to_spec.params.dB_scale = True
pre.pipeline.to_spec.params.scaling = "spectrum"

pre.pipeline.to_spec.params

window_type hann
window_samples 1024
window_length_sec None
overlap_samples None
overlap_fraction 0.5
fft_size None
dB_scale True
scaling spectrum
dtype: object

Why is this?

[sammlapp]

The Spectrogram class has been modified so that instead of restricting the spectrogram values to a range (decibel_limits), it now retains all values, but the Spectrogram.to_image() function provides the opportunity to provide desired decibel range.

However, when I tried to provide an example of how to adapt your code I noticed that the pre.pipeline.to_tensor is not implemented in a way that allows you to provide decibel_limits. Apologies for that, and we'll resolve that issue soon. In the mean time, you can over-ride the default to_tensor action and provide the range argument like this:

from opensoundscape.preprocess.actions import Action
from opensoundscape import Spectrogram
class SpectrogramToTensor(Action):
    """Action to create Tesnsor of desired shape from Spectrogram

    calls .to_image on sample.data, which should be type Spectrogram
    
    **kwargs are passed to Spectrogram.to_image()
    
    """
    def __init__(
        self,
        fn=Spectrogram.to_image,
        is_augmentation=False,
        **kwargs
    ):
        kwargs.update(dict(return_type='torch')) #return a tensor, not PIL.Image
        super(SpectrogramToTensor, self).__init__(fn,is_augmentation,**kwargs)
    def go(self, sample, **kwargs):
        """converts sample.data from Spectrogram to Tensor"""
        # sample.data must be Spectrogram object
        # sample should have attributes: height, width, channels
        # use info from sample for desired shape and n channels
        kwargs.update(
            shape=[sample.height, sample.width],
            channels=sample.channels
        )
        # clip spectrogram values
        spec = sample.data
        if 'range' in self.params:
            # clip values to desired range
            spec = spec._spawn(spectrogram=spec.spectrogram.clip(
                self.params['range'][0],self.params['range'][1])
                )
        sample.data = spec.to_image(**dict(self.params, **kwargs))

# generate an instance of this action with the desired decibel range
my_action = SpectrogramToTensor(range=[-100,-20])

# replace to_tensor action with this action
pre.pipeline.to_tensor = my_action

I opened an issue #881 for us to resolve this missing functionality. Thanks for your message

[canihaveabravo]

thank you for this work around. :)

[sammlapp]

@canihaveabravo I noticed that the above workaround has undesirable behavior if the spectrograms contain NaN values, so I'm updating the workaround code above with a modified version

[canihaveabravo]

Okay, thank you for letting me know. Please share when you have the new code- thanks very much for your help

[sammlapp]

the code above is now updated