Help! Why is a multi-species model behaving so badly for one species?

[KristianBell]

We are developing a multi-species frog detector and have been getting decent success recognising most species, but the model is still very poor with one species in particular, even though this species has quite a distinctive 'growling' call.

To train for this species we have used:

83 positive training files (many of which have other species also calling)
103 negative training files (mostly of similar sounding birds)
745 noise files

Despite similar numbers to the above being sufficient for many other species, the model is still returning almost entirely false positives and at unrealistically high levels of 'confidence' - many at or close to 1.0 - when the actual audio is often just noise.

Why is the model so bad? Why is it so confident of detections that seemingly bear no resemblence to the target species? What would be the most efficient approach to improve the model - more positives, negatives, noise etc?

[abfleishman]

What is the dominant frequency and frequency range of the problem species (and what species ?)?

Is there something different/strange about the training recordings compared to the recordings that you are running inference on?

[KristianBell]

The frequency is generally 1-2KHz range. Its a growling grass frog (Litoria raniformis)

There is nothing notably different about the training recordings. Quite a few were collected off publicly available calls away from the study area but the majority are calls from our exact recordings/sites. This same method has proved fine for most other species. The nature of the misidentifications is particularly confusing as much of it is just noise with no discernable calls and with an associated confidence of 1.0 despite many negative and noise examples.