Enhancing an Avian Sound Recognition Model's Detection Precision via Logistic Regression of Large Acoustic Datasets: A Case Study of the European Robin (Erithacus rubecula)

Summary

The goal of this protocol is to determine species- and site-specific confidence score thresholds using logistic regression to improve detection precision in large acoustic datasets processed with automated acoustic recognition software.

Abstract

Passive acoustic monitoring (PAM) has become an invaluable tool for biodiversity research, enabling the non-invasive collection of vast datasets. However, a significant challenge remains in efficiently and reliably processing this large volume of data to extract species-specific information across varying locations. This paper presents a detailed, step-by-step protocol to address this challenge using a machine learning detector module within a bioacoustics analysis software. The methodology is designed to accurately and confidently identify and validate bird vocalisations from raw acoustic recordings.

Our protocol details the process from initial data collection using autonomous recording units (ARUs) to the final generation of a high-quality annotated dataset. Key steps include configuring the machine learning detector module to generate initial detections, a manual validation procedure to calculate precision tables, and a logistic regression analysis to determine a species-specific and, where appropriate, a location-specific confidence score threshold. This statistically derived threshold is then used to refine the detector’s output, tested on two overlap configurations (0 s and 2 s). We show that applying the derived optimal confidence score thresholds substantially improves the machine learning-based avian sound recognition models detection precision across sites. For the three sites used to illustrate the process (Liverpool Park, Cairngorms, and Glasgow Suburban) precision increased by 26.1%, 17.7%, and 17% for an overlap of 0 s, and by 28.77%, 16.87%, and 15% for an overlap of 2 s. We suggest the resulting methodology is superior to manual counting methods in both speed and reliability. In summary, this paper provides a reproducible framework that facilitates the accessible and effective use of machine learning approaches in bioacoustics, enabling researchers to confidently leverage large acoustic datasets for ecological studies and parameter analysis.

Introduction

Animal vocalisations offer valuable insights into the natural world and act as powerful biological tools allowing researchers to investigate various aspects of animals lives¹. The field of bioacoustics has become increasingly important in biodiversity monitoring and conservation, aiding in species identification and in assessing population trends and ecosystem health^2,3,4. This field of research also contributes to our understanding of animal behaviour, for example, in communication and habitat use^5,....

Protocol

Ethics statement
This protocol follows the ethical guidelines of Liverpool John Moores University. Methodology involved passive acoustic monitoring only, and did not include the handling, disturbance, or experimental manipulation of animals. No hazardous procedures were required to deploy the ARUs. ARUs were deployed in accordance with institutional research and fieldwork guidelines, and appropriate signage was installed at recording sites to inform the public of acoustic monitoring. However, it is highly advised to check weather forecasts before conducting field activity, and to conduct a risk assessment for each field site.

Results

To illustrate the effectiveness of the described protocol, we present representative results from the analysis of acoustic data for the European Robin (Erithacus rubecula) collected at three sites of varying environments and avian species assemblages, including a park in Liverpool (ARU 1), a naturally regenerating woodland site within The Cairngorms National Park (ARU 2), and a suburban site in Glasgow (ARU 3). These examples demonstrate how applying statistically derived confidence score thresholds can improve .......

Discussion

The use of the protocol outlined in this paper provides a reliable, streamlined method for generating species-specific and accurately classified datasets of bird vocalisations, which has proven useful when handling large audio datasets. We chose to test the protocol on the European Robin due to its well-known structural and frequency variability within its song repertoires⁴³. Meaning initially, detections were found to have high mislabelling (false positive) rates (Table 1). We al.......

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Autonomous Recording Units v1.2.0	Open Acoustic Devices	N/A	Autonomous recording unit (ARU)
Bioacoustics Analysis Software v1.6.5	Cornell Lab of Ornithology	RRID:SCR_016190	Sound analysis software
Data Transfer Cable	Various Manufacturers	N/A	Used to transfer data from devices
Device Configuration Applications	Open Acoustic Devices	N/A	Apps for configuring AudioMoths
Machine Learning Detector Module v2.4	Cornell Lab of Ornithology	N/A	Machine learning detector
Rechargeable Alkaline Batteries	Various Manufacturers	N/A	Power supply for recording units
Removable Memory Cards	Various Manufacturers	N/A	Data storage for recordings
Statistical Computing Software	R Core Team	RRID:SCR_001905	Statistical computing software