Enhancing an Avian Sound Recognition Model's Detection Precision via Logistic Regression of Large Acoustic Datasets: A Case Study of the European Robin (Erithacus rubecula)

Our research focuses on developing a framework to help bioacousticians improve the precision of automated acoustic species detection. Without proper configuration, avian sound recognition models struggle to accurately detect vocalizations. This protocol aims to increase precision by determining species and site-specific confidence score thresholds.

To begin, connect each ARU individually to a computer using a data transfer cable and switch the ARUs to USB mode. Use the AudioMoth ARU device configuration application to set the ARUs to determine the sleep intervals and active recording periods. Identify the time windows when target species are most active, and schedule the recording periods accordingly.

Add multiple recording periods and schedule the recording dates by choosing the first and last recording dates. Use the ARU device configuration application to set the devices to record at a sampling rate of 48 kilohertz with medium gain. Insert each ARU device into an antistatic Ziploc bag and apply additional waterproof tape as needed to prevent water damage.

Then, deploy the ARU devices in the field and switch them to custom mode. Transfer all wave files from each removable memory card onto a computer. Open the Raven Pro 1.6 bioacoustics analysis software and select Tools, then Detector, and choose Learning Detector from the dropdown menu.

In the Choose Detector Inputs window, select Browse. In the subsequent Open Sound Files window, select all the wave files within the relevant ARU folder and click Open. This will open the Configure New Sound Window.

In the Configure New Sound Window, under Paging and Page sound, enter the appropriate page size for the length of each recording. Then, under Multiple files, select Open as file sequence in one window and click OK.An Open File Sequence popup and Choose Detector Inputs window appears, showing the addition of selected files to the sequence. In the returning Choose Detector Inputs window, select Waveform under Available Signals and Views and move it to Required Inputs using the Arrow button.

Confirm that the status under Required Inputs changes to Ready and click OK.This will open the Configure Machine Learning Detector window. Within the Configure Machine Learning Detector window, under the Inputs tab, select the desired model from Select Model. Leave the overlap at the default of zero seconds.

Click on the Outputs tab. Open the dropdown menu for output class file and select the output list corresponding to the geographical location of the study. Lower the threshold to 0.1 and select Apply All.

Ensure that all threshold values in the species list are set to 0.1. Select the Suppress All Species option so that all species are selected. Locate the target species and unselect it in the Suppress column.

Select OK to begin the learning detector processing. Monitor the progress manager to track percentage completed and time remaining. Open the saved learning detector selection table and transfer it to the spreadsheet software.

Confirm that only the target species appears under the Label column. Delete all columns, except the Selection and Score columns. Add a new column using the randomization function, and sort the rows based on this column, and then delete the Randomization column.

Retain the first 300 randomized detections or retain all if fewer than 300 exist. Save the validation dataset using the ARU name and an identifier, such as AM1_ValidationDataset. Open the Raven Pro 1.6 bioacoustics analysis software and select File.

Then, Open Sound Files to select all corresponding wave files, and click Open. This will open the Configure New Sound Window. In the Configure New Sound Window under Paging, select Page sound and enter the page size used previously.

Next, under multiple files, select Open as file sequence in one window and click OK.This will add the selected files to the sequence and open the spectrogram of each file among multiple selected files. Navigate to File and select Open Selection Table to open the Learning Detection Table for the ARU being analyzed. In the panel on the left of the sound view under the Layouts tab and Views, deselect the Waveform view, so all files display in Spectrogram view.

Open the validation spreadsheet alongside the bioacoustics analysis software with the prepared data. Create a new column in the spreadsheet and label it Correctly Detected. From the spreadsheet, select a number from the randomized list and enter it in the draw field within the spectrogram view.

Then, click Play. Enter 1 for correctly detected or 0 for incorrectly detected in the Correctly Detected column. Calculate the precision of the detector using the number of correct detections and total validated detections, and multiply by 100 to express as a percentage.

Use the software RStudio to import the validation data set created in the previous section. Run a logistic regression on the data set using the generalized linear modeling function with the family set as binomial. Set the desired probability of precision as 0.9, so that the probability that a detection is a true positive is 90%or above.

Calculate the optimal threshold by solving the fitted logistic regression equation for the predictor value and verify the obtained threshold value. Rerun the learning detector on the ARU being investigated using the previous Detector setup. Select the sound files.

Enter the page size and select Open as file sequence in one window. Next, select Waveform in the Choose Detector Inputs window and move it to Required Inputs using the Arrow button and click OK.Under the Inputs tab of Configure Machine Learning Detector window, select the desired model and verify that the overlap is set at zero seconds. Next, in the Outputs tab, select the geographical location in the output class file and change the threshold for the target species to the threshold determined in the previous step and select Apply all.

Then, select the Suppress All Species option. Unselect the box for target species and click OK to open the Progress Manager window. Finally, create a new list of detections with improved accuracy and fewer false positives using the optimized threshold value.

Representative results from the analysis of acoustic data for the European robin collected at a park in Liverpool, a naturally regenerating woodland site within the Cairngorms National Park, and a suburban site in Glasgow showed how statistically-derived confidence score thresholds improved detection precision across varying acoustic environments. Logistic regression analysis determined an optimal confidence score threshold of 0.54 for the Liverpool Park site and 0.23 for the Cairngorms site to achieve a precision rate of 90%or higher. Rerunning the detector at the Liverpool Park and Cairngorms sites with the optimized threshold increased detection precision to above 90%for both zero and two-second overlap conditions.

Rerunning the detector at the Glasgow suburban site with the 0.36 optimized threshold improved detection precision, but the overall post regression precision remained below 90%for both zero and two-second overlap conditions. This protocol allows researchers to generate accurate detections of target species within large and acoustically diverse data sets. While this protocol reduces false positives, increasing detection precision, it may also miss some true detections, highlighting a key trade off with recall.

Following the production of accurate detections, vocalizations can be annotated within software for parameter extraction and further analysis of acoustic behaviors.