Validationexplorer - A Tool for Simulation-Based Investigations of Study Design Elements to Estimate Relative Activity Rates and Occurrence Probabilities for Bat Assemblages

Abstract

Bats in North America face emerging threats due to the spread of the bat disease white-nosesyndrome (WNS), the expanding footprint of the wind energy industry, and the effects of global change on suitable bat habitats. One goal of the North American Bat Monitoring Program (NABat) is to monitor status and trend indicators for bat species assemblages at varying spatial extents. Autonomous recording units can efficiently gather data from bats but also necessitate the use of auto-classifiers to assign species labels to large numbers of observations, which introduces misclassification error. Statistical models that account for misclassification error require additional information to estimate misclassification rates. For bat acoustic data, providing this information through human verification of a subset of observed machine- generated species labels can result in credible intervals for relative activity and occurrence that are more precise than those resulting from using auxiliary data or informative priors. We define a validation design as a mechanism for probabilistic selection of recordings from the population of observed recordings (i.e., the design type), together with the level of validation effort (e.g., validation sample size, percentage of recordings). We present a statistical framework and associated software tool, Validationexplorer, that — given a predetermined set of measurable objectives — affords comparisons of validation designs (and other study design elements) to provide insight into balancing validation costs with programmatic goals.