Spending degrees of freedom in a poor economy: A case study of building a sightability model for moose in northeastern Minnesota

• Published in: The Journal of Wildlife Management
• Main Authors: Giudice, John H., Fieberg, John R., Lenarz, Mark S.
• Format: Article in Journal/Newspaper
• Language: English
• Published: Wiley 2011
• Subjects: Alces alces
• Online Access: http://dx.doi.org/10.1002/jwmg.213; https://api.wiley.com/onlinelibrary/tdm/v1/articles/10.1002%2Fjwmg.213; https://onlinelibrary.wiley.com/doi/full/10.1002/jwmg.213

Abstract Sightability models are binary logistic‐regression models used to estimate and adjust for visibility bias in wildlife‐population surveys. Like many models in wildlife and ecology, sightability models are typically developed from small observational datasets with many candidate predictors. Aggressive model‐selection methods are often employed to choose a best model for prediction and effect estimation, despite evidence that such methods can lead to overfitting (i.e., selected models may describe random error or noise rather than true predictor–response curves) and poor predictive ability. We used moose ( Alces alces ) sightability data from northeastern Minnesota (2005–2007) as a case study to illustrate an alternative approach, which we refer to as degrees‐of‐freedom (df) spending: sample‐size guidelines are used to determine an acceptable level of model complexity and then a pre‐specified model is fit to the data and used for inference. For comparison, we also constructed sightability models using Akaike's Information Criterion (AIC) step‐down procedures and model averaging (based on a small set of models developed using df‐spending guidelines). We used bootstrap procedures to mimic the process of model fitting and prediction, and to compute an index of overfitting, expected predictive accuracy, and model‐selection uncertainty. The index of overfitting increased 13% when the number of candidate predictors was increased from three to eight and a best model was selected using step‐down procedures. Likewise, model‐selection uncertainty increased when the number of candidate predictors increased. Model averaging (based on R = 30 models with 1–3 predictors) effectively shrunk regression coefficients toward zero and produced similar estimates of precision to our 3‐df pre‐specified model. As such, model averaging may help to guard against overfitting when too many predictors are considered (relative to available sample size). The set of candidate models will influence the extent to which coefficients are ...