Skip to Main Content
Metrics for evaluating performance and uncertainty of Bayesian network modelsAuthor(s): Bruce G. Marcot
Source: Ecological Modeling. 230: 50-62
Publication Series: Scientific Journal (JRNL)
Station: Pacific Northwest Research Station
PDF: Download Publication (1.09 MB)
DescriptionThis paper presents a selected set of existing and new metrics for gauging Bayesian network model performance and uncertainty. Selected existing and new metrics are discussed for conducting model sensitivity analysis (variance reduction, entropy reduction, case file simulation); evaluating scenarios (influence analysis); depicting model complexity (numbers of model variables, links, node states, conditional probabilities, and node cliques); assessing prediction performance (confusion tables, covariate and conditional probability-weighted confusion error rates, area under receiver operating characteristic curves, k-fold cross-validation, spherical payoff. Schwarz' Bayesian information criterion, true skill statistic, Cohen's kappa); and evaluating uncertainty of model posterior probability distributions (Bayesian credible interval, posterior probability certainty index, certainty envelope, Gini coefficient). Examples are presented of applying the metrics to 3 real-world models of wildlife population analysis and management. Using such metrics can vitally bolster model credibility, acceptance, and appropriate application, particularly when informing management decisions.
- You may send email to email@example.com to request a hard copy of this publication.
- (Please specify exactly which publication you are requesting and your mailing address.)
CitationMarcot, Bruce G. 2012. Metrics for evaluating performance and uncertainty of Bayesian network models. Ecological Modeling. 230: 50-62.
KeywordsBayesian network model, uncertainty, model performance, model validation, sensitivity analysis, error rates, probability analysis
- Application of Bayesian methods to habitat selection modeling of the northern spotted owl in California: new statistical methods for wildlife research
- Occupancy models for data with false positive and false negative errors and heterogeneity across sites and surveys
- A Bayesian model to correct underestimated 3-D wind speeds from sonic anemometers increases turbulent components of the surface energy balance
XML: View XML