Skip to Main Content
U.S. Forest Service
Caring for the land and serving people

United States Department of Agriculture

Home > Search > Publication Information

  1. Share via EmailShare on FacebookShare on LinkedInShare on Twitter
    Dislike this pubLike this pub

    Description

    We evaluated the effects of probabilistic (hereafter DESIGN) and non-probabilistic (PURPOSIVE) sample surveys on resultant classification tree models for predicting the presence of four lichen species in the Pacific Northwest, USA. Models derived from both survey forms were assessed using an independent data set (EVALUATION). Measures of accuracy as gauged by resubstitution rates were similar for each lichen species irrespective of the underlying sample survey form. Cross-validation estimates of prediction accuracies were lower than resubstitution accuracies for all species and both design types, and in all cases were closer to the true prediction accuracies based on the EVALUATION data set. We argue that greater emphasis should be placed on calculating and reporting cross-validation accuracy rates rather than simple resubstitution accuracy rates. Evaluation of the DESIGN and PURPOSIVE tree models on the EVALUATION data set shows significantly lower prediction accuracy for the PURPOSIVE tree models relative to the DESIGN models, indicating that non-probabilistic sample surveys may generate models with limited predictive capability. These differences were consistent across all four lichen species, with 11 of the 12 possible species and sample survey type comparisons having significantly lower accuracy rates. Some differences in accuracy were as large as 50%. The classification tree structures also differed considerably both among and within the modelled species, depending on the sample survey form. Overlap in the predictor variables selected by the DESIGN and PURPOSIVE tree models ranged from only 20% to 38%, indicating the classification trees fit the two evaluated survey forms on different sets of predictor variables. The magnitude of these differences in predictor variables throws doubt on ecological interpretation derived from prediction models based on non-probabilistic sample surveys.

    Publication Notes

    • You may send email to rmrspubrequest@fs.fed.us to request a hard copy of this publication.
    • (Please specify exactly which publication you are requesting and your mailing address.)
    • We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
    • This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.

    Citation

    Edwards, Thomas C., Jr.; Cutler, D. Richard; Zimmermann, Niklaus E.; Geiser, Linda; Moisen, Gretchen G. 2006. Effects of sample survey design on the accuracy of classification tree models in species distribution models. Ecological Modelling. 199: 132-141.

    Keywords

    model accuracy, sample survey, study design, classification trees, lichens, accuracy assessment, probability samples, non-probability samples

    Related Search


    XML: View XML
Show More
Show Fewer
Jump to Top of Page