Variance estimates and confidence intervals for the Kappa measure of classification accuracyAuthor(s): M. A. Kalkhan; R. M. Reich; R. L. Czaplewski
Source: Canadian Journal of Remote Sensing. 23(3): 210-216.
Publication Series: Scientific Journal (JRNL)
Station: Rocky Mountain Research Station
View PDF (562.0 KB)
The Kappa statistic is frequently used to characterize the results of an accuracy assessment used to evaluate land use and land cover classifications obtained by remotely sensed data. This statistic allows comparisons of alternative sampling designs, classification algorithms, photo-interpreters, and so forth. In order to make these comparisons, it is important to know how far in error the estimate might reasonably be. This is accomplished by constructing confidence intervals around the point estimate. The decision to use either the asymptotic variance formulae or bootstrapping variance estimates in constructing confidence intervals for the Kappa statistic is not a simple task. This study was designed to help answer this question. Nine error matrices representing three levels of accuracy (poor, average, and good) of Landsat TM Data consisting of 4, 8 and 16 land cover types in North Carolina were used in this study. Each error matrix was sampled, with replacement, using sample sizes of 50, 100, 150, 300 and 800 pixels to obtain estimates of the Kappa statistic and sample variance. Each of the sample error matrices were resampled 500 times to obtain bootstrap estimates of the variance. The asymptotic variance formula for the Kappa statistic and bootstrap variance provided unbiased estimates of the sample variance. In general, the asymptotic variance estimates were larger than those obtained using bootstrapping, even though the differences were not significant. Confidence intervals based on percentiles of the bootstrap distribution provided the best 95 percent coverage rates (92 to 96 percent with a median of 95 percent). The lowest 95 percent coverage rates were obtained using the bootstrap variance estimate (median of 83 percent).
- You may send email to email@example.com to request a hard copy of this publication.
- (Please specify exactly which publication you are requesting and your mailing address.)
- We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
- This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.
Kalkhan, M. A.; Reich, R. M.; Czaplewski, R. L. 1997. Variance estimates and confidence intervals for the Kappa measure of classification accuracy. Canadian Journal of Remote Sensing. 23(3): 210-216.
KeywordsKappa statistic, land use, and cover classifications, remotely sensed data
- Statistical properties of measures of association and the Kappa statistic for assessing the accuracy of remotely sensed data using double sampling
- Using satellite image-based maps and ground inventory data to estimate the area of the remaining Atlantic forest in the Brazilian state of Santa Catarina
- Statistical inference for remote sensing-based estimates of net deforestation
XML: View XML