Skip to Main Content
U.S. Forest Service
Caring for the land and serving people

United States Department of Agriculture

Home > Search > Publication Information

  1. Share via EmailShare on FacebookShare on LinkedInShare on Twitter
    Dislike this pubLike this pub
    Author(s): Bruce W. Pengra; Stephen V Stehman; Josephine A. Horton; Daryn J. Dockter; Todd A. Schroeder; Zhiqiang Yang; Warren B. CohenSean P. Healey; Thomas R. Loveland
    Date: 2019
    Source: Remote Sensing of Environment. doi: 10.1016/j.rse.2019.111261.
    Publication Series: Scientific Journal (JRNL)
    Station: Pacific Northwest Research Station
    PDF: Download Publication  (1.0 MB)


    The U.S. Geological Survey Land Change Monitoring, Assessment and Projection (USGS LCMAP) initiative is working toward a comprehensive capability to characterize land cover and land cover change using dense Landsat time series data. A suite of products including annual land cover maps and annual land cover change maps will be produced using the Landsat 4-8 data record. LCMAP products will initially be created for the conterminous United States (CONUS) and then extended to include Alaska and Hawaii. A critical component of LCMAP is the collection of reference data using the TimeSync tool, a web-based interface for manually interpreting and recording land cover from Landsat data supplemented with fine resolution imagery and other ancillary data. These reference data will be used for area estimation and validation of the LCMAP annual land cover products. Nearly 12,000 LCMAP reference sample pixels have been interpreted and a simple random subsample of these pixels has been interpreted independently by a second analyst (hereafter referred to as “duplicate interpretations”). The annual land cover reference class labels for the 1984-2016 monitoring period obtained from these duplicate interpretations are used to address the following questions: 1) How consistent are the reference class labels among interpreters overall and per class? 2) Does consistency vary by geographic region? 3) Does consistency vary as interpreters gain experience over time? 4) Does interpreter consistency change with improving availability and quality of imagery from 1984 to 2016? Overall agreement between interpreters was 88%. Class-specific agreement ranged from 46% for Disturbed to 94% for Water, with more prevalent classes (Tree Cover, Grass/Shrub and Cropland) generally having greater agreement than rare classes (Developed, Barren and Wetland). Agreement between interpreters remained approximately the same over the 12-month period during which these interpretations were completed. Increasing availability of Landsat and Google Earth fine resolution data over the 1984 to 2016 monitoring period coincided with increased interpreter consistency for the post-2000 data record. The reference data interpretation and quality assurance protocols implemented for LCMAP demonstrate the technical and practical feasibility of using the Landsat archive and intensive human interpretation to produce national, annual reference land cover data over a 30-year period. Protocols to estimate and enhance interpreter consistency are critical elements to document and ensure the quality of these reference data.

    Publication Notes

    • You may send email to to request a hard copy of this publication.
    • (Please specify exactly which publication you are requesting and your mailing address.)
    • We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
    • This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.


    Pengra, Bruce W.; Stehman, Stephen V.; Horton, Josephine A.; Dockter, Daryn J.; Schroeder, Todd A.; Yang, Zhiqiang; Cohen, Warren B.; Healey, Sean P.; Loveland, Thomas R. 2019. Quality control and assessment of interpreter consistency of annual land cover reference data in an operational national monitoring program. Remote Sensing of Environment. 238 (2020) 111261 10 p.  doi: 10.1016/j.rse.2019.111261.


    Google Scholar


    LCMAP, validation, land cover, land use, disturbance, time series, Landsat, TimeSync

    Related Search

    XML: View XML
Show More
Show Fewer
Jump to Top of Page