Skip to main content
U.S. flag

An official website of the United States government

Caveat emptor: On the Need for Baseline Quality Standards in Computer Vision Wood Identification

Author(s):

Prabu Ravindran

Year:

2022

Publication type:

Scientific Journal (JRNL)

Primary Station(s):

Forest Products Laboratory

Source:

Forests journal

Description

Computer vision wood identification (CVWID) has focused on laboratory studies reporting
consistently high model accuracies with greatly varying input data quality, data hygiene, and
wood identification expertise. Employing examples from published literature, we demonstrate
that the highly optimistic model performance in prior works may be attributed to evaluating the
wrong functionality—wood specimen identification rather than the desired wood species or genus
identification—using limited datasets with data hygiene practices that violate the requirement of clear
separation between training and evaluation data. Given the lack of a rigorous framework for a valid
methodology and its objective evaluation, we present a set of minimal baseline quality standards
for performing and reporting CVWID research and development that can enable valid, objective,
and fair evaluation of current and future developments in this rapidly developing field. To elucidate
the quality standards, we present a critical revisitation of a prior CVWID study of North American
ring-porous woods and an exemplar study incorporating best practices on a new dataset covering the
same set of woods. The proposed baseline quality standards can help translate models with high in
silico performance to field-operational CVWID systems and allow stakeholders in research, industry,
and government to make informed, evidence-based modality-agnostic decisions.

Citation

Ravindran, Prabu; Wiedenhoeft, Alex C. 2022. Caveat emptor: On the Need for Baseline Quality Standards in Computer Vision Wood Identification. Forests. 13(4): 632. https://doi.org/10.3390/f13040632.

Cited

Publication Notes

  • We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
  • This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.
https://www.fs.usda.gov/treesearch/pubs/64297