Observer and automated estimation data from drone-based imagery of blackbird flocks foraging in commercial sunflower

Metadata:

Identification_Information:
Citation:
Citation_Information:
Originator: Duttenhefner, Jessica L.
Originator: Klug, Page E.
Publication_Date: 2025
Title:
Observer and automated estimation data from drone-based imagery of blackbird flocks foraging in commercial sunflower
Geospatial_Data_Presentation_Form: tabular digital data
Series_Information:
Series_Name: Research Dataset Series
Publication_Information:
Publication_Place: Fort Collins, Colorado
Publisher: USDA, APHIS, WS National Wildlife Research Center
Online_Linkage: https://doi.org/10.2737/NWRC-RDS-2024-005
Description:
Abstract:
We used drones to capture images of mixed-species blackbird (Icteridae) flocks damaging sunflower (Helianthus annuus) in North Dakota. Images consisted of an airborne flock with a sky background, creating a strong color contrast for automated bird detection and counting. We analyzed imagery from 60 flights using ImageJ and obtained automated counts. We also asked 20 biologists, ranging from 0-25 years of self-reported experience, to provide estimates for all 60 images in a single sitting. They were asked to make quick estimates (5-10 seconds per photo), to count birds within the sky background, and to zoom in on photos when needed. Photos were randomly ordered and all biologists were given photos in the same order. This study was implemented between September 2021 through October 2022 in multiple counties in North Dakota, USA where blackbird damage to sunflowers is prevalent. This data publication contains the data and R code used to analyze these data as well as the 60 drone images.
Purpose:
We designed the study to evaluate the role of flock size, biologist experience, and photo order on the ability of a biologist to make estimates close to the automated count.
Supplemental_Information:
For more information about this study and these data, see Duttenhefner and Klug (2025).

These data were published on 01/22/2025. On 03/12/2025, the metadata was updated to include reference to newly published article.
Time_Period_of_Content:
Time_Period_Information:
Range_of_Dates/Times:
Beginning_Date: 202109
Ending_Date: 202210
Currentness_Reference:
Ground condition
Status:
Progress: Complete
Maintenance_and_Update_Frequency: As needed
Spatial_Domain:
Description_of_Geographic_Extent:
Drone images were captured in multiple counties in North Dakota, USA, where blackbird damage to sunflowers is prevalent.
Bounding_Coordinates:
West_Bounding_Coordinate: -100.84000
East_Bounding_Coordinate: -99.79000
North_Bounding_Coordinate: 48.99000
South_Bounding_Coordinate: 46.08000
Keywords:
Theme:
Theme_Keyword_Thesaurus: ISO 19115 Topic Category
Theme_Keyword: biota
Theme:
Theme_Keyword_Thesaurus: National Research & Development Taxonomy
Theme_Keyword: Ecology, Ecosystems, & Environment
Theme_Keyword: Wildlife (or Fauna)
Theme_Keyword: Birds
Theme:
Theme_Keyword_Thesaurus: None
Theme_Keyword: automated counts
Theme_Keyword: drone
Theme_Keyword: Icteridae
Theme_Keyword: ImageJ
Theme_Keyword: monitoring
Theme_Keyword: observer error
Theme_Keyword: UAS
Theme_Keyword: unoccupied aircraft system
Place:
Place_Keyword_Thesaurus: None
Place_Keyword: North Dakota
Taxonomy:
Keywords/Taxon:
Taxonomic_Keyword_Thesaurus:
None
Taxonomic_Keywords: single species
Taxonomic_Keywords: animals
Taxonomic_System:
Classification_System/Authority:
Classification_System_Citation:
Citation_Information:
Originator: ITIS
Publication_Date: 2024
Title:
Integrated Taxonomic Information System
Geospatial_Data_Presentation_Form: online database
Other_Citation_Details:
Retrieved [December, 20, 2024]
Online_Linkage: https://www.itis.gov/
Online_Linkage: https://doi.org/10.5066/F7KH0KBK
Taxonomic_Procedures:
Taxonomic_Classification:
Taxon_Rank_Name: Kingdom
Taxon_Rank_Value: Animalia
Applicable_Common_Name: Animal
Applicable_Common_Name: animaux
Applicable_Common_Name: animals
Taxonomic_Classification:
Taxon_Rank_Name: Subkingdom
Taxon_Rank_Value: Bilateria
Applicable_Common_Name: triploblasts
Taxonomic_Classification:
Taxon_Rank_Name: Infrakingdom
Taxon_Rank_Value: Deuterostomia
Taxonomic_Classification:
Taxon_Rank_Name: Phylum
Taxon_Rank_Value: Chordata
Applicable_Common_Name: cordés
Applicable_Common_Name: cordado
Applicable_Common_Name: chordates
Taxonomic_Classification:
Taxon_Rank_Name: Subphylum
Taxon_Rank_Value: Vertebrata
Applicable_Common_Name: vertebrado
Applicable_Common_Name: vertébrés
Applicable_Common_Name: vertebrates
Taxonomic_Classification:
Taxon_Rank_Name: Infraphylum
Taxon_Rank_Value: Gnathostomata
Taxonomic_Classification:
Taxon_Rank_Name: Superclass
Taxon_Rank_Value: Tetrapoda
Taxonomic_Classification:
Taxon_Rank_Name: Class
Taxon_Rank_Value: Aves
Applicable_Common_Name: Birds
Applicable_Common_Name: oiseaux
Taxonomic_Classification:
Taxon_Rank_Name: Order
Taxon_Rank_Value: Passeriformes
Applicable_Common_Name: Perching Birds
Applicable_Common_Name: passereaux
Taxonomic_Classification:
Taxon_Rank_Name: Family
Taxon_Rank_Value: Icteridae
Applicable_Common_Name: American Blackbirds
Applicable_Common_Name: Orioles
Applicable_Common_Name: New World Blackbirds
Access_Constraints: None
Use_Constraints:
These data were collected using funding from the U.S. Government and can be used without additional permissions or fees. If you use these data in a publication, presentation, or other research product please use the following citation:

Duttenhefner, Jessica L.; Klug, Page E. 2025. Observer and automated estimation data from drone-based imagery of blackbird flocks foraging in commercial sunflower. Research Dataset Series. USDA, APHIS, WS National Wildlife Research Center. Ft. Collins, Colorado. https://doi.org/10.2737/NWRC-RDS-2024-005
Point_of_Contact:
Contact_Information:
Contact_Organization_Primary:
Contact_Organization: USDA, APHIS, Wildlife Service, National Wildlife Research Center
Contact_Person: Page Klug
Contact_Position: Supervisory Research Wildlife Biologist
Contact_Address:
Address_Type: mailing and physical
Address: 4101 LaPorte Ave.
City: Fort Collins
State_or_Province: CO
Postal_Code: 80521
Country: USA
Contact_Voice_Telephone: 701-630-3776
Contact_Electronic_Mail_Address: page.e.klug@usda.gov
Contact Instructions: This contact information was current as of original publication date. For current information see Contact Us page on: https://doi.org/10.2737/RDS.
Data_Set_Credit:
This research was funded by the United States Department of Agriculture, Animal and Plant Health Inspection Service, Wildlife Services, National Wildlife Research Center (NWRC) (7438-0020-CA and QA-3108) and the National Sunflower Association (Project 20-P03).


Author Information:

Jessica L. Duttenhefner
North Dakota State University, Department of Biological Sciences
https://orcid.org/0009-0006-2672-8421

Page E. Klug
USDA, APHIS, Wildlife Service, National Wildlife Research Center
https://orcid.org/0000-0002-0836-3901
Cross_Reference:
Citation_Information:
Originator: Duttenhefner, Jessica L.
Originator: Klug, Page E.
Publication_Date: 2025
Title:
Observer estimation accuracy from drone-based imagery influenced by size of blackbird flocks foraging in sunflower
Geospatial_Data_Presentation_Form: journal article
Series_Information:
Series_Name: The Wilson Journal of Ornithology
Online_Linkage: https://doi.org/10.1080/15594491.2024.2444028
Analytical_Tool:
Analytical_Tool_Description:
ImageJ - Image Processing and Analysis in Java
Tool_Access_Information:
Online_Linkage: https://imagej.net/ij/
Tool_Access_Instructions:
See website
Tool_Citation:
Citation_Information:
Originator: Rasband, Wayne S.
Publication_Date: 1997
Title:
ImageJ
Geospatial_Data_Presentation_Form: software
Publication_Information:
Publication_Place: Bethesda, Maryland, USA
Publisher: U.S. National Institutes of Health
Online_Linkage: https://imagej.net/ij/
Analytical_Tool:
Analytical_Tool_Description:
R is a free software environment for statistical computing and graphics.
Tool_Access_Information:
Online_Linkage: https://www.r-project.org/
Tool_Access_Instructions:
See website
Tool_Citation:
Citation_Information:
Originator: R Core Team
Publication_Date: 2024
Title:
R: A Language and Environment for Statistical Computing
Geospatial_Data_Presentation_Form: software
Publication_Information:
Publication_Place: Vienna, Austria
Publisher: R Foundation for Statistical Computing
Online_Linkage: https://www.r-project.org/
Back to Top
Data_Quality_Information:
Attribute_Accuracy:
Attribute_Accuracy_Report:
We were unable to get a true count of blackbirds, due to limitations in counting large flocks, distance from ground observers, and physical obscurities (e.g., topography and birds under crop canopy); thus, we used the ImageJ automated count as the baseline for our linear model. Individual birds below the horizon (i.e., ground background) and overlapping birds were obscured and consequently excluded from detection and the count. Photos were randomly ordered when provided to biologists, and no significant relationship existed between flock size and photo order (|r| = 0.40); however, all biologists were given photos in the same order.
Logical_Consistency_Report:
The data are logically consistent. The consistency was verified as part of the quality assurance that occurred during data analysis.
Completeness_Report:
All data is complete.
Lineage:
Methodology:
Methodology_Type: Field
Methodology_Description:
We conducted 60 drone flights in sunflower fields and adjacent cattail marshes in North Dakota, where mixed-species blackbird flocks were actively foraging in sunflower from 01 September to 25 October (2021–2022) between 07:45 and 12:45. We used one drone to lift flocks off the crop and another to capture video of flocks in flight with a sky background. The pilot-in-command (JLD) and another pilot flew a DJI Mavic Air 2 and DJI Mavic 2 Pro (SZ DJI Technology Co. Ltd., Shenzhen, China) manually towards the flock at approximately 2 meters above ground level. Before reaching the flock, we stopped forward motion of the Mavic 2 Pro, hovered, and adjusted the camera to capture video and photos of birds with a sky background. We flew the drone at variable heights with the camera parallel to the ground (i.e., forward-facing), which led to overlap of birds in the images. The degree of overlap varied and was a limiting factor in both biologist and automated counts, but was handled consistently in the automated count methods. We flew the Mavic Air 2 through the center of the flock until adequate footage was collected. Drone speeds varied depending on flock behavior and logistics to capture imagery.
Methodology_Citation:
Citation_Information:
Originator: Duttenhefner, Jessica L.
Originator: Klug, Page E.
Publication_Date: 2025
Title:
Observer estimation accuracy from drone-based imagery influenced by size of blackbird flocks foraging in sunflower
Geospatial_Data_Presentation_Form: journal article
Series_Information:
Series_Name: The Wilson Journal of Ornithology
Online_Linkage: https://doi.org/10.1080/15594491.2024.2444028
Methodology:
Methodology_Type: Lab
Methodology_Description:
Our coordinated drone flights produced imagery (i.e., video frames and photos) of an airborne flock with a sky background, creating a strong color contrast for automated bird detection and counting. In ImageJ, we analyzed the imagery from these 60 flights and obtained an automated count as the baseline. We also used Google Forms to share the 60 photos with 20 biologists, ranging from 0–25 years of self-reported experience to compare manual and automated estimates of flock sizes. Biologists were referred to two online resources for methods to count birds and estimate flock sizes (eBird 2012, USFWS 2024). We asked biologists to provide estimates in a single sitting, to make quick estimates (5–10 seconds per photo), to count birds within the sky background, and to zoom in on photos when needed. Photos were randomly ordered and all biologists were given photos in the same order.
Methodology_Citation:
Citation_Information:
Originator: Duttenhefner, Jessica L.
Originator: Klug, Page E.
Publication_Date: 2025
Title:
Observer estimation accuracy from drone-based imagery influenced by size of blackbird flocks foraging in sunflower
Geospatial_Data_Presentation_Form: journal article
Series_Information:
Series_Name: The Wilson Journal of Ornithology
Online_Linkage: https://doi.org/10.1080/15594491.2024.2444028
Methodology_Citation:
Citation_Information:
Originator: eBird
Publication_Date: 2012
Title:
Bird Counting 201
Geospatial_Data_Presentation_Form: online material
Publication_Information:
Publication_Place: Ithaca, NY
Publisher: Cornell Lab of Ornithology
Other_Citation_Details:
(accessed 13 August 2024)
Online_Linkage: https://ebird.org/news/counting-101/
Methodology_Citation:
Citation_Information:
Originator: United States Fish and Wildlife Service (USFWS)
Publication_Date: 2024
Title:
Estimating flock size
Geospatial_Data_Presentation_Form: online material
Series_Information:
Series_Name: Aerial observer training and testing resources
Publication_Information:
Publisher: United States Fish and Wildlife Service
Other_Citation_Details:
(accessed 13 August 2024)
Online_Linkage: https://www.fws.gov/waterfowlsurveys/forms/counting.jsp?menu=counting
Back to Top
Entity_and_Attribute_Information:
Overview_Description:
Entity_and_Attribute_Overview:
Below you will find a list and description of the files included in this data publication.

VARIABLE DESCRIPTION FILE (1)

1. \Data\_variable_descriptions.csv: Comma-separated values (CSV) file containing a list and description of variables found in all data files. (A description of these variables is also provided in the metadata below.)

Columns include:

Filename = Name of data file

Variable = Name of variable

Units = Units (if applicable)

Precision = Precision (if applicable)

Description = Description of variable



DATA FILES (2)

1. \Data\Duttenhefner_Klug_Corr_Data.csv: CSV file containing data for comparing counts of blackbird flocks foraging in commercial sunflower. Data include both the automated estimate and the mean of 20 biologist estimates of 60 unique photos collected using drones. This file can be used with the provided R code (\Supplements\Duttenhefner_Klug_2025_WJO.R) to analyze the results as reported in Duttenhefner and Klug (2025).

Variables include:

photo_id = Unique identification number for each photo

video_length = Length of video recording from the start of hazing to the end (minutes:seconds)

automated_estimation = Number of birds estimated by the automated method (i.e., ImageJ; count)

mean_bio_estimation = Mean number of birds estimated by 20 biologists (i.e., count)


2. \Data\Duttenhefner_Klug_LMM_Data.csv: CSV file containing data for comparing counts of blackbird flocks foraging in commercial sunflower. Data include both the automated estimate and 20 separate biologist estimates of 60 unique photos collected using drones.

Variables include:

photo_id = Unique identification number for each photo

photo_order = Order in which the photo was provided in the Google Form (i.e., 1 = first, 2 = second, etc.) and the same order provided in Duttenhefner_Klug_drone_images.pdf

biologist_id = Unique identification number for each biologist

experience = Self-reported number of years of previous experience estimating flock size

biologist_estimation = Number of birds estimated by a biologist observer (count)

automated_estimation = Number of birds estimated by the automated method (i.e., ImageJ; count)



SUPPLEMENTAL FILES (2)

1. \Supplements\Duttenhefner_Klug_2025_WJO.R: Text file containing R code. This code can be used with the provided data files to replicate results step by step as they appear in Duttenhefner and Klug (2025).

2. \Supplements\Duttenhefner_Klug_drone_images.pdf: Portable Document Format file containing the 60 unique drone images referenced in this study.
Entity_and_Attribute_Detail_Citation:
Duttenhefner, Jessica L.; Klug, Page E. 2025. Observer estimation accuracy from drone-based imagery influenced by size of blackbird flocks foraging in sunflower. The Wilson Journal of Ornithology. https://doi.org/10.1080/15594491.2024.2444028
Back to Top
Distribution_Information:
Distributor:
Contact_Information:
Contact_Organization_Primary:
Contact_Organization: USDA Forest Service, Research and Development
Contact_Position: Research Data Archivist
Contact_Address:
Address_Type: mailing and physical
Address: 240 West Prospect Road
City: Fort Collins
State_or_Province: CO
Postal_Code: 80526
Country: USA
Contact_Voice_Telephone: see Contact Instructions
Contact Instructions: This contact information was current as of March 2025. For current information see Contact Us page on: https://doi.org/10.2737/RDS.
Resource_Description: NWRC-RDS-2024-005
Distribution_Liability:
Metadata documents have been reviewed for accuracy and completeness. Unless otherwise stated, all data and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. However, neither the author, the Archive, nor any part of the federal government can assure the reliability or suitability of these data for a particular purpose. The act of distribution shall not constitute any such warranty, and no responsibility is assumed for a user's application of these data or related materials.

The metadata, data, or related materials may be updated without notification. If a user believes errors are present in the metadata, data or related materials, please use the information in (1) Identification Information: Point of Contact, (2) Metadata Reference: Metadata Contact, or (3) Distribution Information: Distributor to notify the author or the Archive of the issues.
Standard_Order_Process:
Digital_Form:
Digital_Transfer_Information:
Format_Name: CSV
Format_Version_Number: see Format Specification
Format_Specification:
Comma-separated values file
Digital_Transfer_Option:
Online_Option:
Computer_Contact_Information:
Network_Address:
Network_Resource_Name: https://doi.org/10.2737/NWRC-RDS-2024-005
Digital_Form:
Digital_Transfer_Information:
Format_Name: PDF
Format_Version_Number: see Format Specification
Format_Specification:
Portable Document Format file
Digital_Transfer_Option:
Online_Option:
Computer_Contact_Information:
Network_Address:
Network_Resource_Name: https://doi.org/10.2737/NWRC-RDS-2024-005
Digital_Form:
Digital_Transfer_Information:
Format_Name: R
Format_Version_Number: see Format Specification
Format_Specification:
Text file containing R code
Digital_Transfer_Option:
Online_Option:
Computer_Contact_Information:
Network_Address:
Network_Resource_Name: https://doi.org/10.2737/NWRC-RDS-2024-005
Fees: None
Back to Top
Metadata_Reference_Information:
Metadata_Date: 20250312
Metadata_Contact:
Contact_Information:
Contact_Organization_Primary:
Contact_Organization: USDA, APHIS, Wildlife Service, National Wildlife Research Center
Contact_Person: Page Klug
Contact_Position: Supervisory Research Wildlife Biologist
Contact_Address:
Address_Type: mailing and physical
Address: 4101 LaPorte Ave.
City: Fort Collins
State_or_Province: CO
Postal_Code: 80521
Country: USA
Contact_Voice_Telephone: 701-630-3776
Contact_Electronic_Mail_Address: page.e.klug@usda.gov
Contact Instructions: This contact information was current as of original publication date. For current information see Contact Us page on: https://doi.org/10.2737/RDS.
Metadata_Standard_Name: FGDC Biological Data Profile of the Content Standard for Digital Geospatial Metadata
Metadata_Standard_Version: FGDC-STD-001.1-1999
Back to Top