Featured Topic
The Pack Test
This paper reviews the development and validation of a job-related work capacity test for wildland firefighting.
HistoryIn 1965 MTDC and the University of Montana Human Performance Laboratory began work on a test to determine a candidate's fitness to perform arduous wildland firefighting tasks. Field measurements of the metabolic, cardiovascular, and thermal demands of firefighting were made on firefighters working on controlled (prescribed) burns. The results indicated that wildland firefighting tasks fell into the category of hard work, with average energy expenditures of 7.5 kilocalories per minute. These measurements and a review of the literature indicated that for the firefighters who were evaluated, aerobic fitness (maximal oxygen intake or VO2 max) was the primary limiting factor in their ability to sustain hard work throughout long shifts.
The Astrand-Rhyming Step Test was modified, validated, field tested, approved by the Civil Service Commission (now the Office of Personnel Management), and adopted in 1975 as the test to determine firefighter fitness for duty. Because workers cannot sustain day-long workloads above 50 percent of their maximum capacity, the average cost of firefighting duties (7.5 kilocalories per minute or 22.5 milliliters per kilogram minute) was doubled to determine the minimum score (45 milliliters per kilogram minute) for wildland firefighters. Soon after the test procedure was implemented, concerns arose that some workers lacked the muscular strength to do the job. These concerns coincided with the integration of women into the wildland firefighting workforce. Field studies of muscular fitness and work capacity (Sharkey, Jukkala, Putnam, and Tietz 1980) confirmed the relationship of strength and lean body weight to performance in firefighting. The average female has 50 to 60 percent of the upper body strength of the average male. Recommendations to add muscular fitness measures to the selection process were not adopted. The step test (and alternative 1.5-mile run) remained the fitness measures used in the selection of firefighters.
During the great fires of 1910 an old ranger phoned Elers Koch in Missoula to request some firefighters. When Koch asked how many, the ranger replied:
"Send me 10 men if they wear hats, and if they wear caps, I'll need 30."
The distinction was that the respectable lumberjack always wore a felt hat... whereas the pool hall boys and general stew bums... usually wore caps and shoved their hands deep in their pockets.
From Year of the Fires
by Stephen Pyne, 2001.
In 1994 MTDC was assigned to review test procedures and revise training materials to ensure compliance with new laws and regulations, and with recent research. Research, new laws, and comments from employees called for replacement of the step test. It violated the Americans with Disabilities Act (by using biomedical data such as the heart rate, according to a ruling by the Equal Employment Opportunity Commission—EEOC No. 915.002, May 1994). The test was not directly job related, and there was evidence of cheating (breathing maneuvers and the use of drugs to lower the heart rate). A safety awareness study (Tri Data 1998) showed that workers were dissatisfied with the step test and eager for a change. MTDC revised the wildland firefighter job task analysis and conducted a series of laboratory and field studies to develop and validate a job-related work capacity test.
Job Task AnalysisThe job task analysis was revised with input from subject matter experts associated with Federal and State land management agencies. Respondents rated the importance, as well as the intensity, duration, and frequency of tasks. New categories in the revised analysis included:
Performing under adverse conditions (including long work shifts; rough, steep terrain; heat, cold, altitude, smoke; insufficient food, fluids, sleep), and Emergency responses (fast pullout to safety zone, rescue, or evacua-tion assistance to others).
The analysis indicated that the most important firefighting tasks included:
- Building fireline with handtools
- Performing under adverse conditions
- Hiking with light loads
- Lifting and carrying light loads
Tasks receiving lower ratings, primarily because they occurred less frequently, included:
- Packing heavy loads
- Emergency responses
- Chain sawing
Wildland firefighting clearly deserves the definition of arduous work:
"Duties involve field work requiring physical performance calling for above-average endurance and superior conditioning. These duties may include an occasional demand for extraordinarily strenuous activities in emergencies under adverse environmental conditions and over extended periods of time. Requirements include running, walking, climbing, jumping, twisting, bending, and lifting more than 50 pounds; the pace of work typically is set by the emergency condition." (NWCG 310.1)
Job-Related Work Capacity TestsThe test development and validation process followed the Uniform Guidelines for Employee Selection published by the U.S. Department of Labor (1979). The job task analysis and data from past field studies were used to identify potential tests. Based on the relationships to other firefighting tasks, a fireline construction (pulaski) test for upper body performance and a load carry (pack test) for lower body performance were selected for further laboratory and field evaluation. The fireline construction test used a small treadmill to simulate line building with a pulaski.
Laboratory Studies—The fireline test and pack test had energy costs similar to those required on the job (7.5 kilocalories per minute). Both tests were significantly correlated to laboratory measures of aerobic and muscular fitness, and to performance on firefighting tasks. The tests were judged to be valid, reliable, objective, and job-related measures of work capacity. However, because of its reliance on upper body strength, the fireline test was found to have an adverse impact on females (based on the EEOC 80-percent rule; the female pass rate was less than 80 percent of the male pass rate). The fireline test also had a higher administrative cost (for equipment and time). The pack test used available equipment and did not show evidence of adverse impact to any group of candidates. The flat version of the test was highly related to performance on a hilly course (r = 0.87), but the hilly course had an adverse impact on female candidates. A score of 45 minutes on the pack test was equivalent to a score of 45 on the step test (the established standard for wildland firefighters). Based on these studies (Sharkey, Rothwell, and DeLorenzo-Green 1994; DeLorenzo-Green and Sharkey 1995), the pack test was scheduled for field trials.
Source: University of Montana Human Performance Laboratory| Energy Cost of Firefighting Tasks | |
|---|---|
| Task | Energy cost (kilocalories/minute) |
| Digging line with a handtool | 7.5 |
| Chain sawing | 6.2 |
| Shoveling | 6–8 |
| Chopping | 8–10 |
| Lifting and carrying a light load | 6–8 |
| Hiking with a light load Flat (hill) | 6 (12) |
| Hiking with a heavy load Flat (hill) | 7 (15) |
During the 1995 fire season, field trials were conducted on 320 firefighters from three Federal agencies, three regions of the Forest Service, and one State. The sample represented the gender and ethnic distribution of the firefighter population (Sharkey, Rothwell, and Jukkala 1996).
Ethnicity did not appear to be a factor in test performance. For a passing score of 45 minutes, 84.4 percent of males passed compared to 71.9 percent of females. Females passed at 85.2 percent of the male pass rate, which does not constitute adverse impact as defined by the EEOC (table 1). The field trial was not a condition of hire, so some individuals did not give their best effort. Also, field experience has shown that subjects improve substantially when retested, so those within 1 minute of the passing score would be likely to pass on a retest (89.5 percent of the males and 79.7 percent of the females had scores under 46 minutes, yielding a potential female pass rate of 89 percent of the male pass rate, table 2). The scores were also analyzed for the effects of age, height, and weight.
| Pack test (< 45 min) |
F/M* | Pack test** (< 46 min) |
F/M | |
|---|---|---|---|---|
| Males | 216/256 = 84.4% | 229/256 = 89.5% | ||
| Females | 46/64 = 71.9% | 85.2% | 51/64 = 79.7% | 89.1% |
**The passing score is 45 minutes. Those who have scores of 46 minutes or less can usually pass on a retest.
Age—The age of test participants ranged from 18 to 63, including 30 individuals who were over 40. Those over 40 averaged 41.4 minutes on the pack test, which was better than the overall average (41.8 minutes). Of the 30 individuals over 40, 5 scored over 45 minutes and 25 scored under 45 minutes, for a pass rate of 83.3 percent. This pass rate was slightly better than the pass rate for all individuals (81.9 percent).
Height—The height of test participants ranged from 61 to 79 inches with an average of 69.7 inches for all test participants (70.6 inches for males, and 66.3 inches for females). While the data for all individuals suggested a low negative relationship between height and pack test performance (r = -0.294), analysis of scores above 45 minutes revealed no significant relationship (r = -0.022, r2 = 0.0005). The coefficient of determination (r2) indicates the proportion of the variance in performance accounted for by a relationship. Far less than 1 percent (0.05 percent) of the variation in performance among test scores over 45 minutes can be attributed to height.
Weight—Weights ranged from 104 to 270 pounds, averaging 170.9 pounds for all test participants (178.7 pounds for males and 140.9 pounds for females). There was no relationship between weight and performance on the pack test for all individuals, for males, for females, or for those who scored over 45 minutes.
A 1998 field evaluation of more than 5,000 firefighters verified the results of the 1996 field trial. The test was used to qualify firefighters; average scores (41.8 minutes) and pass rates (91.5 percent) were substantially higher than the earlier field trial. However, the trial identified a somewhat lower pass rate for 101 individuals shorter than 5 feet 3 inches tall. Subsequent analysis of 33 individuals who did not pass indicated that 18 did not finish, 10 were overweight, several were underweight, and 27 trained less than 12 hours for the test, including 9 who did no training at all. The results suggested that those factors and the low lean body weight associated with short stature contributed to the somewhat lower pass rate for persons shorter than 5 feet 3 inches. In the 1998 trial, those who did not pass were allowed to take the step test or 1.5-mile run to qualify for employment.
Pack Test Summary and RecommendationsThe pack test is a valid, job-related test of work capacity. The test uses a common firefighting tool (pack) and requires an energy cost similar to that required on the job. Pack test scores were correlated to laboratory measures of fitness (aerobic and muscular fitness, lactate threshold), and to performance of the firefighting tasks identified in the job task analysis. The duration of the test ensures the capacity to perform prolonged arduous work, under adverse conditions, with a reserve to carry out emergency responses. Pack test scores are not adversely influenced by gender, ethnicity, age, height, or weight. MTDC recommended:
- Introduce the pack test in a national interagency implementation program.
- Develop and implement tests for other fire-related positions (table 3).
- Require work capacity tests for a red card (a card used to certify firefighters).
- Develop materials to support implementation of tests.
Both the American Heart Association and the American College of Sports Medicine have recommended a health screening questionnaire designed to identify the small number of individuals who should seek medical advice before becoming involved in moderately strenuous physical activity. Use of the questionnaire by apparently healthy adults substantially reduces the risk of taking exercise tests or beginning training. Candidates for fitness training, firefighting, or field work should complete the questionnaire before beginning strenuous training or taking a work capacity test. The American Heart Association and the American College of Sports Medicine have raised the recommended age for medical evaluation from 40 to 45. A medical exam may be recommended for individuals older than 45, individuals with one or more heart disease risk factors (for example smoking, high blood pressure, or elevated cholesterol), those who have been inactive, or those for whom the test, training, or work represent a significant increase in intensity. For many others, the questionnaire provides assurance of the readiness to engage in training, work, or a job-related work capacity test.
Medical Examinations—The need for more extensive health screening, physician examinations, and medical tests for wildland firefighters is being studied. Federal agencies are considering the need for comprehensive medical standards, medical history, medical tests, and a physician's examination for entry-level firefighters. The medical history would be updated annually, and the physician's examination and some medical tests would be repeated every 5 years until age 45, then every 3 years thereafter (under the current proposal). Costs include several hundred dollars per candidate for medical tests and examinations. Benefits could include early detection of health problems, some reduction in worker's compensation costs, and assignment of candidates to more appropriate positions, when possible. Problems include the waste of scarce resources on a young, generally healthy population, false positive results (indication of problems that may not exist), and the costs of additional testing needed to clear candidates for arduous work. Alternatives to comprehensive examinations include risk stratification by low-cost screening (such as a health screening questionnaire) or a more comprehensive medical history, with tests and examinations for those at higher risk (such as those older than 45).
Training for the Pack Test—Before training, candidates should complete the health screening questionnaire or an equivalent questionnaire and consult a physician, if indicated. They should begin training at least 4 to 6 weeks before they report for duty. If previously inactive, candidates should exercise at moderate intensity for the first few weeks. They can train by hiking, wearing the ankle-height footwear they will use during the test.
They can begin by hiking a 3-mile flat course without a pack; when they can cover the course in less than 45 minutes they should add a pack with 25 pounds. They should increase the pack weight until they can hike 3 miles in 45 minutes with a 45-pound pack. In addition, they can hike hills (with a pack) to build leg strength and endurance, jog the flat course (without a pack) to build aerobic fitness, and cross train (for example mountain bike or lift weights) to build stamina and strength.
Adoption of the Work Capacity TestsAfter extensive field tests MTDC recommended:
- Retiring the step test
- Adopting the work capacity tests
- Developing administrative materials (a test booklet and an information brochure)
The pack test has been adopted by five Federal land management agencies, by some States, by the Province of British Columbia, and by Australia, where it has been endorsed as the minimum national standard.
Note: Brian Sharkey, Ph.D., is professor emeritus of the University of Montana Human Performance Lab and a project leader at MTDC. Versions of this paper were presented at meet-ings of the American College of Sports Medicine (1996), the International Association of Wildland Fire (1999), and the Canadian Society of Exercise Physiology (2000).
