The question of "sustainability’ has become a key consideration in most human endeavors. Whether it is in forests or farmland, rivers or oceans, we are increasingly challenged to consider the consequences of our actions. Humans are part of ecosystems. The key question is not how much should we harvest or how much should we protect, but rather is the overall system sustainable. Many organizations, nations, and industrial groups have been trying to develop sets of criteria and indicators to assess sustainability of forest ecosystems. This document contains the results of an independent review of a range of different sets of criteria and indicators of sustainable forest management. The review was conducted under the auspices of The Center for International Forest Research (CIFOR). This review constitutes CIFOR’s North American test of sustainable forest management, which is the seventh worldwide CIFOR test. The test was hosted by the Boise National Forest and was conducted in the area of southwest Idaho, USA. Field testing for the project began on June 8, 1998 and was concluded on July 10. Funding was provided by the United States Agency for International Development (USAID), the USDA - Forest Service Office of International Programs and USDA – Forest Service Research.

The North American test is part of the second phase of the CIFOR research project to develop criteria and indicators of sustainable forest management. The primary focus of CIFOR’s criteria and indicators project has been in tropical countries. However, the principles are potentially applicable to temperate forest management as well. The USDA Forest Service participates on the scientific advisory board for the criteria and indicator research project and offered to host the North American test at an appropriate location in the United States. A team of experts was identified from Canada, Mexico, and the United States to test the CIFOR criteria and indicators, along with other appropriate sets of criteria and indicators.

The North American test includes some unique characteristics that influence the results when compared to previous CIFOR criteria and indicator tests. With the potential exception of tests in Germany and Austria, the North American test occurred in a markedly different social, legal, political, economic, and ecological venue than the tests that CIFOR has carried out in tropical countries. The North American test also occurred after CIFOR developed a preliminary synthesis of the earlier tests (CIFOR's phase 1). Therefore the North American test had the benefit of previous experience, but also very different socio-ecological conditions.

Mexico, Canada, and the United States support the North American Test of the CIFOR project through cooperation by Instituto Nacional de Investigaciones Forestales, Agricolas y Pecuarias, the Canadian Forestry Service, Parks Canada, and the USDA Forest Service.

1.1 Why use Criteria and Indicators? Do they have value?

The 1994 "Montreal process", and the subsequent 1995 "Santiago Declaration", committed the temperate and boreal forest nations, including Canada, the United States, and Mexico, to use criteria and indicators to provide a common understanding of what is meant by sustainable forest management. According to this declaration, criteria and indicators are to provide a framework for describing, assessing, and evaluating a country's progress toward sustainability. The main focus of the Montreal Process is on national level criteria and indicators.

This report is the result of an independent test of sets of criteria and indicators of sustainability, in this case the sustainable management of forests in a particular place. Before considering our test results, or test approaches, it is first important to examine the assumptions behind our work. First and foremost, we must ask the question "are criteria and indicators useful to help us understand anything?". Second, "do criteria and indicators have relevance to a concept such as sustainability?". The team debated these questions during its work. We tried to provide insights, from our own disciplines, into what would help us assess the sustainability of the systems in which we live. The answers to questions on the value of criteria and indicators are not easy, but we will at least try to clarify the assumptions under which we worked.

Sustainability has been rightly criticized as being a "fuzzy idea". It has been called unscientific, immeasurable, and even useless. There is some truth in those accusations. However, using fuzzy ideas to define our hopes, dreams, and values has never deterred humanity in the past. Notions such as honor, or justice, or truth are the common currency of our lives. They are certainly not more rigorous in their definition than sustainability. Societies regularly use the best of their knowledge systems to support and clarify these fuzzy ideas. For example, the justice system uses biological and physical sciences in the applied forensic sciences to aid in determinations of guilt or innocence. Governments use economics and sociology to help design tax legislation. These are just two examples of the use of science to help make decisions, but, in neither case does science have the last word. So it is with sustainability. The use of criteria and indicators aims to apply knowledge systems to help society understand its position relative to sustainability. Those knowledge systems include economics, sociology, biology and the physical sciences. Debates on the merits of one type of science relative to another seem trite. We need all our understanding to deal with a notion such as sustainability and the underlying problems that prompted its rise in popularity.

The idea of sustainability is based on having a relationship with the earth that will persist. It is an effort to avoid a future that seems increasingly bleak as, worldwide, we record dramatic losses of the biological diversity on our planet and a decline in the flow of goods and services that come from intact ecosystems. Human population numbers are soaring and individual human consumption levels seem insatiable. We are now 5.8 billion people and projected to double by the year 2040. Of those who have tried to understand the carrying capacity for humans on our planet, the vast majority feels such a doubling of the human population would be disastrous. Sustainability is an attempt to define a way of living on our planet that avoids bleak consequences. With all its flaws, sustainability has at least gained international currency. There appears to be no other competing model.

Because sustainability is such an inclusive idea, it draws on a wide range of the "sciences". These sciences describe ecological and social systems as highly complex. A common criticism of indicators is that they are an inadequate tool to deal with such complexity. That is perhaps so, but there are few alternative solutions to deal with such complexity. Certainly there are models and approaches that explicitly try to incorporate or accommodate such complexity. We would argue that those model outputs are simply better indicators. There need not be anything simple or unscientific about indicators. They are the best "gauges" we can find to understand the "performance" of the system.

Indicators are used by people for many different reasons, but they certainly pervade our lives. Some of our most celebrated indicators are weak and easy targets for ridicule. The Gross National Product (GNP) of the United States increased after the Exxon Valdez leaked millions of gallons of crude oil onto the shores of Alaska. This has been used as an indictment of the GNP as an indicator. Perhaps it is more an indictment of a use of the indicator. The Gross National Product was never designed to be an indicator of the state of the nation. It simply is a measure of the volume of transactions within the economic system. It should be never used for more. A better example of an indicator might be human body temperature. It has been shown to be a very sensitive indicator of a whole range of system dysfunctions, and it has well established norms and variances. However, by itself, body temperature never tells the whole story about human condition.

We do not have one comprehensive indicator for sustainability, and that is surely a good thing. Sustainability does not have a unified theoretical basis, and thus no common denominator, despite the efforts of non-traditional economists to use dollars. Understanding sustainability requires a suite of indicators, each chosen to say something important about the system and where it is headed. Managing for sustainability requires paying attention to all the indicators in the suite. At least for the present, the individual indicators cannot be averaged, or chosen between, or put on a common scale. Certainly some are more important, or better, than others. We see the suite of indicators as a work in progress. As we learn more, we should develop better ones.

We are working under the assumption, perhaps naively, that better information on the ecosystem (which includes human social systems) will lead to management decisions that have a better chance of being sustainable. If we assume all management (decisions in the generic sense) is a choice between alternative futures, the value of indicators is to provide information on the direction and future state of the system. Simply put, for any given decision, where is the system likely to go? We are assuming that managers will not make "bad" or "unsustainable" decisions on purpose. We assume that "bad" or "unsustainable" decisions are made because of (1) weakness of models (of the mind or in a computer) used to create the forecasts that underlie the design of management of forests and of related environmental systems, and (2) insufficient attention to, or impossibility of following, those forecasts during implementation. People do not make dumb choices, so much as they choose from among dumb forecasts and/or make insufficient effort to ensure that the actions are invoked to "cause" the chosen forecast to happen (Baskerville, 1997).

1.2 CIFOR - History of the C&I Testing Project

1.2.1 Criteria and Indicators Testing

A major project of CIFOR has been the development of sets of locally appropriate criteria and indicators (C&I) at the forest management unit (FMU) level for different parts of the world. C&I are tools, that can be used to conceptualize, evaluate and implement sustainable forest management. The principal aim of C&I field-testing is to identify C&I that are objective, cost-effective, and relevant to the sustainable management of forests. The focus of the testing procedure should be to identify the smallest number of C&I needed to reliably assess forest management in a cost-effective manner.

The process of identifying appropriate C&I is based on the evaluation of existing sets of C&I. If gaps exist, or if existing C&I are not suitable, new or substitute C&I can be developed. This iterative process involves multiple stakeholders in the region or countries concerned. The methods chosen by CIFOR were developed by using interdisciplinary teams of experts acting within the framework of a well-defined iterative process (Prabhu et al., n.d.).

Expected gains from the process include:

• ability to discern clearly between sustainable and unsustainable forest management practices, and so reduce uncertainty concerning environmental and social costs of logging. This will enable greater productive use to be made of forests;
• preferential treatment or better prices for forest products, especially timber, from sustainable sources through certification;
• reduction in environmental impacts and degradation through an improved ability to diagnose the factors affecting sustainability of management practices; and
• reduction in social inequities and enhancement of opportunities for income generation and improved quality of life.

1.2.2 Summary of Previous CIFOR Tests

(for more detailed information on previous CIFOR Tests see Volume 2)

The results of the first phase of field tests in Germany, Indonesia, Côte d’Ivoire, Brazil, and Austria showed considerable commonality on the relevance of C&I related to policy and legal frameworks, and ecological and production aspects to all test sites. There was a marked divergence regarding the social aspects of forest management. The social C&I selected at each site, not surprisingly, showed the least amount of commonality.

The tests also showed that more work is needed to make the conceptual framework of principles, criteria, indicators and verifiers more consistent and operational. The ITW and Woodmark sets were the sources with the largest number of references in the proposals made by the experts at each of the three tropical test sites. The ITW set with over 600 C&I was generally considered to be the most comprehensive, in terms of conventional forest management. The Woodmark set was considered to be the most prescriptive of the five base sets used in these initial tests.

The sixth test in Cameroon served to refine the operational requirements for C&I field-testing, and further tested several improved C&I related to biodiversity and genetics.

Based on these field experiences, CIFOR suggests that an appropriate set of C&I address issues within the following four categories:

• matters largely outside the influence of the forest management unit: policy, planning, and legal frameworks,
• ecological impacts of the management of forests,
• impacts on the social environment, including economic impacts, and
• C&I related to the production of goods and services, including C&I related to financial performance.

The generic set of C&I that arose from the Phase I work is included in Appendix B.

1.3 The North American Test

The North American C&I test was conducted in the area of the Boise National Forest, Idaho, from June 8 to July 10, 1998. Preparatory work for the field test began in late March, 1998. A more detailed description of the test area is presented in Volume 2 of this report.

The Boise Study Area represents a sophisticated level of forest management. Most landholders maintain comprehensive resource management plans generally aimed towards long-term productivity and ecological health of the forest. The area also has a comprehensive database and a high level of stakeholder involvement. The forest represents a valued resource for a wide range of users, supplying local people with revenue from timber products, outdoor recreational opportunities, fuelwood and other forest products. The area also serves as a refuge for many animals and plants and protects ecosystems and natural processes, which may be declining, in adjacent lands.

The lead agency for the evaluation was the United States Department of Agriculture’s Forest Service (USDA – Forest Service). Project team members were selected from a wide range of disciplines and originated from throughout the United States, Canada, and Mexico. The core test team consisted of two ecologists, one social scientist, one economist, two forest managers, and one forest geneticist. Additional specialists included a carbon biochemist, an anthropologist, a systems ecologist, and a forest ecologist.

The objectives of the test were: i) to conduct a scientific evaluation of various sets of C&I; ii) to define C&I useful to forest managers and those interested in forest outcomes; iii) to attempt to integrate social and ecological C&I into a comprehensive sustainability model; and, iv) to develop new or refined criteria and indicators.

1.4 Which Sets of Indicators Were Tested

The sets of C&I selected for evaluation included: 1) those that emerged from the CIFOR Phase I synthesis; 2) CIFOR’s basic assessment guide for human well-being; 3) Canadian Council of Forest Ministers (CCFM) Criteria and Indicators of Sustainable Forest management in Canada (which are similar, but not the same as, the Montreal Process - see following paragraph); 4) local/regional indicators including the Idaho Forest Practices Act; and, 5) the Greater Fundy Ecosystem Guidelines developed for the Fundy Model Forest. (see Appendix B).

This project desired to test C&I that had come from the Montreal Process. These C&I were developed following the 1992 United Nations Conference on Environment and Development (UNCED), where there was international agreement to formulate guidelines or criteria to ensure sustainable forest management. One of the groups to take up this challenge was the Working Group on Criteria and Indicators for the Conservation and Sustainable Management of Temperate and Boreal Forests ("or the Montreal Process" Working Group) which was formed in Geneva in June 1994. It developed from the work of an International Seminar of Experts on Sustainable Development of Boreal and Temperate Forests, held in Montreal, Canada, in September 1993. The Montreal Process Working Group membership now includes 12 countries covering over 90 per cent of the world's temperate and boreal forests, including Australia, Canada, Chile, China, Japan, the Republic of Korea, Mexico, New Zealand, the Russian Federation, the United States of America and Uruguay.

In Canada, The Canadian Council of Forest Ministers had an ongoing project to develop C&I (1992) which was presented at Rio. In 1993, following UNCED, the Montreal process began with the Canadian Government initiative. The CCFM project predates the Montreal Process and the indicator sets are not identical. However they are substantially similar, were further developed and thus more testable. For this test, therefore, we used the CCFM indicator set. The CCFM has six criteria, 22 elements and 83 indicators while the Montreal Process has seven criteria and 67 indicators. Any Montreal Process indicators not covered by the CCFM set, were generally covered by the significant overlap with the CIFOR Phase 1 synthesis set.

The field test allowed hands-on testing of the candidate C&I against real data in a working situation. It also allowed the team to interact with forest managers, local experts, individuals, and interest groups.

With the completion of the field test, the group finalized their C&I "tool-box" for future presentation of the results to area managers and stakeholders during a September 1998 workshop in Boise, Idaho. The results of the test and comments arising from the workshop were incorporated into this final report.

It is hoped that the C&I that emerge from the Boise National Forest test will contribute to the development of unbiased and objective systems to assess the sustainability of forest management in other parts of North America and elsewhere in the world. They are aimed ultimately to serve as tools for those wishing to develop or improve their own C&I.

1.5 Support for the Project

The United States Department of Agriculture, Forest Service, served as the sponsor and lead agency for the test exercise. The Forest Service selected the location for the C&I test and picked the members of the expert test team. Throughout the test process, the Forest Service provided logistical support to the expert team, provided multi-attribute data for analysis, and made available database expertise. The Forest Service is a principal landowner in the area, with parts of three national forests extending over the area of the test site. The Boise National Forest makes up the majority of the test area and acted as host for the test.

The Idaho Department of Lands and the Boise Cascade Corporation supported the project through supplying data and information, staff time, and logistical support for field tours.

Work facilities, including access to a GIS lab and GIS staff, were provided for the test team at Boise State University in Boise, Idaho.

1.6 Scale Issues – how big is a forest management unit?

Criteria and Indicators of sustainability have been considered and described at several scales. The most well known scale is associated with the Montreal Process, a system that develops reports for nations such as Canada, Mexico and the United States. These national reports are based on sub-national ecological units within which descriptive data are collected on various indicators. Numerous other scales could be used for evaluating sustainability attendant with the scale of an assessment on sustainability (see Table 1). The North American test reported on in this document was conducted at what has been referred to as the Forest Management Unit (FMU). The FMU is considered to be comprised of one or more ownerships that make decisions about how the landscape will be affected by land and resource management activities. Social and economic conditions are considered as the direct effects of actions on the FMU. Ownership size varies but in the case of this study, various ownership sizes are provided in the description of the Boise Test Site in Volume II of this report. The scale of the analysis area for evaluating criteria and indicators directly affects the definition of the individual criteria and indicators. In some cases criteria and indictors are only applicable at a small range of scales and in other cases they can apply across a wide range of scales. When criteria and indicators are considered across scales, the definition of the criteria and indicator itself may change scale or only the metric used to quantify the criteria or indicator may change with no definition change. In the North American test, no consideration was given to how the Criteria and indicators were related to other scales such as the Montreal process since such an evaluation was outside the scope of this project.