2.1 Assessment Team

 Additional Experts:

2.2 Description of Methods

Prior to the initiation of the North American field test, several meetings were held to inform and seek participation in the test. In March, 1998, the CIFOR coordinator (Dr. Moad), Project Coordinator, (Dr. Hoekstra), Team Leader (Dr. Woodley), and Boise National Forest personnel made a series of presentations to potential cooperators and interested participants in the North American field test. Interested participants included agencies, corporations and others who might provide data, present research papers or observe the test. In addition, a meeting was held for groups that could have an interest in knowing what the test would be about but who would not be expected to participate. Finally a briefing was held for both State of Idaho and U.S. Congressional staff to provide information about what the North American test was and would expect to accomplish. In addition, the CIFOR test leaders met with the Boise National Forest and Boise State University personnel to make all the logistical arrangements for the period when the entire team would be in Boise during June –July, 1998.

Methods generally followed those prescribed by CIFOR (1996) with modifications to fit the site and North American context. Field testing of criteria and indicators (C&I) involved four stages, with each stage acting as a separate screening or filter. The filters used evaluation parameters to examine each criterion and indicator. There were separate evaluation forms for criteria and indicators. At each stage a particular criterion or indicator could be rejected, merged, or passed to a more detailed evaluation in the next stage. These filters were conceived to be more than a simple, staged sifting process. The four stages explicitly allowed creative inputs and modifications to C&I, provided these were also subjected to the evaluation process. The four stages are described below:

Stage No. 1: Pre-fieldwork

Stage 1 was essentially a desk exercise to carry out a preliminary evaluation of all C&I in the initial sets selected. First each team member was asked to read a comprehensive set of information on the study site, as well as local planning and evaluation documents. Then each criterion and indicator was numerically ranked against a set of parameters as shown in Form 1 (Appendix E). The parameters were as follows:

• Class definition? - (Biophysical, Social, Planning & Policy, Damage control management, Yield control management)
• Closely and unambiguously related to the assessment goal?
• Does it fit into a theoretical framework?
• Easy to detect, record and interpret?
• Provides a summary or integrative measure?
• Adequate response range to changes in levels of stress?

Each parameter was given a score of 1-5 and general comments were given. The results of Form 1 were tabulated and averaged in a spreadsheet and made available to the team at the start of the fieldwork stage.

Stage No. 2: Initial Fieldwork:

The team members assembled at the Boise study site and were given a detailed 3-day orientation workshop on the local social, economic, and ecological conditions, as well as a summary of data available for the test. The third day of the workshop was devoted to a field trip, which looked at forest conditions and harvest practices on all land ownership’s. The agenda for the initial workshop is provided in Appendix G. Following the orientation workshop, the team worked in sub-groups and discussed the tabulated results from Form 1. After debate, individual criteria and indicators were rejected, merged or allowed to go to the next, more detailed evaluation (Form 2, which was completed in stage 3).

Terminology was not consistent between the sets of criteria and indicators tested. To test the same elements between the indicator sets, members used the definitions of principles, criteria, and indicators provided by Tropenbos (1997), that are as follows:

• A principle is a fundamental law or rule, serving as a basis for reasoning and action. Principles have the character of an objective or attitude concerning the function of the forest ecosystem or concerning a relevant aspect of the social system that interacts with the ecosystem. Principles are explicit elements of a goal, e.g. sustainable forest management or well-managed forests.

• A criterion is a state or aspect of the dynamic process of the forest ecosystem, or a state of the interacting social system, which should be in place as a result of adherence to a principle. The way criteria are formulated should give rise to a verdict on the degree of compliance in an actual situation.

• An indicator is a quantitative or qualitative parameter which can be assessed in relation to a criterion. It describes in an objectively verifiable and unambiguous way features of the ecosystem or the related social system, or it describes elements of prevailing policy and management conditions and human driven processes indicative of he state of the eco- and social system.

• A norm is the reference value of the indicator and is established for use as a rule or a basis for comparison. By comparing the norm with the actual measured value, the result demonstrates the degree of fulfillment of a criterion and of compliance with a principle. Note: We used target and verifier in the same context.

At this point we regrouped the criteria and indicators under three main principles, following the Tropenbos definitions. The three main principles followed an amalgamation of principles provided by various sets of indicators. The principles were (1) maintenance of ecological integrity, (2) yield and quality of forest goods are sustainable, and (3) society accepts responsibilities for resource management. These three generalized principles are common to virtually all definitions of sustainability.

After regrouping the results of the first filtering of candidate C&I, the team worked in groups, or singly, to conduct a detailed evaluation of the each of the remaining criteria and indicators. Most emphasis was on field testing indicators. Members worked with reference material, and in discussion groups, to critique or refine the theoretical basis for each criterion or indicator. In some cases, this meant directly contacting experts in the local area or reaching others by telephone or e-mail. For each indicator, team members also attempted to use data from the Boise Study Area to assess its practicality. Wherever possible, team members talked to local resource people to get their views on the value of the indicator. In practice, this stage of the field work consisted of a cyclic repetition of inductive and deductive approaches, as team members applied their existing knowledge to the C&I, then tested their conclusions against field realities and their colleagues’ views, returning to re-evaluate the C&I with their now-improved knowledge, in an iterative process.

For each indicator tested, team members filled out assessment forms (see Form 2, Appendix F). The detailed test results are presented in Volume II of this report.

State No. 4: Post-fieldwork stage - workshop:

Following the field-testing and modification of candidate C&I, the process turned to a discussion/modification stage where team members discussed proposed C&I with other participants. A total of 60 new participants attended a post-fieldwork closing workshop. The participants represented a wealth of experience from academia, industry and government, characterized by their knowledge and interest in sustainable forest management.

During this two-day workshop, working groups discussed the proposals made by the team concerned. This provided peer review to the team members’ work and also a first view of the wider applicability of the proposed C&I.