Participatory Evaluation

Lawrenz, (2003). How Can Multi-Site Evaluations be Participatory?

This article takes a look at 5 NSF funded multi-site programs, and asks the question whether they can be considered truly participatory since participatory evaluation requires stakeholder groups to have meaningful input in all phases including evaluation design, defining outcomes and selecting interventions. The authors highlight the fact though that these projects were funded through a competitive process and selection of successful projects was not based on their facilitation of successful (central) program evaluation. The programs investigated were: Local Systemic Change (LSC), Collaboratives for Excellence in Teacher Preparation Program (CETP), the Centers for Learning and Teaching (CLT), and Advanced Technological Education (ATE) and the Mathematics and Science Partnerships (MSP). 
Criteria used to evaluate whether these programs were particiatory in their evaluation practices drew on two frameworks, Cousins and Whitmore's three-dimensional formulizations of collaborative inquiry (1998) and Bourke's participatory evaluation spiral design using 8 key decision points (1998). Four types of decision making questions were used to compare the degree by which each of the individual projects were involved with the program evaluation. These were
1) the type of evaluation information collected, such as defining questions and instruments; 
(2) whether or not to participate; 
(3) what data to provide; and 
(4) how to use the evaluation information.

Findings showed that the programs were spread across a continium from no participation to full participation. So the authors next asked 'in what ways can participation contribute to the overall quality of the evaluation' (p.476). They suggest four specific dimensionsof quality evaluation: 
1) objectivity, 
(2) design of the evaluation effort, 
(3) relationship to site goals and context and 
(4) motivation to provide data.
 and go on to discuss these in relation to the literature. Finally they propose a model for participatory multi-site evaluations which they name a 'negotiated evaluation approach'. The approach consists of three stages, creating the local evaluations (each project), creating the central evaluation team and negotiation and collaboration on the participatory multi-site evaluation.   This enables the  evaluation plan to evolve out of the investigations at the sites and results in instruments and processes which are grounded in reality of the program as it is implemented.