A Project Profile Approach to Evaluation

Accountability is a common driver for evaluation particularly as funding bodies strive to obtain measurable gains for their investments, in teacher content knowledge, change in practice and of course student learning. The authors of this study insist that individual project profiles are needed to take into account the unique contextual variables of a project whilst comparing projects across a funded program.

The context for this study was to look at professional development of teachers in the K-12 sector under the  Improving Teacher Quality State Grants Program. Each grant recipient is required to conduct some internal evaluation processes and also to be part of an external evaluation. This paper reports on the design of the latter. Goals were:

(1) how well projects attained their objectives; 

(2) the quality of the PD that was delivered, and 

(3) what outcomes were achieved for teachers and students.

Nine projects were investigated and a profile for each was constructed. The profile consisted of 6 sections: Project background; Project Design; participants and their schools; Quality of implementations; satisfaction survey; Outcomes and Recommendations. In other words do not comapare outcomes alone since the the teachers and the school settings can vary significantly across school sites and therefore outcomes alone do not tell the whole story.

Then the model of using project profiles was compared to a model for evaluating professional development programs (Guskey, 2000). Guskey's hierarchical model includes 6  levels, moving from the simple to the more complex: 

1. Participants' reactions

2. Participants' learning

3. Organization support and change

4. Participants' use of new knowledge and skills

5. Student learning outcomes

The authors mapped their model and found that they needed to modify Guskey's model to make it  more holistic. They created a central core with steps 1,2,4 and 5 each fed by step 3. And then an outer layer of content, context and process. (p.152)