2nd focus group for phase 3

After the first focus group I revisited the planning instrument and took on some of the feedback to rethink the content and structure.

The main concern from the focus group was that it was too lengthy for a small project. So some steps were combined and others removed. The resultant checklist has 6 steps:

1. What is the purpose and scope of the evaluation? Consider also how the information will be used. (i, ii, iii, iv, v, vi.)

2. Who are the stakeholders of the project and of the evaluation? Are they also part of the study audience? i, v, vi.

3.  What are the Key Evaluation Questions? i

4. What data and evidence will be collected and how will it be analysed? i, ii, v

5. What are the criteria for judgment? i, ii.

6. What dissemination strategies will be used and how will this help you? i, vi.

i.     Chesterton, P., & Cummings, R. (2011). OLT Grants Scheme - Evaluating Projects.key to literature

ii.    Owen, J. M. (2006). Program Evaluation. Forms and Approaches. Crows Nest: Allen and Unwin.

iii.   Patton, M. (1994). Developmental Evaluation. American Journal of Evaluation, 15(3), 311–319.

iv.   Scriven, M. (1991). Evaluation thesaurus. Newbury Park, Calif. Sage Publications.

v.     Saunders, M. (2000). Beginning an evaluation with RUFDATA: Theorising a practical approach to evaluation planning. Evaluation 6(1) 7-21.

vi.  Stufflebeam, D. L. (2011). Meta-Evaluation. Journal of MultiDisciplinary Evaluation, 7(15). 

Some key points to come from this focus group included:

• allow users to begin at any point as some felt step 1 (outcomes) was not their logical starting point.
• use the langauge of progress rather than achievement.
• add back in a question about recommendations for the future - for people who want to pick up where this project leaves off.
• an iterative approach to the steps would also be useful. Feedback loops. Once you start your project you usually revisit these steps and answers may change as you go.
•  collecting evidence - not only what but also how (many people forget ethics approval)