Q13 - other comments

This was an interesting final question. There was a number of themes that came from each interviewees final responses.

A number of people talked about the need for support and resources on evaluation. Both during the time of application and also during the project. Suggested ideas included templates and guidelines as well as information available of different frameworks and their benefits.

A few people mentioned having time to look back at the project revisit it an look at impact - but I guess that would depend on whether the project produced an 'product' that could be evaluated for impact.

A few people mentioned the importance of incorporating the evaluation into the research cycle. And there was the mention of the importance of receiving constructive feedback from the university. This could be viewed as a need for identifying study audiences and stakeholders which was another theme that came out re the importance if one required some traction in implementing the outcomes of a project.

Another theme was the forms - make evaluation compulsory and give it importance by having a section on the application, but more importantly on the reporting pro-forma.

And finally the theme of networking was also evident. Participants mentioned the benefit of sharing findings with people who were interested ie L&T related research etc.

Q5 - Stakeholders (primary and secondary)

This question asked whether stakeholder and study audiences were identified (and hence consulted - though this wasn't asked but implied) and furthermore whether it was both primary and secondary stakeholders.
The aim of this set of questions was to see whether there was any understanding of the terminology and also the relevance of identifying these groups.
There is a good paper on working with Evaluation Stakeholders (Bryson, Patton & Bowman, 2011) which offers a rationale, step-wise approach and toolkit. They define stakeholders as 'individuals, groups or organisations that can affect or are affected by an evaluation process and/ or its findings. (p.1)

In Stufflebeam's 1974 Seminal paper on Meta-Evaluation (republished in 2011), one of his eleven essential criteria for technical adequacy of evaluations, is Relevance: 'This criterion concerns whether the findings respond to the purposes of the evaluation. What are the audiences? What information do they need? To what extent is the evaluation design responsive to the stated purposes of the study and the information requests made by the audiences? The concern for relevance is crucial if the findings are to have more than academic appeal and if they are to be used by the intended audiences. Application of the criterion of relevance requires that the evaluation audiences and purposes be specified. Such specifications essentially result in the questions to be answered. Relevance is determined by comparing each datum to be gathered with the questions to be answered.'

Themes from the interviews:

• Six projects mentioned students
• Some confusion about this topic
• Not thought of or implicitly stated in the application.