What can be evaluated?

The following excerpt is taken from: Johannessen, E.M. (2001). Guidelines for evaluation of education projects in emergency situations. The Norwegian Refugee Council. http://toolkit.ineesite.org/toolkit/INEEcms/uploads/1039/Guidelines_for_Evaluation_of_Educ_Projects.PDF

5 WHAT CAN BE EVALUATED?
It is common in many evaluations to focus on the OECD criteria: relevance and fulfillment of objectives, development efficiency, effectiveness, impact and sustainability. Relevance refers to whether the project addresses the needs of the beneficiaries, efficiency to how productively the resources have been used while effectiveness relates to whether the objectives of the project were achieved. Impact is the long-term effect  while sustainability relates to the maintenance of the changes after the project or program has been terminated (Dale 1998, Lewin 1993, Samset 1993).   Sida adds "lessons learned" to the criteria which refer to important general experience the evaluation has yielded. All or some of the criteria are commonly addressed in evaluations of education projects although the terms may be used differently.
Although the concepts seem relevant and reasonable, there are several problems associated
with them. First of all, they need to be defined and specified in terms of identifiable activities when the project is being planned. This is often not the case. Secondly, the concepts "relevance", "impact" and "sustainability" may be difficult to investigate. The project may have been going on for a too short period and/or there is not enough time available to find answers to this type of questions.  Another objection is that the OECD concepts are related to final products and not to description of the processes that have taken place. No projects/programs, whether in research or evaluations, develop neatly in a linear way. The description of the project and the steps we plan in the beginning are always transformed as they are being implemented. Major decisions regarding adjustments and turning points should be described and justified and be part of the evaluation.

I think it will be useful in the next paper when I'm analysing the themes and relating them back to the literature.

The following excerpt is from p12. The author uses the terminology 'Terms of Reference' (ToR) to describe to evaluation plan. This is in line with other articles and will fit into the paper on lit review/gaps in the lit.

Often the terms of reference are too vast and ambitious taken the time and resources into
consideration. Sometimes questions and topics are suggested that are not possible to answer, at least not within the limited time available. As a result the evaluators know already before they start working that it is not feasible to follow the ToR

And again on p18:

8 WHEN TO EVALUATE
Ideally the planning of an evaluation should start parallel with the activities/project. But more
often the opposite is the case; the coordinators start too think about evaluation when the
project  is concluded or about to finish.