evaluation or research

Leading on from last weeks ramblings on the differences between research and evaluation, Owen (2006 p.64) says that the inclusion of the planning and communicating stages is what differentiates evaluation from social research. The 'middle section' is the research which uses similar ranges of data collection and analysis techniques.

I want to concentrate on this difference. Try and prove that concentrating on the first and 3rd sections of the pie (picture) will help improve evaluation practices. ie the interactive instrument helps with these sections. and help with including time to do them.

Need more readings around time taken for each of these sections. Need to continue reading Owen.

Update: Nov 2011

Alkin & Taaut, 2003 write that the goal of research is generizable knowledge but the purpose of evaluation is context-specific (p.3). They Quote Cronbach & Suppes (1969) and conclusion-oriented research vs. decision-oriented evaluation.

update March 2012

Alkin (2011 - Evaluation Essentials form A to Z) states on p.8 that research seeks conclusions and evaluation leads to decisions. researchers ask their own questions in order to seek conclusions they can use to add to the knowledge bank. Evaluation answers questions that are important to a particular person - the stakeholder or client, say.

Alkin talks briefly about the definition of evaluation (which are goal orientated, around merit and worth) but he directs the readers focus to the processes which allow one to reach the point of being ready to judge merit and worth.

Update August 2012

reading Mackenzie, N. M. & Ling, L. M. (2009). The research journey: A Lonely Planet approach. Issues In Educational Research, 19(1), 48-60. http://www.iier.org.au/iier19/mackenzie.html

quote Mertens (Mertens, D. M. (2005). Research methods in education and psychology: Integrating diversity with quantitative and qualitative approaches (2nd ed.). Thousand Oaks, CA: SAGE.) though i can't give a page number of this as its in html (open access)


"This highlights the decisions we take as researchers when we aim for convergent outcomes or divergent outcomes. If our research is so prescribed and directed as to push us towards particular desired outcomes, it is convergent and in fact, may potentially not be research at all. Mertens (2005) makes a distinction between research and other forms of activity such as evaluation.

The relationship between research and evaluation is not simplistic. Much of evaluation can look remarkably like research and vice versa. Both make use of systemic inquiry methods to collect, analyse, interpret and use data to understand, describe, predict, control or empower. Evaluation is more typically associated with the need for information for decision making in a specific setting, and research is more typically associated with generating new knowledge that can be transferred to other settings. (p.2)

In fact, much of the prescribed and funded so-called research we undertake for convergent outcomes which fit the agenda of funding bodies is probably more akin to evaluation than research. Research which does not work towards pre-determined or prescribed outcomes and thus can produce divergent outcomes more in the spirit of what we understand as true research."

some words

empirical : derived from or relating to experiment and observation rather than theory
epistemology: The branch of philosophy that studies the nature of knowledge, its presuppositions and foundations, and its extent and validity.

Accepted

Wow, amazed that I didn't have to make any modifications to my proposal! After a few technical hiccups, I received the following email:

Dear Elaine,

I am pleased to advise you that you have been offered a place in the PhD in Education program for Semester 2, 2011 at Macquarie University.

And then:

Dear Elaine,

I am pleased to advise you that you have now been enrolled as at 30 September 2011in the Doctor of Philosophy in Education for Semester 2, 2011 as a Part Time On-site Domestic candidate.  Your expected date of completion will be 30 September 2019.

wow that's a loooong time away!

More mq projects

I revisited the MQ Grants 'previous winners' webpage to find it updated with many more final reports. There are now 29 in total from across the competitive and strategic grants categories.

I've been thinking about how to best make use of these reports. I could compare the final report with the initial application (were available, and when not, I could approach the awardee and ask them for a copy). This may highlight changes, things they said they would do but haven't.

I've been grappling with the 'criteria' - should i use the excellent questions provided by Chesterton and Cummings and supplement with Datta etc? Stufflebeam has an excellent Metaevaluation checklist but it is too detailed with 300 items to be checked. If we could use this it would provide excellent quantitative data. But I thik we cannot.

The main issue for me at the moment is my first question is 'what evaluation forms and approaches were used in this project?' And I have a feeling my answer for majority is 'none'. Maybe its about terminology. Some things that are covered by the word evaluation are things like data collection - isn't this research though? Now I'm struggling with the difference between research and evaluation.....
some more searches required me thinks.

An article by Julia Coffman, Consultant, HFRP based on Scriven's work:

http://www.hfrp.org/evaluation/the-evaluation-exchange/issue-archive/reflecting-on-the-past-and-future-of-evaluation/michael-scriven-on-the-differences-between-evaluation-and-social-science-research

How are evaluation and social science research different?Evaluation determines the merit, worth, or value of things. The evaluation process identifies relevant values or standards that apply to what is being evaluated, performs empirical investigation using techniques from the social sciences, and then integrates conclusions with the standards into an overall evaluation or set of evaluations (Scriven, 1991).
Social science research, by contrast, does not aim for or achieve evaluative conclusions. It is restricted to empirical (rather than evaluative) research, and bases its conclusions only on factual results—that is, observed, measured, or calculated data. Social science research does not establish standards or values and then integrate them with factual results to reach evaluative conclusions. In fact, the dominant social science doctrine for many decades prided itself on being value free. So for the moment, social science research excludes evaluation.¹However, in deference to social science research, it must be stressed again that without using social science methods, little evaluation can be done. One cannot say, however, that evaluation is the application of social science methods to solve social problems. It is much more than that.