Chapter 2

Just looking at Chapter 2, the lit. review.

Some feedback from Marina, try to use some more action words to stress the need for what i did and why.

I then came across a table summarising the articles i used in the lit review - this will be fabulous as an appendix - or perhaps even as part of the intro to chapter 2?

Whilst doing that I recalled and reread an article from Oliver, McBean, Conole & Harvey (2002). They had developed an online toolkit to help evaluators of projects get to grips with evaluation. On the premise that a one size does not fit all:

The toolkit involves six steps, derived from the literature and from research into evaluation practice:

1. Identification of the audience for the evaluation
   
2. Selection of an evaluation question
3. Choice of an evaluation methodology
4. Choice of data collection methods
5. Choice of data analysis methods
6. Selection of the most appropriate format(s) for reporting the findings to the audience 

(p.200-201).

This is eerily similar to mine. however their tool took 4.5 hours to complete and the authors agree that "Although it could be argued that this reflects the complex demands of evaluation, and thus is not unreasonable, it does mean that the toolkit is ill-suited to small, quick studies." (p.207).

Narrative

compiling some literature and words around the use of narrative

The Narrative Construction of Reality (Bruner, 1991):

"...we organise our experience and our memory of human happenings mainly in the form of narrative—stories, excuses, myths, reasons for doing and not doing..." (Bruner, 1991, p4). 

"Narratives, then, are a version of reality whose acceptability is governed by convention and "narrative necessity" rather than by empirical verification and logical requiredness..." (Bruner, 1991, p4). 

Bruner provides 10 features of a narrative:

1. Narrative diachronicity - an account of events occurring over time.
2. Particularity - context or a particular embodiment
3. Intentional state entailment - so no causality, just the basis for interpreation of what happens
4. Hermeneutic composability - interpretation through intention attribution and back- ground knowledge
5. Canonicity and breach - (not sure how this works...)

6. Referentiality
7?
8. Normativeness

9. Context sensitivity and negotiability

10. Narrative accrual

In this paper Bruner has described how reality is described through narrative principles, how thought is enunciated (through discourse). He concludes that his work has just begun and that he wants to show how narrative can organise 'the structure of human experience' (p.21).

The Value of Narrativity in the Representation of Reality (white, 1980):

"narrative might well be considered a solution to a problem of general human concern, namely, the problem of how to translate knowing into telling" (White, 1980, p.6)

So what is the difference between narrative and discourse? The latter is subjective and identifies an 'ego' whereas the former is objective and is just the logical progression of facts and information that join to tell the account.

"Benveniste shows that certain grammatical forms like the pronoun "I" (and its implicit reference "thou"), the pronominal "indicators" (certain demonstrative pronouns), the adverbial indicators (like "here," "now," "yesterday," "today," "tomorrow," etc.) and, at least in French, certain verb tenses like the present, the present perfect, and the future, find themselves limited to discourse, while narrative in the strictest sense is distinguished by the exclusive use of the third person and of such forms as the preterit and the pluperfect." p.7.

So, i think I am using discourse not narrative if I am using 'I'?

 Next I need to read the work of Albert Bandura? - Actually not much use - more about social cognative therapy...

This article may have something of interest:

Narrative Means to Preventative Ends: A Narrative Engagement Framework for Designing Prevention Interventions
Michelle Miller-Day  & Michael L. Hech

http://doi/10.1080/10410236.2012.762861

PGRF

Found out yesterday that I got my funding application accepted. It was a competitive process, very time consuming to get it right and a bit of a pain running around to get signatures, but worth it in the end. The funds will support me to go to the Canadian Evaluation Conference and present my thesis. But I also get to go on a side trip and visit UCLA. I wrote to Professor Christy and Prof. Alkin who are going to host me for 2 days in April to present my research and also meet with their research students. YAY!

But the best news was that I received a DVC (research) Commendation for my application (and an extra payment!)

Now I just have to write the conference abstract and get it accepted.... eek.

Evaluative Thinking

Today I read an article given to me by my supervisor and found that it rang more than a few chords with me, both in terms of my research and my work.

Buckley, Archibald, Hargreaves & Trochim (2015). Defining and Teaching Evaluative Thinking: Insights From research on Critical Thinking. American Journal of Evaluation 36(3), 375-388. 

The authors begin by reminding us that links between evaluative thinking (ET) and critical thinking are not entirely new. Scriven has long insisted on the use of critical thinking in evaluation and the authors take the stance of offering ways in this paper of incorporating ET more intentionally in all aspects of evaluative work. They begin by reviewing current definitions of ET and providing their own succinct version.

The literature on ET reached a peak in 2013 which interestingly aligns with my own search of the literature (in 2013) where one of the themes to come from my review was evaluation capacity building.

The authors explain how ET is often compared to reflective practice, "questioning reflecting, learning and modifying.."(p377). I liken this to Wadsworth's MERI (monitor, evaluate, reflect, implement). They talk about systematic inquiry (Preskill & Boyle, 2008), an analytical way of thinking that infuses everything that goes on (Patton, 2005), skeptical questioning (Weiss, 1998), a combination of commitment and expertise comprised of evaluative know-how and an evaluative attitude (Davidson, Howe & Scriven, 2004).

They believe that the lack of meaningful discussion around ET and that is is not researched, measured, promoted or taught is due to the absence of a widely agreed upon definition (p.377). In order to build their own definition they have borrowed from the critical thinking literature.

Evaluative thinking is critical thinking applied in the context of evaluation, motivated by an attitude of inquisitiveness and a belief in the value of evidence, that involves identifying assumptions, posing thoughtful questions, pursuing deeper understanding through reflection and perspective taking, and informing decisions in preparation for action.(p378)

 They make an interesting statement about not everyone needing to be an evaluator or do evaluative work on a program or team but if they employ evaluative thinking they will have a better chance of success.

Next the authors describe constructivism and how knowledge and understanding is built on what we know and believe. Then onto levels of thinking and that the highest order (Blooms) is evaluative thinking or evaluativist-level thinking. Humans are not born with an ability to engage in such high level critical thinking - it needs to be cultivated or acquired. Similar to walking and sprinting. One we do automatically, the other required training and not everyone reaches the top. (Analogy by David Perkins in Hunkins, 1995). So ET needs to be learned and practiced.

Another interesting point brought to light by the authors is that of "belief preservation" whereby the brain has a natural tendency to disbelieve evidence if it goes against an existing belief (p.380). Metacognition is required (another high-level thinking skill) to overcome this deficiency (Lord, Ross & Leper, 1979).

So the desire to practice ET needs to be intentional and done regularly as with any other thinking-skill, if learning is to occur.

The 5 guiding principles are aimed at ECB practitioners and other 'promoters' such as teachers and facilitators.

1. be opportunistic about engaging learners in ET processes in a way that can maximise intrinsic motivation
2. incorporate incremental experiences by using 'scaffolding'.
3. offer opportunities to intentionally practice ET
4. work to overcome assumptions and belief preservation
5. apply and practice ET in multiple contexts and alongside peers and colleagues.

The authors discuss these practical strategies for promoting ET but the acknowledge that time and resources can be daunting - this aligns with my findings about conducting Evaluation in small projects (Huber & Harvey, 2013).

In terms of changing an organisation's culture one needs to encourage all members to become ETs. But methods and strategies to teach and learn ET have the potential to improve the quality of an organisation through evaluation.