Found out yesterday that I got my funding application accepted. It was a competitive process, very time consuming to get it right and a bit of a pain running around to get signatures, but worth it in the end. The funds will support me to go to the Canadian Evaluation Conference and present my thesis. But I also get to go on a side trip and visit UCLA. I wrote to Professor Christy and Prof. Alkin who are going to host me for 2 days in April to present my research and also meet with their research students. YAY!
But the best news was that I received a DVC (research) Commendation for my application (and an extra payment!)
Now I just have to write the conference abstract and get it accepted.... eek.
Friday, September 23, 2016
Saturday, September 17, 2016
Evaluative Thinking
Today I read an article given to me by my supervisor and found that it rang more than a few chords with me, both in terms of my research and my work.
Buckley, Archibald, Hargreaves & Trochim (2015). Defining and Teaching Evaluative Thinking: Insights From research on Critical Thinking. American Journal of Evaluation 36(3), 375-388.
The authors begin by reminding us that links between evaluative thinking (ET) and critical thinking are not entirely new. Scriven has long insisted on the use of critical thinking in evaluation and the authors take the stance of offering ways in this paper of incorporating ET more intentionally in all aspects of evaluative work. They begin by reviewing current definitions of ET and providing their own succinct version.
The literature on ET reached a peak in 2013 which interestingly aligns with my own search of the literature (in 2013) where one of the themes to come from my review was evaluation capacity building.
The authors explain how ET is often compared to reflective practice, "questioning reflecting, learning and modifying.."(p377). I liken this to Wadsworth's MERI (monitor, evaluate, reflect, implement). They talk about systematic inquiry (Preskill & Boyle, 2008), an analytical way of thinking that infuses everything that goes on (Patton, 2005), skeptical questioning (Weiss, 1998), a combination of commitment and expertise comprised of evaluative know-how and an evaluative attitude (Davidson, Howe & Scriven, 2004).
They believe that the lack of meaningful discussion around ET and that is is not researched, measured, promoted or taught is due to the absence of a widely agreed upon definition (p.377). In order to build their own definition they have borrowed from the critical thinking literature.
Buckley, Archibald, Hargreaves & Trochim (2015). Defining and Teaching Evaluative Thinking: Insights From research on Critical Thinking. American Journal of Evaluation 36(3), 375-388.
The authors begin by reminding us that links between evaluative thinking (ET) and critical thinking are not entirely new. Scriven has long insisted on the use of critical thinking in evaluation and the authors take the stance of offering ways in this paper of incorporating ET more intentionally in all aspects of evaluative work. They begin by reviewing current definitions of ET and providing their own succinct version.
The literature on ET reached a peak in 2013 which interestingly aligns with my own search of the literature (in 2013) where one of the themes to come from my review was evaluation capacity building.
The authors explain how ET is often compared to reflective practice, "questioning reflecting, learning and modifying.."(p377). I liken this to Wadsworth's MERI (monitor, evaluate, reflect, implement). They talk about systematic inquiry (Preskill & Boyle, 2008), an analytical way of thinking that infuses everything that goes on (Patton, 2005), skeptical questioning (Weiss, 1998), a combination of commitment and expertise comprised of evaluative know-how and an evaluative attitude (Davidson, Howe & Scriven, 2004).
They believe that the lack of meaningful discussion around ET and that is is not researched, measured, promoted or taught is due to the absence of a widely agreed upon definition (p.377). In order to build their own definition they have borrowed from the critical thinking literature.
Evaluative thinking is critical thinking applied in the context of evaluation, motivated by an attitude of
inquisitiveness and a belief in the value of evidence, that involves identifying assumptions, posing
thoughtful questions, pursuing deeper understanding through reflection and perspective taking, and
informing decisions in preparation for action.(p378)
They make an interesting statement about not everyone needing to be an evaluator or do evaluative work on a program or team but if they employ evaluative thinking they will have a better chance of success.
Next the authors describe constructivism and how knowledge and understanding is built on what we know and believe. Then onto levels of thinking and that the highest order (Blooms) is evaluative thinking or evaluativist-level thinking. Humans are not born with an ability to engage in such high level critical thinking - it needs to be cultivated or acquired. Similar to walking and sprinting. One we do automatically, the other required training and not everyone reaches the top. (Analogy by David Perkins in Hunkins, 1995). So ET needs to be learned and practiced.
Another interesting point brought to light by the authors is that of "belief preservation" whereby the brain has a natural tendency to disbelieve evidence if it goes against an existing belief (p.380). Metacognition is required (another high-level thinking skill) to overcome this deficiency (Lord, Ross & Leper, 1979).
So the desire to practice ET needs to be intentional and done regularly as with any other thinking-skill, if learning is to occur.
The 5 guiding principles are aimed at ECB practitioners and other 'promoters' such as teachers and facilitators.
1. be opportunistic about engaging learners in ET processes in a way that can maximise intrinsic motivation
2. incorporate incremental experiences by using 'scaffolding'.
3. offer opportunities to intentionally practice ET
4. work to overcome assumptions and belief preservation
5. apply and practice ET in multiple contexts and alongside peers and colleagues.
The authors discuss these practical strategies for promoting ET but the acknowledge that time and resources can be daunting - this aligns with my findings about conducting Evaluation in small projects (Huber & Harvey, 2013).
In terms of changing an organisation's culture one needs to encourage all members to become ETs. But methods and strategies to teach and learn ET have the potential to improve the quality of an organisation through evaluation.
They make an interesting statement about not everyone needing to be an evaluator or do evaluative work on a program or team but if they employ evaluative thinking they will have a better chance of success.
Next the authors describe constructivism and how knowledge and understanding is built on what we know and believe. Then onto levels of thinking and that the highest order (Blooms) is evaluative thinking or evaluativist-level thinking. Humans are not born with an ability to engage in such high level critical thinking - it needs to be cultivated or acquired. Similar to walking and sprinting. One we do automatically, the other required training and not everyone reaches the top. (Analogy by David Perkins in Hunkins, 1995). So ET needs to be learned and practiced.
Another interesting point brought to light by the authors is that of "belief preservation" whereby the brain has a natural tendency to disbelieve evidence if it goes against an existing belief (p.380). Metacognition is required (another high-level thinking skill) to overcome this deficiency (Lord, Ross & Leper, 1979).
So the desire to practice ET needs to be intentional and done regularly as with any other thinking-skill, if learning is to occur.
The 5 guiding principles are aimed at ECB practitioners and other 'promoters' such as teachers and facilitators.
1. be opportunistic about engaging learners in ET processes in a way that can maximise intrinsic motivation
2. incorporate incremental experiences by using 'scaffolding'.
3. offer opportunities to intentionally practice ET
4. work to overcome assumptions and belief preservation
5. apply and practice ET in multiple contexts and alongside peers and colleagues.
The authors discuss these practical strategies for promoting ET but the acknowledge that time and resources can be daunting - this aligns with my findings about conducting Evaluation in small projects (Huber & Harvey, 2013).
In terms of changing an organisation's culture one needs to encourage all members to become ETs. But methods and strategies to teach and learn ET have the potential to improve the quality of an organisation through evaluation.
Subscribe to:
Posts (Atom)