Evaluative judgement paper reaches its first 100 citations!
19 May 2020
On February 28, 2020, I was excited to see that my paper ‘Developing Evaluative Judgement: Enabling students to make decisions about quality of work’ had reached 100 citations on Google Scholar. It’s been just over 3 years since it was published early online and, at that time, I wasn’t sure if my colleagues were just being rather kind about an idea that I’d started to advance during my PhD (in a previous paper). Of course, evaluative judgement comes out of a strong and well-respected set of ideas, developed by D. Royce Sadler, David Boud and David Nicol, amongst others. It seems that in its latest incarnation, evaluative judgement has resonated with a lot of people and, specifically, researchers. It’s my most cited paper by a long run and so I thought, to celebrate its first century, I’d have a quick look at how the idea of evaluative judgement (EJ) has been taken up.
https://twitter.com/DrJoannaT/status/1233289856756748288?s=20
Firstly, a few statistics thanks to Harzing’s Publish or Perish and Google Scholar: the paper has been cited by 104 other papers, which collectively have been cited a total of 415 times. I suppose this says something about the potential reach of the paper (and what the ARC might count as engagement…). Looking at the most cited papers which cite the EJ paper, most of them are indeed authored by close colleagues: the top three are Boud & Carless‘ paper on feedback literacy; Dawson et al. on effective feedback; and our own edited book on evaluative judgement. The data get a bit messy on publication sources, but it seems that 15 citing papers have been published in Assessment & Evaluation in Higher Education and 4 in Studies in Higher Education. It seems that quite a few journal proceedings have also referred to it. Crunching the titles of all these citing articles, it seems that ‘feedback’ is the most commonly used word (40 times), followed by ‘assessment’ (36 times), ‘students’ (29 times) and ‘learning’ (25 times).
On the whole, it would seem that the work on EJ is being interpreted and used by people in the fields we might expect. So, what are these citing articles? For the whole list, you can check out Google Scholar.
It looks like two papers have recently included the word ‘ecological’ in their titles – this paper on an ecological perspective of students’ feedback literacy by Sin Wang Chong; and this paper on ecological teaching evaluation and data, by Tim Fawns & colleagues. Both of these papers take new perspectives which may help to explore existing issues in more detail.
A student perspective on evaluative judgement has also been published by James Bartoli-Edwards, a BA student at the University of Sussex. To me, this was a huge milestone – while my research has been on how students develop EJ, to have a current student spend time writing an article about it really indicates the meaningfulness and intelligibility of the concept.
Finally, it’s also really awesome that there are several papers that aren’t in English. I can’t claim to speak Italian or Spanish, and my school German is extremely rusty, so I’m humbled that researchers have made the time and effort to consider work that’s published only in English:
Bruna CE, Villarroel VA, Bruna DV, Martinez JA. Experiencia de Diseño y Uso de una Rúbrica para Evaluar Informes de Laboratorio en Formato Publicación Científica
Calderon-Garrido D, Gustems-Carnicer J, Carrera X. Música y tic en los grados de maestro: contenidos y actividades en los planes docentes
Lluch Molins L, Pons Segui L, Cano Garcia E. La formación del juicio evaluativo como elemento básico de la competencia de aprender a aprender: El impacto de los criterios de evaluación
Serbati A, Grion V, Fanti M. Peer feedback features and evaluative judgment in a blended university course
Wiesner C, Schreiner C, Paasch D, Breit S, Freunberger R. Der kompetenzorientierte Unterricht in Mathematik in der österreichischen Volksschule aus Sicht der Schüler/innen: Impulse für eine reflexive Unterrichtsentwicklung und -forschung
I think that’s enough navel-gazing for me. I’m so glad that work I’ve been part of has seemingly made a reasonable contribution to the field of higher education assessment and feedback research. When I started – and even handed in – my PhD I never could have imagined that this work would go so far, so quickly!
