CRADLE in journal metrics
7 July 2020
When the new round of journal Impact Factors, published by Clarivate Analytics, was recently released, I went to have a look to see how my ‘favourite’ journals (the ones who have accepted my papers!) had fared. I discovered quite quickly that the CRADLE team featured prominently in some of them.
The first step was to actually get access to the impact factors – thankfully, we have institutional access to Journal Citation Reports (JCR) – though it took a couple of goes on the library website to find the link! Generally, the way that Impact Factor is calculated is:
where the X time period is more recent than the Y time period (or how would you know if anything got cited?!). JCR provides a yearly Impact Factor (e.g. for 2019, it is 2019 citations for articles published 2017 – 2018), and a 5 Year Impact Factor (5YIF) (e.g. for 2018, it is 2018 citations for articles published 2013 – 2017). The 5YIF would smooth out differences between individual years – say, if someone published a really popular special issue.
The journal you’d probably expect to find a Centre for Research in Assessment and Digital Learning paper in is, obviously, one on assessment. So I had a look at Assessment and Evaluation in Higher Education (AEHE). Their JIF for 2019 is now 2.32, and the 5YIF is 2.526. The neat thing about JCR is that it lists JIF contributing items individually, which tells us which articles were the ones that got cited the most.
2019 Journal Impact Factor contributing items for AEHE
As you can see, CRADLE team members David Boud, Rola Ajjawi and Phillip Dawson take out the top three spots – with our Honorary Professor David Carless sharing that top paper on feedback literacy. What’s interesting here is that to an extent, all three papers are focusing on advancing a conceptual idea, even if it includes empirical data. I haven’t read all of the rest of the top 10 cited items but I suspect there’s a similar theme with many of the others. This fits nicely with the idea that good research has a “so what?” to it.
Beyond individual articles, Deakin University was also the top contributing organisation to AEHE, with 15 papers in the past two years. The runner up was the University of Hong Kong with nine articles, closely followed by the University of Queensland with eight articles. I suppose we could conclude from this that it is likely that CRADLE manages to do pretty well both on quality and quantity of published work – at least, in AEHE. I do also note that, due to the vagaries of journal issue allocation, the papers by Rola and David, and Phill, were Online First in late 2015 – so they’ve actually had a substantial amount of time online to gain citations prior to the 2019 calculation period.
CRADLE also features prominently in some other journals’ Impact Factor calculations. My paper on developing evaluative judgement in Higher Education is the top contributor of citations (a whole 16!) towards the 2019 JIF of 2.856, and while University of London contributed the most (16 papers) to the journal, Deakin features at equal 9th with six publications (along with Griffith, University of Cape Town, and University of Technology Sydney – I note this is one of Dave’s other affiliations).
My final journal for today is Higher Education Research & Development – its 2019 JIF is 2.129. Beverley Oliver and Trina Jorre de St Jorre’s paper on graduate attributes was 6th on the citable items list, contributing nine citations to the calculation. Again, Deakin University made the most contributions (17!), with the remaining top 10 institutions all being from Australia and New Zealand.
While Journal Impact Factors can tell us a little about how the journal is going overall, the data used to calculate them can tell us even more about how our individual papers are going – and perhaps, with sufficient data and investigation, are another way to investigate which type of work gets cited more, or less. There’s been some interesting research lately on the impact of Twitter promotion on citation rates, and previously, some work on the impact of article titles as well.
Overall, it seems like discussing journal metrics has become a theme for me. This probably won’t be my last post on this topic either – I understand CiteScores were also released recently!
Articles from the CRADLE team referred to in this post:
Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment and Evaluation in Higher Education, 43(8), 1315–1325. DOI:10.1080/02602938.2018.1463354
Ajjawi, R., & Boud, D. (2017). Researching feedback dialogue: an interactional analysis approach. Assessment & Evaluation in Higher Education, 42(2), 252–265. DOI:10.1080/02602938.2015.1102863
Dawson, P. (2017). Assessment rubrics: towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education, 42(3), 347–360. DOI:10.1080/02602938.2015.1111294
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: enabling students to make decisions about the quality of work. Higher Education, 76(3), 467–481. DOI:10.1007/s10734-017-0220-3
Oliver, B., & Jorre de St Jorre, T. (2018). Graduate attributes for 2020 and beyond: recommendations for Australian higher education providers. Higher Education Research and Development, 37(4), 821–836. DOI:10.1080/07294360.2018.1446415
