Shaking power structures through summative self-assessment: CRADLE Seminar Series

15 April 2019

Summative self-assessment was the fascinating and provocative topic of the recent CRADLE seminar presented by the University of Helsinki’s Juuso Nieminen. Here, CRADLE Fellow Dr Kate Anderson offers a thoughtful reflection on Juuso’s seminar – and if you weren’t able to make the seminar on the day, a recording is now available! The recording, along with a link to Juuso’s slides, is available to view at the bottom of the page.

For me, the best CRADLE seminars provoke ongoing discussion, debate, and speculation. I can usually tell just how provocative a seminar has been by how many cups of tea it takes me to digest it. Last week, we were fortunate to hear from Juuso Nieminen about his doctoral work at the University of Helsinki, in which he is pioneering the use of summative self-assessment in mathematics higher education. Juuso’s work also explores how summative self-assessments disrupt traditional power structures within higher education, and the impacts this can have on student learning.

This intriguing presentation was most certainly a “three-cup seminar”.

Formative self-assessment methods are gaining good traction in teaching research and practice, and are increasingly valued as a vehicle to support student self-monitoring, reflective learning and the development of evaluative judgement (see prior CRADLE seminars on self-assessment and evaluative judgement). Like previous authors (e.g. Kelvin Tan), Juuso proposes summative self-assessment as a more active mechanism for students to engage in the assessment process. His presentation also highlighted the alignment of summative self-assessment measures with university objectives around sustainable, life-long learning. Despite this, the academic community has largely resisted adopting self-assessment as a means of summative evaluation, particularly in fields like pure mathematics where objective teacher-led assessment methods (such as examinations) are strongly favoured by both students and staff. As such, embracing summative self-assessment demands a significant paradigm shift for all involved.

So what can summative self-assessment offer, beyond its formative counterpart? Using a randomised trial design coupled with qualitative interview methods, Juuso and his colleagues were able to compare the outcomes for students completing the same introductory linear algebra course under two separate conditions: 1) formative self-assessment activities culminating in a traditional summative exam, versus 2) formative self-assessment activities culminating in summative self-assessment. I’ll leave Juuso to detail the results in his full seminar recording, but some of the standout findings for me were that:

1. Students who completed the summative self-assessment did not out-perform students in the traditional examination group, suggesting that these students were not artificially inflating their performance scores (…in other words, summative self-assessment did not automatically result in cheating for this cohort).
2. Students in the different summative conditions demonstrated different approaches to learning, with more students in the summative self-assessment condition showing greater tendencies towards deep- versus surface-learning approaches.
3. For students who exhibited deep or lower-deep learning profiles, being assigned to the summative self-assessment condition correlated with higher degrees of self-efficacy than in the traditional examination condition.
4. Qualitative findings showed that summative self-assessment exhibited a complex relationship with student empowerment.

This latter point intrigued me, and elicited a significant degree of reflection during my first cup of tea after the seminar. As an educator it could seem intuitive that giving students more “power” in the assessment dialogue (in this case, more responsibility to have the final say over their end grade), would be “empowering”. Yet Juuso showed that the experience of summative self-assessment had both empowering and disempowering consequences, which were visibly framed through sovereign, epistemological, and disciplinary notions. For instance, in the case of sovereign power, students could be seen “taking back” power as they harnessed study for personal learning, as opposed to study for assessment. On the other hand, students also expressed some doubt as to whose opinion was most salient when it came to assessment of ability – their own opinions, or those of their expert teacher? Additionally, students hinted at the tension between emergent assessment methods (such as self-assessment) and institutional or industry epistemologies about what assessment “should look like” in the discipline. What does this mean, then, when students are empowered locally, but the overarching hegemony is not shaken? I feel that Juuso’s findings reinforce a concept well-known to the humanitarian sector: you can’t empower people simply by handing over the reins – appropriate scaffolds, supports, and infrastructure must be in place. I also agree with Juuso that a further unpacking of the relationship between self-efficacy and empowerment (terms that are frequently synonymous in the literature) is warranted here.

Over a second cup of tea I eavesdropped on the intellectual fallout from the presentation. It was clear that Juuso’s ideas had sparked deep introspection and energetic debate amongst audience members. For instance:

• Are all students equally empowered within the summative self-assessment forum?
• Is the integrity and accuracy of such summative self-assessment likely to remain stable over time, and what moderation might be required to ensure this?
• What is the experience like for struggling students, who face failing themselves at the end of the course?
• What is the impact of gender, culture and other personal factors on students’ approaches and responses to self-assessment?

As someone who experiences imposter syndrome (and sees this regularly in my students), I found myself pondering how I would fare in tertiary learning situations where my own self-assessments had the ultimate say. Could I learn to challenge my imposter feelings with some recalibration from my teachers, or would I always be seeking out that examination score as validation? Reflecting back on Juuso’s findings around empowerment and disempowerment, we pondered as a group what different types of students might gain or lose from their engagement with the summative assessment process, and how we as educators might support this within the feedback dialogue.

Heading back upstairs, I realised I still wasn’t ready to let this topic go. I made for the kitchen and, over a third and final cup, considered why the concept of summative self-assessment is such a challenging one. As Phillip Dawson pointed out, Juuso’s topic is indeed contentious in the current Australian climate around academic integrity, yet I found it interesting that concerns around integrity sparked the least debate today. I suspect this reflects the contrast between communities of scholarship and communities of practice in teaching and learning around the way students are positioned – and trusted – within the assessment process. The question remains as to whether current epistemological paradigms of academic integrity will, can, or even should adapt to accommodate summative self-assessment models in the future.

Considering the depth of reflection above, I believe our journey into the scary lands of summative self-assessment has been more than worth the caffeine-induced sleeplessness ahead. Juuso has also shared with us a rigorous and considered methodology for comparing self-assessment models, and his points on learner heterogeneity provide some important considerations for future research and practice in this area. Meanwhile, tales of disruption in the power structures of higher education are always refreshing… almost as refreshing, in fact, as a hot cup of tea.

{{CODE-JN-9.4.19}}

View slides from Juuso’s seminar here.