National Qualifications Experience 2020: Rapid Review

This independent review by the University of Stirling looks at the processes through which National Qualifications were awarded in 2020 after exams were cancelled due to the coronavirus pandemic.

In March 2020, in the face of the disruption caused by the COVID-19 pandemic, and following the cancellation of the 2020 examinations diet, the Scottish Qualifications Authority (SQA) was commissioned by the Deputy First Minister and Cabinet Secretary for Education and Skills to provide alternative means for awarding qualifications, based on three principles:

• fairness to all learners;
• safe and secure certification of qualifications, while following the latest public health advice;
• maintaining the integrity and credibility of the qualifications system, ensuring that standards are maintained over time, in the interest of learners.

The SQA subsequently developed the Alternative Certification Model (ACM), comprising the following steps:

• Step 1 — Estimates
• Step 2 — Awarding
• Step 3 — Results and certification
• Step 4 — Appeals

This review considers the evidence and provides recommendations around those issues which are most pertinent to consideration of awarding methodology in 2021 if there is further significant disruption to learning and teaching and/or the cancellation of exams.

Key Findings:

• This report draws upon a range of evidence, including stakeholder testimony (generated in panel and individual interviews) and analysis of relevant documentation (including government and SQA emails).
• SQA, the government, local authorities and schools faced an extremely difficult set of circumstances, within which there were no easy solutions. In this context, a workable system for qualifications, the Alternative Certification Model (ACM), was developed. This was based on three core principles and four stages.
• All parties involved in the process were found to have acted with integrity, with the best interests of students in mind.
• Respondents (teachers, lecturers, head teachers and local authority officials) generally found that SQA guidance was clear and useful.
• The generation of estimated grades, while clearly undertaken with integrity in the majority of centres, has been subject to variation (in the types of evidence available, the processes followed for internal moderation and the support given by local authorities), which has impacted on reliability and consistency of assessment at this stage.
• The statistical approach to moderation could have been more transparent earlier in the process, and moreover it has led to anomalies in grade adjustment, especially at the level of subject cohorts within centres and individuals.
• There is widespread criticism by respondents of SQA for a perceived lack of transparency and a failure to engage in participative development of solutions with stakeholders.
• While the application of the appeals process offered an in-principle technical solution to address these anomalies, it paid insufficient attention to the severe impact on those students obliged to undergo it (in terms of mental health and wellbeing, missed opportunities to transition into Higher Education, etc.).
• Principles relating to what data is appropriate to be held by certain organisations at certain points in time.(i.e. SQA, the Scottish Government), which make perfect sense in normal times (e.g. arrangements around data sharing), appear to have impeded the development of actions that might have led to an earlier anticipation and mitigation of subsequent problems.
• The equalities implications of an over-reliance on a statistical approach, premised on comparison with historical cohort data, had been raised repeatedly from April onwards, but seem to have been under-emphasised by both the government and SQA until late in the process.
• Many stakeholders believe that, subsequently, opportunities were missed (or dismissed) to engage in qualitative moderation of the statistical process (e.g. sense-checking of anomalous cohort patterns by local authorities).
• There has been an erosion of trust/confidence in SQA amongst teachers and young people, and damaged relations in some cases between young people and their teachers.
• Communications (with professionals and with young people and their families) has been a constant source of criticism.
• The overall assessment is that, despite the extremely difficult environment for decision making, there are points in the process where different decisions may have led to better outcomes and at least partially avoided the controversy that ensued in August 2020. The primary intention is to illustrate how the system can benefit from lessons learned in 2020 to avoid a similar predicament in 2021.

<--- The article continues for users subscribed and signed in. --->