Do multiple choice tests have a role to play in academic legal education?

by Imogen Moore (University of Bristol, Law School) and Lee Price (University of West of England)

Multiple choice tests (MCTs) can get a bit of a bad rap, sometimes seen as little more than quizzes to test basic knowledge, with no real place in a respectable law programme. But the acceleration of changes to teaching and assessment in response to the pandemic should prompt further consideration of the role of MCTs in academic legal education. And such consideration is particularly timely with MCTs now a key element of assessment for professional legal qualification under the recently approved new Solicitors Qualifying Examination (SQE).

MCTs in learning and teaching

The move to online or blended learning has seen greater formative use of MCTs within law teaching, and not just an added ‘extra’ but as important structural poles in more scaffolded learning. Including MCTs in a curriculum has been seen to improve students’ learning performance and perceptions of the quality of their learning experience, as reflected in student surveys of blended learning experiences this year.

Formative MCTs can help to encourage active learning, academic engagement and stimulate students’ self-managed learning. Whether in a blended or traditional educational world, MCTs can help keep students engaged with learning outside the classroom (whether physical or online) by enabling them to test understanding in a zero-risk environment at multiple staging points. Even very basic MCTs, provided aligned to the course, allow students to check their knowledge base before proceeding. Well-designed MCTs have significant further value by enabling students to test higher level skills of understanding and application, encouraging re-engagement with difficult topics and self-identification of support needs. Some platforms permit the creation of more sophisticated formative work, for example by uploading materials or developing scenarios to test comprehension, interpretation and application. Although time-consuming to create, these more complex MCTs provide a rich learning experience.

The value of MCTs as a learning tool is significantly enhanced by the method and quality of feedback provided. Automatic feedback (a function of even simple platforms) advising whether an answer is wrong, right, or not the ‘best’ answer available and why, encourages students to reflect on their understanding, with immediacy encouraging engagement while issues are fresh – something that traditional assessment formats can rarely supply. As well as explaining and reassuring, targeted feedback can link to materials to encourage resolution of problem areas or further learning, creating further developmental opportunities.

Platforms also invariably offer overviews of responses to tutors, enabling identification of areas of particular confusion or difficulty for a cohort. This is information that might otherwise only emerge, if at all, ad hoc or through more subjective polling and enables more adaptive, focused teaching. Student engagement thus directly influences course activity and direction, while lack of engagement might highlight possible deeper issues within a course or at individual level.

More widespread and sophisticated use of MCTs is not without potential downsides. MCTs may be seen as ‘yet another’ burden, and demotivate if perceived as too onerous. This will be exacerbated if teaching and assessment methodologies are insufficiently aligned and communicated. It is therefore vital that MCTs are designed into the course, students informed of their purpose and function, and expectations managed accordingly.

MCTs as summative assessment

The changing educational and professional environment seems likely to lead to increased summative use of MCTs in academic law programmes. Many institutions, not just those offering ‘SQE-ready’ programmes, want to expose students to a broad range of assessments, including formats used in professional training. Institutions with growing law student numbers may be tempted by MCTs as a pragmatic response to increased marking burdens, although the time required to create appropriately clear, directed and challenging questions should not be overlooked. The simplicity of administering MCTs is also appealing, although strict assessment conditions remain critical to deter collusion.

Carefully designed MCTs can also offer more objective evaluation than traditional assessment, and may level up differing abilities in written English. But a particular concern, seen in discussions surrounding introduction of the SQE and more generally, is whether MCTs are capable of assessing the critical evaluation, reasoning, and provision of nuanced advice and argument to be expected of a law graduate or professional. MCTs clearly differ markedly from more traditional assessment of legal competencies and their limitations should not be overlooked. They do not tend to promote creativity, exploration, argument, or engagement with wider social issues, nor are they well suited to assessing communication skills beyond basic levels. But their limitations should also not be overplayed. Provided they are well designed (a critical proviso) MCTs can certainly move beyond testing ‘mere’ knowledge to assess understanding, problem-solving, reasoning and evaluative skills.

Some summative use of MCTs within academic law programmes is therefore justifiable and beneficial, if designed and reviewed with appropriate care and expertise. But their limitations in promoting and testing some skills which remain critical to graduate outcomes create a natural constraint on their use within a programme. The need to align assessment as a whole to course and programme outcomes remains.

Conclusion

The forced re-evaluation of how and what we teach, combined with changes to professional legal training, has encouraged fresh engagement with MCTs as a tool of learning and assessment in law. We should resist losing their educational benefits even if academic legal education otherwise returns to ‘normal’.  But to be effective MCTs must be created with care, appropriately aligned to course and level, and require regular monitoring and reflection. This requires significant investment of staff time that needs to be fully accounted for, and that is often in short supply. With these caveats MCTs can and should play a supporting role in academic legal education and offer valuable opportunities both in formative and summative use.

Leave a Reply

Your email address will not be published. Required fields are marked *