Project Investigator(s): Mark Lam, Lecturer, Department of Psychology; Katherine Lyon, Instructor, Department of Sociology; Brett Todd, Lecturer, Vantage College Academic English Program; Jennifer Lightfoot, Instructor, Vantage College Academic English Program
Project Description
Multiple-choice questions (MCQs) are a commonly used assessment tool in post-secondary institutions and are often lexically dense and grammatically complex (containing highly packed nominal groups), reflecting common university discourse. The purpose of the proposed project is to examine whether such linguistic features found in MCQs (e.g. complexity, structure, vocabulary, etc.) limit comprehension of the question for students who have English as an additional language (EAL) and whether the clause structure of such questions can be revised and clarified, thus improving the understanding of test questions for EAL learners.
Research Questions
1. Does the complexity, structure, and lexis of multiple-choice questions limits understanding for EAL students?
2. Can such questions can be revised to increase comprehension, and thus exam performance?
Impact on teaching and learning at UBC
Multiple-choice questions are a commonly used assessment tool at post-secondary institutions, where large class sizes are often the norm. Beyond the scope of the Vantage One program, this research could extend to investigate whether the linguistic features of multiple-choice questions increase question difficulty for direct entry international students. This is particularly relevant, given the continually increasing number of international students at UBC and post-secondary institutions more broadly. Findings from this research could also inform the development of educational materials (e.g. workshops, online resources, lessons, etc.) designed to help EAL students better understand the linguistic features commonly found in multiple-choice questions. A likely next step would be to extend this research to direct entry sections of first year Psychology and Sociology taught by Drs. Lyon and Lam before expanding to similar courses offered by additional instructors.
Project Outcomes
“The idea for our project originally came from the classroom itself, noticing the challenges our students were facing and wondering how we can best help as instructors. One particular challenge noted by instructors in our Vantage One program was how our international students who use English as an additional language (EAL) seemed to require additional time to complete MCQ exams, ask more clarifying questions, and score lower on such exams compared to their direct entry peers. As our program is collaborative in nature, with disciplinary instructors working closely with Academic English instructors to support international students throughout their first-year of university studies, these instructors wondered whether unpacking the language of the test questions would have an impact on student performance. This then sparked the idea for a SoTL Seeds supported project, directly connecting research to practice where there is often a gap. This SoTL Seeds support was key to our success in completing this research project. With positive results, showing that unpacking linguistic complexity in test questions helps students better represent their content knowledge, we have presented our work at conferences both within our Canadian community and abroad, and have recently published an article in the Journal of English for Academic Purposes. One key outcome of this collaboration has been greater awareness of the comprehension challenges that assessments pose for students, particularly EAL students, highlighting the importance of providing linguistic space for novice students to demonstrate their knowledge of disciplinary content with instruction and assessments which meet their linguistic needs.”