Euroexam presents novel scoring method at EALTA 14th Annual Conference

The top event in the European language testing experts’ calendar EALTA’s 14th Annual Conference was held this year in Sèvres, France, and focused on issues around mobility and social integration. Part of an exciting and busy programme, Euroexam International’s leading research expert, Zoltán Lukácsi, gave a very well-received presentation about a novel test scoring method to improve the consistency and reliability of assessment.

The event, which is Europe’s most prestigious in the field, was divided into three parts and started with a three-day workshop series that offered participants some hands-on practical activities in three distinct areas of language assessment: (i) using statistical analysis to improve testing (led by John de Jong, Language Testing Services, UK and Ying Zheng, University of Southampton, UK), (ii) standards setting for language tests (led by Steve Ferrara, Measured Progress, USA) and (iii) integrated testing and assessment (led by Sathena Chan and Chihiro Inoue, CRELLA, University of Bedfordshire).

Next, in a busy schedule, the Special Interest Groups held their events and a series of papers were presented in each session: Assessing Speaking, the CEFR SIG and a joint meeting for the Assessing Writing and Academic Purposes groups.

The theme of the conference proper was the role of language testing and assessment in mobility and social integration with some excellent and thought-provoking talks about, among other things, how different European countries address the challenges of preparing migrants and refugees for citizenship tests or commencing their studies in higher education in a foreign language. 

New methodology to improve consistency of scoring

Noteworthy among the general research paper presentations was a talk by Euroexam’s research and validation analyst Zoltán Lukácsi, PhD, who presented the outcome of a two-year research project that aimed to increase the transparency and accountability of the exam board’s high-stakes tests.

While in general testing practice Writing and Speaking rating scales were intended to increase reliability by replacing the formerly applied impressionistic scoring, recent research has shown the weaknesses of these scales and motivated the Euroexam team to explore alternative methods. Focusing on the general level B2 Writing paper, a list of characteristic statements was developed and several rounds of trial and statistical analysis have demonstrated that the list constitutes a novel and viable way to enhance consistency between raters and the validity of scoring.

An exceptionally large audience followed the presentation, followed by a Q&A session with some interesting ideas for further development from Norman Verhelst (Senior Researcher, Eurometrics, the Netherlands) and also from Martina Hulesova (Charles University, Prague), who expressed great interest in studying and possibly adopting the method. As part of a subsequent stage of this general project, the Euroexam team will be looking at how to improve the scoring of Speaking tests.

On the whole, the conference was a wonderful opportunity to exchange ideas with leading testing experts and our heartfelt thanks go to the organisers at the Centre International D’Études Pédagogiques.