Projects
Benchmarked Performances
AHS
BHS
Probing the cognitive validity of the Linguaskill Writing component: exploring test takers’ cognitive processes using eye-tracking and keystroke logging
March 2019 - December 2019, funded project
The research project focusses on the cognitive validity of the computer-based tasks of the Linguaskill Writing component in relation to test taker characteristics. In particular, it will investigate the cognitive processes engaged by a mixed-proficiency sample of L2 learners when responding to the Linguaskill tasks, and in how far they resemble a full model of the L2 writing process. In addition, it aims to uncover the links between computer writing behaviour (i.e. fluency, pausing and revision behaviours of candidates) and the outcome of the L2 writing process. The study will do so by triangulating data from eye-movement analysis, keystroke logging, and stimulated recalls.
LTRGI members involved: Benjamin Kremmel, Kathrin Eberharter, Matthias Zehentner, & Franz Holzknecht
Project partners: Cambridge Assessment English
Text and task features of university reading materials in relation to Aptis and Lexile® scores
September 2017 - September 2018, funded project
The Language Testing Research Group Innsbruck has been awarded a Research into Reading Grant by the British Council and Metametrics to investigate the text characteristics of Aptis tests and the text and task features of first semester undergraduate university reading materials as well as the relationship between these test scores and indices.
LTRGI members involved: Benjamin Kremmel, Carol Spöttl, Matthias Zehentner, & Franz Holzknecht
Project partners: British Council Aptis & Metametrics
Assessing writing at lower levels: Research findings, task development locally and internationally, and the opportunities presented by the extended CEFR descriptors
June 2017 - June 2018, funded project
The Language Testing Research Group Innsbruck has been awarded an Assessment Research Grant by the British Council to investigate the practices, challenges, and opportunities of assessing writing at lower levels.
LTRGI members involved: Carol Spöttl, Eva Konrad, Franz Holzknecht, & Matthias Zehentner
Project partners: British Council Aptis
Looking into listening: Using eye-tracking to establish the cognitive validity of the Aptis Listening Test
September 2015 - September 2016, funded project
The Language Testing Research Group Innsbruck has been awarded an Assessment Research Grant by the British Council to investigate the cognitive validity of their APTIS listening test section.
LTRGI members involved: Carol Spöttl, Franz Holzknecht, Kathrin Eberharter, Benjamin Kremmel, Eva Konrad, & Matthias Zehentner
Project partners: British Council
Book chapters on teacher involvement in high stakes exams
2015-2016
The Language Testing Research Group Innsbruck is authoring four chapters for a book on teacher involvement in high stakes exams. The book is edited by Daniel Xerri and Patricia Vella-Briffa and published by Springer. The four chapters are:
- Fostering assessment literacy through teacher involvement in high stakes exams
- Potentials and challenges of teacher involvement in rating scale design for high stakes exams
- The role of classroom teachers in standard setting and benchmarking
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Franz Holzknecht, Eva Konrad, & Benjamin Kremmel
Project partners: Carmen Konzett (University of Innsbruck, for the chapter on rating sales)
Cross-validation of Austrian and Finnish rating scales
since 2013
The aim of this project is to cross-validate two EFL rating scales for assessing writing that were designed in Austria and Finland and to investigate features of EFL learner language use at the different CEFR levels.
LTRGI members involved: Franz Holzknecht, Michael Maurer
Project partners: Ari Huhta (University of Jyväskylä, Finland), Iasonas Lamprianou (University of Cyprus, Greece)
Assessing Speaking: Task design, rater training and interlocutor training
since 2012, funded project
In cooperation with three teacher training colleges (Tirol and Wien) two groups of English teachers were trained to design speaking tasks at the CEFR B-level, rate with the new B2 CEFR-linked Austrian Assessment Scale (CLAAS), and act as a reliable interlocutor during speaking test simulations. The project’s main aim is to train teachers to disseminate their knowledge and skills in their schools and contribute to improving the quality of the non-standardized speaking part of the Austrian school leaving examination, the Matura.
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Michael Maurer & Matthias Zehentner
Regional coordinators: Sabine Hosp (Pädagogische Hochschule Tirol), Silvia Jindra (Pädagogische Hochschule Wien)
B1 user language profiles: Italian
2012-2016
In cooperation with the Università per Stranieri di Siena a corpus of B1 learners of Italian is currently being compiled and entered. The aim of this project is to establish and compare language profiles of learners of Italian in an Austrian and Italian context. Extending this project to other contexts and/or languages is under consideration.
LTRGI members involved: Carol Spöttl, Michael Maurer
Project partners: Monica Barni (Università per Stranieri di Siena)
Foreign Language Learner Corpora: English, French, Spanish and Italian
2011-2016, funded project
The BIFIE Vienna commissioned the University of Innsbruck to compile four language learner corpora on the basis of trial and live performances that were collected during the development and implementation of the new Austrian standardized school-leaving examination, the Matura. The performances are based on CEFR-linked standardized writing tasks at B1 and B2 level. Each corpus is projected to reach 500,000 words by the end of the project in December 2015. The corpus will be used by the BIFIE to inform future test development decisions. The LTRGI team is planning to publish some of the insights from this project for teachers of foreign languages.
LTRGI members involved: Carol Spöttl, Michael Maurer, & Benjamin Kremmel
Project partner: BIFIE Vienna
The literature curriculum in standardized language assessment
since 2011, funded project
More information will follow soon.
Fostering Inter Rater Reliability: Project 1 and 2
2011-2014, funded project
With the cooperation of the BIFIE and the Pädagogische Hochschule Tirol, two teams of Austrian teachers of English were trained with the new B2 CEFR-linked Austrian Assessment Scale (CLAAS). The four week training program focused on providing in-depth knowledge of the rating tool and the rating process in general, and introducing basic concepts of statistical methods and the principles of language testing. Many of the participants are now involved in the training of other teachers and supply expert opinions for school authorities during the examination periods.
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Franz Holzknecht, & Benjamin Kremmel
International trainer: Rita Green
Project coordinator: Michael Schober (Pädagogische Hochschule Tirol)
Standardisierte Reifeprüfung – Project IV
funded project
More information will follow soon.
Standardisierte Reifeprüfung – Project III
funded project
More information will follow soon.
Standardisierte Reifeprüfung – Project II
funded project
More information will follow soon.
Standardisierte Reifeprüfung – Project I
funded project
More information will follow soon.