Projects
Language Tests for migration and integration
October 2025 - ongoing, funded project
This study will conduct a comprehensive comparative analysis of language policies and language tests, with a particular focus on Austria and Belgium. It will examine the rationales, constructs and operationalizations of language tests for migrants, as well as the social impact of integration policies and language tests on migrants' lives. This research aims to promote fair language testing practices in migration contexts and support equitable integration processes.
LTRGI members involved: Eva Konrad, Viktoria Ebner, Benjamin Kremmel
Project partners: FWF, FWO, Bart Deygers (Ghent University), Laura Schildt (Ghent University)
Anxiety in the online IELTS Speaking Test
September 2025 - ongoing, funded project
This study examines candidate anxiety in the IELTS Speaking Exam, comparing the traditional face-to-face format with the online, remotely assessed speaking exam. It explores how anxiety manifests itself in different parts of the exam and investigates its relationship with speaking performance. The study aims to provide insights into performance-related anxiety and identify potential threats to the validity of the two assessment methods.
LTRGI members involved: Kathrin Eberharter, Doris Moser-Frötscher, Viktoria Ebner, Julia Kugl, Benjamin Kremmel
Project partner: IELTS partners
Reforming educational standards monitoring in Austria
February 2020 – ongoing, funded project
In this project, we work with the Federal Institute for Quality Assurance in the Austrian School System (IQS) on the development and the implementation of the iKMPLUS which is a newly introduced instrument in Austria designed to assess the attainment of educational standards for 4th and 8th grade students. Built on a modular framework, it is complemented by extensive resources and professional development opportunities for interpreting and applying results. The goal is to enhance the effectiveness of teaching, support educational planning, and facilitate more precise monitoring of both educational outcomes and the school system.
LTRGI members involved: Kathrin Eberharter, Eva Konrad, Simone Baumgartinger, Doris Moser-Frötscher, Elisa Guggenbichler, Benjamin Kremmel
Project partners: Federal Institute for Quality Assurance in the Austrian School System (IQS)
How does item presentation affect anxiety in French listening tests with self-pacing option? A biosensor study
2023 – ongoing, funded project
Digital test environments offer the opportunity for candidates to control the audio in foreign language listening tests. Allowing test takers to pause and rewind audio recordings during a listening test task might reduce anxiety levels; however, findings to date rely on self-reported data and do not investigate different item presentation formats. As part of the INNALP Education Hub (https://www.innalp.at/), this project probes the effect of this innovative accommodation idea in listening assessment. The study aims at analyzing test takers’ anxiety levels while completing standardized B1 listening tasks in French. 27 upper-secondary learners completed tasks with different item presentation formats (no item preview, preview of item stem and full item preview) while their anxiety levels were measured by using electrocardiography, electrodermal activity and facial expression data. In the analysis, self-reported and biosensor data is triangulated.
LTRGI members involved: Simone Baumgartinger, Elisa Guggenbichler, Sarah Egger, Daniel Ensmann, Benjamin Kremmel
Project partner: Österreichische Forschungsförderungsgesellschaft (FFG): Innovationslabore für Bildung
Investigating migrants’ digital literacy and language needs
July 2024 - February 2025
Considering the omnipresence of digital communication and use of digital devices, digitalizing national integration tests for adult migrants and refugees might be the way forward. As part of this project, a needs analysis was conducted in Austria to inform future test development. More than 400 participants who were enrolled in preparatory language and integration courses (A1 – B2 level) took part in an online survey. The results provided useful insights into the target group’s digital literacy and communication behaviors.
LTRGI members involved: Doris Moser-Frötscher, Eva Konrad, Viktoria Ebner, Sarah Egger, Simone Baumgartinger, Benjamin Kremmel
Project partner: Austrian Integration Funds (ÖIF)
Developing tasks for English, Italian and German (B1 – C1) for a university language center
October 2024 – January 2025, funded project
The LTRGI developed test tasks for English, Italian and German for the language exams of the Free University of Bozen-Bolzano. Listening, reading, writing and speaking tasks were developed for three different language levels (B1, B2, C1).
LTRGI members involved: Carol Spöttl, Simone Baumgartinger, Viktoria Ebner, Sarah Egger, Eva Konrad, Doris Moser-Frötscher, Benjamin Kremmel
Project partner: Free University of Bozen-Bolzano, Stefanie Hollenstein, Sara Ritrovati
Voices from test takers: What do memes tell us about test takers’ perception of a high-stakes exam?
2023 – 2025
Students’ reactions to high-stakes exams may provide valuable insights into their beliefs, emotions, and perspectives of a test. This study explores test takers’ experience of taking the Austrian Matura by investigating memes that test takers post on the social media platform Instagram. The primary research interest is to find out if and to what extent Matura memes can provide information on test takers’ perception of the Austrian Matura.
LTRGI members involved: Eva Konrad, Viktoria Ebner, Kathrin Eberharter, Benjamin Kremmel
Tracing emerging assessment literacy in pre-service language teachers: Evaluation of a university course on testing and assessment
2022 - 2023
Although assessing is an integral part of teaching, language teachers’ knowledge about testing is often experience-based, and practical courses on language testing are rare in teacher training curricula. This study evaluated the effectiveness of a university course on testing and assessment for pre-service language teachers. Pre-service teachers’ assessment literacy was traced in three sub-studies: study one provided insights into their item writing skills, study two showed how participants applied course content when revising reading items, and study three explored participants’ opinions about the course and their self-perceived language assessment literacy.
LTRGI members involved: Doris Moser-Frötscher, Eva Konrad, Elisa Guggenbichler, Viktoria Ebner, Kathrin Eberharter, Benjamin Kremmel
Monitoring and evaluation of an assessment training programme in Croatia
May 2021 – August 2022, funded project
This project evaluated a series of item writer workshops for the school-leaving exam in Croatia. The participants were teachers and item writers for English as an L2, Croatian (L1), and Mathematics and Physics. Alongside a 10-month training cycle, the development of participants’ assessment literacy and item writing skills was traced. Questionnaires and assessment tasks that were developed by the participants provided information on participants’ professional development.
LTRGI members involved: Doris Moser-Frötscher, Eva Konrad, Benjamin Kremmel
Project partner: British Council, Florian Stampfer (IFD), Agnes Tauscher (Osteuropazentrum UIBK)
The impact of self-pacing on the validity and accessibility of the TOEFL Junior test for candidates with different first language literacy profiles
March 2021 – December 2022, funded project
For young learners, authentic listening in a second language (L2) increasingly involves listening to online materials, which can be paused and rewound for comprehension purposes at the will of the listener. In addition to increasing authenticity, exercising control over the listening input might also support L2 learners with L1 reading-related difficulties. Based on this rationale, this study set out to investigate whether using self-paced listening might enhance the validity and accessibility of the TOEFL Junior listening test. The project focused on the two following aspects:
- The impact of self-pacing on the listening performance of candidates with different first language literacy profiles.
- The interplay between first language literacy skills, listening strategy use, and anxiety in self-paced versus single-play L2 listening tests.
LTRGI members involved: Kathrin Eberharter, Benjamin Kremmel, Eva Konrad, Viktoria Ebner, Elisa Guggenbichler, Simone Baumgartinger, Doris Moser-Frötscher
Project partners: Educational Testing Service, Judit Kormos (Lancaster University, UK), Shungo Suzuki (Waseda University, Japan)
Evaluating test tasks for German, Italian and English for a university language center
July 2019 – November 2020, funded project
The Free University of Bozen-Bolzano commissioned the LTRGI to evaluate a sample of German, Italian and English tasks of the language exam that students must take prior to entering the university. The aim was to evaluate the validity and reliability of the language exams and analyze the representation of the construct in the exam by means of quantitative and qualitative item analysis.
LTRGI members involved: Eva Konrad, Carol Spöttl, Veronika Schwarz, Benjamin Kremmel
Project partner: Free University of Bozen-Bolzano, Carmen Konzett-Firth (IfD)
Probing the cognitive validity of the Linguaskill Writing component: exploring test takers’ cognitive processes using eye-tracking and keystroke logging
March 2019 - December 2019, funded project
The research project focusses on the cognitive validity of the computer-based tasks of the Linguaskill Writing component in relation to test taker characteristics. In particular, it will investigate the cognitive processes engaged by a mixed-proficiency sample of L2 learners when responding to the Linguaskill tasks, and in how far they resemble a full model of the L2 writing process. In addition, it aims to uncover the links between computer writing behaviour (i.e. fluency, pausing and revision behaviours of candidates) and the outcome of the L2 writing process. The study will do so by triangulating data from eye-movement analysis, keystroke logging, and stimulated recalls.
LTRGI members involved: Benjamin Kremmel, Kathrin Eberharter, Matthias Zehentner, & Franz Holzknecht
Project partners: Cambridge Assessment English
Developing a work-place numeracy test
February 2019 – December 2019, funded project
The remit of this project was to develop a computer-based numeracy test for work-place purposes in Singapore. It involved the development and validation of a localized competency framework and over 400 numeracy test items across 8 different levels in collaboration with mathematics experts of the University of Innsbruck. The items were developed by subject specialist item writers trained at the University of Innsbruck, then trialled on a large sample of test takers in Singapore. Successful items were then standard set by a LTRGI-moderated panel of Singaporean experts in Singapore before being included in the final item bank of the test. The test launched across Singapore in January 2020.
LTRGI members involved: Carol Spöttl, Matthias Zehentner, Franz Holzknecht, Benjamin Kremmel
Project partners: British Council, the Singapore government agency Skillsfuture Singapore, Florian Stampfer (University of Innsbruck)
Text and task features of university reading materials in relation to Aptis and Lexile® scores
September 2017 - September 2018, funded project
The Language Testing Research Group Innsbruck has been awarded a Research into Reading Grant by the British Council and Metametrics to investigate the text characteristics of Aptis tests and the text and task features of first semester undergraduate university reading materials as well as the relationship between these test scores and indices.
LTRGI members involved: Benjamin Kremmel, Carol Spöttl, Matthias Zehentner, & Franz Holzknecht
Project partners: British Council Aptis & Metametrics
Assessing writing at lower levels: Research findings, task development locally and internationally, and the opportunities presented by the extended CEFR descriptors
June 2017 - June 2018, funded project
The Language Testing Research Group Innsbruck has been awarded an Assessment Research Grant by the British Council to investigate the practices, challenges, and opportunities of assessing writing at lower levels.
LTRGI members involved: Carol Spöttl, Eva Konrad, Franz Holzknecht, & Matthias Zehentner
Project partners: British Council Aptis
Looking into listening: Using eye-tracking to establish the cognitive validity of the Aptis Listening Test
September 2015 - September 2016, funded project
The Language Testing Research Group Innsbruck has been awarded an Assessment Research Grant by the British Council to investigate the cognitive validity of their APTIS listening test section.
LTRGI members involved: Carol Spöttl, Franz Holzknecht, Kathrin Eberharter, Benjamin Kremmel, Eva Konrad, & Matthias Zehentner
Project partners: British Council
Book chapters on teacher involvement in high stakes exams
2015-2016
The Language Testing Research Group Innsbruck is authoring four chapters for a book on teacher involvement in high stakes exams. The book is edited by Daniel Xerri and Patricia Vella-Briffa and published by Springer. The four chapters are:
- Fostering assessment literacy through teacher involvement in high stakes exams
- Potentials and challenges of teacher involvement in rating scale design for high stakes exams
- The role of classroom teachers in standard setting and benchmarking
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Franz Holzknecht, Eva Konrad, & Benjamin Kremmel
Project partners: Carmen Konzett (University of Innsbruck, for the chapter on rating scales)
Cross-validation of Austrian and Finnish rating scales
2013-2018
The aim of this project was to cross-validate two EFL rating scales for assessing writing that were designed in Austria and Finland and to investigate features of EFL learner language use at the different CEFR levels.
LTRGI members involved: Franz Holzknecht, Michael Maurer
Project partners: Ari Huhta (University of Jyväskylä, Finland), Iasonas Lamprianou (University of Cyprus, Greece)
Publication: https://www.sciencedirect.com/science/article/abs/pii/S1075293518300424
Assessing Speaking: Task design, rater training and interlocutor training
2012, funded project
In cooperation with two teacher training colleges (Tyrol and Vienna) two groups of English teachers were trained to design speaking tasks at the CEFR B-levels, rate with the new B1 and B2 CEFR-linked Austrian Assessment Scales for speaking, and act as a reliable interlocutor during speaking test simulations. The project’s main aim was to train teachers to disseminate their knowledge and skills in their schools and contribute to improving the quality of the non-standardized speaking part of the Austrian school leaving examination, the Matura.
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Michael Maurer & Matthias Zehentner
Project partners: Sabine Hosp (Pädagogische Hochschule Tirol), Silvia Jindra (Pädagogische Hochschule Wien)
B1 user language profiles: Italian
2012-2016
In cooperation with the Università per Stranieri di Siena a corpus of B1 learners of Italian is currently being compiled and entered. The aim of this project is to establish and compare language profiles of learners of Italian in an Austrian and Italian context. Extending this project to other contexts and/or languages is under consideration.
LTRGI members involved: Carol Spöttl, Michael Maurer
Project partners: Monica Barni (Università per Stranieri di Siena)
Foreign Language Learner Corpora: English, French, Spanish and Italian
2011-2016, funded project
The BIFIE Vienna commissioned the University of Innsbruck to compile four language learner corpora on the basis of trial and live performances that were collected during the development and implementation of the new Austrian standardized school-leaving examination, the Matura. The performances are based on CEFR-linked standardized writing tasks at B1 and B2 level. Each corpus is projected to reach 500,000 words by the end of the project in December 2015. The corpus will be used by the BIFIE to inform future test development decisions. The LTRGI team is planning to publish some of the insights from this project for teachers of foreign languages.
LTRGI members involved: Carol Spöttl, Michael Maurer, & Benjamin Kremmel
Project partner: BIFIE Vienna
The literature curriculum in standardized language assessment
2011 - 2018, (funded between 2011 and 2014)
As a response to the perceived marginalization of literature in foreign language teaching, this project aimed at bringing together specialists in literature, the teaching of literature, and language assessment to explore how literature and literary competences could be integrated meaningfully into standardized assessment tasks. The project involved the development of a model of literary competences that attempted to bring together theories of literary competence and the Council of Europe’s Common European Framework of Reference Companion Volume descriptors, as well as the design and piloting of writing tasks on literary topics with pre- and in-service teachers.
Project partners: Austrian Ministry of Education, Ulla Ratheiser (University of Innsbruck), Grit Alter (University of Innsbruck/PH Tirol), Andrew Milne-Skinner (University of Innsbruck), Birgit Mertz-Baumgartner (University of Innsbruck)
Publication:
https://academic.oup.com/eltj/article-abstract/73/4/377/5528098?redirectedFrom=PDF
Fostering Inter Rater Reliability
2011-2014, two funded projects
In collaboration with the BIFIE (Austrian Federal Institute for Educational Research, Innovation and Development) and the Pädagogische Hochschule Tirol, two cohorts of Austrian teachers of English were trained with the new B2 CEFR-linked Austrian Assessment Scale (CLAAS). The four-week training program focused on providing in-depth knowledge of the rating tool and the rating process in general, and introducing basic concepts of statistical methods and the principles of language testing. Many of the participants are now involved in the training of other teachers and supply expert opinions for school authorities during the examination periods.
LTRGI members involved: Carol Spöttl, Kathrin Eberharter, Franz Holzknecht, Benjamin Kremmel
International trainer: Rita Green
Project partners: Pädagogische Hochschule Tirol (Michael Schober, project coordinator), BIFIE
Standardisierte Reifeprüfung
2007 – 2008, funded project
This was a pilot project funded by the Austrian ministry for Education to investigate the potential of introducing a standardised Austrian national school-leaving exam for the foreign languages in academic upper-secondary schools (for details about the reform see Spöttl, Kremmel, Holzknecht, and Alderson (2016) in the publications section). In the course of this first project, 15 practicing teachers from all regions of Austria were trained in designing standardized listening and reading tasks for English at CEFR B2 level and for French at B1 level. The tasks went through several quality control procedures before entering two field trials. Successful tasks were selected from both trials to go forward to standard setting in December 2007, which was led by the project’s international consultant, Professor J.C. Alderson. The new exam in listening and reading was first administered in May 2008 in 56 participating pilot schools across Austria.
Project team University of Innsbruck: Carol Spöttl, Doris Frötscher, Carmen Konzett and Theresa Weiler
Project partners: Charles Alderson (international consultant) and Rita Green (item-writer trainer), Austrian ministry of Education