Publications
forthcoming/in preparation
Chen, Y., Gao, J., Hui, B., Kremmel, B., Ma, S., Maie, R., Puimège, E., & Rogers, J. (proposal accepted). First language effects on incidental vocabulary learning through bimodal input: A multisite, preregistered, and close replication of Malone (2018). A special issue on replication in Studies in Second Language Acquisition
Eberharter, K. (forthcoming). Formative Evaluation. In Vogt, K., & Rossa, H. (Eds.): Handbuch Leistungsmessung, Evaluation und Feedback im Fremdsprachenunterricht. Klett/Kallmeyer.
Eberharter, K., Guggenbichler, E., Zehentner, M., Holzknecht, F., Schwarz, V., & Kremmel, B. (in prep). Exploring the cognitive validity of a computerized writing test using eye-tracking, keystroke logging and stimulated recalls.
Isbell, D.R., Kim, J., & Kremmel, B. (in prep). Remote proctoring in language assessment: implications for validity, fairness and justice.
Kremmel, B. (forthcoming). Language Assessment Literacy. In Vogt, K., & Rossa, H. (Eds.): Handbuch Leistungsmessung, Evaluation und Feedback im Fremdsprachenunterricht. Klett/Kallmeyer.
Kremmel, B. (in prep). Words and counting units. In The Encyclopedia of Applied Linguistics, C.A. Chapelle (Ed.).
Kremmel, B. (forthcoming). Wortschatz. In Vogt, K., & Rossa, H. (Eds.): Handbuch Leistungsmessung, Evaluation und Feedback im Fremdsprachenunterricht. Klett/Kallmeyer.
Kremmel, B., Indrarathne, B., Kormos, J., & Suzuki, S. (IPA). Unknown vocabulary density and reading comprehension: Replicating Hu & Nation 2000. Language Learning. https://osf.io/fmq7e
Moser-Frötscher, D. (forthcoming). Summative Evaluation. In Vogt, K., & Rossa, H. (Eds.): Handbuch Leistungsmessung, Evaluation und Feedback im Fremdsprachenunterricht. Klett/Kallmeyer.
Schmitt, N., Dunn, K., O'Sullivan, B., Anthony, L., & Kremmel, B. (in press). Knowledge-based Vocabulary Lists. Equinox.
2023
Eberharter, K., Kormos, J., Guggenbichler, E., Ebner, V. S., Suzuki, S., Moser-Frötscher, D., Konrad, E., & Kremmel, B. (2023). Investigating the impact of self-pacing on the L2 listening performance of young learner candidates with differing L1 literacy skills. Language Testing, 0(0). https://doi.org/10.1177/02655322221149642
Guggenbichler, E., Eberharter, K., & Kremmel, B. (2023). Investigating cognitive processes during writing tests: Methodological considerations when triangulating data from eye-tracking, keystroke logging, and stimulated recalls. In Manchon, R. & Roca de Larios, J. (Eds.): Research methods in the study of writing processes (pp. 247-268). John Benjamins.
Kremmel, B., Eberharter, K., Konrad, E., Guggenbichler, E., Moser-Frötscher, D., Ebner, V., & Spöttl, C. (2023). The CEFR Companion Volume: Opportunities and challenges for language assessment. In Kaltseis, M., Fuchbauer, J., Hirzinger-Unterrainer, E., Zink, A., & Bergmann, A. (Eds.): An den Schnittstellen von Didaktik und Philologie: Gesammelte Beiträge zu Ehren von Wolfgang Stadler (pp. 65-84). Innsbruck: DiSlaw Special Issue I 2023. https://www.dislaw.at/index.php/ds/article/view/56/54
2022
Boers-Visker, E., Eberharter, K., Hammer, A., Harding, L., & Kremmel, B. (2022). Discussion on Language Assessment Literacy. In Haug, T., Mann, W., & Knoch, U. (Eds.): Handbook of Language Assessment across Modalities (p. 395-399). OUP.
Durrant, P., Siyanova-Chanturia, A., Sonbul, S., & Kremmel, B. (2022). Research methods for vocabulary studies. Amsterdam: John Benjamins.
Harding, L., Kremmel, B., & Eberharter, K. (2022). Language Assessment Literacy in second language assessment contexts. In Knoch, U. & Haug, T. (Eds.): Handbook of Language Assessment across Modalities (pp. 373-382). OUP.
Holzknecht, F., Gugenbichler, E., Zehentner, M., Yoder, M., Konrad, E., & Kremmel, B. (2022). Assessing reading ability and reading demands for EMI degree programs: Lecturers as mediators between test results, readability indices, and local challenges. Journal of English as a Medium of Instruction
Kremmel, B., Eberharter, K., & Holzknecht, F. (2022). Pre-operational testing. In Fulcher, G., & Harding, L. (Eds.): The Routledge Handbook of Language Testing (2nd edition) (pp. 415-429). New York: Routledge.
Spöttl, C., Konrad, E., Schwarz, V., Guggenbichler, E., & Kremmel, B. (2022). Authenticity in Language Assessment: Reconceptualizing the Implementation of a Key Principle. In: Will, L., Stadler, W., & Eloff, I. (Eds.), Authenticity across Languages and Cultures. Themes of Identity in Foreign Language Teaching and Learning.
2021
Harding, L., & Kremmel, B. (2021). SLA researcher assessment literacy. In: Winke, P., & Brunfaut, T. (Eds.), The Routledge Handbook of Second Language Acquisition and Language Testing (pp. 54-65). New York: Routledge.
Kremmel, B. (2021). Selling the (word) family silver? A response to Webb’s 'lemma dilemma'. Studies in Second Language Acquisition, 43(5), 962-964. doi:10.1017/S0272263121000693
Kremmel, B. (2021). Does one size fit all? Comparing two computer-adaptive algorithms for a diagnostic vocabulary test. In Barclay, S., & Szudarski, P. (Eds.): Vocabulary theory, patterning and teaching (pp. 48-64). Bristol: Multilingual Matters.
Kremmel, B., & Pellicer-Sanchez, A. (2021). Measuring vocabulary development. In: Winke, P., & Brunfaut, T. (Eds.), The Routledge Handbook of Second Language Acquisition and Language Testing (pp. 211-222). New York: Routledge.
Leitner, K., & Kremmel, B. (2021). Avoiding scoring malpractice: supporting reliable scoring of constructed-response items in high-stakes exams. In: Lanteigne, B., Coombe, C., & Brown, J.D. (Eds.), Challenges in Language Testing Around the World (pp. 127-145). Springer.
Paran, A., Spoettl, C., Ratheiser, U., & Eberharter, K. (2021). Measuring literary competences in SLA. In: Winke, P., & Brunfaut, T. (Eds.), The Routledge Handbook of Second Language Acquisition and Language Testing (pp. 326-337). New York: Routledge.
Schmitt, N., Dunn, K., O'Sullivan, B., Anthony, L., & Kremmel, B. (2021). Introducing Knowledge-based Vocabulary Lists. TESOL Journal. Online First. https://doi.org/10.1002/tesj.622
2020
Holzknecht, F., McCray, G., Eberharter, K., Kremmel, B., Zehentner, M., Spiby, R., & Dunlea, J. (2020). The effect of response order on candidate viewing behaviour and item difficulty in a multiple-choice listening test. Language Testing. https://doi.org/10.1177/0265532220917316
Isbell, D. R., & Kremmel, B. (2020). Test review: Current Options in At-Home Language Proficiency Tests for Making High-Stakes Decisions. Language Testing. https://doi.org/10.1177/0265532220943483
Kremmel, B., Harding, L. & Guggenbichler, E. (2020). A survey to detect what different groups need to know about language assessment. OASIS Summary of Kremmel, B. & Harding, L. (2020) in Language Assessment Quarterly. https://oasis-database.org/concern/summaries/dj52w484n?locale=en
Schwarz, V. & Holzknecht, F. (2020). Comparing ratings of writing performances with different rating scales across European countries. OASIS Summary of Holzknecht, F., Huhta, A. & Lamprianou, I. (2018) in Assessing Writing. https://oasis-database.org/concern/summaries/p8418n34x?locale=en
Schwarz, V. & Holzknecht, F. (2020). Testing listening with multiple choice tasks: Is the order of the answer options important?. OASIS Summary of Holzknecht, F. et al. (2020) in Language Testing. https://oasis-database.org/concern/summaries/nc580m86b?locale=en
2019
Chapelle, C., Kremmel, B., & Brindley, G. (2019). Assessment. In: Schmitt, N., & Rodgers, M. (Eds.), An Introduction to Applied Linguistics. New York: Routledge.
Ehling, R., Amprosi, M., Kremmel, B., Bsteh, G., Eberharter, K., Zehentner, M., Steiger, R., Tuovinen, N., Gizewski, E.R., Benke, T., Berger, T., Spoettl, C., Brenneis, C., & Scherfler, C. (2019). Second language learning induces grey matter volume increase in people with multiple sclerosis. PLoS ONE 14(12): e0226525. https://doi.org/10.1371/journal.pone.0226525
Holzknecht, F. (2019). Book review: Assessing L2 listening - Moving towards authenticity. Language Testing. doi: 10.1177/0265532219892879
Kremmel, B. (2019). Measuring vocabulary learning progress. In Webb, S. (Ed.): The Routledge Handbook of Vocabulary Studies (pp. 406-418). New York: Routledge.
Kremmel, B., & Harding, L. (2019). Towards a Comprehensive, Empirical Model of Language Assessment Literacy across Stakeholder Groups: Developing the Language Assessment Literacy Survey. Language Assessment Quarterly, 1-21. doi: 10.1080/15434303.2019.1674855
Schmitt, N., Nation, P., & Kremmel, B. (2019). Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 1-12. doi:10.1017/S0261444819000326
2018
Eberharter, K. (2018). A qualitative approach to validating multiple-choice items assessing language in context. In Sigott, G. (Ed.): Language testing in Austria: taking stock (pp. 585-606). Berlin: Peter Lang.
Eberharter, K., Holzknecht, F., & Kremmel, B. (2018). Evaluating the effectiveness of a training program for double-raters. In Sigott, G. (Ed.): Language testing in Austria: taking stock (pp. 607-628). Berlin: Peter Lang.
Eberharter, K., Kremmel, B., & Konzett-Firth, C. (2018). Produktive Fertigkeiten überprüfen und bewerten. In Hinger, B. & Stadler, W. (Eds.), Testen und Bewerten fremdsprachlicher Kompetenzen: Eine Einführung (pp. 87-116). Tübingen: Narr Francke Attempto.
Eberharter, K., Kremmel, B., & Zehentner, M. (2018). Die Erstellung von Testaufgaben: Der Testentwicklungszyklus. In Hinger, B. & Stadler, W. (Eds.), Testen und Bewerten fremdsprachlicher Kompetenzen: Eine Einführung (pp. 57-68). Tübingen: Narr Francke Attempto.
Holzknecht, F. (2018). Reading strategies in the standardized Austrian matriculation examination (Matura) for English. In Sigott, G. (Ed.): Language testing in Austria: taking stock (pp. 563-584). Peter Lang.
Holzknecht, F., & Harding, L. (2018). IELTS listening sub-test. In Liontas, J. I., DelliCarpini, M., & Riopel, J. C. (Eds.), The TESOL Encyclopedia of English Language Teaching, online edition. Wiley.
Holzknecht, F., Huhta, A., & Lamprianou, I. (2018). Comparing the outcomes of two different approaches to CEFR-based rating of students' writing performances across two European countries. Assessing Writing, 37, 57-67.
Holzknecht, F., Kremmel, B., Konzett, C., Eberharter, K., Konrad, E., & Spöttl, C. (2018). Potentials and challenges of teacher involvement in rating scale design for high-stakes exams. In Xerri, D., & Vella Briffa, P. (Eds.), Teacher involvement in high stakes language testing (pp. 47-66). Springer.
Konrad, E., Holzknecht, F., Schwarz, V., & Spöttl, C. (2018). Assessing writing at lower levels: Research findings, task development locally and internationally, and the opportunities presented by the extended CEFR descriptors. ARAGs Research Reports Online. London: The British Council.
Konrad, E., Spöttl, C., Holzknecht, F., & Kremmel, B. (2018). The role of classroom teachers in standard setting and benchmarking. In Xerri, D., & Vella Briffa, P. (Eds.), Teacher involvement in high stakes language testing (pp. 11-29). Springer.
Kremmel, B. (2018). In the classroom - Vocabulary: principles and practice 9. ETprofessional, Issue 117, 43-45.
Kremmel, B., Eberharter, K., Holzknecht, F., & Konrad, E. (2018). Fostering language assessment literacy through teacher involvement in high-stakes test development. In Xerri, D., & Vella Briffa, P. (Eds.), Teacher involvement in high stakes language testing (pp. 173-194). Springer.
Kremmel, B., Eberharter, K., & Maurer, M. (2018). Righting Writing Practices? An exam reform’s impact on L2 writing teaching and assessment. In Ruecker, T., & Crusan, D. (Eds.), The Politics of English Second Language Writing Assessment in Global Contexts (pp. 122-137). New York: Routledge.
Stadler, W., & Kremmel, B. (2018). Testprinzipien. In Hinger, B. & Stadler, W. (Eds.), Testen und Bewerten fremdsprachlicher Kompetenzen: Eine Einführung (pp. 39-56). Tübingen: Narr Francke Attempto.
Spöttl, C., Eberharter, K., Holzknecht, F., Kremmel, B., & Zehentner, M. (2018). Delivering reform in a high stakes context: from content-based assessment to communicative and competence-based assessment. In Sigott, G. (Ed.): Language testing in Austria: taking stock (pp. 219-240). Berlin: Peter Lang.
Zehentner, M. (2018). Examining the effect of partner choice in peer-to-peer assessment. In Sigott, G. (Ed.): Language testing in Austria: taking stock (pp. 629-650). Berlin: Peter Lang.
2017
Eberharter, K., Zehentner, M., & Spöttl, C. (2017). Die Entwicklung illustrativer mündlicher und schriftlicher Performanzen für die neue Standardisierte Reifeprüfung in Englisch. In: Dalton-Puffer, C., Boeckmann, K.-B., & Hinger, B. (Eds.), Sprachlehr-lernforschung in Österreich. Symposium anlässlich des 10-jährigen Bestehens der ÖGSD, Wien 19.Mai 2017. ÖGSD Tagungsberichte 2.
Holzknecht, F., Eberharter, K., Kremmel, B., McCray, G., Zehentner, M., Konrad, E., & Spöttl, C. (2017). Kognitive Prozesse während Hörverständnisaufgaben: Eine Eye-Tracking und Stimulated-Recall Untersuchung. In: Dalton-Puffer, C., Boeckmann, K.-B., & Hinger, B. (Eds.), Sprachlehr-lernforschung in Österreich. Symposium anlässlich des 10-jährigen Bestehens der ÖGSD, Wien 19.Mai 2017. ÖGSD Tagungsberichte 2.
Holzknecht, F., Eberharter, K., Kremmel, B., Zehentner, M., McCray, G., Konrad, E., & Spöttl, C. (2017). Looking into listening: Using eye-tracking to establish the cognitive validity of the Aptis Listening Test. ARAGs Research Reports Online. London: The British Council.
Kremmel, B. (2017). Entwicklung und Validierung eines computer-adaptiven Wortschatztests. In: Dalton-Puffer, C., Boeckmann, K.-B., & Hinger, B. (Eds.), Sprachlehr-lernforschung in Österreich. Symposium anlässlich des 10-jährigen Bestehens der ÖGSD, Wien 19.Mai 2017. ÖGSD Tagungsberichte 2.
Kremmel, B., & Schmitt, N. 2017. Vocabulary Levels Test. In Liontas, J. I., DelliCarpini, M., & Riopel, J. C. (Eds.), The TESOL Encyclopedia of English Language Teaching, online edition. Wiley.
2016
Harding, L., & Kremmel, B. (2016). Teacher assessment literacy and professional development. In Tsagari, D., & Banerjee, J. (eds.), Handbook of Second Language Assessment (pp. 413-428), Berlin: De Gruyter.
Kremmel, B. (2016). Word families and frequency bands in vocabulary tests: Challenging conventions. TESOL Quarterly 50(4), 976-987. http://onlinelibrary.wiley.com/doi/10.1002/tesq.329/full
Kremmel, B., & Schmitt, N. (2016). Interpreting vocabulary test scores: What do various item formats tell us about learners’ ability to employ words? Language Assessment Quarterly 13(4), 377-392. http://dx.doi.org/10.1080/15434303.2016.1237516
Kremmel, B., Frick, A., Lutz, S., & Parhammer, S. (2016). Erforschen testen – Tests erforschen. In Hinger, B. (Ed.), Zweite Tagung der Fachdidaktik 2015 "Sprachsensibler Sach-Fach-Unterricht – Sprachen im Sprachunterricht", Innsbrucker Beiträge zur Fachdidaktik 2 (pp. 233-255), Innsbruck, iup.
Spöttl, C., Kremmel, B., Holzknecht, F., & Alderson, J.C. (2016). Evaluating the achievements and challenges in reforming a national language exam: The reform team’s perspective. Papers in Language Testing and Assessment, 5(1), 1-22.
2015
Kremmel, B., Brunfaut, T., & Alderson, J.C. (2015). Exploring the Role of Phraseological Knowledge in Foreign Language Reading. Applied Linguistics, 1-24.
2013
Alderson, J.C., & Kremmel, B. (2013). Re-examining the content validation of a grammar test: The (im)possibility of distinguishing vocabulary and structural knowledge. Language Testing, 30(4), 535-556.
Eberharter, K., Konrad, E., Kremmel, B., Maurer, M., & Spöttl, C. (2013). Matura in the making - Die Entstehung kompetenzorientierter Testaufgaben in den lebenden Fremdsprachen. ILS-Mail - Institutszeitschrift zu aktuellen Themen der LehrerInnenbildung und Schulforschung 1/13, 12-13.
Kremmel, B., & Holzknecht, F. (2013) Strengths and weaknesses of the CEFR in guiding test task design: What the can dos can do and what they can’t (yet)”, In J. Colpaert, Simons, M., Aerts, A., & Oberhofer, M. (Eds.): Conference Proceedings, Language testing in Europe: Time for a new framework? University of Antwerp, Belgium, May 27-29 2012 (pp. 153-156). Antwerp: University of Antwerp.
Spöttl, C., & Eberharter, K. (2013). CEFR performance descriptors and the missing formulae. In J. Colpaert, Simons, M., Aerts, A., & Oberhofer, M. (Eds): Conference Proceedings, Language testing in Europe: Time for a new framework? University of Antwerp, Belgium, May 27-29 2012 (pp. 203-208). Antwerp: University of Antwerp.
2012
Eberharter, K., & Frötscher, D. (2012). Quality control in marking open-ended listening and reading test items: Issues and practice. In Tsagari, D., Papadima-Sophocleous, S. & Ioannou-Georgiou, S. (eds.) International experiences in language testing and assessment. Frankfurt am Main: Peter Lang.
Unpublished MA Theses
Eberharter, K. (2008). Exam reform and its impact on teaching. University of Innsbruck.
Eberharter, K. (2011). A qualitative approach to validating multiple-choice items assessing language in context. Lancaster University.
Holzknecht, F. (2009). A needs-analysis for Austrian English teachers in developing and assessing writing tests for the Matura. University of Innsbruck.
Holzknecht, F. (2012). Establishing one part of a validity argument: Reading strategies tested in the standardized Austrian Matura exam. Lancaster University.
Konrad, E. (2012). Effects of the use of dictionaries. Lancaster University.
Kremmel, B. (2010). Item difficulty in reading tests at CEFR level B2. University of Innsbruck.
Kremmel, B. (2012). Explaining variance in reading test performance through linguistic knowledge: The relative significance of vocabulary, syntactic and phraseological knowledge in predicting second language reading comprehension. Lancaster University.
Maurer, M. (2015). How do test takers write a blog? A corpus-based study of linguistic features in performances based on the task type blog. Lancaster University.
Zehentner, M. (2015). The Partner Effect – Investigating the impact of partner choice on paired speaking task performance ratings. University of Innsbruck.
Zehentner, M. (2017). No c(l)ue what to do – Examining the effects of situational cues in writing task prompts on students’ writing behaviour and their attitude towards the task. Lancaster University.