LEAP/XR – Language-driven Embodied AI for Perceptual Lighting Design in eXtended Reality
Project leader: Bartenbach GmbH
Project leader EEB: Sascha Hammes
Project partners:
Bartenbach GmbH
NEED immersive reality GmbH
University of Innsbruck, Department of Computer Science
University of Innsbruck, Unit of Energy Efficient Building
Funding agency: Federal Ministry for Innovation, Mobility, and Infrastructure represented by the Austrian Research Promotion Agency (FFG)
Funding program: Digital Technologies 2025
Funding amount: 499.884,00€ (UIBK-EEB: 70.625,00€)
Duration: 01.01.2027 to 31.12.2028
Project website: https://projekte.ffg.at/projekt/99ee443e-f36b-1410-865a-00d3c07d99bb
Summary
The planning of artificial lighting requires the translation of qualitative, often language-based design intentions into quantitative technical parameters. Terms such as atmospheric, safe, or restrained play a central role in early planning phases but can only be formalised to a limited extent. At the same time, immersive technologies such as virtual reality (VR) are increasingly used as tools for the spatial evaluation of lighting effects. However, existing approaches reach their limits, as linguistic meaning, spatial structure, perception, and normative requirements are still insufficiently integrated.
Against this background, the LEAP/XR project investigates whether and under which conditions language-based lighting intentions can be operationalised in immersive environments using artificial intelligence. The aim is to systematically explore the feasibility of AI-supported, language-driven interaction in XR environments. At the core of the project is an embodied AI approach, in which AI acts as a spatially situated agent within the virtual environment, interpreting linguistic input in relation to spatial structure, observer position, and perceptually relevant parameters. The innovative contribution lies in the integrative consideration of language, space, perception, and normativity, as well as in the investigation of their translation limits within an exploratory, interdisciplinary framework.
As an outcome, LEAP/XR provides a structured assessment of the technical, methodological, and conceptual prerequisites for language-driven lighting planning in XR. The project is expected to generate robust insights into which classes of linguistic descriptions can be mapped consistently to spatial and perceptual representations, where ambiguities or systematic limitations arise, and which planning decisions can be meaningfully supported by immersive, AI-assisted interaction. The results form a sound basis for decision-making regarding subsequent research and development activities and contribute to the advancement of context-aware, energy- and resource-efficient immersive planning technologies.