As part of a university cooperation between the TU Berlin (Stage Design_Scenic Space course) and the HTW Berlin (Communication Design course), 18 students realised a highly complex and creative practical project in the summer semester of 2024 with ‘MUSICVERSE’. With the help of artificial intelligence, they have brought creative live visualisations to the 360-degree dome of the Zeiss-Großplanetarium Berlin to immerse the audience in a new world of sound and images. The one-hour live concerts of the 4 newcomer bands Mogli, Ásgeir, orbit and Alice Merton were to be experienced in a new way with this special spatial sphere and were recorded by the television producer DEF Media for the German-French broadcaster ARTE. Mogli, Ásgeir and Orbit are already available in the media centre.

Under the direction of lecturer and visual artist Lukas Ruoff and accompanied by Franziska Ritter (TU Berlin) and Prof. Pablo Dornhege (HTW Berlin), the 18 interdisciplinary young talents from the fields of architecture, theatre, stage design, communication design, film and VR worked on the media concert design for the four up-and-coming bands in a six-week intensive work phase in the studios of the TU Berlin and the HTW Berlin and then implemented it in the planetarium.

For the Berlin multi-talent Mogli and her previously unreleased album, the audience is transported into psychedelic natural worlds that were actually recorded in real time and enhanced with the help of artificial intelligence to overlay effects and illustrations. The heavily pregnant artist can be seen surrounded by unreal projections at the zenith of the dome. The band orbit (aka Marcel Heym & his friends) benefited from a much more synthetic visual language through AI-generated sequences to match its atmospheric synthesizer leads. Morphings and image changes that would not be possible with conventional methods were combined with the VJs’ organic sense of rhythm in the live concert, creating an artificially real world between growing mushrooms and swimming goldfish. For Alice Merton and her band, whose globally successful debut song ‘No Roots’ is in everyone’s ears, expansive landscapes and atmospheres were AI-generated, which were repeatedly shaken up by surreal situations. For example, the gentle tilting of a desert landscape on its own axis played with the audience’s perception. The performance by Icelandic dream pop musician Ásgeir impressed with billowing particle clouds reminiscent of the Northern Lights: Forces of nature met pixels and created memories of clouds, waves and rain that literally washed over the audience.

The main aim of the project realisation was to complement acoustic and visual media synaesthetically in such a way that dialogue spaces are opened up and immersive atmospheres (3D live and 2D on screen) are created. A special focus was placed on researching the possible applications of artificial intelligence. An impressive range of innovative tools and technologies were used, which each of the four teams utilised individually – depending on the concept:

In the research phase at the beginning of the project, for example, Perplexity.ai and ChatGPT were used to research in-depth information as efficiently as possible with the AI assistants and AI-supported search engines or to enter into a playful dialogue to find concepts. Conventional design methods such as film, 2D and 3D animation and image-generating AI tools were used to produce the visual content. Initially, text-to-image as such as Midjourney were used to create textures and environments. The animation of the content was carried out using a wide variety of methods. In addition to working with classic animation tools, artificial intelligence was also used here, partly for video-to-video, i.e. the stylisation of existing moving images, but also text-to-video, i.e. the text-based generation of moving images. The resulting collection of content was transferred to the planetarium’s 12 projectors using a system specially created in Touchdesigner and could be controlled live by the students.

The 4 project results:

Students: Bruno Ammon, Lennard Becking, Talile Bekele, Anna Braun, Carmen Hartmann (TU), Tabea Jorcke (TU), Jessica Kaczmarek (TU), Senta Kammerer, Marie Kauke, Florencia Martina (TU), Danijela Matović (TU), Evrigiorgio Panzera, Anna Patzak, Emma Planckaert (TU), Hannah Riedel, Vincent Schneider, Charlotte Marie Töpfer, Mara Vorberg

Production: DEF Media
Direction: Miriam Dehne
Tutor, Creative Direction & Technical Lead: Lukas Ruoff
Project leadership: Prof. Pablo Dornhege, Franziska Ritter

Fotos: Victoria Basel u.a.