Researchers from Potsdam develop AR training for deployment scenarios
Extreme weather situations such as heavy rain and flooding are becoming more frequent. The disaster control teams in affected districts then have to react quickly and make decisions under great stress. For an initial overview, it helps to take a look at the map: Where is the water likely to collect around a location? Which zones could be particularly affected? All of this requires a topographical understanding and the ability to make quick decisions, which many administrative staff are not exposed to on a day-to-day basis. How could they? Such situations are often difficult to simulate in advance.
A team from the Film University Babelsberg KONRAD WOLF has developed an augmented reality (AR) training application for the oKat-SIM funding project. The entire project, which promotes the transfer of geoscientific findings into the training of crisis teams, was initiated under the leadership of the Institute of Geosciences at the University of Potsdam and the network spokesperson Dr. Gerold Zeilinger. The University of Lübeck, the Institute of Cognitive Science at the University of Potsdam and, as an associated partner, the company KSK Stabskonzepte, which coordinates crisis team training throughout Europe, are also involved.
Quick familiarization with the situation
“Our application is intended for administrative staff who, in their position, may be confronted with sudden crises such as flood disasters. In such situations, they are confronted with information that they have never had to deal with before, they have to make decisions that they have never made before and under conditions that they have never experienced before,” says Professor Dr. Björn Stockleben, who is leading the project on behalf of the Film University.
In traditional training courses for professional crisis teams, elaborate role plays with actors and materials are prepared. In contrast, the Film University team sees the deliberately lean AR application as an introductory training course to create scenarios for a broad target group that help them to read maps, understand landscapes and explore decision-making options. It can be carried out virtually anywhere and with little prior knowledge and materials, provided there are enough AR glasses available.
Play through possible decisions
A digital plastic model is projected onto a table using AR and the participants gather around it equipped with HoloLenses. The model shows the town of Kemnitz near Görlitz, which is prototypically affected by a heavy rainfall event. They can see where the rain catchment basin is located and how the masses of water make their way along the hills through the streets or where flooding of the transformer house suggests a power failure. A trainer guides the participants through a series of events that build on each other in the escalating scenario. The water level on the dyke is rising. Should it be secured first or should the site be evacuated immediately? What are the possible decisions – and more importantly, how do you communicate with each other and reach a consensus quickly? In addition to improving imagination thanks to the three-dimensional map model, interaction is also practiced here. In principle, this creates a simulation game or a social AR application with a multiplayer option. Televisions, pinboards, whiteboards or other information resources can also be integrated into the training. Participants can interact with the application using printed QR codes that are placed on the “playing field”. For example, alternative map displays or the display of evacuation zones can be selected. XR designer Miriam Jud explains: “This is how we train people to understand the landscape and see potential sources of danger. But the most important goal is that participants learn to make decisions quickly – and not just analyze everything.” The decisions can then be triggered using additional QR codes and the scenarios on the digital table model change. They now show the immediate consequences of the previous step. For example: evacuation zone A is flooded if the rainwater retention basin breaks, but evacuation zone B is also affected. What does the staff decide? The trainer can then also raise the water level accordingly using a QR code. In contrast to reality, everything can be reversed in training to test other solutions.
Cinematic-didactic exaggeration instead of exact replication
“The depiction of the masses of water, for example, cannot take place in real time. We sometimes condense an event lasting six hours into a few minutes or seconds and still try to create an authentic representation. Otherwise a flood, for example, would only appear as a short blue snake of water moving through the picture. This requires didactic exaggeration or attenuation elsewhere. Scientific accuracy meets cinematic realization, where we have to find a balance,” says David Schornsheim from the Film University team. “It’s not an exact replica of the crisis team situation, but it fills a gap. It’s an introductory training course that helps people to get into situations like this,” adds Stockleben. In addition to the AR application, the Film University also worked on the didactic design of the visualization and an interaction concept that allows the application to be seamlessly integrated into KSK’s existing training approaches.
A particular challenge was the data visualization of the geodata for AR. As there are practically no tools on the market that convert geoscientific data into film and games standards, a lot of the work had to be done by hand. Here, too, it was necessary to combine the demand for scientific visualization with the demand for aesthetic visualization and make it accessible for the HoloLens.
The training sessions already held with the Kemnitz prototype have been well received and the feedback from city councils shows that the training concept works and is motivating. The project, funded by the Federal Ministry of Research and Education, has successfully made scientific research findings available for training purposes and created further training opportunities with AR.
To find out more about the project, listen to the eleventh episode of the Film University’s “Dritte Klappe” podcast, which features a conversation between Prof. Björn Stockleben (New Media, Film University), Oliver Oswald (KSK) and David Schornsheim (HoloLens Developer oKat-SIM, Film University) about the development process, training and options for action.
More about the MTH Blog
The media technologies of the future are already being used today – not only in the entertainment sector but in a wide variety of industries. For our MediaTech Hub Potsdam blog, we talk to tech enthusiasts, entrepreneurs and researchers once a month and tell the stories behind their innovative business models, ideas, projects or collaborations.
