New tools for doctors to process medical data: €8 million research project advances the use of AI and extended reality in medicine

The volume of medical data has increased exponentially with the advent of digital imaging. However, the development of diagnostic tools capable of making effective use of this data has lagged behind. 

The three-year MEDALLION project, which is now drawing to a close, investigated how virtual reality and AI can help to unlock new ways of using medical data.

“As doctors, we urgently need better tools. We are drowning in data but lack the necessary analytical and diagnostic software. We need AI-based tools that enable the efficient visualisation and manipulation of multidimensional medical data,” says Dr Jorma Järnstedt, Senior Radiologist at Tampere University Hospital and Specialist Dentist in Oral and Dental Radiology. He served as an adviser to the project on the clinical applicability of AI and XR technologies.

Professor Roope Raisamo, Director of the Tampere University Computer-Human Interaction Research Center (TAUCHI), coordinated the MEDALLION project. According to Raisamo, the project produced significant results in areas such as the validation and standardisation of XR technologies, interaction technology development and communication between doctors and patients.

“A key achievement was the scientific demonstration that XR enhances the teaching of radiological practice and that AI can improve the diagnosis of pancreatic cancer. We also succeeded in reducing hallucinations generated by large language models,” says Raisamo.

During the project, Raisamo’s research group explored more effective methods for analysing digital images using XR technologies.

“The three-dimensional visualisation of medical data is an area in which computers can genuinely assist clinicians. Virtual reality models enable a more intuitive understanding of complex datasets, while augmented reality tools can facilitate planning,” says Raisamo.

The tools can also support virtual collaboration over the internet, both locally and internationally, thereby making remote working and data sharing easier for healthcare professionals.

MEDALLION integrated language intelligence into XR and VR environments

The MEDALLION project delved into various aspects of AI, including how machines understand and generate medical text, such as medical dictations and reports. 

Throughout the project, the researchers worked closely with industry partners to seamlessly integrate language intelligence into XR and VR environments.

“Through language processing and enrichment, we have developed improved solutions that help to ensure we can continue to provide reliable, user-friendly, interoperable and understandable content for these environments,” says Tiina Lindh-Knuutila, Data Lead at Lingsoft.

XR tools must be validated before clinical use

Medical devices and software undergo rigorous testing and evaluation before they are approved for clinical use.

Optical devices, such as computer displays, must also meet exacting quality standards when used for diagnostic purposes. Similar requirements apply to XR devices.

Using its unique technology, OptoFidelity helped to accelerate the development of XR devices while improving the immersiveness and user-friendliness of the tools.

“The MEDALLION project provided us with an excellent opportunity to contribute to the development of solutions for the medical technology sector. We have been seeking new business opportunities and applications for our products as well as for our expertise in optical metrology,” says Hans Kuosmanen, CEO at OptoFidelity.