AI projects at the Medical Faculty Heidelberg

Project description:

In this project, an AI-supported assistance system for skin cancer diagnostics is being developed, which is to be transferred from university research to marketability in compliance with the MDR. The software supports doctors in the assessment of potentially malignant skin lesions and emphasises explainability in order to strengthen trust in the technology. Together with HEINE Optotechnik GmbH & Co KG, the solution will be integrated into digital dermatoscopes to enable broad application in skin cancer screening. The aim is to achieve a measurable improvement in diagnostics and thus a concrete benefit for patients, practitioners and the healthcare system. At the same time, the project serves as a blueprint for other research institutions and develops recommendations for regulatory and health policy frameworks.

Vision:

AI is important for our research because it can help support early and more accurate detection of melanoma and avoid unnecessary biopsies. Our vision is a trustworthy, explainable and MDR-compliant solution that is used in the digital dermatoscope as a complementary decision-making aid. In the future, AI should be used in healthcare under clear clinical framework conditions: transparent, validated and with a view to fairness and data security. In this way, it can support doctors and potentially contribute to improved treatment outcomes.

Contact:

PD Dr Titus J. Brinker: titus.brinker(at)nct-heidelberg.de

Project description:

The project aims to develop an AI-supported system for the automated analysis of cardiac deformations and the precise phenotyping of cardiovascular diseases based on cardiac MRI data. The focus is on a heart-phase-specific strain analysis based on deformable image registration models, which enables both the automatic detection of characteristic heart phases and vector field-based strain calculation.The aim is to evaluate the suitability of this methodology for the characterisation of complex clinical pictures, in particular heart failure with preserved ejection fraction (HFpEF), and to investigate the extent to which the integration of multimodal information from cardiac MRI, echocardiography and clinical data can increase the diagnostic and prognostic value. The innovative approach combines deformable image registration models for strain analysis and cardiac phase detection with multimodal data fusion to create comprehensive, patient-specific diagnostics.

Vision:

As the population grows and ages, the need for precise diagnostics and personalised treatment decisions increases. AI offers the opportunity to support doctors with intelligent, data-driven analysis systems so that more people have faster access to better and personalised medical care and doctors have more time for their patients.

Contact:

Sarah Kaye Müller

PhD Student of Institute "Artificial Intelligence in Cardiovascular Medicine" [AICM]

Heidelberg University Hospital | Im Neuenheimer Feld 410 | D-69120 Heidelberg

SarahKaye.Mueller(at)med.uni-heidelberg.de

Sandy.Engelhardt(at)med.uni-heidelberg.de

Project description:

The project aims to develop generative AI models that generate realistic medical image data without disclosing patient data. Through a resource-efficient diffusion approach and optimised evaluation metrics, the project contributes to a sustainable and efficient application of generative AI in medicine. A particular focus is on reducing data memorisation, improving image quality and diversity and analysing the ecological footprint of such models.

Vision:

Artificial intelligence, in particular generative AI, plays a central role in medical imaging, as it can recognise complex structures and generate realistic, diagnostically relevant information from limited data. In the future, generative AI systems will go beyond pure image synthesis and create patient-centred, multimodal representations. By integrating anatomy, physiology and different image modalities into common latent spaces, AI can discover new correlations, bridge modalities and enable digital patient twins. This creates a foundation for deeper, data-driven and personalised medical research.

Contact:

Marvin.seyfarth(at)med.uni-heidelberg.de

SalmanUl Hassan.Dar@med.uni-heidelberg.de

Sandy.engelhardt(at)med.uni-heidelberg.de

Project description:

The development of AI systems for analysing surgical image and video data is an active field of research. Such systems enable the extraction of information about the course of operations that was previously inaccessible. This information can be used to support operating theatre staff in real time in order to increase the safety of surgical procedures and optimise logistical processes in the operating theatre. Furthermore, the retrospective analysis of procedures opens up the possibility of gaining valuable information from the recordings and making it available to trainee surgeons.

As part of this project, we are investigating the development of multimodal AI models that can simultaneously process and interpret image and text data from cardiac surgery video recordings.

Vision:

AI models that can simultaneously process and analyse surgical image and text data form an important basis for novel interactive learning platforms and decision-support systems.

Contact:

Prof Dr Sandy Engelhardt
Institute for Artificial Intelligence in Cardiovascular Medicine
Department of Cardiac Surgery
Heidelberg University Hospital
Im Neuenheimer Feld 420
69120 Heidelberg
+49 6221 56-37173, sandy.engelhardt(at)med.uni-heidelberg.de

Georgii Kostiuchik
Institute for Artificial Intelligence in Cardiovascular Medicine
Department of Cardiac Surgery
Heidelberg University Hospital
Im Neuenheimer Feld 420
69120 Heidelberg
+49 6221 56-310341, georgii.kostiuchik(at)med.uni-heidelberg.de

Project description:

The project, started at the DZHK and now internationalised, aims to predict clinical complications of transcatheter aortic valve implantations (TAVI). Data from different institutions and different modalities (CT, ECG, prosthesis information, etc.) are included without leaving the institution. The model, which has been trained and orchestrated in Heidelberg, can be trained and evaluated across several centres.

Vision:

Artificial intelligence, especially deep learning, plays a central role in medical imaging, as it can recognise complex structures and predict realistic, diagnostically relevant information from limited data. Federated methods enable the training of such deep learning models on data from different institutions without exchanging patient-specific information. By adding multi-modal capabilities to such federated models, existing data from different diagnostic procedures (e.g. CT and ECG) can be combined to significantly improve the predictions of deep learning models. The combination of federated and multi-modal techniques enables extensive collaboration between clinical departments and centres, and lays the foundation for deeper and data-driven medical research and diagnostics.

Contact:

Yannik.Frisch(at)med.uni-heidelberg.de

Sandy.en gelhardt@med.uni-heidelberg.de

Project overview:

The WSI-Babel-Shark pipeline is a modular framework for automated metadata extraction from digital pathology slides (WSIs).
It integrates barcode decoding, ROI-based optical character recognition, stain detection, and slide ID generation to create structured registries from raw histopathology data.
By combining deep-learning-based layout classification with rule-based text parsing, it achieves robust recognition of complex label layouts across diverse scanners and staining protocols.
The system emphasises reproducibility, modular design, and transparent logging, providing a scalable tool for AI-driven digital pathology workflows.
It serves as a successor to the earlier WSI-BabelFish prototype with improved open-set handling and ROI fallback logic.

Vision:

Artificial intelligence enables reproducible, large-scale integration of pathology data for research and clinical practice.
By automating metadata acquisition and harmonization, Babel-Shark bridges the gap between raw digital slides and structured datasets ready for downstream AI analysis.
We envision a fully automated digital pathology infrastructure where metadata, image content, and clinical context are seamlessly linked.
Such systems will accelerate AI model training, enable transparent validation, and support precision diagnostics across institutions.

Contact:

Shahram Aliyari¹

¹ Section Computational Pathology Heidelberg, Institute of Pathology Heidelberg, University Hospital Heidelberg, University of Heidelberg, Heidelberg, Germany.

Shahram. aliyari@med.uni-heidelberg.de

Project overview:

DiagnosticReportHarvester is a self-hostable clinical text search platform to identify relevant clinical cases from free-text diagnostic reports. It combines Elasticsearch indexing with clinical NLP to normalise terminology, expand queries using UMLS synonyms, and detect negations or modifiers (via medspaCy or using LLM-supported semantic search). The system supports semantic retrieval with clinical LLM embeddings and ranks reports to reduce manual screening time. Designed for compatibility with SQL-based clinical databases, it offers a secure web interface tailored for routine and research workflows.

Vision:

By turning unstructured clinical text into searchable, semantically rich data, DiagnosticReportHarvester accelerates cohort discovery and hypothesis generation. We envision a privacy-preserving search layer that integrates with LIS/PACS and downstream AI pipelines. The result is faster, reproducible case selection with transparent evidence trails that clinicians can trust.

Contact:

Maximilian Legnar Msc.¹

¹ Section Computational Pathology Heidelberg, Institute of Pathology Heidelberg, University Hospital Heidelberg, University of Heidelberg, Heidelberg, Germany.

Maximili an.Legnar@med.uni-heidelberg.de

Project overview:

Large language models (LLMs) are increasingly applied in medical documentation and have been proposed for clinical decision support. A main focus in this field has been on the assessment of closed-sourced frontier LLMs, while the interest in open-source alternatives is rising. We are evaluating the performance of small language models as building blocks for novel open compound-AI systems, which can be operated at the edge.

Vision:

LLM based medical compound AI system must be based on transparent and controllable open-source models. Openness enables medical tool developers to control the safety and quality of underlying AI models, while also allowing healthcare professionals to hold these models accountable. For these reasons, the future of medical compound AI system must be open.

Contact:

lars.rie demann@med.uni-heidelberg.de

Project overview

This project aims to develop novel predictive and prognostic biomarkers for PDAC using advanced deep learning applied to histology and immunohistochemistry (IHC) data. By integrating multimodal image features, we seek to predict molecular characteristics directly from slides. The approach will map spatial niches where specific cell phenotypes coexist and evaluate how their heterogeneity influences clinical outcomes. This AI-driven framework addresses PDAC's complex, stroma-rich microenvironment and has the potential to advance personalised treatment strategies through image-based biomarker discovery.

Vision:

AI is revolutionising how we study disease by turning routine images into rich sources of molecular and clinical insight. It enables us to decode the tumour microenvironment and predict outcomes with unprecedented precision. The future of AI in research lies not in replacing scientists, but in expanding the boundaries of what science can reveal.

Contact:

Prof. Dr Jakob N. Kather - jakob.kather(at)med.uni-heidelberg.de

PD. Dr Nathalia Giese -Nathalia.Giese(at)med.uni-heidelberg.de

Dr Anna-Katharina König -Anna-Katharina.Koenig(at)med.uni-heidelberg.de

Project overview

Our project leverages decentralised deep learning approaches, particularly swarm learning, to collaboratively train AI models across institutions without sharing raw data. This ensures that sensitive information remains local while still enabling the creation of powerful, multicentric datasets. Our current initiatives apply swarm learning to a variety of clinical data types, including histopathology whole-slide images, CT, MRI scans, surgical videos and single-cell analyses. Through this approach, we aim to advance medical AI while maintaining the highest standards of data privacy and security.

Vision:

AI is revolutionising how we study disease by turning routine images into rich sources of molecular and clinical insight. It enables us to decode the tumour microenvironment and predict outcomes with unprecedented precision. The future of AI in research lies not in replacing scientists, but in expanding the boundaries of what science can reveal.

Contact:

Prof. Dr Jakob N. Kather - jakob.kather(at)med.uni-heidelberg.de

Dr Oliver Saldanha - Oliver.Saldanha(at)med.uni-heidelberg.de

Dr Silvia Barbosa - silvia.barbosa(at)med.uni-heidelberg.de

Project overview

The goal of this project is to develop a fully automated AI model for accurate glioma prognosis by integrating pre-operative MRI and post-operative WSI, addressing the critical clinical bottleneck of manual image segmentation. The central innovative AI approach is a fully automated, segmentation-free pipeline that employs pre-trained foundation models to extract deep features directly from raw imaging data. The project systematically investigates the optimal data fusion strategy by evaluating novel fusion architectures against single-modality models and simpler fusion baselines. This research aims to determine if this foundation model-driven approach can provide a more accurate, objective, and scalable tool for clinical risk stratification than current methods.

Vision:

Artificial intelligence is essential for our research because it provides a practical way to quantitatively integrate the complementary information from macroscopic MRI and microscopic WSI data. Our specific use of foundation models is critical as it allows us to bypass the manual segmentation bottleneck, enabling a fully automated and reproducible analysis pipeline. The future we envision is one where such automated systems are seamlessly integrated into standard clinical workflows, analysing routinely acquired imaging data in the background to provide consistent, objective decision support.

Contact:

Prof. Dr Jakob N. Kather - jakob.kather(at)med.uni-heidelberg.de

Dr Silvia Barbosa - silvia.barbosa(at)med.uni-heidelberg.de

Prof. Dr Shuixing Zhang -zsx7515(at)jnu.edu.cn

Project overview

The goal of this project is to harness AI to predict colorectal cancer prognosis and, in the future, treatment response. Using data from the ColoCare Study, we focus on imaging phenotypes derived from CT scans and digital pathology to identify patterns linked to clinical outcomes such as recurrence and disease-free survival. Our research addresses key questions about how imaging-based features reflect underlying tumour biology and patient outcomes. The innovative aspect of this work lies in its fully automated, agnostic AI approach, which allows the discovery of novel, data-driven imaging biomarkers without relying on predefined assumptions.

Vision:

AI is essential to this research because it enables us to uncover complex, hidden patterns in medical images that are beyond human perception. By applying AI to CT scans and digital pathology data, we aim to better predict colorectal cancer prognosis and, in time, treatment success. This work will establish a foundation for integrating multiple data types within the ColoCare Study to enhance precision medicine. Ultimately, we envision a future where AI-driven insights guide truly individualised care for colorectal cancer patients.

Contact:

Dr Biljana Gigic - biljana.gigic(at)med.uni-heidelberg.de

Dr Victoria Damerell - victoria.damerell(at)med.uni-heidelberg.de

Prof Dr Jakob N. Kather - jakob.kather(at)med.uni-heidelberg.de

Dr Silvia Barbosa - silvia.barbosa(at)med.uni-heidelberg.de

Project overview

The goal of the LIMBS project is to reduce the time- and labour-intensive process of extracting structured information from proprietary document formats, such as PDFs and unstructured text, by employing a generative AI framework with human-in-the-loop oversight. Can generative information extraction enhance efficiency and documentation quality in real-world clinical settings?

The LIMBS project is implementing a biomedical information extraction framework combining different machine learning (ML) techniques like optical character recognition (OCR), large language models (LLMs) and vision language models (VLMs) into one agentic system to extract structured information from unstructured PDF documents.

The limbs-framework's modular architecture enables cost efficient generalisation to new clinical settings via prompt engineering and allows for easy extension or customisation by third-parties.

Vision:

AI is essential for our research because it can unlock clinically valuable information trapped in unstructured, heterogeneous records like discharge summaries and physician letters, which currently limit interoperability and complicate the integration and analysis of patient data. By transforming this information into structured, analysable insights, generative machine learning enables optimised data sharing, better clinical decision-making, and accelerated research.

Contact:

Dr Keno März - k.maerz(at)Dkfz-Heidelberg.de

Project description:

The project aims to develop an automated, AI-based system for assessing the severity of lung disease in cystic fibrosis (CF) in MRI images to overcome the limitations of current visual scoring methods, which are time-consuming and subjective. Deep learning, in particular Convolutional Neural Networks (CNNs), will be used to detect key MRI features such as bronchial wall changes, mucus plug formation, consolidations and perfusion defects. A database of around 850 standardised chest MRIs from 200 CF patients, which were previously evaluated by experts, will serve as a reference (ground truth) for training and validation. The resulting software should enable a fast, objective and user-independent assessment and at the same time serve as a decision-making aid for clinical specialists. Finally, the prototype will be tested in the clinical workflow and validated in a multi-centre setting with previously unknown data to ensure robustness and transferability.

Vision:

Artificial intelligence is crucial in this research project to replace the previously subjective and time-consuming visual assessment of CF MRIs with an automated, objective and reproducible procedure. In the future, AI will transform radiological diagnostics by supporting doctors with precise, data-based analyses and thus enabling faster, standardised and patient-oriented diagnostics.

Contact:

Dr Urs Eisenmann (urs.eisenmann@med.uni-heidelberg.de)

Institute for Medical Informatics

Im Neuenheimer Feld 130.3

69120 Heidelberg

Project description:

The project is developing an AI-based system for the automated determination of lung perfusion scores from MRI images in various chronic lung diseases. The aim is to use image processing methods and AI approaches to enable MRI-based lung lobe definition and reliable classification of perfusion defects. The multi-centre data collection should result in generalisable models that include clinical lung function parameters in addition to the image data. A particular focus is on the traceability of the decision-making processes, which is why Explainable AI methods are used.

Vision:

The aim is to simulate and support medical decision-making processes in the assessment of lung perfusion defects with the help of AI, especially where classic image processing methods reach their performance limits. AI offers the ability to recognise complex patterns in MRI and clinical data and to incorporate additional data types into the decision-making process. This creates the potential for reproducible, transparent and more efficient diagnosis of chronic lung diseases.

Contact:

Dr Urs Eisenmann (urs.eisenmann@med.uni-heidelberg.de)

Institute for Medical Informatics

Im Neuenheimer Feld 130.3

69120 Heidelberg

Project description:

This is not a single project, but rather a number of ongoing and planned investigations. The aim is to address intensive care and perioperative issues using AI-supported methods. The focus is on explorative analyses of diverse clinical data sets and the use of machine learning algorithms for predictive and decision-supporting applications. In the long term, the aim is to translate these findings into everyday clinical practice and thus further improve the quality and safety of patient care.

Vision:

Artificial intelligence opens up new possibilities for visualising complex medical relationships. Our vision is to raise clinical decisions to a new level of precision and knowledge on the basis of AI-supported examinations and thus to be able to treat patients more individually, safely and predictively.

Contact us at

melanie. marhofer@med.uni-heidelberg.de