You never know when you're the exception

“You never know when you’re one of the underrepresented subgroups,” says DIREC Postdoc Jakob Ambsdorf, who trains the project’s models. “It could be that your disease presents differently than it usually does, or that you look different from the patients the model was trained on. Or it could simply be that they’re using a different scanner at that one hospital, and that’s why you get a less accurate diagnosis.”

From robots to real patients

Jakob studied human-computer interaction in Hamburg and got hooked on deep learning while writing his bachelor’s thesis. During his master’s he worked with humanoid robots. He found them fascinating, but they were a long way from making a difference in people’s lives. When a position opened in ExplainMe, the DIREC project that preceded FairFM, it came with a medical use case he had never worked with before.

“As a computer scientist, you’re sometimes used to just working with datasets. Seeing that there are real patients getting real scans and real diagnoses gave me a lot of purpose for the PhD. It still does.”

Taking on the hardest problems

Predicting spontaneous preterm birth is one of the most important tasks in fetal medicine, and currently one of the hardest. When doctors know a baby is at risk of arriving early, there is a lot they can do, but only if they know in time. Estimating a baby’s weight is another key task, since a baby that is too small or too large for its gestational age needs different care. FairFM’s models have been applied to both tasks and are now being tested by partner hospitals around the world.

AI has been used in medical imaging for years, and the same problem keeps appearing.

“If we train a model on data from one hospital and move it to the hospital next door, it already performs a bit worse,” says Jakob.

Foundation models could change that. Traditional models learn from a small set of images annotated by experts. Foundation models first learn from large amounts of unlabelled data and are only adapted to a specific clinical task afterwards.

“That way we can cover all the data that doesn’t usually make it into our datasets,” says Jakob, “like a rare scanner type or patient characteristics we don’t see very often.”

According to Jakob, the collaboration between computer scientists, clinicians and an industry partner is what makes the project work: “We can do a study and show an effect, but that doesn’t change how patients are treated. Commercialisation is ultimately how you make an impact.”

If the models hold up, being the exception should no longer mean getting a less accurate scan.