In brief Neural networks can correctly guess a person’s race just by looking at their bodily x-rays and researchers have no idea how it can tell.

There are biological features that can give clues to a person’s ethnicity, like the colour of their eyes or skin. But beneath all that, it’s difficult for humans to tell. That’s not the case for AI algorithms, according to a study that’s not yet been peer reviewed.

A team of researchers trained five different models on x-rays of different parts of the body, including chest and hands and then labelled each image according to the patient’s race. The machine learning systems were then tested on how well they could predict someone’s race given just their medical scans.

They were surprisingly accurate. The worst performing was able to predict the right answer 80 per cent of the time, and the best was able to do this 99 per cent, according to the paper.

"We demonstrate that medical AI systems can easily learn to recognise racial identity in medical images, and that this capability is extremely difficult to isolate or mitigate," the team warns [PDF].

"We strongly recommend that all developers, regulators, and users who are involved with medical image analysis consider the use of deep learning models with extreme caution. In the setting of x-ray and CT imaging data, patient racial identity is readily learnable from the image data alone, generalises to new settings, and may provide a direct mechanism to perpetuate or even worsen the racial disparities that exist in current medical practice."

The Register®
Software AI + ML
AI algorithms uncannily good at spotting your race from medical scans, boffins warn
Plus: British MP wants to ban AI deepfake smut tools
Katyanna Quach
Sun 8 Aug 2021 // 11:01 UTC
16 comment bubble on white
In brief Neural networks can correctly guess a person’s race just by looking at their bodily x-rays and researchers have no idea how it can tell.

There are biological features that can give clues to a person’s ethnicity, like the colour of their eyes or skin. But beneath all that, it’s difficult for humans to tell. That’s not the case for AI algorithms, according to a study that’s not yet been peer reviewed.

A team of researchers trained five different models on x-rays of different parts of the body, including chest and hands and then labelled each image according to the patient’s race. The machine learning systems were then tested on how well they could predict someone’s race given just their medical scans.


They were surprisingly accurate. The worst performing was able to predict the right answer 80 per cent of the time, and the best was able to do this 99 per cent, according to the paper.

"We demonstrate that medical AI systems can easily learn to recognise racial identity in medical images, and that this capability is extremely difficult to isolate or mitigate," the team warns [PDF].

"We strongly recommend that all developers, regulators, and users who are involved with medical image analysis consider the use of deep learning models with extreme caution. In the setting of x-ray and CT imaging data, patient racial identity is readily learnable from the image data alone, generalises to new settings, and may provide a direct mechanism to perpetuate or even worsen the racial disparities that exist in current medical practice."

https://www.theregister.com/2021/08/08/in_brief_ai/