Detect, Score, Explain: Anatomy of an ML Pipeline for Hand X-Rays

The brief model: With one week and a crew of 4, we constructed an end-to-end pipeline that scores erosive hand osteoarthritis from X-rays: detecting every finger joint, evaluating it towards the scientific GUSS scale, and producing GradCAM heatmaps that present the place the mannequin seemed. Two modeling approaches, one strict analysis protocol, a fivefold enchancment over the baseline on the metric that issues clinically. Here’s what we constructed, how we constructed it, and what it taught us about medical imaging AI.

Teaching AI to see arthritis

Every yr, in that unusual quiet week between Christmas and New Year, one thing uncommon occurs at ML6. We shut the door on shopper work, brew rather a lot of espresso, and spend a couple of days constructing issues simply because we need to. It’s our annual Christmas initiatives week, equal elements hackathon, playground, and excuse to chase the concepts that by no means fairly make it onto a roadmap.

This yr, underneath our ML6 For Good program, we pointed that vitality at initiatives with a transparent constructive impression. One of them sits squarely in a site we care about rather a lot: medical imaging. The objective was to see how far a small crew might push an automated scoring pipeline for erosive hand osteoarthritis, a painful, under-recognized illness, utilizing solely hand X-rays as enter.

Erosive hand osteoarthritis, EHOA for brief, impacts as much as 10% of girls and three% of males, and but most individuals have by no means heard of it. It’s an aggressive cousin of the extra acquainted osteoarthritis, largely affecting postmenopausal girls, that quietly eats away on the small joints of the fingers. It’s painful, it’s damaging, and proper now there’s no remedy. Treatment is essentially about managing the ache, with surgical procedure as a final resort.