One Patient, One Record
Hospitals rarely lose data. They duplicate it: a misspelt surname at admission, a swapped date of birth at the lab, a new address at the pharmacy. I built a Master Patient Index pipeline that collapses 5,000 patient-style records back into the people they describe, and measured every step against the known truth.
Where to draw the line
Every compared pair by match score. Move the threshold to see the trade-off between false merges and missed duplicates.
Steward review queue
Pairs scoring between 0.55 and 0.70 go to a person. Field-by-field similarity helps them decide; the truth is shown for scoring.
Golden records
The largest clusters found, each collapsed to one golden record by survivorship: the most frequent non-empty value per field. Red cells differ from the golden value.
Why synthetic data
Real patient registries cannot be published. FEBRL was built for exactly this problem: realistic Australian-style identities with the typos, swaps and gaps that real registration desks produce, plus the truth needed to score a matcher.
Why thresholds matter
A false merge joins two patients' medical histories, which is a clinical safety risk. So the automatic threshold sits where testing showed zero false merges, and the grey zone goes to a person.
What I would add next
Learn the field weights from the data with Fellegi-Sunter or a small classifier, and add a nickname table (bob, robert) and an address reference file for suburb spelling.
Source: FEBRL dataset 3 (Christen, 2008), distributed with the Python recordlinkage package. Record IDs were used only to score the result, never inside the matching.