Why this topic
At the Mayo Clinic there is a network that can look at ten seconds of an ordinary heartbeat and tell you the pump is failing, quietly, years before the patient has thought to complain about anything, and while the cardiologist reading the same ten seconds sees nothing at all worth mentioning. That is a strange and rather wonderful thing to have built.
Heart disease kills more people than anything else on earth, and it has accordingly become one of the most crowded workshops in medical AI. Whatever anyone in this community happens to be building, some version of it arrives in their own field soon enough.
The club has spent the year circling a single question, approaching it from whichever direction the month's paper allowed: what has to be true of a body of data before anybody ought to act on it? Cardiology is a fine place to ask, for the uncomfortable reason that the quantity of work is enormous and the quantity that has changed a patient's life is not.
What kind of paper this is
It is a review, which is to say it gathers up other people's studies rather than running any of its own. Every number in it is therefore on loan, and a borrowed number arrives carrying whatever manners it learned at home.
A narrative review, more precisely, meaning the authors selected by judgement rather than by a declared search. What you have in your hands is a well-informed argument about a field, made by ten people who have read a great deal of it. That is a different object from a census, and worth knowing which you are holding.
Four jobs, four diseases
The authors could have sorted the field by algorithm, which is how computer scientists tend to prefer it. They sorted it instead by where along a patient's care the model actually stands, which is how a clinician would prefer it, and which turns out to be the more illuminating arrangement.
| The job | What it means | Example |
|---|---|---|
| Predicting | Spotting risk before anything has happened | Genetic risk scores; flagging high blood pressure two years early |
| Diagnosing | Working out what is wrong, and which kind | EchoNet-Dynamic on heart ultrasound; sorting heart failure into subtypes |
| Treating | Choosing the drug and when to start | Matching drugs to heart failure subtypes; how long to keep someone on blood thinners |
| Managing | Watching continuously instead of at appointments | A chest patch flagging heart failure; follow-up over a messaging app |
Those four run across blocked arteries, heart failure, atrial fibrillation and high blood pressure. And here is the habit that makes the paper worth an afternoon: each section builds its case with enthusiasm and then, in the closing paragraph, takes most of it back. The last paragraph is generally the honest one.
Steve walked the room through the results that make people sit forward in their chairs, and then drew a line the field blurs almost as a matter of habit. Helping a clinician read a trace more accurately is a real contribution and nobody should be sniffy about it. It is also an entirely different occupation from understanding what causes the disease, or from producing something new to prescribe. Conflating the two is how this field oversells itself, cheerfully and without noticing.
What goes in
Three kinds of data
- Tidy. Hospital records, labs, cohorts like UK Biobank.
- Messy. Scans, the free text a doctor types, wearable streams.
- New. Genetics, blood chemistry, microbiome, where you live.
Three ways to combine them
- Early. One long row, one model.
- Middle. Each type gets a sub-model, merged inside the network.
- Late. Separate models, then let them vote.
The finest technical writing in the paper concerns something almost aggressively unglamorous: agree the format while you are collecting the data, not in the hungover months afterwards. Skip that step and, as the authors put it with some restraint, "even technically advanced platforms can devolve into data silos."
The review closes its systems chapter with an architecture diagram of the kind that looks marvellous on a slide: every source of data feeds a single platform, and out the other end comes one clear instruction for the patient. Steve's verdict was that this is highly optimistic. It asks hospitals, companies and regulators to cooperate at a level the industry has never once managed.
His list of what stands in the way was less dramatic and considerably more convincing. No agreed standards between institutions. Records missing, and missing most reliably for the patients you most want to find. The same condition written down differently depending on who is holding the pen. And a catch with a certain bleak symmetry: the FDA and NIH set rules about who may join a study precisely in order to reduce bias, and those rules narrow who ends up in the data, which introduces a bias of a different shape. He was blunt that nobody has solved this, and that the field's answer so far has been to go and collect more data.
Three rungs
Nearly every claim in this paper rests on one of three rungs, and the distance between the rungs is the whole story. It is also considerably greater than the prose lets on.
Tier 1 The model does the maths well
The measure is AUC, a mark out of one for telling two groups apart, where 0.5 means the model is flipping a coin with some conviction. The great majority of the 260 studies live on this rung, and they live there comfortably.
These are handsome results and there is no call to be grudging about them. The paper's own verdict on the rung, however, is delivered without ceremony: "relatively abundant technical validation but insufficient high-level clinical effectiveness evidence."
Tier 2 Doctors did something different
Tier 3 Patients were better off
Wang Xiang offered the comparison that explains the entire shape of the ladder. In cancer, a mutation picks the drug, so a model that finds the mutation walks you directly to a treatment decision. Cardiology enjoys very little of that convenience.
The room divided on what follows from that. Should the effort go into better prediction, so that the treatments already on the shelf begin sooner? Or into the biology, so that there is eventually something new to prescribe? Most leaned towards prediction as the achievable near-term goal, with laboratory work continuing alongside. Nobody sounded entirely satisfied with that as the whole of the answer, and nobody should have been.
The 32% problem
There is one sentence in this review that performs, in miniature and with perfect innocence, the exact offence the review spends its final third warning against. It is the best teaching object in the paper.
The sentence reports that an AI study "demonstrated that AI-optimized revascularization strategies could improve outcomes by ~32%." Now go and read what Peyman Ghasemi and his colleagues actually published.
The study took the records of 41,328 patients and asked what a computer might have chosen in place of what the doctors chose, then scored that imaginary policy against what in fact became of the real patients. The method is entirely legitimate, the authors describe it with care, and they went to the trouble of constraining the computer to propose only things a working hospital could actually deliver.
But no one was treated by it. Not one person. The 32% is a score in a simulation.
Carry this to the session as a mirror rather than a weapon. Ten specialists, writing a paper whose entire thesis is that the field mistakes good scores for good outcomes, produced a sentence mistaking a good score for a good outcome. They were not careless. They were human, near a deadline, working quickly through material they knew well. If understanding the trap is insufficient protection against it, then reading faster is certainly not going to help.
Who is actually in the data?
The review never says. Not once, across some 68,500 characters. The words were counted rather than merely complained about, on the principle that a grievance with a number attached is harder to wave away.
What the paper offers by way of remedy amounts to four sentences, tucked into the governance chapter. Data that fails to represent people "can systematically disadvantage women, racial or ethnic minorities, or resource-constrained populations," and therefore "bias assessment" ought to be embedded in governance. The diagnosis is correct. The prescription is a noun. It arrives with no measurement attached, no groups named, no threshold anybody must clear, and no mention of TRIPOD+AI, PROBAST+AI, CONSORT-AI or DECIDE-AI, which are the four checklists built for precisely this purpose and which somebody went to considerable trouble to write.
A paper that opens by projecting 35.6 million deaths across the whole world turns out to rest on evidence gathered almost entirely in wealthy health systems, with China and Japan for company.
The one real fix on offer
A polygenic risk score gathers thousands of small genetic differences into a single number. For years these worked admirably for Europeans and poorly for everybody else, for the unremarkable reason that Europeans were who had been studied. GPS-Mult, from Patel and colleagues in Nature Medicine 2023, was built to correct that, and it does beat every previous version in every ancestry group it was tested in.
Steve raised it against his own presentation, which is precisely why it landed. The standard answer to bias in medical AI is to go and collect more data. His worry is that if you do so without changing who the data comes from, what you build is a model that has learned the workings of a hospital. Who gets referred. Who gets tested. How a doctor phrases a note on a Thursday afternoon. Not the heart.
Christine Wilson agreed and then went considerably further. If you study only clinical data, she said, you are studying only those who already have access and have already consented, and everybody else remains invisible to you. Laboratory research has to run alongside the data work, in order to go and find out whether the patterns mean anything at all. She described the current position as "the first of many baby steps", which is both modest and accurate.
She also put a question that nobody quite answered: why has the personalised medicine that works so well in oncology never arrived in cardiology? Fiona Connolly offered that genotypic and expression-based roles in the progression of heart disease very probably exist, but are poorly defined, and that genetics plays a smaller hand here than it does in cancer. Christine's reply was that the shortage of research interest is itself the gap, and that the multimodal approaches under discussion are exactly how one would find out whether the investment is deserved.
One conflation is worth naming before leaving this section. The paper does take fairness seriously in exactly one place, which is digital health literacy, noting that some 13 million older adults in the United States may struggle with telemedicine at all, and proposing sensible remedies: design the thing with the people who will use it, keep a telephone option open, teach people. That is real content and it should not be waved away.
But it answers a different question. It concerns who can reach the treatment, not who was inside the data that built it. Two failures, two remedies, and the review treats solving the first as though it settles the second.
A model trained on UK Biobank and delivered flawlessly to a well-supported patient in Lagos has still never seen anybody remotely like her.
The strongest argument in the paper
Set aside everything that only ever reached the first rung and a single chain survives, with real evidence at every link along it. The review assembles this case and then, curiously, never quite commits to it, because it has organised itself by disease rather than by the strength of what it found.
Three things make this chain sturdier than anything else in the paper. The scan is already being taken for another reason entirely, so the additional cost is close to nothing. What comes out of it lands on a drug decision that already has evidence behind it. And the machinery for converting a risk score into a prescription exists and works, which is not nothing.
Set that beside the heart failure chapter, where the review concedes that "most evidence for differential drug responses derives from post hoc analyses of retrospective data." Which is to say: somebody went back through the old records looking for patterns. A perfectly respectable way to generate a hypothesis. Not a way to prove one.
The problem underneath all of this
Aditya Nanda presented second, on work of his own, and it appears at first to be a change of subject. It is nothing of the kind. Everything in the pages above depends on somebody, somewhere, being able to lay hands on patient data.
Measure how strongly each of a person's twenty thousand genes is switched on and you have produced something enormously useful and very nearly impossible to share. Roughly half of the relevant biomedical data sits locked away. An application for access consumes months. And the rare, mixed populations that translational work actually requires are, for the most part, not in the public collections at all.
Then came the part that visibly surprised the room. That table of numbers is not anonymous. A paper from 2012 demonstrated that you can work backwards from gene activity to a person's individual DNA variants, producing what its authors called a genetic barcode. Gene activity data has the appearance of a spreadsheet and the behaviour of a fingerprint.
The idea, and why the obvious tools fail
The proposal is to manufacture patients. Not invented from nothing, but computed so that the artificial group behaves like the real one in aggregate while no single record corresponds to a living person. Share the manufactured version freely and keep the original under lock. This is workable because most analysis has no interest in individuals, only in the patterns they collectively make.
Two methods scored identically and were doing entirely different things. The same mark on the examination, quite different biology underneath. That is the finding worth carrying away: accuracy is a weak test of manufactured data, because a model may arrive at the right answer for altogether the wrong reasons.
Note also which method finished last. The most sophisticated one. Compressing twenty thousand genes into three hundred numbers and expanding them out again loses too much on the return journey. Sophistication was never the binding constraint. Information loss was.
Asked whether the method might rescue a very small trial, five responders and five non-responders and ten controls, he declined the invitation without hesitation. Every statistic in the manufactured data descends from the real data beneath it, so ten patients remain ten patients' worth of information however many rows one prints. There was a good deal on offer there for a man selling a method, and he did not take any of it.
Nobody has yet proved that one cannot work backwards from manufactured gene activity data to a real person's DNA. Until somebody does, this remains a convenience for researchers rather than a guarantee of privacy, and the distinction is not a small one.
Whether regulators would accept synthetic data in a submission is unclear. Performance drops for the smallest patient groups, often the ones you most want to study. And asked how the method handles a genetic variant when the variant is what you are predicting, he explained that in the lung cancer data the mutation status was already a separate clinical column rather than computed from expression. How you would generate that variation inside synthetic data is unanswered.
Nanda A, Saha S. Synthetic RNA-seq cohorts for data sharing: a discovery-aware benchmark at transcriptome scale. bioRxiv 2026.05.22.726357. Preprint · Code. Worth saying plainly: dbTwin, the best-performing method, is the corresponding author's own, and the benchmark and code are fully open, which is the right answer to that. Sage Bionetworks published comparable work this year.
Where it falls down, and where it doesn't
Falls down
- No method. No search strategy, no dates, no inclusion rules, no quality appraisal. With 260 references you cannot tell "the literature" from "what ten people found interesting."
- Accuracy figures pass through untouched, from studies of wildly different size and rigour, with no weighting.
- Nobody asks who was in the data.
- A significant randomised trial is missing from a section complaining about missing randomised trials.
- Both cautionary trials are older than the technology.
- An undeclared lens. Nine of the ten authors work in traditional Chinese medicine institutions, and a model reading tongue and pulse features sits alongside CT and MRI without comment. Not a flaw. Useful context for which examples got picked.
Doesn't
- It refuses to celebrate. The self-undercutting at the end of every section is honest and rare.
- The vocabulary travels, as the card above shows.
- Fixing data formats at collection rather than afterwards is correct, boring and ignored everywhere.
- It takes seriously the problem of regulating a model that keeps changing.
- Its heart failure subtype table is useful work in its own right: eight studies, four countries, six methods, the same three patient types falling out.
The session opened on a subject that appeared unrelated and was not. Matt McCormick reported slow progress recreating July's project, on account of only part of the original code having been published, which left him guessing at the steps in between. Fiona Connolly matched it with her own experience in protein engineering, where the binding sequences printed in a paper did not correspond to the data.
The group's diagnosis was deadline pressure compounded by tacit knowledge, which is the understanding a research team holds in common and never troubles to write down, because everybody in the room already has it. A review with no stated method and a paper missing half its code are the same failure wearing different clothes. What is absent from the record remains invisible until somebody sits down and tries to use it.
What the room left open
Words you will need
AUC
Calibration
Surrogate endpoint
Off-policy evaluation
Randomised controlled trial
Healthy volunteer effect
Ejection fraction
Angiogram
Plaque
FFR-CT
Fat attenuation index
Polygenic risk score
HFpEF
MACE
Atrial fibrillation
Dataset shift
Federated learning
Differential privacy
FHIR and OMOP-CDM
Alert fatigue
How this was checked
Two sources, different standards, kept visually apart. Purple blocks carry insights from the BiB community, as recorded in the notes taken on the day. Everything else comes from the paper or the primary studies behind it, each checked against the original rather than against the review: BEAT-HF, Persell, Ghasemi, TARGET, SCOT-HEART, ORFAN, Attia, both EchoNet papers, the 2050 projection and its missing qualifier, PREVENT, UK Biobank's volunteer statistics, GPS-Mult, and the four reporting checklists. The word counts and country counts were computed from the text of the PDF.
Three things stayed unverified. Whether GPS-Mult still underperforms in smaller ancestry groups. The quality of the two plaque accuracy figures, quoted only to show the review passes them through unexamined. And the China count in the geography chart is deliberately conservative, because several single-centre Chinese studies resist clean counting.
And Table S1, which nobody has yet opened. It would settle the question this whole guide turns on. Time ran out.