Health records predicted who stays on buprenorphine only modestly, and less well when moved to a different set of records

The short version

In this study, about 6 in 10 treatment episodes on buprenorphine-naloxone ended within six months. Researchers asked whether a computer reading health records could tell, at the first prescription, which episodes would end early.

Inside one large database, the models did reasonably well. Moved to the other set of records, they did only modestly better than chance. Three addiction specialists, reviewing a sample of Stanford charts, scored in the same modest range.

What they did

The team used two sets of records. One came from Stanford's health system: 1,800 treatment episodes. The other came from NeuroBlu, a database of de-identified records from 166 U.S. mental health care sites, owned by the company Holmusk: 7,957 episodes. These are episodes, not people. One person can start and stop more than once.

Staying in treatment meant filling buprenorphine-naloxone prescriptions for at least 180 days with no gap longer than 30 days.

The models saw only what sits in structured fields at the first prescription: diagnoses, medications, procedures, and four demographic details. They were trained on earlier years and tested on later years, and then each site's models were tested on the other site's patients.

For a human benchmark, three board-certified addiction medicine physicians reviewed 200 Stanford charts up to the first prescription and predicted who would stay. After removing patients they already knew, 147 predictions were scored.

What it found

Stopping was common. Treatment ended within six months in 61% of Stanford episodes and 58% of NeuroBlu episodes. At Stanford, half of episodes ended by about two months. The paper's text gives a median of 64 days, and its abstract says 65.

Accuracy was scored with ROC-AUC, written here on a 0 to 100 scale, where 50 is a coin flip and 100 is perfect.

• Inside NeuroBlu, the best model scored 75.8.
• Inside Stanford, the best model scored 63.4.
• Trained on Stanford and tested on NeuroBlu, it scored 61.2. Trained on NeuroBlu and tested on Stanford, 60.2.
• The three physicians scored 67.8, with a wide margin of uncertainty (59.0 to 76.9). They were right about 61% of the time.

Read the comparison with the physicians carefully. The 75.8 was measured on NeuroBlu patients; the physicians judged Stanford patients. On Stanford's later test years, the best model scored 63.4, and models brought in from the other site scored about 60. The paper does not report how the models scored on the physicians' 147 charts. The authors describe the models as comparable to the physicians and more consistent. The numbers support "comparable." They do not show the models doing better on the same patients.

Some signals stood out. A coded diagnosis of opioid dependence went with staying longer. Signs of a hospital stay around the first prescription, such as IV fluids, went with stopping earlier. The authors warn these are markers, not causes. A prescription for extended-release oxycodone went with staying, for example. The authors suggest it may mark people who started buprenorphine for prescription opioid misuse rather than illicit opioid use, not that the prescription itself helped.

What it does not show

No one's care changed in this study. It shows how well records sort people after the fact, not whether flagging someone as likely to stop, and reaching out, keeps them in treatment.

The models saw only structured data. The physicians said they leaned on things the models could not see: attendance history, how closely a person was followed, involvement in support programs, and how engaged the person seemed in clinical notes.

The study includes everyone prescribed buprenorphine-naloxone, for any reason, and the authors say it cannot isolate opioid use disorder. A person who moved to another health system can look like a dropout; the physicians found that in fewer than 10% of the cases they reviewed.

The study was funded through the National Institute on Drug Abuse Clinical Trials Network, and one author works at the institute's clinical trials center. Four authors are employees of and hold equity in Holmusk Technologies, which owns the NeuroBlu data. The strongest result came from that database, and access to it requires a license from Holmusk.

Why it matters

Health systems are being offered tools that promise to flag patients at risk of dropping out of treatment. Even with thousands of records, predicting who stays on buprenorphine from the chart alone was hard, and accuracy fell when a model built on one set of records was tested on the other. For clinicians and programs weighing such a tool, two questions follow. Was it tested on patients like yours, from a system like yours? And was it compared with your own staff on the same patients?

For someone on buprenorphine, a chart does not decide whether you stay in treatment. If you are thinking about stopping, talk with your prescriber first. The authors note that the period after stopping treatment carries a high risk of death.