Paul Alper shares this story:
From the NYT:
Dr. Stephen M. Hahn, the commissioner of the Food and Drug Administration, said 35 out of 100 Covid-19 patients “would have been saved because of the administration of plasma.”
He later walked this back because of confusion between Absolute Risk Reduction and Relative Risk Reduction, a common error usually promoted by drug manufacturers because relative improvement appears more dramatic to the beholder.
He [Hahn] clarified that his earlier statements suggested an absolute reduction in risk, instead of the relative risk of a certain group of patients compared with another.
The chart, analyzing the same tiny subset of Mayo Clinic study patients, did not include numerical figures, but it appeared to indicate a 30-day survival probability of about 63 percent in patients who received plasma with a low level of antibodies, compared with about 76 percent in those who received a high level of antibodies.
From the FDA:
“there appears to be roughly a 35 percent relative improvement in the survival rates of patients” who received the plasma with higher versus lower levels of antibodies.
As best as I [Alper] can figure out, the absolute risk reduction is
.37-.24 = .13
The relative risk reduction is
(.37-.24)/ .37 = .35
The number needed to treat, a figure of merit which is often omitted from discussion, is
1/.13 = 7.69
Statisticians and scientists said that Dr. Hahn, in saying at the news conference that 35 out of 100 sick Covid-19 patients would have been saved by receiving plasma, appeared to have overstated the benefits.
I looked up Hahn on the internet and he’s an oncologist:
Hahn completed an internal medicine residency at the University of California, San Francisco School of Medicine where he eventually served as chief resident before embarking on a fellowship in medical oncology at the National Institutes of Health.
After completing his fellowship, Hahn worked as a medical oncologist in Santa Rosa, California. He was then recruited by his mentor, Dr. Eli J. Glatstein to complete a separate residency in radiation oncology at the NIH between 1991 and 1994, where he eventually attained the rank of commander in the U.S. Public Health Service Commissioned Corps between 1989–1995. During the period of 1993–1999, he served as chief of NCI’s Prostate Cancer Clinic in the Clinical Pharmacology Branch . . .
I don’t see any formal statistics training. That’s fine! Alper and I don’t have any oncology training and here we are posting on this. But in any case the commissioner of the FDA might well too busy to be carefully reading the individual studies. I assume the fault is whatever assistant prepared the numbers for him.
The story with the “A/Chairman @WhiteHouseCEA” was worse, because that guy didn’t just mess up some numbers, he went all-in on the attack. The FDA story is embarrassing, but such things happen. Whoever prepared the FDA commissioner’s briefing must feel pretty bad about this one.
from Statistical Modeling, Causal Inference, and Social Science https://ift.tt/2FYwSYH
via IFTTT
Comments
Post a Comment