Skip to main content

Statistics is hard, especially if you don’t know any statistics (FDA edition)

Paul Alper shares this story:

From the NYT:

Dr. Stephen M. Hahn, the commissioner of the Food and Drug Administration, said 35 out of 100 Covid-19 patients “would have been saved because of the administration of plasma.”

He later walked this back because of confusion between Absolute Risk Reduction and Relative Risk Reduction, a common error usually promoted by drug manufacturers because relative improvement appears more dramatic to the beholder.

He [Hahn] clarified that his earlier statements suggested an absolute reduction in risk, instead of the relative risk of a certain group of patients compared with another.

The chart, analyzing the same tiny subset of Mayo Clinic study patients, did not include numerical figures, but it appeared to indicate a 30-day survival probability of about 63 percent in patients who received plasma with a low level of antibodies, compared with about 76 percent in those who received a high level of antibodies.

From the FDA:

“there appears to be roughly a 35 percent relative improvement in the survival rates of patients” who received the plasma with higher versus lower levels of antibodies.

As best as I [Alper] can figure out, the absolute risk reduction is

.37-.24 = .13

The relative risk reduction is

(.37-.24)/ .37 = .35

The number needed to treat, a figure of merit which is often omitted from discussion, is

1/.13 = 7.69

Statisticians and scientists said that Dr. Hahn, in saying at the news conference that 35 out of 100 sick Covid-19 patients would have been saved by receiving plasma, appeared to have overstated the benefits.

I looked up Hahn on the internet and he’s an oncologist:

Hahn completed an internal medicine residency at the University of California, San Francisco School of Medicine where he eventually served as chief resident before embarking on a fellowship in medical oncology at the National Institutes of Health.

After completing his fellowship, Hahn worked as a medical oncologist in Santa Rosa, California. He was then recruited by his mentor, Dr. Eli J. Glatstein to complete a separate residency in radiation oncology at the NIH between 1991 and 1994, where he eventually attained the rank of commander in the U.S. Public Health Service Commissioned Corps between 1989–1995. During the period of 1993–1999, he served as chief of NCI’s Prostate Cancer Clinic in the Clinical Pharmacology Branch . . .

I don’t see any formal statistics training. That’s fine! Alper and I don’t have any oncology training and here we are posting on this. But in any case the commissioner of the FDA might well too busy to be carefully reading the individual studies. I assume the fault is whatever assistant prepared the numbers for him.

The story with the “A/Chairman @WhiteHouseCEA” was worse, because that guy didn’t just mess up some numbers, he went all-in on the attack. The FDA story is embarrassing, but such things happen. Whoever prepared the FDA commissioner’s briefing must feel pretty bad about this one.



from Statistical Modeling, Causal Inference, and Social Science https://ift.tt/2FYwSYH
via IFTTT
Labels:

Comments

Post a Comment

Popular posts from this blog

Controlling legend appearance in ggplot2 with override.aes

[This article was first published on Very statisticious on Very statisticious , and kindly contributed to R-bloggers ]. (You can report issue about the content on this page here ) Want to share your content on R-bloggers? click here if you have a blog, or here if you don't. In ggplot2 , aesthetics and their scale_*() functions change both the plot appearance and the plot legend appearance simultaneously. The override.aes argument in guide_legend() allows the user to change only the legend appearance without affecting the rest of the plot. This is useful for making the legend more readable or for creating certain types of combined legends. In this post I’ll first introduce override.aes with a basic example and then go through three additional plotting scenarios to how other instances where override.aes comes in handy. Table of Contents R packages Introducing override.aes Adding a guides() layer Using the guide argument in scale_*() Changing multiple aesthetic par...
Post a Comment
Read more

Zooming in on maps with sf and ggplot2

(This article was first published on r-bloggers – WZB Data Science Blog , and kindly contributed to R-bloggers) When working with geo-spatial data in R, I usually use the sf package for manipulating spatial data as Simple Features objects and ggplot2 with geom_sf for visualizing these data. One thing that comes up regularly is “zooming in” on a certain region of interest, i.e. displaying a certain map detail. There are several ways to do so. Three common options are: selecting only certain areas of interest from the spatial dataset (e.g. only certain countries / continent(s) / etc.) cropping the geometries in the spatial dataset using sf_crop() restricting the display window via coord_sf() I will show the advantages and disadvantages of these options and especially focus on how to zoom in on a certain point of interest at a specific “zoom level”. We will see how to calculate the coordinates of the display window or “bounding box” around this zoom point. Loading t...
Post a Comment
Read more