Base rates: incidence, prevalence and the surprising positive screen

Transcript

SoheeA college counseling center decides to screen every incoming student, all 10,000 of them, for a condition that about 1 percent of students have. The screen is good. It correctly identifies 90 percent of students who have the condition, and 90 percent of students who don't.

RyanAnd the results come back with about 1,080 positive screens.

SoheeSo, how many of those 1,080 students actually have the condition?

RyanAbout 90. Roughly one in twelve. The other thousand or so are false alarms, from a test that is working exactly as designed.

SoheeWelcome to EPPP Prep, the podcast. I'm Sohee.

RyanI'm Ryan. Last time we did sensitivity and specificity. Today we add the missing ingredient: the base rate, how common a condition is in the group you're testing. And that takes us into epidemiology, so let's get the vocabulary right first.

SoheeStart with incidence. The National Library of Medicine puts it plainly: incidence refers to the number of new cases of disease in a population. The epidemiology glossary from the Centers for Disease Control and Prevention adds the time window: how often a new case occurs in a population over a period of time.

RyanThe key word is new. People who already had the condition before the period started aren't counted.

SoheePrevalence is different. It's the proportion of a population with a condition. Point prevalence counts all existing cases, new and pre-existing, at a single point in time. Period prevalence counts them over a stretch of time.

RyanAnd then the two you'll see in mental health statistics. Lifetime prevalence, which OpenStax defines as the percentage of people in a population who develop a disorder in their lifetime. And past-year prevalence, people who had the disorder at any point in the past year.

SoheeThe National Institute of Mental Health gives a great pair of numbers for anxiety disorders, from diagnostic interviews in the National Comorbidity Survey Replication. An estimated 19.1 percent of United States adults had any anxiety disorder in the past year. And an estimated 31.1 percent experience one at some time in their lives.

RyanSame people, same disorders, different time windows. That's why the lifetime number is bigger. It includes everyone who ever had it, even if they recovered long ago.

SoheeThose statistics also show group differences, which the knowledge statements ask about. Past-year prevalence of any anxiety disorder was higher for females, 23.4 percent, than for males, 14.3 percent.

RyanAnd it isn't only adults. For adolescents aged 13 to 18, the institute reports a lifetime prevalence of any anxiety disorder of an estimated 31.9 percent.

RyanAnd for a sense of scale, OpenStax reports a finding that nearly half of all Americans will meet the criteria for a disorder in the American diagnostic manual at some point in their lives. The textbook adds that this finding likely fuels skepticism that new diagnoses might be turning common human problems into mental illnesses.

SoheeHere's a subtle one the National Library of Medicine points out. Prevalence can actually rise even if incidence stays the same or falls. That generally happens when people begin to live longer with a disease.

RyanHere's my own way to picture it: new cases are water flowing into a bathtub. Prevalence is how much water is in the tub. If the drain slows, the tub fills even with the same flow.

SoheeSo rising prevalence doesn't automatically mean a condition is spreading. It can mean people are living longer with it.

SoheeOne more term: relative risk. The glossary defines it as a comparison of the risk of some health-related event, such as disease, in two groups.

RyanPause and think. A researcher asks a large community sample whether they have ever, at any time in their lives, had a particular disorder, and counts everyone who has, including people who recovered decades ago. What is she estimating?

SoheeLifetime prevalence. Ever, at any time in life, means lifetime, recovered or not. If she'd counted only people who developed it for the first time during the study year, that would be incidence.

RyanSo in these questions, the time window is your clue. Ever means lifetime. In the past year means past-year. At one moment means point prevalence. And only new cases means incidence.

RyanNow back to our counseling center, and let's walk the numbers slowly, in words.

SoheeTen thousand students. One percent have the condition. So 100 have it, and 9,900 don't.

RyanOf the 100 who have it, the screen catches 90 percent. That's 90 true positives. Ten are missed.

SoheeOf the 9,900 who don't have it, the screen correctly clears 90 percent. Which means 10 percent get flagged anyway. Ten percent of 9,900 is 990 false positives.

RyanAdd them up. Ninety true positives plus 990 false positives is 1,080 positive screens. Only 90 of them are real.

SoheeNothing is wrong with the test. The problem is arithmetic. When almost everyone being screened is healthy, even a small false positive rate, applied to a huge healthy group, swamps the true cases.

RyanParikh and colleagues show the same thing with a medical test. Their new test had 90 percent sensitivity and 95 percent specificity, measured in a sample where half the people had the disease. In that sample, the positive predictive value was 95 percent.

SoheeThen they apply it to a million people where only 1 percent have the disease. The positive predictive value drops to 15 percent. In their words, although the sensitivity and specificity of the test have not changed, the positive predictive value has changed drastically.

RyanAnd remember our example from the last episode, where exactly half the people had the condition? Parikh's test was checked in that same kind of group, half with the disease. The trouble starts when a test moves from a group like that into a population where the condition is rare.

SoheeThe rule from Parikh: assuming all other factors remain constant, positive predictive value increases with increasing prevalence. And negative predictive value moves the other way. In their low-prevalence example, it rose to 99 percent.

RyanSo, here's how the exam builds these. A screening questionnaire for bipolar disorder was validated in a specialty mood clinic, where most patients had the disorder. A primary care practice, where few patients have it, adopts the screener. A follow-up study confirms its sensitivity and specificity are the same there. Scoring is automated. Yet most patients who screen positive don't have bipolar disorder. Why?

SoheeThe low base rate in primary care. And look how the stem shuts the other doors. Sensitivity and specificity are confirmed unchanged, so a drop in either is ruled out. Scoring is automated, so inconsistent scoring is ruled out. What's left is prevalence.

RyanSecond pause and think. A county jail screens new admissions with a computer-scored screener for serious mental illness. It was validated on psychiatric inpatients, nearly all of whom had serious mental illness. In the jail, the illness is far less common, and research there shows the screener's accuracy is unchanged. Most admissions who screen positive don't have the illness. Most likely reason?

SoheeThe lower base rate among jail admissions has lowered the positive predictive value. Same test, different population.

RyanWhy does this matter so much clinically? Parikh warns that when prevalence is low, false positives can far outnumber true positives, which leads to problems like labeling healthy people as ill and unnecessary treatment.

SoheeAnd Trevethan stresses making sure the people a test was studied on match the people being screened. So in practice, a positive screen in a low base rate setting is a reason for a closer look, not a diagnosis.

RyanWhich gives you a habit for every score you ever interpret. Before asking what this result means, ask how common the condition is in the people sitting in front of you. A specialty clinic, a primary care waiting room, and a freshman orientation can look completely different on that question.

RyanLet's recap. Incidence counts new cases over a period. Prevalence counts all existing cases, at a point in time, over a period, in the past year, or over a lifetime.

SoheeFor anxiety disorders, about 19 percent of United States adults in the past year, about 31 percent across a lifetime, with past-year rates higher in females than males.

RyanPrevalence can rise when people live longer with a condition, even if incidence holds steady. Relative risk compares two groups.

SoheeAnd base rates drive predictive value. Keep sensitivity and specificity constant, lower the prevalence, and positive predictive value falls, so more positive results are false alarms.

RyanIf you had to replay that worked example in your head a couple of times, good. That's exactly how this sinks in. Try sketching it on scrap paper: 10,000 people, split them, then split again.

SoheeOnce you can rebuild it yourself, base rate questions become some of the most satisfying ones on the test. Take a breather, and we'll see you next time on EPPP Prep.

Loading the read-along timings…