Case studies, correlations and studying change over time

Transcript

SoheeAsk almost anyone who has worked a night shift in an emergency room, and you may hear the same story. Full moon nights are chaos. People act strange.

RyanIt feels true. You can probably picture a specific wild night.

SoheeWelcome to EPPP Prep, the podcast. I'm Sohee.

RyanAnd I'm Ryan. Today we're looking at the research designs that observe and measure without manipulating anything: case studies, correlational studies, and studies of change over time. And we'll come back to that full moon.

SoheeLet's frame it with one sentence from the OpenStax psychology textbook. After describing case studies, observation, surveys and archival research, it says all of these methods are correlational in nature.

RyanMeaning they can show important relationships between variables. But the same passage says correlational data cannot be used to make claims about cause-and-effect relationships.

SoheeHold on to that, because it's the heart of the exam questions in this area. Let's start with the most intimate design, the case study.

RyanThe textbook says researchers are conducting a clinical or case study when they focus on one person or a few individuals. Some scientists spend whole careers studying ten to twenty people.

SoheeThe example in the book is Krista and Tatiana Hogan, conjoined twins connected at the head. There's evidence they're connected in the thalamus, a major sensory relay center.

RyanSo researchers wondered whether one twin might experience the other's sensations. Over time it became clear they share some sensory experiences and motor control, yet remain two distinct individuals.

SoheeAnd you can see the strength immediately. The textbook says the richness of information in case studies is unmatched by any other single research method.

RyanAnd the weakness is the same feature, turned around. Case studies often focus on people with rare characteristics, so they're not like most other people. That makes it hard to generalize.

SoheeGeneralizing, in the textbook's words, is the ability to apply the findings of a particular study to larger segments of society.

RyanThere's a reasoning angle here too. The textbook links case studies to inductive reasoning: gathering lots of observations and looking for patterns, which become new ideas.

SoheeWhile experiments lean on deductive reasoning, testing a hypothesis against the world. So case studies are great for generating hypotheses, not for confirming causes.

RyanNow to correlation. The textbook defines it as a relationship between two or more variables, and adds that this relationship does not necessarily imply cause and effect.

SoheeWe measure it with a correlation coefficient, usually written as the letter r. It runs from negative one to positive one.

RyanAnd r carries two separate pieces of information. The number tells you strength. The sign tells you direction.

SoheeStrength first. The closer the number is to one, whether positive or negative, the more strongly related the variables are, and the more predictable one is from the other. The textbook's example: 0.9 is a far stronger relationship than 0.3.

RyanDirection second. Positive means the variables move together. Height and weight, or age and number of wrinkles. Negative means they move in opposite directions.

SoheeThe book's real example of a negative correlation comes from student researchers at the University of Minnesota. They found a weak negative correlation, r equals negative 0.29, between how many days a week students got fewer than five hours of sleep, and their grade point average.

RyanAnd the book stresses that a negative correlation is not the same as no correlation. Hours of sleep and shoe size would probably show no correlation at all.

SoheeIf you picture a scatterplot, the stronger the correlation, the closer the dots sit to a straight line.

RyanOkay, pause and think. Which shows the stronger relationship: a correlation of negative 0.70, or positive 0.40?

SoheeNegative 0.70. The sign only tells you direction. Strength is how close the number is to one, and 0.70 is closer than 0.40. Keep that in mind whenever two correlations are compared.

RyanHere's a second, subtler trap. The OpenStax statistics textbook warns that a correlation of zero means there's likely no linear relationship. But data with a curved pattern can also produce a correlation near zero.

SoheeSo r measures straight-line relationships. Always look at the scatterplot.

RyanNow the big one. Why can't a correlation tell us about cause? The psychology textbook opens the topic with ice cream. As ice cream sales go up, so does the crime rate.

SoheeIt even jokes about the reverse direction, committing a crime and then treating yourself to a cone.

RyanThe much more likely story is temperature. When it's warm, more people are out and interacting, and sometimes committing crimes. And when it's warm, people want ice cream.

SoheeThe textbook calls temperature a confounding variable, some other factor that's actually causing the systematic movement in both variables.

RyanAnd it's careful to add that even when you can't point to a clear confound, you still shouldn't assume a correlation means one variable causes the other.

SoheeIts example is breakfast cereal. Research found that people who eat a certain cereal may have a reduced risk of heart disease. Does cereal cause better health, or are there other explanations for the health of people who eat it?

RyanAnd the textbook's bottom line: the only way to establish cause and effect is to conduct an experiment.

SoheeThat doesn't make correlations useless. The book says correlations have predictive value. An admissions committee can use the relationship between test scores and college grades to predict who's likely to succeed.

RyanNow back to the full moon. This is what the textbook calls an illusory correlation: believing a relationship exists when it doesn't.

SoheeIt cites a meta-analysis of nearly 40 studies that consistently showed the relationship between the moon and our behavior does not exist. We may pay more attention to odd behavior on full-moon nights, but the rates of odd behavior stay constant across the lunar cycle.

RyanAnd why do we believe it? Partly confirmation bias, looking for evidence that supports a hunch and ignoring evidence against it. That wild night is memorable. The quiet full moons fade away.

SoheeThe textbook also notes that illusory correlations, where behaviors are wrongly attributed to certain groups, are involved in forming prejudiced attitudes. So this isn't only a statistics problem.

RyanLast design family for today: studying change over time. Two classic options.

SoheeLongitudinal research tests the same group of people repeatedly over an extended period. The book's example is surveying people about their diet at 20, again at 30, and again at 40.

RyanCross-sectional research compares different groups at the same time. You'd study 20-year-olds, 30-year-olds and 40-year-olds all at once.

SoheeCross-sectional is faster. But the textbook names its weakness: differences between generations, or cohorts, that have nothing to do with age itself, and instead reflect different social and cultural experiences.

RyanIts example: younger people tend to be more supportive of same-sex marriage than older people. Does that mean we become less open as we age? Or that older people grew up in a different social climate?

SoheeA cross-sectional design can't separate those. Longitudinal research follows the same people, so cohort differences are much less of a concern.

RyanBut longitudinal studies have their own costs. They take enormous time and money, results may not be known for years or decades, and many researchers can't commit the resources to finish.

SoheeAnd people drop out. They move, get ill, or choose to stop. The textbook calls this attrition, the reduction in the number of participants due to dropouts, and says attrition rates in longitudinal studies are quite high and increase over time.

RyanWhich is why researchers recruit many more participants than they need, and keep checking whether the sample still represents the population.

SoheePause and think, second one. A researcher surveys 25-year-olds and 70-year-olds on the same day and finds the older group uses far fewer smartphone apps. What's the biggest problem with concluding that aging reduces app use?

RyanIt's cross-sectional, so cohort differences are confounded with age. The two groups grew up with very different technology. Following the same people over time would separate aging from generation.

SoheeSo how does the exam test this? Expect a study description and the question: what can the researcher conclude?

RyanThe tempting wrong answers claim a cause from correlational data, treat a negative correlation as weak or as no relationship, or treat a cross-sectional age difference as proof of a developmental change.

SoheeRecap time. Case studies give unmatched depth about one or a few people, generate hypotheses, and are hard to generalize from.

RyanA correlation coefficient runs from negative one to positive one. Its size is strength, its sign is direction, and it describes straight-line relationships.

SoheeCorrelation does not establish cause. A confounding variable may drive both. Only an experiment establishes cause and effect.

RyanIllusory correlations, like the full moon, feel real because of confirmation bias.

SoheeLongitudinal designs follow the same people, so cohort differences are much less of a worry, but they face cost and attrition. Cross-sectional designs are quick but mix age with generation.

RyanIf you've ever second-guessed yourself on a correlation question, that's very normal. The sign trick catches a lot of careful people. Now you have a rule you can trust.

SoheeNext time you see a study in the news, ask yourself which design it used and what it can really claim. That habit is exam practice too. Thanks for listening to EPPP Prep.

Loading the read-along timings…