Schedules of reinforcement and resistance to extinction

Transcript

RyanIt's eleven at night. Maya tells herself she'll check her social media feed one last time and go to sleep. Nothing new. She puts the phone down. Two minutes later she picks it up again. Sometimes there's something new, sometimes there isn't. She never knows when.

SoheeAnd the never knowing is the whole point. That uncertainty is a reinforcement schedule, and it's remarkably good at keeping a behavior going.

RyanWelcome to EPPP Prep, the podcast. I'm Ryan.

SoheeI'm Sohee. Today, schedules of reinforcement. It's a topic with a lot of vocabulary, but it has a clean logic underneath. Once you see the logic, the four schedules almost name themselves.

RyanLet's start with the simplest schedule. Continuous reinforcement means the organism gets a reinforcer every time it shows the behavior. The OpenStax psychology textbook calls it the quickest way to teach someone a behavior, especially a new one.

SoheeAnd timing matters. If you're teaching a dog to sit, you'll be most successful if you give the treat immediately after he sits, so he connects the behavior with the consequence.

RyanThen, once a behavior is trained, trainers often switch to partial reinforcement, also called intermittent reinforcement. Now the behavior isn't reinforced every time.

SoheeAnd here's the logic. Partial schedules vary on two dimensions. First, fixed versus variable. Fixed means the number of responses, or the amount of time, between reinforcements is set and unchanging. Variable means it varies.

RyanSecond, interval versus ratio. Interval means the schedule is based on the time between reinforcements. Ratio means it's based on the number of responses.

SoheeSo a memory hook. Ratio is about responses, how many times you do it. Interval is about the clock. Fixed is predictable. Variable is unpredictable. Two by two, four schedules.

RyanFixed interval first. Behavior is rewarded after a set amount of time. The textbook's example is June, recovering from surgery with a patient-controlled painkiller. The doctor sets a limit of one dose per hour.

SoheeSo when June presses the button doesn't matter much until the hour is up. In the textbook's words, there's no point in the behavior when it won't be rewarded.

RyanAnd that shows up in the response pattern. Fixed interval produces a scallop-shaped pattern, with a significant pause after each reinforcement. The textbook's table describes it as a moderate response rate with significant pauses after reinforcement.

SoheeVariable interval. Reinforcement comes after varying, unpredictable amounts of time. Manuel manages a fast-food restaurant. Every once in a while, someone from quality control shows up, and if the place is clean and the service is fast, everyone on that shift earns a twenty-dollar bonus.

RyanManuel never knows when the inspector will come. So he always keeps the restaurant clean and the service prompt. His effort is steady.

SoheeVariable interval gives a moderate, steady response rate. And the textbook's table lists checking social media as its example. That's Maya at eleven at night.

RyanFixed ratio. A set number of responses must occur before reinforcement. Piecework is the classic, a factory worker paid for every set number of items made. The textbook's example is Carla, who earns a commission every time she sells a pair of glasses.

SoheeFixed ratio produces a high response rate, with a short pause after reinforcement. And notice a trade-off the textbook points out. Carla's commission is based only on the number of pairs sold, not the quality of what she sells.

RyanSo fixed ratios are better suited to maximize the quantity of output. A fixed interval, where the reward isn't based on quantity, can lead to higher quality of output.

SoheeAnd the fourth, variable ratio. The number of responses needed for a reward varies. The textbook calls it the most powerful partial reinforcement schedule, and its example is gambling.

RyanSarah visits Las Vegas for the first time, not a gambler at all, and feeds quarters into a slot machine out of curiosity. Nothing. She's about to quit. Then the machine lights up and pays out fifty quarters.

SoheeA few minutes later she's used up her gains and she's ten dollars in the hole. And she keeps going, because she never knows when the next reinforcement is coming.

RyanVariable ratio yields high, steady response rates, with little if any pause after reinforcement. The textbook calls this one of the reasons gambling is so addictive, and so resistant to extinction.

SoheeWhich brings us to extinction in operant conditioning. Extinction of a reinforced behavior happens at some point after reinforcement stops. And how fast it happens depends on the schedule.

RyanOn a variable ratio schedule, extinction comes very slowly. On the other schedules it may come quickly. If June presses the button before the hour is up and gets nothing, extinction is quick, because reinforcement didn't come when expected.

SoheeSo the textbook's bottom line, and it's a good one to memorize. Among the schedules, variable ratio is the most productive and the most resistant to extinction. Fixed interval is the least productive and the easiest to extinguish.

RyanPause and think. A farmworker is paid a set amount for every bushel of apples picked. Which schedule, and what response pattern would you expect?

SoheePayment depends on the number of responses, bushels, and the number is set. Fixed ratio. Expect a high response rate, with a short pause after each payment.

RyanHere's one with real clinical weight. Skinner himself used gambling to show the power of variable ratio schedules. He wrote that if a gambling establishment can't persuade a patron to hand over money with no return, it can get the same effect by returning part of the money on a variable ratio schedule.

SoheeAnd the textbook brings it home to parenting. Imagine how often a child would have tantrums if a parent gives in even once. The occasional reward makes the behavior almost impossible to stop.

RyanPause and think on that. A parent usually ignores tantrums in the grocery store, but every so often, worn out, buys the candy. Which schedule is the child on, and what does it predict?

SoheeThe number of tantrums before a payoff varies, so it's variable ratio. And that predicts high, persistent tantrums that are very resistant to extinction. Giving in only sometimes can make the behavior very hard to stop.

RyanNow, a careful note on the science. Skinner even claimed he could turn a pigeon into a pathological gambler. But recent research in rats has failed to support the idea that variable ratio training alone causes pathological gambling.

SoheeOther research links gambling to activation of the brain's reward centers that use dopamine. And near misses, almost winning but not winning, have been shown to increase activity in the ventral striatum and other dopamine reward centers.

RyanBut the textbook is honest about the limits. You can't run a true experiment, because turning randomly assigned people into problem gamblers would be unethical. So causation could run the other way. Gambling itself might change neurotransmitter levels.

SoheeSo the fair summary is that some combination of brain chemistry and reinforcement schedule may lead to problem gambling. Schedules matter, but they aren't the whole story.

RyanSo how might all this be tested? Often you'll get a short scenario and need to name the schedule. Ask two questions. Is the payoff tied to responses or to time? Is it predictable or not?

SoheeAnd watch for the tempting mix-ups. Someone paid on a set time, like a regular paycheck, is on an interval schedule, not a ratio one, because working faster doesn't bring the reward sooner. And unpredictable doesn't automatically mean ratio. Manuel's inspector is unpredictable, but it's about time. Variable interval.

RyanThen the comparison questions. Fastest acquisition? Continuous. Most resistant to extinction? Variable ratio. Easiest to extinguish? Fixed interval. A scalloped pattern with a pause after reinforcement? Fixed interval.

SoheeLet's recap. Continuous reinforcement teaches a new behavior fastest. Partial reinforcement varies on fixed versus variable, and interval versus ratio.

RyanFixed interval: a set time, a scalloped pattern with significant pauses, the least productive and easiest to extinguish.

SoheeVariable interval: unpredictable time, a moderate, steady rate. The inspector, or the social media feed.

RyanFixed ratio: a set number of responses, a high rate with a short pause. Piecework and commission, good for quantity.

SoheeVariable ratio: an unpredictable number of responses, a high, steady rate, the most productive and the most resistant to extinction. Slot machines, and tantrums that sometimes pay off.

RyanThis topic has a lot of labels, and it's normal if they slip at first. But the logic underneath never changes. Responses or time. Predictable or not. You've got this.

SoheeGo try a few practice questions, maybe on a schedule of your own choosing. See you next time on EPPP Prep.

Loading the read-along timings…