Card 07

🎯 🔄 💥

When might paying people to quit smoking increase smoking?

💭 How to Think About This

Goal: Reduce smoking. Solution: Pay people to quit! But what if people START smoking just to get paid to quit? This is a PERVERSE INCENTIVE - when a well-intentioned reward creates the opposite behavior you wanted!

Can incentives be designed to avoid backfiring?

Good incentives reward actual outcomes, not proxies. Instead of paying for grades, reward demonstrated mastery. Instead of counting sales calls, measure customer satisfaction. When incentive and goal point the same direction, gaming becomes helping.

Use multiple metrics that are hard to game simultaneously. A hospital can track wait times, patient outcomes, and satisfaction. Gaming one while maintaining others is difficult. Balanced scorecards prevent single-metric optimization.

Before implementing, ask: "How would I game this?" Find the loopholes yourself. What's the easiest path to the reward? What behavior am I actually rewarding? Red team your own incentive design. If you can't find exploits, ask someone else to try.

People are creative at gaming systems. As soon as you close one loophole, they find another. The cobra effect, rat bounty, and countless corporate examples show that even well-designed incentives get gamed eventually. Is perfect incentive design possible, or is it an arms race?

Good design reduces perverse incentives - but perfect design may be impossible since people constantly find new ways to game systems!

Design principles that help:

• Align incentives with actual outcomes, not proxies

• Use multiple measures that are hard to game simultaneously

• Think like a gamer: find exploits before implementing

• Build in adaptability: adjust when gaming emerges

The limitation:

• Human creativity at gaming exceeds designer foresight

• Complex systems have unexpected loopholes

• Good design reduces but doesn't eliminate gaming

The insight: Design carefully AND expect to adapt. Incentive design is ongoing, not one-time. Monitor for gaming and adjust continuously.

"When a measure becomes a target, it ceases to be a good measure." Any metric that matters will attract gaming. The act of measuring changes what's being measured. This isn't failure - it's a fundamental property of incentive systems.

Incentive design is an arms race. Designer creates incentive → people find loopholes → designer closes loopholes → people find new ones. Tax codes get longer every year. Anti-cheating software gets more complex. The race never ends - you can only stay ahead for a while.

Good enough beats perfect. Some gaming is acceptable if the incentive mostly works. A speed limit that 90% of people follow is better than no limit. An imperfect bonus system that mostly motivates is better than no motivation. Manage gaming, don't try to eliminate it.

The best systems expect gaming and adapt. Monitor for unintended behaviors. Adjust incentives when gaming emerges. Use random audits. Sunset clauses force re-evaluation. Design for evolution, not perfection. The question isn't "can I prevent all gaming?" but "can I adapt faster than gamers?"

Gaming is inevitable (Goodhart's Law), but adaptive design can stay ahead - it's an arms race, not a one-time fix!

Why perfect prevention is impossible:

• When measure becomes target, it ceases to be good measure

• Human creativity at gaming exceeds designer foresight

• Closing loopholes creates new ones

How to stay ahead:

• Monitor for unintended behaviors continuously

• Adjust incentives when gaming emerges

• Use multiple measures and random audits

• Accept "good enough" over perfect

The insight: Design for adaptation, not perfection. The question is "can I adapt faster than gamers?" not "can I prevent all gaming?"

People are optimizers. Given any incentive, they'll find the easiest path to the reward. This isn't cynicism - it's human nature. We evolved to get maximum reward for minimum effort. Any system that ignores this will fail. Gaming isn't a bug, it's a feature of human behavior.

Real systems are too complex to model. There are always interactions and loopholes the designer didn't anticipate. The cobra effect, rat bounty, and Wells Fargo scandals weren't caused by stupid designers - they were caused by complexity exceeding human foresight. You can't design for what you can't predict.

The act of measuring changes behavior. Once people know what's measured, they optimize for it. Teach to the test. Game the algorithm. Hit the metrics. The only way to avoid this is to not measure - but then you can't create incentives at all. Measurement is inherently corrupting.

Some incentives work well enough. Piece-rate pay increases output. Sales commissions motivate selling. Speed limits reduce speeding (mostly). Not all incentives produce cobra effects. Maybe the question isn't "will this be gamed?" but "is this still useful despite gaming?" Perfect isn't required for useful.

All incentives get gamed eventually - but many are still useful despite imperfect behavior!

Why gaming is inevitable:

• Human nature is to optimize for reward

• Complexity exceeds designer foresight

• Measurement itself changes behavior (Goodhart's Law)

What's still possible:

• Useful incentives despite some gaming

• "Good enough" outcomes even if not perfect

• Adaptation faster than gaming evolves

The reframe: The question isn't "will this be gamed?" (yes, always) but "is this still useful despite gaming?" Many imperfect incentives create net positive behavior.

🤔 Which thinking lens(es) did you use?

Select all the lenses you used:

Learn more about the 7 Thinking Lenses →

👨‍👩‍👧 For Parents & Teachers

🌱 A Small Everyday Story

A reward is offered.
"Do this, get that."
Someone finds a shortcut.
The reward is claimed.
The goal remains unmet.
The system adapts.

See more guidance →

🧠 Thinking habits this builds:

Recognizing that incentives can produce unintended consequences
Understanding that people optimize for what's measured, not what's intended
Seeing how systems adapt around interventions
Designing incentives that align behavior with goals

🌿 Behaviors you may notice (and reinforce):

"Wait, what are we actually rewarding here?" questions
Spotting perverse incentives in daily life (school, work, media)
Designing better systems: "How could someone game this?"
Recognizing Goodhart's Law in action

How to reinforce: When they spot a perverse incentive, ask them to explain what behavior is actually being rewarded. Help them see the gap between intention and outcome.

🔄 When ideas are still forming:

Some learners may become cynical about all incentives, missing that well-designed incentives can work. Others may struggle to see how systems adapt, thinking solutions are permanent.

Helpful response: "What behavior is this actually rewarding? How could someone game it?" Help them think like a systems designer, not just a critic.

🔬 If you want to go deeper:

Analyze incentives in your own systems: school, family, community
Design better incentives: "How would you reward the actual goal?"
Explore: When do incentives work? When do they backfire?

Key concepts (for adults): Perverse incentives, Goodhart's Law, cobra effect, unintended consequences, incentive design, systems adaptation, measurement vs. goals.

When might paying people to quit smoking increase smoking?

📚 All Perspectives

Yes, With Careful Design

Sometimes, But Gaming Is Inevitable

No, Incentives Always Get Gamed

🤔 Which thinking lens(es) did you use?