The Monty Hall Problem is cognitive Three-Card Monte

The Monty Hall Problem is a rather famous brain teaser, a rare mathematical puzzle that entered pop culture so successfully it was featured in a movie. I’m not that interested in providing yet another explanation of its counterintuitive correct answer; there are dozens on the Internet, for example here and here and here and here and here.

What fascinates me is why so many smart people get the answer wrong: I’ll admit, the first time I encountered it, I picked the intuitive, incorrect answer. When The Problem first became famous in a newspaper column in 1990, over 1,000 Ph.D.s wrote letters to argue argue that the correct solution was, in fact, wrong; one world famous mathematician took a lot of convincing before he came around. Like most people when their backs up are against a wall, they dig their heels in, and when they are finally shown their error to their satisfaction (often with the help of computer simulations), they grumble that the question was posed ambiguously (for example, it doesn’t explicitly state that space aliens aren’t manipulating Monty Hall’s behaviour… okay, I’m exaggerating, but that’s the tenor of it).
Here’s a quick rundown of The Problem. You’re on a game show (presumably Let’s Make a Deal, which Monty Hall hosted off and on for over 30 years), and there are three doors: you get whatever is behind the door you choose. One has a car, the other two have goats. (The delicious cheese isn’t a bad consolation prize, if you ask me.)
You make a preliminary choice, then Monty opens one of the other two doors to reveal a goat. You can either stick with your original choice or choose the other unopened door. Should you change? Should you stay? Does it matter?

Short answer: it matters. You should change to the other door, you double your chances of winning the car. Most people instinctively think it doesn’t matter whether they stick or change.Once it’s explained, it takes a while to process, but eventually it’s like an optical illusion: you couldn’t see it at all at first, and now you can’t not see it, and it’s difficult to imagine why other people don’t see it too. (Again, I’m not going to get into the explanation, other people have done it far more thoroughly than I ever could, click the links above or Google it if you need it proven to you).
Studies have shown that psychology plays a factor in perception (people want to stick with their first choice for emotional reasons), and I’ll buy that, but it doesn’t explain why really, really smart people are so prone to making fools of themselves trying to prove a fallacy.

Actually, I think the answer is rather simple: it’s a mental card trick. The Problem is like a short con artist playing Three-Card Monte, making you think the queen is moving when it’s staying in one hand. The problem has the appearance of randomness when it is anything but: the participation of Monty Hall himself in the Monty Hall Problem is what changes the odds.

If you first pick a door with a goat behind it, you don’t know what you’ve just picked — but Monty does. He has to, otherwise he has a 50% chance of ruining the game by revealing the car. (Unless that makes you win the car, but that would be a different game, and much less of a Problem in both senses of the word). Here is the kicker: Monty is constrained in what door he opens for you; it is not a random selection. He has to open a door with a goat: by doing so, he has passed information to you and changed the odds of the game. He has the appearance of a random player, but he is anything but.

(At this point, most people still don’t believe Monty has passed information to the player. He has, I guarantee it, Google it and eventually your brain will warp enough to understand how. I’ll dispel the most common objection right now: if you picked the door with the car first, Monty’s choice is indeed random, but you’ve only got a 1 in 3 chance of that happening. Two out of three times, Monty is 100% constrained, which means overall he’s 66% constrained. Now back to your regular programming.)

There’s been a lot of academic attention on Bayesian probability in the past decade, for good reason: it works. The Monty Hall problem is a textbook (literally) demonstration as to the value of Bayes’ theorem. I won’t get into the details, I’ll just pass along the best analogy of the Bayesian approach I ever heard: If you toss a coin and get heads nine times in a row, a traditional mathematician will say the chance of a head on the next toss is 50% because each toss is an independent event. A Bayesian mathematician will say the chance of a head on the next toss is just about 100%, because given the fact that you got nine heads in a row, it’s a virtual certainty that you’re using a two-headed coin.