Mar 28, 2013 General Nonsense |

The famous Monty Hall problem in the field of statistics goes like this: Monty Hall is a game show host. You are given a choice of three doors. One has a car behind it, the other two have goats. If you pick the door with the car, you win it. Your odds are 1-in-3.

So you pick a door, but before it opens, Monty opens one of the other two doors to reveal a goat. He asks if you want to switch from the door you initially picked to the other closed door. Your brain says the odds are the same for any closed door, so you stay. But in fact, the odds are twice as good if you switch doors.

You can see the math of it here. But if you are normal, you'll never reconcile in your mind how one closed door could have better odds than the other. If there are two closed doors remaining, how can the odds be anything but 50-50?

This reminds me of the Schrodinger's Cat thought experiment in which a cat in a sealed box (presumably with air holes) exists in a state of being simultaneously alive and dead depending on the results of a randomized event happening inside the box. How can a cat be alive and dead at the same time? Math says it can happen, my brain says no.

The pattern recognition part of my brain is connecting the Monty Hall problem with the Shrodinger's Cat thought experiment because both situations feel like proof that our brains are not equipped to understand reality at its most basic level.

Most of us accept the idea that math is a better indicator of truth than our buggy personal perceptions. Math doesn't lie, but our brains are huge scam artists. The Monty Hall problem and Schrodinger's Cat are examples in which our perceptions of reality and the math of reality disagree in a big way. It makes me wonder how much of the rest of my so-called reality disagrees with math without me knowing.

If I were programming a computer simulation full of artificial humans who believe they are real, I would need to take some shortcuts in creating their context and history. It would be nearly impossible to invent consistent histories for seven billion people spanning back to the primordial ooze. A far smarter approach would be to craft the history as you go, based on the present, in whatever minimum way is necessary to make all histories consistent.

For example, let's say you learn that you are the grand winner of a lottery. At the moment you realize you are the big winner, history becomes limited to only the possibilities that got you to that winning moment. Before you learned you were a winner, the reality at the lottery headquarters was only a smear of possibilities - like Shrodinger's Cat - where you were both a winner and a loser, just like everyone else. As soon as you learn you won, your history and everyone else's harden to conform to it. No one else can perceive that they won the grand prize in that particular lottery.

If I were the programmer of this simulation that you call your reality, I would make the history dependent on the present just to streamline my work. All I need from my fake history is that it is consistent with all the other fake histories so there is no "tell" left by the programmer.

I realize the simpler explanation for my confusion about Monty Hall and Schrodinger's cat is "Math be hard." But I like the psychological freedom of feeling as though I am the author of my own history and not its bitch.

Here's the cool part: I get to keep my interpretation of reality - in which my history is a manufactured illusion - until something in my present experience is inconsistent with that view.

Recently I heard of two senior citizens with mild dementia who became friendly at a senior care facility. Their fragile minds concocted an elaborate history of being childhood acquaintances that had found each other through fate. No one tries to dissuade them of this illusion because it works for them. They successfully rewrote their histories without any repercussions.

I wonder how often the rest of us rewrite our histories. Our only limitations are that our new histories have to be consistent with whatever scraps of history have already hardened.

So you pick a door, but before it opens, Monty opens one of the other two doors to reveal a goat. He asks if you want to switch from the door you initially picked to the other closed door. Your brain says the odds are the same for any closed door, so you stay. But in fact, the odds are twice as good if you switch doors.

You can see the math of it here. But if you are normal, you'll never reconcile in your mind how one closed door could have better odds than the other. If there are two closed doors remaining, how can the odds be anything but 50-50?

This reminds me of the Schrodinger's Cat thought experiment in which a cat in a sealed box (presumably with air holes) exists in a state of being simultaneously alive and dead depending on the results of a randomized event happening inside the box. How can a cat be alive and dead at the same time? Math says it can happen, my brain says no.

The pattern recognition part of my brain is connecting the Monty Hall problem with the Shrodinger's Cat thought experiment because both situations feel like proof that our brains are not equipped to understand reality at its most basic level.

Most of us accept the idea that math is a better indicator of truth than our buggy personal perceptions. Math doesn't lie, but our brains are huge scam artists. The Monty Hall problem and Schrodinger's Cat are examples in which our perceptions of reality and the math of reality disagree in a big way. It makes me wonder how much of the rest of my so-called reality disagrees with math without me knowing.

If I were programming a computer simulation full of artificial humans who believe they are real, I would need to take some shortcuts in creating their context and history. It would be nearly impossible to invent consistent histories for seven billion people spanning back to the primordial ooze. A far smarter approach would be to craft the history as you go, based on the present, in whatever minimum way is necessary to make all histories consistent.

For example, let's say you learn that you are the grand winner of a lottery. At the moment you realize you are the big winner, history becomes limited to only the possibilities that got you to that winning moment. Before you learned you were a winner, the reality at the lottery headquarters was only a smear of possibilities - like Shrodinger's Cat - where you were both a winner and a loser, just like everyone else. As soon as you learn you won, your history and everyone else's harden to conform to it. No one else can perceive that they won the grand prize in that particular lottery.

If I were the programmer of this simulation that you call your reality, I would make the history dependent on the present just to streamline my work. All I need from my fake history is that it is consistent with all the other fake histories so there is no "tell" left by the programmer.

I realize the simpler explanation for my confusion about Monty Hall and Schrodinger's cat is "Math be hard." But I like the psychological freedom of feeling as though I am the author of my own history and not its bitch.

Here's the cool part: I get to keep my interpretation of reality - in which my history is a manufactured illusion - until something in my present experience is inconsistent with that view.

Recently I heard of two senior citizens with mild dementia who became friendly at a senior care facility. Their fragile minds concocted an elaborate history of being childhood acquaintances that had found each other through fate. No one tries to dissuade them of this illusion because it works for them. They successfully rewrote their histories without any repercussions.

I wonder how often the rest of us rewrite our histories. Our only limitations are that our new histories have to be consistent with whatever scraps of history have already hardened.

Apr 19, 2013

My apologies for the double post, My TimeWarner connection is not working properly.
Apr 19, 2013

The crux of this problem stems from case where your initial guess is correct. If you are willing to deny the fact that Monty Hall has two choices of goat to display, and then you have two choices off of this, and then you choose to combine these four equal realities into two (without properly weighing them), and setting them against the probabilities of your initial guess being false, then you will get the answer it is better to switch. Doesn't this act violate Bayes theorem, the "Prime Directive" of statistics?What if your goal was to actually win a goat? Or even better, that pretty goat you admiringly named Frank?

Math in itself doesn't lie. It is how we try to apply reality in the form of math that leads us to trouble. We spent our early scientific lives in the belief that everything literally and figuratively revolved around us. We were quite happy with this belief till we found out that to maintain this model, the math got really complicated, and things like planetary "do si dos" were a bit too much for our minds to bear. Thankfully laziness (the good kind) saved us when we moved everything to revolve around the sun. Things went pretty smoothly for awhile until somebody had to muck things up with string theory and dark matter.

Bill Bryson wrote an excellent book on the politics and history of our science. "A Short History of Nearly Everything". Its an entertaining read.

Probability isn't really good at predicting the past. No one will ever ask you to determine the probability that I had a bagel for breakfast yesterday. I was under the misunderstanding that Schroedingers Cat has nothing to do with math, but in what happens at the instant before you do a measurement, akin to the point just before you read your winning ticket. But my absurdity is approachng that of Schroedinger.

If I were the programmer of this simulation called reality, I'd be turning miscreants into pillars of salt, guiding a bunch of people around in circles for no good reason, and placing images of myself (and my family) in highly processed foodstuffs and mundane inanimate objects. Who cares about history when you can distract them with a new Iron Man movie?

Nothing is stopping you from rewriting your own history. If you don't tell anyone about it, then they can't spend all that time disproving it. I don't seem to follow you on your notion that future events force us to rewrite our past.

There is nothing more beautiful and more dangerous than a shared lie. History is rife with them, or not. It all depends on your desire to ignore/acknowledge them.

Apr 19, 2013

The crux of this problem stems from case where your initial guess is correct. If you are willing to deny the fact that Monty Hall has two choices of goat to display, and then you have two choices off of this, and then you choose to combine these four equal realities into two (without properly weighing them), and setting them against the probabilities of your initial guess being false, then you will get the answer it is better to switch. Doesn't this act violate Bayes theorem, the "Prime Directive" of statistics?What if your goal was to actually win a goat? Or even better, that pretty goat you admiringly named Frank?

Math in itself doesn't lie. It is how we try to apply reality in the form of math that leads us to trouble. We spent our early scientific lives in the belief that everything literally and figuratively revolved around us. We were quite happy with this belief till we found out that to maintain this model, the math got really complicated, and things like planetary "do si dos" were a bit too much for our minds to bear. Thankfully laziness (the good kind) saved us when we moved everything to revolve around the sun. Things went pretty smoothly for awhile until somebody had to muck things up with string theory and dark matter.

Bill Bryson wrote an excellent book on the politics and history of our science. "A Short History of Nearly Everything". Its an entertaining read.

Probability isn't really good at predicting the past. No one will ever ask you to determine the probability that I had a bagel for breakfast yesterday. I was under the misunderstanding that Schroedingers Cat has nothing to do with math, but in what happens at the instant before you do a measurement, akin to the point just before you read your winning ticket. But my absurdity is approachng that of Schroedinger.

If I were the programmer of this simulation called reality, I'd be turning miscreants into pillars of salt, guiding a bunch of people around in circles for no good reason, and placing images of myself (and my family) in highly processed foodstuffs and mundane inanimate objects. Who cares about history when you can distract them with a new Iron Man movie?

Nothing is stopping you from rewriting your own history. If you don't tell anyone about it, then they can't spend all that time disproving it. I don't seem to follow you on your notion that future events force us to rewrite our past.

There is nothing more beautiful and more dangerous than a shared lie. History is rife with them, or not. It all depends on your desire to ignore/acknowledge them.

Apr 13, 2013

@Dalebert7 It's not t hat there is some merit to the problem. It's the answer.
Apr 11, 2013

I have been thinking about this problem a lot and I have come to the conclusion that there may be some merit to it. Therefore I proclaim myself the title of A-hole of the month!
Apr 10, 2013

It's funny that people are only discussing the Monty Hall problem. (Statistics be hard.)I don't think anybody rewrites their history. A lot of people (including myself) have bad memory. But even if I change my perceived history because of that, my actual history still remains the same.

Your life is not an illusion. It is real. And you better start behaving accordingly.

Apr 9, 2013

The monty hall problem is indeed an example of our intuition being inconsistent with logic. But the Schreodinger's cat thing isn't math at all - it's physics, which is based on observations, not logic. Math is just a language used to describe our conclusions from these observations, nothing else.
Apr 8, 2013

@Dalebert7, if you compare Monty Hall to the "coin toss probability" thing, you've misunderstood how the problem works. 3 doors, right? 2 goats, one car. That means you have a 2/3rds chance of choosing a goat.

After you've picked, another door is opened revealing a goat. Now there's just one goat and one car, and two doors. That means that if you switch, you're GUARANTEED to get what you didn't originally pick. You have 2/3rds chance of having picked a goat door. So that means a 2/3rds chance that if you switch after the first door is opened, you get a car.

Apr 8, 2013

Faza,sounds like you used Notepad to edit - better use Wordpad for pure text ;-)

Nice explanation, i.e. the middle part that shows the problem the 50-50ers are actually solving. I doubt the switches will clear up anything, though ;-)

Apr 8, 2013

Sorry about the mangled paragraphs - had to prepare this offline (timeouts are a killer) and apparently some extra newline characters got introduced in the copy-paste.
Apr 8, 2013

@Dalebert7Oh dear, let's try a simpler explanation.

First, a visual analogy: imagine a two-way toggle switch with the poles marked "Car" and "Goat" (instead of On and Off). If it's in the "Goat" position and you flip it,

it becomes "Car" and if it's in the "Car" position, it becomes "Goat" when flipped. With me so far?

The second decision in Monty Hall is just such a switch, except that the pole labels have been covered up, so you don't know whether the switch is set to "Goat" or

"Car". If it's set to "Goat" and you flip it, you win. If it's set to "Car" and you flip it, you lose. Still following?

(Just in case it isn't absolutely clear: you've already picked one door and you only have one other door left to choose from after Monty opens the third one. The two

doors are the two poles of your switch, "your" door - the one you picked - is the position the switch is in and the goat and car behind the two doors are the labels under the masking tape.)

Now, what you're thinking is that since the switch can have two possible settings then the probability distribution of it being in either of the two settings is 50/50

(just like flipping a coin). This is where you're mistaken: whoever said anything about an equal probability distribution?

The number of choices has nothing to do with the probability of getting the predicted result. For example, the switch might ALWAYS be set to "Goat" (this would involve

some rushed backstage herding). Or it might always be set to "Car" (rather more tricky in the doors scenario, but nothing you couldn't do with a couple of hydraulic lifts - and possibly a spare car). If we got rid of the doors altogether and just had an actual switch that we have to flip, we could assign all sorts of equal or unequal probability distributions to the two settings. It's REALLY important that you understand that two choices DOES NOT equal 50-50 chances. I mean, if you have two teams facing off in any kind of competitive sport, either one or the other can win (we'll assume tie-breaks). So you can bet on one or the other (and you don't know in advance which one will win), but you'll seldom decide this by flipping a coin. You generally know whom the odds favour - and if not, any bookie will tell you.

Back to our goats: we have a choice of two doors, but the probability distribution isn't 50-50. This is because the original setting of our "switch" above is a result of your prior decision. If you picked a goat door, it's set to "Goat". If you picked the car door, it's set to car. The chances of picking a goat initially are twice as high as the car, because there are twice as many goats. You never get a 50-50 chance of picking goat or car on the first try, because you're choosing out of three doors not two. Two-thirds of the time you will pick a goat on your first guess, one-third of the time it'll be the car. I hope you're not going to dispute that, at least.

As a quick aside, let me describe to you the situation you're thinking of, 'coz it'll show the key difference. It goes like this:

1. You get a choice of three doors, knowing that there's one car and two goats behind them. You pick a door, write the number down and - this is important - DON'T TELL

MONTY (you'll see why it's important in a little bit).

2. Monty opens any one of the two goat doors.

3. Obviously, some of the time Monty will open the door you chose, in which case you crumple up your piece of paper, put it in your pocket and not tell anyone. After

Monty has opened the door, he asks you to pick one of the remaining two. Your odds here are 50-50, regardless of whether you originally chose the door Monty opened or not.

So what's different? The fact that Monty's decision above is independent of yours. Kinda like two coin flips. Not so in Monty Hall Classic: here, Monty's decision which

door to open is constrained by two factors:

a. He can't open the door you chose,

b. He can't open the door with the car.

So if you picked goat, he has only one door he can possibly open (the other is the car) and if you picked car he can choose between the other two (both of them having

goats). Monty's decision which door to open in Classic DEPENDS ON YOUR ORIGINAL CHOICE. Regardless of what you chose, however, he can reduce the problem to the Car/Goat

toggle switch version described above ('coz he does have a spare goat) and the probability of the switch being in either of the two positions initially will depend on the probability distribution in your first choice (2/3 goat, 1/3 car).

It's not a question of the car "becoming more likely" to be behind one of the doors, it's simply the matter of you first guess being more likely to be wrong and your

second decision being of the kind that if you were wrong on your first guess and you change your mind you become right ( you can't switch from goat to goat, only goat to car). That's all there is to it.

I hope this helped you understand how things really work.

TL;DR: When you initially choose a goat door - 2/3 of the time - Monty has only one door he can possibly open: the other goat door. He can't open your door and he can't reveal the car. Those 2/3's of the time, switching means you win the car since you started with a goat door. QED

Apr 8, 2013

Btw, if you want a problem that is REALLY controversial, try this one: http://en.wikipedia.org/wiki/Sleeping_Beauty_problem
Apr 8, 2013

@AtlantaDudeActually, your tree isn't the full tree, for the simple reason that the contestant is choosing the door, not the riding vehicle directly. Also, you are assuming independent pick/reveal choices, when they are actually not. By the definition of the problem, reveal is dependent on a. the pick (reveal <> pick) and b. the placement (reveal <> car); i.e Monty Hall has to make a choice in between the contestant's, which you (and a lot of other commenters here) ignore.

To get to the full tree, first look at how the vehicles can be distributed. Looking from left to right, the 6 permutations are

1. CMG

2. CGM

3. MCG

4. GCM

5. MGC

6. GMC

Lets call the doors A, B and C. We have to overlay the pick reveal on top of the above distributions, although it's enough to look at "pick A", since the rest are just variations.

You included pick X, reveal X in your tree, which is disallowed by the definition of the problem. That leaves pick A, reveal B and pick A, reveal C - your tree actually has 12 (!) branches when you look at the problem the way it is posed. Each of the above permutations has two branches:

1a. (A,B) Pick C, reveal G - lose on switch, win on stay

1b. (A,C) Pick C, reveal M - lose on switch, win on stay

2a. (A,B) Pick C, reveal M - lose on switch, win on stay

2b. (A,C) Pick C, reveal G - lose on switch, win on stay

3a. (A,B) Pick M, reveal C - not allowed

3b. (A,C) Pick M, reveal G - win on switch, lose on stay

4a. (A,B) Pick G, reveal C - not allowed

4b. (A,C) Pick G, reveal M - win on switch, lose on stay

5a. (A,B) Pick M, reveal G - win on switch, lose on stay

5b. (A,C) Pick M, reveal G - not allowed

6a. (A,B) Pick G, reveal C - not allowed

6b. (A,C) Pick G, reveal M - win on switch, lose on stay

The trick is that because of the "not allowed", permutations 3-6 all default to the "win on switch" situation. That is only the case because Monty Hall is choosing and the contestant knows he is choosing.

The way I usually explain it to my friends is with a variation of the problem: imagine we are looking for a needle in a haystack - a very uniform haystack with straws of the same size and length, and where the needle is actually hidden in one of the straws. The haystack has a million straws.

You choose one. The odds that that is the right one is 1:999,999 (i.e. a one in a million probability). Reversely, the odds that the needle is in the rest of the stack is 999,999:1.

Now comes Monty Hall, says "I'll show you where it isn't", takes the huge stack sans your straw and breaks 999,998 empty straws open. Now we have your original pick (still with odds of 1:999,999) and one lone straw from the rest of the stack, which gained all the information that was in the stack before. It should be pretty obvious now why switching is better. (If switching and not switching had the same odds, i.e. no information was gained by breaking the straws/opening the door, we could save the strawbreaking effort and just check the original pick.

Try it with 10 (cups, cards, whatever), ideally with friends who insist the odds are 50:50, so they shouldn't object if they aren't allowed to switch. Have them wager 100$, giving them 200$ if they win- which is a fair bet with 50:50 odds. Do it 100 times - you'll make lots of money... ;-)

Cheers

Chris

P.S. Scott, as you are Mensa I find it hard to believe that you would find the problem counterintuitive... ;-)

Apr 8, 2013

@langley - Exactly. The same as the coin-flip fallacy (a standard stat 101 example). Maybe they should use this one as an example, too!
Apr 8, 2013

The problem with math is that you have to understand how reality works in order to apply it. You can "prove" anything if you don't understand how things really work. You only had a 2/3 chance of being wrong between all three doors. Your chance of being wrong between any two of the doors was ALWAYS 50%, even in the beginning. You just have to think about it some more. Sorry, but no matter how much you want to believe it is true, this is still just a hoax! Maybe some of you would have less trouble understanding this if you thought about what would happen if you didn't guess. If you said "I can't guess, so show me a door." Now you see a goat was behind door number 3. So, now is there a greater chance for the car to be behind door number 1 or door number 2? If you didnâ€™t guess, is the probability different than if you had guessed? No, because your original guess has no influence on the probability of the car being behind one of two doors, any more than the outcome of the last coin flip has on the next coin flip. No "math" you can dream up can get past that fact. If your math doesn't fit reality, it is wrong.
Apr 8, 2013

The simplest explanation of the confusion around the Monty Hall problem is it's a sequence of two events and two decisions disguised as one. Schrodingers Cat isn't real! Quantum physics is full of miss leading metaphors, and anthropomorphic rubbish that the physicists themselves encourage so the seem like cool necromancers.

Apr 7, 2013

@Dalebert7 and other folks who are still struggling with Monty Hall.The actual fallacy is that we think about two separate choices, when in fact we are simply asked to re-evaluate our original choice (the question posed is "Do you want to switch?")

Moreover, Monty opening one of the goat doors is a key element - not because of new information, but because of constraining our choice. If originally we chose wrong (a 2/3 probability), we can ONLY switch to the right choice. Without the door being opened, we could go from one goat to the other goat - leaving us back where we started. After Monty opens the door, we can only go from goat to car and vice versa.

Thus if we originally chose a goat door (2/3), switching means it becomes a car. Conversely if we originally chose car (1/3), switching turns it into a goat. The question is how confident we are in our original choice and the odds are clearly in favour of us choosing a goat on the first try. This is why switching is a winning strategy, statistically - despite the fact that a third of the time, people will find that they'd have gotten the car if they stuck with their original choice.

Also a quick aside about Shroedinger's cat: the point here is to show that we can't extrapolate quantum level events into the macro scale directly. The easy way to understand it is this: you might not know if the cat is alive or dead until you check, but the CAT certainly knows. Plus, the Geiger counter must be able to tell whether the atom has decayed or not, or the whole experiment becomes moot. In short: we shouldn't confuse HUMAN observation with "observation" meaning that various quantum-level and higher systems interacting - arriving at a deterministic result.

Apr 4, 2013

I can't believe how many people in the comments are contradicting the Monty Hall Problem. Sure it's counterintuitive at first, but the math works if you follow it. The illustration about halfway down the wikipedia article shows it well. Schrodinger's Cat is something different, it's not so much a "math be hard" scenario, it's a thought experiment relating to the mysterious behaviour of subatomic particles.

Apr 4, 2013

Well, that is the American myth/reality. In America we --can-- reinvent ourselves.
Apr 4, 2013

It's hard to believe that anyone would fall for this phony "Monty Hall solution". I kind of wondered if it was an early April Fools joke you were playing on us! It is the same fallacy as proving the odds of a random coin toss coming up heads is less if it previously came up heads a number of times in a row. You can mathematically "prove" that the likelihood of a coin coming up heads a certain number of times in a row is less than 50%, but the odds on the next toss are still 50-50.This fallacy is based on the idea that your previous guess was more likely to be wrong, so it influences the odds of the second guess. But, of course, it doesn't. Your previous guess is irrelevant. The second time you are presented with a scenario of two doors and one car. The car has equal chance of being behind either door. If you are shown a goat behind door #3, the car is not more likely to be behind door #2 because you previously guessed door #1, nor is it more likely to be behind door #1 because you previously guessed door #2. You can't influence the position of the car by guessing; it is where it is. (That is, unless they are cheating and moving them around behind the doors, which I wouldn't put past them!)