In other words, find the n that makes this event most likely. That's called the maximum likelihood estimator. So we want to find the n that makes this probably the largest, because that's what happened. So the most probable thing is probably what happened.

So we find the n that maximizes p of a.

So one way to do this, we'll call this f of n. It's a function of n, right? And now, when you're working with binomial coefficients,

what do they look like? They're products of numbers from 1 to n, or from 1 to 10, and they divide by the products of numbers 1 to 4, and then 1 to 6, et cetera. They often lead to lots of cancellations if you do the right thing.

Because if you get like, what's n plus 1 factorial divided by n factorial? n plus 1, right? So in here, we'll have an n minus 10 factorial, an n factorial. If we can get rid of all the stuff except the n and the n minus 10 somehow, that would be nice.

So a lot of times what you might do is look at ratio of this to maybe f of n plus 1. So let's look at f of n plus 1 over f of n.

So we want to do what? How do we pick n? Suppose this ratio is bigger than 1. If this ratio is bigger than 1, which event is more likely, the one with n plus 1 or the one with n? One with n plus 1, right? So we'd go with n plus 1.

But then we'd want to maybe check to see maybe n plus 2 gives a bigger result. So we're going to look to see if f of n plus 2 over f of n plus 1 is bigger than 1. So what we want to look at is, when is this bigger than 1? When it stops being bigger than 1, that'll be our estimator. When we find an n so that's bigger than 1,

but f of n plus 2 over f of n plus 1 is less than 1, then n plus 1 would be our estimator. Because what does that say? f of n plus 1 is bigger than f n.

So that's the event with n plus 1 is the population more likely. But this one says f of n plus 1 is bigger than f of n plus 2. So things start going down. They are going up until n plus 1. And they're going down after n plus 1 if this happens. So let's look at this ratio. We should expect miracles.

Well, OK, in the numerator, this

will occur in both f of n and f of n plus 1. So when I divide 1 by the other, the 10 choose 4 will cancel. So I'm not going to write down the 10 choose 4.

So this ratio, so it'd be what? n minus 9 over 16 divided by n plus 1 over choose 20.

That's with the n plus 1. Why n minus 9? Right. Right. And then we divide by what we get when you have n in there. And that would just be when you divide by a fraction, you would multiply, right?

So I get n choose 20 in the numerator here. And down here, I'd get n minus 10 choose 16. And now we're really in business because if we write out the expressions here,

this becomes that term. This one is down here. We get, let me write it over here.

This term, that's from here.

And then this one I'll put here. OK. So this and this cancel.

This and this cancel. Now let me maybe invert and multiply.

So I get n minus 9 factorial, n factorial. This goes in the numerator, n minus 26 factorial. This goes in the numerator, n minus 19 factorial.

And then in the denominator, we have n minus 25 factorial. I'll put that here. That's from here. n minus 10 factorial, I'll put that here. n plus 1 factorial, I'll put that here. And then n minus 20 factorial.

And what happens now? This ratio leaves what? n minus 9 in the numerator.

So we've taken care of this part. What about this ratio? So n plus 1 in the denominator. This ratio, n minus 25 in the denominator.

And this ratio, n minus 19 in the numerator. That's our ratio then. Not bad. So what did we want to do? We want to check when is this ratio bigger than 1. We want to find the last n plus 1 for which this

is bigger than 1. We pick the first n for which that is less than 1. Well, I mean, after it gets.

So let's check. When is that bigger than 1? It's bigger than 1 when n minus 9 times n minus 19 is bigger than n plus 1 times n minus 25. That ratio is bigger than 1 if the numerator is bigger than the denominator.

Let's multiply out both sides. Get n squared minus 28n plus 9 times 19. That's 71.

The n squareds cancel.

Let me subtract 25. I'll subtract n squared from both sides. Subtract 25 from both sides. 25 from this gives 146. Is that right? OK, I'll add.

196 is bigger than 4n, or n is less than 4 into this goes what?

40 what? 49 times, OK? So as long as n is less than this, ln plus 1, fn plus 1 over fn is bigger than 1. After that, it's less.

So what's the biggest integer n that's less than 49? 48. So what's our estimate? Not n, but n plus 1. So 49 is our estimate. So the estimated population size would be 49.

OK? The largest n that's less than 49 would be 48. But we want to take n plus 1, because the probability with n plus 1 is bigger than the probability with this n. So that would be 49. After 49, the probability starts going down.

So 49 is the population. This is actually used as kind of, OK, well, if binomial is good, multinomial is even better.

And what's the multinomial theorem?

Question, how many ways are there to do this?

So maybe think of the clauses as being boxes or something.

OK, so we're going to put n1 objects in here, n2 objects in here, n3 in here, et cetera, nr in there.

Is that where you understand? OK. So we just used the multiplication principle. Let's count how many ways there are to put n1 things in here first. And then for every one of those ways, we count how many ways there are to put n2 in here.

And we multiply them together. And for every one of the ways we've done that and that, we count how many ways there are to do this and just keep it going. Just use the multiplication principle. So how many ways are there to put n1 objects in here? We have to select, do what? Select n1 objects from n. So that would be n choose n1, right?

And now, given any one of those choices, how many ways are there to put n2 objects in here? Right, n minus n1 choose n2. And now once we've done those two jobs, how many ways are there to do this one? You guys are good.

And then down to here, we get to n minus n1 minus n sub minus 1 choose n sub r. But actually, what would this number be? How many objects are left after we've done all the other jobs? n objects, so we have, this is 1.

Because what's this number after all? If I subtract the first r minus 1 of these, well, if I subtract the first r minus 1 from here, what do I get? n sub r. n minus the first r minus 1 would be n sub r. So that last number is 1. OK, so that's a pretty long, complicated-looking expression.

Would you like to compute it that way? Or you'd probably want to simplify it first, right? Because you all have the same instinct as a mathematician. Mathematician always wants to make things simple. You believe that? The goal of mathematics is to make description of the world simpler.

So let's simplify. And like many expressions involving multiplying and dividing factorials, this admits a big simplification. Let's write these out. This is n factorial over n minus n1 factorial times n1

factorial. This would be n minus n1 factorial times n minus n1 minus n2 factorial times n2 factorial. The next one is n minus n1 minus n2 factorial

times n minus n1 minus n2 minus n3 factorial times n3 factorial, et cetera, all the way down to, well, 1.

And look at this. It's like falling dominoes. That cancels that. That cancels that. This one cancels the next one. So what will be left? Well, in the numerator, what survives?

In the denominator, n1 factorial, n2 factorial, n3 factorial, all the way down to n sub r factorial. That's called the multinomial coefficient.

The multinomial theorem says that x1 plus x2 plus dot,

dot, dot plus xr to the n is the sum over. Oh, and by the way, we use this symbol here.

Say n choose n1, n2, n3 through nr.

So we sum over all sets of non-negative integers, n sub i that add up to n, this expression.

That's just like the binomial theorem, only more. And same argument. OK. So where would the multinomial theorem ever come up? Well, who wants to come over Saturday night

to my house to play poker? Anybody? One, any more? Two, three, four, five? OK, that's enough. So five of you said you'll come and I'll play.

So that's how many? Six. And so we're going to play five-card poker. I mean, any poker game you wish with five cards. So I'm going to deal from a deck of 52. I'm going to deal out six hands, five cards each.

And before you can bet, I'm going to ask you how many ways are to do that? How many ways are there to deal with six hands with five cards each from a deck of 52? That's one of these problems. Instead of boxes or closets, I'm

going to deal to six people, five cards each. So how many ways are there to do that? Well, what's the little n? What's the total number of objects I have?

I have six players. Am I going to deal out all the cards? No. I'm going to put five cards in here, five here, five here, five here, five here, five here. What else do I do? I put the other rest of them on the table and we use those for drawing. So I'm going to have a seventh box here

where the remainder go. So I'm going to put five cards here, five cards here, five, five, five, five, and then how many? 22. So the number of ways to deal six poker hands would be 52 choose five, five, five, five, five, five.

I should have done this with something that had nines because of the Beatles song, number nine, number nine. And then 32. That's the stack that remains after you.

One, two, three, four, five, six. So it would be, oh. Sometimes I do that to see if you're watching. 52 factorial over what? Five factorial to the sixth and then 22 factorial.

Another example would be playing bridge. By the way, I'm busy Saturday. I can't have you all over. Sorry. I just remembered that. What about bridge? Anybody here play bridge?

You play with a deck of 52. Nobody plays bridge? No? Well, do you know how many cards each player gets? Do you know how many players there are? No. Does anybody know how many players are in a bridge game? Right, four. And how big is the deck? And they all get the same number, so they each get 13.

So the number of possible bridge hands would be, that's an enormous, enormous number.

So just one last little question. If you're tossing a coin, how long do you expect to toss until you get a head if it's a fair coin? How many tosses do you think you should expect to have to make before you get a head on average?

Well, the answer is two. If you're rolling a die, how many times do you think you have to roll it before you get a one?

Six times, yeah, six times. So in bridge, how long do you have to wait until you get a particular hand? Say, where the first person gets all spades, the second one gets all hearts, the third one gets all diamonds, and then the last one gets all clubs. It'd be one over this number.

That'd be the average time until that happens. Turns out the universe isn't that old. I mean, it takes some time to deal out a hand. Or at least the Earth isn't old enough for that to have happened. Yet I remember years ago watching TV, and a group of elderly women came on

and said that that happened when they were pregnant. I'm suspicious. OK, so let's take a break, and we'll go back a couple minutes after 10. OK, it's time to talk about conditional probability

and independence.

Notions in probability theory, one of them is positive.

This, and it's equal to probability of A intersect B over probability of B.

So let's do a simple example.

It's probably you get a six, given that your roll was an even number. Any guesses? Yeah, 1 third. Why 1 third? And is there any reason for one being

more likely than the other? No. So this is one of these examples of a probability where all outcomes are equally likely. There's really no bias among the even numbers or among the odd numbers. There's no reason to suspect there is. So the answer should be 1 third here.

Can we get that from here?

B is a subset of B, is it not? This would be probability of A, probability of B. And what's the probability of A if the die is fair?

1 sixth? And the probability of an even number? Well, this is one of those problems where the probability is faced with equally likely outcomes. So all we have to do to complete the probability of B is count how many ways there are for that to happen, divided by the total number of outcomes. How many possible outcomes are there when you roll a die? Six. How many ways are there for B to occur?

Three. So this is 1 half. And 1 sixth divided by 1 half is 1 third.

This is the probability of A intersect B is the probability of A given B times the probability of B formula.

But sometimes you might want to compute this. And it's not completely obvious what the answer is. But this side becomes simpler. The probability of A given B sometimes might be easier than probably A intersect B. And then if this is also simple to compute,

you just multiply the two simple answers together to get the more complicated one.

A useful tool for computing probabilities is the following.

B and N are mutually disjoint events. Their union is the whole sample space. And each has a positive probability.

A can be computed using these.

It's called the law of total to see why this is true with the thing I've underlined. This side, right?

A intersect Bi. What about the sets A intersect Bi? Are they mutually disjoint?

So maybe let me take one moment. Then this side over here is also this is equal to this.

Why is that? That's the definition of a probability. These are disjoint sets. If I intersect A with Bi, if the Bi's are disjoint,

then their intersection with A is also disjoint. And the probability of a union of disjoint sets is the sum of the probabilities of those sets that was in the definition of probability. This could be rewritten. A intersect that union.

De Morgan's laws, they're a bunch of rules for how the unions intersect, sections intersect. And what's this set here? The union of Bi's? It's all of omega.

So what's A intersect omega? It's A, probability of A. See why that's true? Let's try to use it in models of statistical mechanics.

Here's an urn problem.

In here, a bunch of balls inside. White ones, if you want.

The white ones are the same.

Return that one and two more of the same color.

Collection is repeated.

Two of the balls have been put back in. You draw one more out. If G2 is the event, the second gumball is green.

Probability is, there are five green balls and eight white balls to begin with.

It's easier if we know what happened the first time, isn't it? That means we should condition them what happened the first time. We should use this law of probability where the B's are what? Indicate what happens. They should exhaust all the possible outcomes

from the first draw. What could happen first draw? Green is drawn or white is drawn? So our sets here B should be, maybe white would be good choices for describing green as drawn the first time. Or it could be a good way to denote the event.

White is drawn the first time. W1, okay? So naturally, we take G1 as the event. White is drawn first time.

Drawn first time. W1 as white is drawn first time. And of course, G1 union W1 exhausts all the possibilities.

If we keep drawing, either we have to have white the first time or green the first time. What about the intersection of these two things? Is it empty? Because the ball can't be simultaneously white and green.

What about the probabilities? Are they positive? What's the probability of G1? Because this is one of those examples of a probability space where all outcomes are equally likely.

So you just have to count the total number of possible draws for the first time. How many different possibilities are there when you're drawing one thing out of 13? You choose one or 13, right? And how many ways are there to get a green one? Five ways. So five out of 13. And probably W1 is again, well, eight over 13, right?

So they're both positive. So we can apply the law of total probabilities here with B1 equal to G1, B2 equal to W1. And N is two in our case, right?

So the probability of green the second time would be, well, there are two sum Ns. I'm not gonna use summation notation. It'd be probability of green the second time given green the first time times probability of green the first time plus probability of green the second time

given white was drawn the first time times the probability that white is drawn the first time. Now, are these probabilities easy to compute? We already did these two. What about probably a green the second time given green the first time? What would that equal?

Yeah, because what's the urn look like after green is drawn the first time? We replaced that green ball and put two more in. So now how many green balls would be here? Seven. And how many balls all together? How many? I replaced the one I did draw and then I put two more in.

So 15, right? So that conditional probability is seven out of 15. And then times the probability of green is five out of 13. What about this conditional probability? What does the urn look like on the second draw if white was drawn the first time?

How many green balls are there now? Five still, and all together they'd be 15. So the probability of green the second time if we had white the first time would be five over 15?

Or is this eight over this one? Okay, so remember we're putting the ball we selected back and two more.

So we started with 13 and we add two more so that now in the second draw they're 15. Any other questions?

Let's repeat it again. What's the probability the third one is green? Is this easy to do right off? I mean, just by inspection. I think it requires some analysis, a little bit.

This is easier to do if we know what we're drawing from, right? It'd be easier to know what the probability

of getting green the third time is if we know what the urn looks like on the third draw. Let me update this.

Do the composition of the urn after two draws, can this probability be easy? Is it to be the number of green balls over the number of balls? How many balls will be in the urn for the third draw? So last time we conditioned on

what happened on the first draw. This time we should condition on what happened on the first two draws, right? We should condition what happened on the first two draws. So what could happen on the first two draws? We need this G1 and W1. We also need G2 and W2

which will have natural meanings that I don't have to write down. So what are the possible things that could happen the first two times? Four possibilities, are there four possibilities?

Okay, thank you. G1 intersect G2, that would be our B1. G1 intersect W2, that would be something like B2. W1 intersected with G2 and then W1 intersected with W2.

Those are the four possibilities for the outcomes of the first two draws. Green then green, green then white, white then green, white then white. These are mutually destroying, correct? Their union is all possibilities, right?

The union of these would be omega. Is each probability positive? Yes, so we can use the law of total probability.

Very consecutively this means intersection.

It's like a probability of G2 and G1.

The conditional probabilities, what's this conditional probability?

All you have to do is ask yourself how many green balls are in the urn now? First time I added two more, next time I added two more. I started with five, so now I'll have nine in an urn with 17. The chance of getting a green would be nine out of 17.

And now this one I'm gonna write as probability of G2 given G1 times probability of G1. Just use the definition of conditional probabilities. That was one of the formulas I wrote earlier, right? The probability of an intersection is probably one given the other times the probability of the other.

Why do I do that? Is this easy to compute? Yes. Is this one easy to compute? Yes it is, because I know what's in the urn. If I condition on what happened first. Any question on this? This is just using the probability of this intersected

with that is probably this, given that, times probably that. This probability, because now there are only two additional greens have been put back in. There are 17 balls all together, so this would be seven over 17 times the probability of one given,

I'm sorry, W2 given G1 is the probability of G1. What's this probability, G3 given W1, G2?

Well here, two green balls have been put in. So it would be seven over 17 again, right? And then times the probability of G2 given W1 times the probability of W1. Plus, here we get what? No more green balls are in, so there's still five.

In an urn of 17, probably getting a green then would be five over 17. And then the probability of W2 given W1 times the probability of W1. About the rest, now we get nine over 17.

What's this probability? There are 15 balls in the urn, not in the second draw, right? And seven greens, so seven over 15,

and this would be five over 13. Plus seven over 17 times, well, how many whites are there? Same as there ever was, eight over 15.

Times the probability of getting green the first time, that's five over 13. Here, we get seven over 17. Here, five over 15, is that right? Because there are no additional green balls, but there are 15 in the second draw. And here would be eight over 13. Plus five over 17.

Here, we get 10 over 15. And here we get eight over 13. So a somewhat difficult problem becomes a series of, or somewhat difficult computation,

probably green the third time becomes a series of very simple ones.

The useful formula when doing conditional probabilities is called Bayes' rule.

They're the same conditions as the law of total probability.

So if we take A to be something like one and B2 are destroyed,

we have positive probabilities, or union is the whole sample space. And we might ask, what's the probability you had white the first time, given you got green the second?

Just reverse those, it'd be easy to do, right? Sort of like, this is the effect and that's the cause.

Asking what was the possible cause, given the effect. Reminds me of a story I heard on the radio as I was driving somewhere. Seems like I'm driving all the time.

It said, report said that children of working mothers, I was wondering, what is it about obese kids

that makes their moms want to go out to work? So it's, it's obvious that people think the effect is the other way, conditional probabilities in this order. Now this is what Bayes' rule does.

It kind of reverses the way the conditioning is going. Puts it kind of in an order that's easier to compute. So this would be the probability of what? G two, given W one, times the probability of W one,

over, well we just have a sum of two things, so I'm not going to write the summation notation. It'd be probability of G two, given W one, times the probability of W one, plus the probability of G two, given G one,

times the probability of G one. Now these are more natural and easier to compute directly. This is like we've been doing.

White was drawn the first time, there are now 10 white balls, and they're earned with 15. Probably getting a green would be five out of 15. And they're probably getting white the first time was eight out of 13. This term goes here,

and then here, probably getting green the second time, would be seven out of 15. And then probably green the first time was five out of, so this problem was a little bit confounding,

became a bunch of simpler ones. So whenever you come across a conditional probability, that looks like it'd be easier to solve if the conditioning and the event were reversed, then use Bayes' rule.

This is equivalent to the probability of A given B is probability of B, I'm sorry, A.

What does this say? If you know that B happened, what's your estimate of the chance that A has happened? Does information about the occurrence of B tell you anything about the probability or likelihood of A happened? Tells you nothing, gives you no information.

So let's think of some examples. Suppose I toss a coin twice. Comes up, I tell you it comes up heads the first time. What's the probability it comes up heads the second time? Has it changed? In order for the probability to change. Something about the first toss would have to affect.

Does the coin remember what it did the first time? You better say no. There's no memory. The coin has no recollection of the first event. Even if it did, could it do anything to control the outcome of the second event? No, not likely.

Or it's not something we consider possible. Let's actually consider the sample space of two coin tosses. So how many possible outcomes are there when you toss a coin twice? And they're equally likely, right?

So let's try to see if that or that holds. Where A is a head's first toss.

Well, maybe I should do this the other way around. Let's compute the probability of A intersect B.

What? Heads, heads. That's one out of four equally likely possibilities. So the probability of that is one fourth, right? What's the probability of A?

Maybe I should call two more. Let me do two more events. C is tail on the first toss.

Well, that'll be enough. That's all I need. It can be decomposed as probability of A intersect B plus probability of A intersect C. Heads of second toss is the union of heads the second toss and heads the first toss,

together with heads the second toss, tails the first toss. The reason I do that is now this is in our sample space. This is one of four equally likely outcomes, namely heads, heads.

This is one of four equally likely outcomes, namely tails, heads. Each of these has what probability? One fourth, so that would be one half. And what about the probability of B?

Our sample space is two tosses. I don't think I have to go through it. What's the probability of heads the first time? So what's the probability of A times the probability of B?

Fourth, right? And that happens to be also the probability of A intersect B. So the outcome on consecutive tosses of a coin would be independent, as if you toss it twice. What if you toss it three times? All we use here was this equally likely property.

So any two tosses would be independent. What about rolls of a die? We do the same computations, but instead of we get one sixth, right? And if you roll the die twice, each outcome would have probability one thirty sixth,

and one sixth times one sixth would be one thirty sixth. So consecutive rolls of a die would be independent. Or if you roll die ten times, the first roll on the tenth roll would be independent. Right? Similar computations. What about in the urn problem? What about G2 and G1? Are they independent?

What was the probability of G2? Did we compute that? Or G1? Is that the same as the probability of G2?

This one's bigger, right?

Does the outcome of the first draw affect the outcome of the second one? Very much. So there's no independence here. So if there's some sort of memory or mechanism for carrying over information from the past to the present, then you probably won't have independence. But if there is no such mechanism,

seventy times, each time you have probably 0.05 of hitting the bull's eye, the trial is independent of the other trials. Probably you get at least one.

That sometimes, the probability of A, it's easier to compute the probability of A complement. A looks a little bit complicated.

Probably a lot of ways for you to get at least one bull's eye. It could be the first time, the twentieth time, the second and the eighteenth, and the thirty-fifth time. Sounds like a pretty complicated counting argument. How many ways are there to ever get a bull's eye?

One way every time you miss. So if there's at least one, this would be that there are none. Let's think about something like rolling and tossing a coin.

What's the probability you don't get ahead the first time? What's the probability you don't get ahead the first two times? Fourth, because the trials are independent, right? And each one has the probability of half, and the probability of each one, probably both happening would be the product of the probabilities.

What about not getting ahead if you roll it three times? One-eighth. By independence, it would be one-half, times one-half, times one-half. The probability of the intersection of those three events, heads the first time, intersect heads the second time, intersect heads the third time, would be the product of their probabilities. What's the probability of not getting ahead four times?

So it would be one-half to the n, if you're looking at the probability of not getting ahead n times the roll. Now let's go to a die. What's the probability of not getting one the first time? One-sixth. The probability of not getting one the first two times?

One-sixth squared. First three times, one-sixth to the third. N times, one-sixth to the n. Now let's say we have an experiment where there's success or failure. So the probability of P of success. What's the probability of not succeeding the first time?

One minus P. What's the probability of not succeeding two times the roll? One minus P squared. Three times the roll. One minus P cubed. N times the roll. One minus P to the n. Success here would be hitting a bullseye, right? The probability of success is 0.05. The probability of failure is 0.95.

So what's the probability of failure n times the roll here? 0.95 to the n. This is the probability of failing n times the roll. So it would be one minus 0.95. And as n goes to infinity, it goes to zero.

So if you're persistent, eventually you'll get a bullseye. Consider this. A monkey is randomly typing letters as an infinite scroll of paper in the ribbon.

Well, this is an old joke. A monkey has a computer and a nice word processor and he's tapping the keys. And he's been doing this ever since the universe started. There's some chance that starting with one letter, he will type out all the works of Shakespeare perfectly, right?

If he keeps going. I mean, given maybe the first work of Shakespeare starts with the letter A and it's capitalized and he gets all the punctuation correct. There's some chance that he could do that. But it's pretty small, right? But if he keeps doing it, eventually he'll write all the works of Shakespeare.

Because the probability of failing each time is something like this. The probability of failing forever would be zero. Alright, so see you Friday.