Limit Theorems
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Teil | 7 | |
| Anzahl der Teile | 16 | |
| Autor | ||
| Lizenz | CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben. | |
| Identifikatoren | 10.5446/12892 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | |
| Genre |
1
2
3
4
5
6
7
9
10
11
12
13
14
15
16
00:00
Offene MengeMathematikGleitendes MittelMengenlehreSpannweite <Stochastik>Leistung <Physik>TermGruppenoperationFrequenzFamilie <Mathematik>MereologieFigurierte ZahlRechter WinkelOrdnung <Mathematik>Dichte <Physik>MatchingPotenz <Mathematik>Numerische MathematikMultiplikationsoperatorJensen-MaßUniformer RaumRegulator <Mathematik>FunktionalOvalKategorie <Mathematik>GammafunktionKonditionszahlTeilbarkeitDifferenteWeg <Topologie>Grothendieck-TopologieDifferentialKugelkappeMathematikVerschiebungsoperatorSchwebungGrenzschichtablösungUnendlichkeitRelativitätstheorieDivergente ReiheFaktor <Algebra>Ext-FunktorÜbergangKontraktion <Mathematik>EinfügungsdämpfungSortierte LogikAsymmetrieBerechenbare FunktionFormation <Mathematik>SterbezifferVorzeichen <Mathematik>WasserdampftafelDistributionenraumStochastische AbhängigkeitFlächeninhaltProdukt <Mathematik>FaserbündelÄhnlichkeitsgeometrieEntscheidungstheorieSchlussregelDezimalzahlSummierbarkeitMittelwertGammaverteilungDifferenzkernGeometrische ReiheWahrscheinlichkeitsfunktionNatürliche ZahlForcingGleichverteilungStirling-ReiheOrdnungsstatistikDichtefunktionalZufallsvariableAlgebraische StrukturIndexberechnungGanze ZahlParametersystemErwartungswertExponentialverteilungKartesische KoordinatenFolge <Mathematik>QuaderBeobachtungsstudieObjekt <Kategorie>ModulformStichprobenumfangInverser LimesDerivation <Algebra>UngleichungPolynomialverteilungEbeneMinimumGrößenordnungRichtungFreier LadungsträgerQuadratzahlPunktMeterMaß <Mathematik>Klasse <Mathematik>SpieltheorieAffiner RaumEinsVollständiger VerbandMedianwertZentrische StreckungAusdruck <Logik>Arithmetischer AusdruckGarbentheorieModelltheorieOffene MengeEreignishorizontDruckverlaufRandomisierungGruppendarstellungGüte der AnpassungGerichteter GraphNeunzehnGefrierenKonstanteWirbelströmungAggregatzustandAbstimmung <Frequenz>Kappa-KoeffizientArithmetisches MittelPrimidealStatistikTropfenGeradeRuhmasseDeskriptive Statistikt-TestReelle ZahlMomentenproblemProjektive EbeneIntegralPunktspektrumLokales MinimumNichtunterscheidbarkeitWurzel <Mathematik>AsymptotikKalkülGleitendes MittelLoopZweiWahrscheinlichkeitsverteilungPartielle IntegrationGeometrische VerteilungGeometriePolynomZählenBetrag <Mathematik>GleichheitszeichenKoeffizientKettenregelTopologieAbstandQuotientPascal-ZahlendreieckGrenzwertberechnungRandwertSymmetrieVorlesung/Konferenz
Transkript: Englisch(automatisch erzeugt)
00:08
So the exam will take place in your discussion sections 8 o'clock till 850 Friday and Do you want to have it chapters 1 through 4 chapters 1 through 3?
00:26
It sounds pretty uniform, but not very loud Sounded like he said He sounded a little defeated
00:41
I Like you're choosing between two forms of execution chapters 1 through 3
01:02
With height with probably about three fourths who asked that question With probably about three fourths, it'll be evenly distributed Yeah, if you have a discussion in the other room you meet in that room, okay, so
01:32
Let's finish Chapter three then and The last topic is called order statistics
02:07
So pose we have a sequence of random variables, so these are outcomes of some random phenomena like the role of a die Bernoulli trials one
02:21
application would be insurance claims X 1 is The amount that Client number one claims in a year from his insurance company This is the amount client to claims the amount client three claims. This would be the amount client end claims
02:43
What would be of interest to insurance companies is the distribution of the largest claim they get in one year Why would they be interested in how The largest and the distribution of the largest thing what what does an insurance company want to happen?
03:04
Yeah, they want to probably the largest claim to be small or if it's big if they're probably the largest claim is big They'll they'll have to charge more Money for the premiums. So they're very interested in the distribution of the size of the claims in particular
03:20
they're very interested in the distribution of the size of the largest claim so What they might do is? reorder the sample
03:42
so that X 1 is the largest and then Next largest we'll call X 2 but I'm putting parentheses around the indices here the subscripts
04:30
You still have the same set of values. You've just reordered them from top to bottom So this is the largest of the Random variables you see here
04:40
This would be the second third and then the smallest So let's do a couple simple things let's first Do the following if?
05:10
these are independent and Identically distributed so we'll often have to write this independent and identically distributed and
05:32
Since this comes up so often we use I ID for that term independent identically distributed
05:43
With distribution function F
06:07
Okay, so this is a restatement of identically distributed. They all have the same distribution function here this line
06:22
Just means that I'm taking the same set of values But I've reordered them so I this is to indicate that this is a reordering of this set of values So I have the same numbers Okay This is maybe the order in which they arrived and this is their ordering by their magnitudes or size
06:42
Any other question? No identical means they all have the same F the same distribution function identically distributed
07:03
Is this property here? It could be Exponential gamma normal, but as long as it's same for all the random variables Which is often a reasonable assumption you might be looking at a certain class of
07:28
Outcomes for events and they all fall within that same class so they all have the same distribution like the role of a die roll the die Ten times The distribution of each role is the same right the probability of one is a one six probably with two is one six
07:45
And this doesn't depend on which role You're making it could be there'd be the same on the first roll second roll, or if you're tossing your coin And then you'd have two outcomes each time probably of heads or tails each time as a half So they all have the same
08:01
Distribution function as long as you have the same coin you're tossing or you might be testing lifetime of light bulbs and Maybe the light bulbs are all from the same manufacturer, and so they should all have this same exponential distribution Okay, so that's what identical distribution means you have a sequence of random variables. They all have the same distribution function
08:25
What would what does independent mean? factors Yeah, exactly
09:08
Here's the joint distribution function. What is that that is? this probability
09:23
Independence is that this joint distribution function splits into the product of the individual ones
09:43
Okay, that's definition of independence Okay, so now with that
10:20
Let's see if we can say what the distribution is for
10:24
the largest of the sample or sequence of random variables and the Smallest and remember, what are what are you being asked for when you ask for the distribution of a random variable?
10:42
You're asked for the distribution function So let's start with the largest It says that the largest in the sample and random variables Has to be less than or equal to X
11:03
So what what about X 1 is it can that be bigger than X or less than X? Sorry, which which X 1 am I talking about? What about this one? If this is less than or equal X, what about that one?
11:21
That's also a single X the biggest one is less than or equal X. So this one has to be what about X 2? All of them have to be less than or equal to X if the largest one is less than or equal to X all of them are If the tallest person in this room is less than seven feet tall, then we're all less than seven feet tall I think the tallest person is less than seven feet in this room, right?
11:40
Okay, but it doesn't matter if the tallest person is smaller than shorter than seven feet then we all are So this would say I think that this is true
12:01
in other words this event The largest is less than or equal to X is the same as all of them are less than or equal to X Those are the same events the largest is less than or equal to X is the same as all of them are less than or equal to X And now what can you do with?
12:20
that Right because of independence you could write it that way, but the same Thing appears here every time right here at X 1 X 2 X n so I'd have divide X 1 X 2 X through X And I'd have X 1 here F at X 2 here and F of X n there
12:44
But now I have X X X X everywhere how many terms do I have? So this would be to the nth power so that's the distribution function for the largest, it's the common distribution function raised to the nth power, so
13:05
let's Before looking at examples, let's see if we can say something similar about The smallest one being less than or equal to X, and this is one of those cases where we use this
13:33
Very trivial, but useful fact the probability of A is 1 minus probability of A complement Here this would be the probability of A
13:40
And I claim computing the probability of A complement is easier What would the event A complement be if this is A? Yeah, just the smallest is Bigger than X so this would if X 1 through X n were the heights of the people in this room
14:01
This would be something like the statement The shortest person in the room is taller than I don't know four feet, okay? Now how can we use independence to try and the and the distribution function for the random variables to try to express that?
14:22
Up here Here we said the tallest is less than or equal to X means all of the individuals have to be shorter than X Here we're saying the shortest is bigger than X. So is there a similar statement? Everybody has to be taller than X right everybody's taller than four feet if the shortest person is taller than four feet
14:43
Everybody is taller than four feet so this would be the probability that X 1 minus probably X 1 is bigger than X X 2 is bigger than X All the way up to X n is bigger than X and now we can use independence here
15:25
And we can express each of those Terms in the product using the distribution function this is Yeah 1 minus F of X and all of them are 1 minus F of X so in the end we get the probability that
15:41
the smallest is Less than or equal to X is 1 minus 1 minus F of X to the end this one is that one and Each of these is 1 minus F of X and there are n of them
16:00
I'm multiplying them together so I get 1 minus F up to the end and Let me put here. Oh, let's suppose that let's do a couple examples
16:32
Or better yet, let's suppose we're in the continuous case And we have a
16:42
Density, what would the densities be for X 1 and X n?
17:12
How do you get to a density if you have the distribution function? Take the derivative density is a derivative of the distribution function so here
17:21
We have the distribution functions all we need to do is differentiate so the density for the smallest one We just differentiate this One goes away. We get a minus sign, but hold on
17:41
We'll get another one. We'll get we're differentiating something to the nth power so an n comes down We get whatever appeared inside to the n minus 1 and then we multiply by the derivative of what appeared inside this chain rule derivative of capital F is
18:01
Little F, but with a minus sign then knocks out that one so we get little F here There's the density for the smallest What about for the largest We just differentiate this and that's a simple application of the chain rule
18:37
giving that find the distribution of the largest and you have exponential random variables, so
20:11
This would be something like the following light bulbs are commonly Models the lifetime of light bulbs is commonly modeled as having an exponential distribution in this room. We might have
20:28
three three light bulbs in each fixture and one two three four five six seven eight nine ten Fifteen eighteen times three is 52 they're probably 52
20:45
Okay All right 54 54 light bulbs, so this is 54 and X 1 would be the time when This one that one burns out next would be the time when the middle one burns on x3 would be the time when that one
21:01
burns out What would Let's find the distribution of x and you can change it this way this is the This is which light bulb This is the first one to burn out this would be when the first one burns out the smallest lifetime
21:23
The shortest lifetime would be when the first one burns out, so we're finding the distribution of the Lifetime of the shortest live light bulb in all these sockets. That's what this would represent if n was 54 okay, so
21:42
We can use these formulas Maybe this one here or this one. What is the distribution function in this case well in this case?
22:03
X 1 minus capital F is e to the minus lambda X This is for X bigger than 0 and the density is this
22:35
so here we just
22:42
Plug in the formula to get the density for the shortest loop we get n and then we get a This term to the nth power n minus first power right and would that look like this So this part is here this one is this and then we multiply by little F. That would be
23:12
lambda e to the minus lambda X and this is for X bigger than 0 and Let's try to simplify this
23:21
And where do we get well these two are constants? Let's write those in front and lambda Then we get an exponential Well there will be a minus sign There will be a Lambda because there's a lambda for each term there will be an X there's an X for each term and then n minus 1 plus 1 is
23:43
n and So what's the distribution of this random variable? Smallest of n independent exponentials and Lambda it's exponential with parameter n lambda. Let's do
24:24
the maximum the density for the maximum here would be
24:50
This use this we get n and then 1 minus e to the minus lambda X raised to the n minus 1 and then this is lambda e to the minus lambda X and this again is for
25:02
X bigger than 0 for X less than 0 the density is 0 because none of the random variables are can be negative and We don't have a name for this But that's it so that's how you get to just density for the largest or the smallest
25:29
Now let's well, let's do one more example So let's call that example one. Let's do example two
26:06
so this Capital U of 0 1 means all these are uniformly distributed on the interval from 0 1 that means the distribution function of any one of these Looks like this
26:35
This is for all I
26:42
Okay, that's what uniform distribution is So let's write down the density for the smallest and the largest
27:21
Okay, the density for the largest one is here It's n times the common distribution function of n minus 1 times the density So we'll write this down for X between 0 and 1 this would be n
27:42
X to the n minus 1 and then what's the density for this for these random variables? 1 if you're here okay, so I multiply by 1 and of course this density is 0 if X is not in that interval. Okay, pretty simple one. This is f of X, right?
28:07
to the n minus 1 n capital F to the n minus 1 times little f little f is 1 Okay, what about for the smallest did you supply this formula n?
28:31
1 minus X to the n minus 1 times the density is 1 this is if 0 is between X is between 0 and 1 and
28:40
It's of course 0 if X is not in the interval from 0 1 Okay, so pretty easy to write down the Densities for the largest and smallest in a sample uniform
29:01
Okay, what about something in the middle? What about the ith largest? you think you can write down the density for the ith order statistic If you know the distribution function and the density so this I is some number could be 1 could be n
29:55
We've done those already, but probably somewhere in the middle like the third or the tenth or in the case of this room with 26
30:02
There's something like that There were 54 light bulbs in here X27 would be what that's called the median Right X27 would be the median. So be something like what's the distribution of the median or if we're
30:21
computing your scores your scores are random variables What's X1? X1 with parentheses, I mean X1 with parentheses would be the score of the best score X The N with parentheses would be the lowest score if there were
30:44
90 students in the class X45 with parentheses parentheses would be the Median scores you might be asking what's the distribution of the median? Which is an interesting statistic to look at a lot of times You in fact often ask me to publish. Well, I mean that you but you will and you probably ask your other professors
31:04
Can you publish the statistics on triple-e right? You've asked that probably you want to know what the median is So that means you're looking at the density Or something like this the ith order statistic where I might be n over 2. Okay
31:22
So enough talk how about some action how do you do this well Maybe I should draw a picture higher so people in the back can see
31:57
okay, so Well, here's a line
32:01
here's X the Density of the ith order statistic is the limit as Epsilon goes to 0 of this ratio
32:35
Why is that what's this new what's this oops
32:42
What's this ratio called Newton quotient? This is the this is computing the derivative. It's called the Newton quotient Which reminds me of a joke my son told me okay, so they're quiet your class is getting so quiet
33:01
I think I could liven you up a bit so Einstein Newton and Pascal are gonna play a game of hide-and-seek and And Einstein's going to be it first Newton and Pascal have to hide so Einstein closes his eyes counts to 100 Pascal runs way off in the distance somewhere hides behind a tree Newton stands right in front of Einstein, but he draws a square meter
33:27
Around himself and stands there right in front of Einstein. Einstein opens his eyes stares at Newton says ah Newton I found you and Newton says no no you found Newton over a square meter. That's a Pascal you found Pascal
33:50
Okay good now. This is the derivative right?
34:14
Okay, now For continuous random variables are probably finding two independent ones in a very short interval is about like
34:23
Epsilon squared or finding three in a very short interval is about like epsilon cubed Typically in a small interval you only find one in a sample of n. So if this happens You can ignore the possibility that there are other samples
34:41
Between here here when epsilon is really small so that means That our ith largest Falls in here somewhere and the rest fall outside of that interval
35:02
So where do the rest go are there going to be any over here? How many are gonna go over here? Well, let's suppose I is seven how many would have to go over there?
35:25
This means it's the seventh Largest maybe I should look over here how many if this is seven if I seven how many go bigger? This is the seventh largest. That means it's there are how many bigger ones? well
35:42
Bigger ones. Oh, yeah n minus seven. So this is over here. There would be n minus I Bigger ones right up to here. There are I so it means over here. There are n minus I samples Or random variables and down here. How many smaller ones are there I?
36:05
Minus one, so what does our Sample of n random variables have to do
36:32
We have three boxes our sample has to do what it has to place I minus one random variables here
37:02
One random variable here and n minus I random variables here, okay? Now for any particular Outcome where I minus one fall here. I what one falls here and minus I falls there. What's the probability of that event?
37:26
For any particular one. I mean it could be X2 X 5 X 100 X So I just pick out I minus one of them within here one goes here What's the probability of any one of them happening? Well? What's the probability that I minus one random variables fall below this level?
37:44
They're independent all right Probably that one falls below. There's capital f of x the fault probability that 2 would be f of x capital f of x squared the probability that I minus 1 fall below Would be f of x to the I minus 1
38:03
The probability that one falls in here is about Well, it's this it's this limit here, so let's just say it's f of x What's the probability that one falls in here? It's f of x times epsilon, but I'm going to
38:24
Divide by what what I should actually probably do is That would be the probability that one falls between x plus epsilon and x and then what's the probability that n minus one of them?
38:44
Are above here? Well, what's it probably any one of them is above here would be 1 minus f of x plus epsilon to the n minus I Okay
39:02
Now let's count how many ways there are to do that Distribute n things in these what three boxes where I put I minus 1 in this box One in this box and n minus I in that box have you ever seen anything like that before? You have capital and you have little n objects
39:21
And you're going to distribute them in three places I minus one in the first place one in the second place and and minus I and fast that's n choose I minus 1 1 and n minus I multinomial coefficient and we this is the
39:42
Probability that now we want to find the density so that means we want to divide by epsilon and let epsilon goes to 0 Now this number here is That factorial divided by this factorial times this factorial times that factorial so be n factorial
40:01
over I Minus 1 factorial times 1 factorial which is 1 so write it n minus I Factorial then we get the distribution function At raised to the power I minus 1 Here we get the density function And here we get 1 minus the distribution function
40:23
to the n minus I and If you can't read that I'll write it up here
41:21
So let's look at maybe the example of the uniform that's what the
42:27
Density looks like for the ith largest
43:21
okay, so This may seem like I'm asking a question out of blue. What's what's the value of that interval? Can you do that one well it must be related to what I was just talking about
43:51
So what was I just talking about well up here Is this related to that in any way?
44:03
Yes, I suppose because there's an X and there's a 1 minus X and they both have powers This has the power of 15 that has power 17 Can you choose an n and I so you get 15 and 17 here well this must be I?
44:24
Minus 1 So I must be 16 if I is 16 n is 30 33 so I is 16 n is 33
44:41
What is this this is this is what yeah, but What's the other word I'm looking for it's a density, I'm sorry, this is a density right what what happens when you integrate a density
45:06
Integrate to 1 so the integral this is 1 so What's this integral we get that for free
45:35
More or less I think this derivation is easier than doing that integral every see why?
45:47
This is I have if I invert this number and put it over here, that's a density. I'm integrating a density. I have to get one Okay, the integral, but just using the fact that the integral of this over zero one is one
46:02
And then that's true whatever and then I are so if I take I to be 16 and to be 33 we get this fact okay, also Let's think for just a moment about what these densities look like let's say
46:23
I is n over 2 Then for convenience let's take n even so that I as an integer then This is the density of the median. What do you think that looks like?
47:25
When it gets bigger and bigger and bigger I keep picking more and more points independently and uniformly in the interval from 0 to 1 do you think there? How do you think that those points would look? If you had a picture of them
47:40
You give the unit interval Take maybe 10,000 points in the interval independent identically distributed uniform distribution Would they be clustering somewhere or would they kind of be uniformly spread through the interval it should be uniformly spread
48:01
So where should the median be? Right about a half. So what does that mean about the density? I think the density will be flat If the density is flat that means the median is as likely to be one place as any other place So this will not be like function one
48:22
But rather the median is going to be getting more and more focused near a half. So what should the density be doing? It should be small away from half and get really big at a half It won't be normal, but if this is 0 from 0 to 1 it may I guess maybe
48:47
Vaguely it would look something like that. Whereas a very high narrow peak As n goes to infinity we get more and more concentrated near a half If X is a number between 0 1 what happens to X to the n over 2 it goes to
49:09
0 right Number less than 1 raised to high power goes to 0 this one goes to 0 But what prevents the whole thing from going to 0? This stuff out in front Okay, we'll do this later
49:26
There's an important formula called Stirling's formula It gives the asymptotic Expression for n factorial. I think there's a square root pi here or maybe it's root 2 pi
49:43
I can't remember. So let me just put a constant. I think it's root pi. So I'll go out on a limb and say root pi
50:05
n factorial looks like this n to the n plus a half e to the minus n if you use that in Here here and here You can deduce that the density begins to look like this very high peak around one-half and very narrow as n goes to infinity
50:21
For the other ones the largest and the smallest the largest one the density Looks like this and for the smallest one the density will look Like that. Thank you a question
50:44
No, so this would be n over 2 that's n here No, it'd be n over 2. Oh, it should be minus 1. Is that what you said? Sorry, yeah n over 2 minus 1. Yeah, I forgot the minus 1
51:05
Okay So one one last little flourish and then we'll take a break
51:27
suppose I is less than J, what would the and little x then less than little y
51:43
What about the joint density of the ith order statistic and the jth Here you might be interested in quartiles like I might be n over 4 and J might be 3 n over 4 so this would be like our how are the four?
52:04
first and last third quartile Markers distributed well for this It's not like that
52:21
derivation there We have to take our sample of size n Put place one sample here that happens with probably let's say a little of that little f of x heuristically We have to put one here that happens with probably a little f of y So that's two gone, and they're n all together the ith largest goes here
52:44
So how many go over here? This is this is where x I will go The ith largest will fall here how many fall over here then I minus one How many fall over here if this is going to be the jth largest and minus J?
53:06
So so far we've used n minus J. Plus I minus 1 plus 2 Right I minus 1 over there 1 here 1 here n minus J. That's n minus J plus I plus 1
53:22
Go in here, okay So the probability of any particular sample of doing this would be Well the probability you have I minus 1 down here would be capital f of x to the I minus 1 probably getting one
53:41
There's little f of x Probably getting this many in here would be well the probably getting one in here would be f of y minus f of x That's the probably getting one random variable here probably getting that many in there where they're independent would be this number to
54:08
that power Probably getting one here would be f little f of y And then we have n minus J out there the probably getting one bigger than y would be
54:24
1 minus capital f of y But we get this many of them, and they're independent so We get that now We just have to count how many ways are they put I minus 1 in that box one in this one and minus J plus I minus 1 here 1 here and and
54:42
Minus J out there, and that's this multinomial coefficient and choose I minus 1 1 I'm running out of room here, but this should be n minus J plus I plus 1 and Here and then 1 and then next year would go n minus J. I guess they have room for that
55:05
Okay, so that's the multinomial coefficient, so it'd be n factorial I minus 1 factorial 1 factorial this factorial 1 factorial J factor on
55:20
Okay, so this could be used Maybe I'll do an example when we come back It's a lot like the previous derivation before we just had
55:44
The ith one goes here I minus one had to go there, and then the rest had to go here now. We have two things to to fix and Because this is the ith largest going here The jth largest going here. We know how many go here here and here And we can compute the probabilities of each one of those. This is the probability of having I
56:05
Minus 1 in here. This is the probability of having that many in here This is the probability of having That many Out there, okay, and then this would be sort of the probability of landing exactly here probably of landing exactly there
56:29
I thought this would mean that's the first but it means this is smallest x1 is the smallest not the largest Sorry, so in your notes reverse that inequality
56:41
Now what else do you have to change? Here we divide x1 the density for x sub 1 before was this Just change right over one put an end there and where you had n change that to one All the arguments work. You just have to change the labels
57:03
Sorry about that so x sub Parenthesis n is the largest So the density for the largest is this before I said that would be x sub parenthesis one so you know let's put parentheses n here and before I said this was the Density for the largest, but I saw her for the xn, but it's
57:24
Actually x1. That's the smallest So you had to put changes from n to 1 and then I? Rushed in the end here. I put the wrong power here This was this this was a probability falling between here here. I had to count how many were in there I
57:47
Counted how many weren't in there? There were I minus I minus one over here n minus J here one here and One here that meant the remainder right here. This is how many I've used so far n minus J
58:05
plus I minus 1 plus 1 plus 1 is 2 that's how many have been used how many haven't been used and minus that and minus that so this should be and Minuses to become J minus I minus 1
58:22
J minus I minus 1 sorry about that So again, I wrote the inequalities the wrong direction. I thought that this would be the largest that the smallest, but it's actually the opposite All the arguments still are true. Just have to change the label here before in your notes. It would have x parenthesis 1 and
58:41
This would have had in your notes x parenthesis n and changes to x parenthesis n and one switch them and then again I Power here is J minus I minus 1 okay, so
59:11
Another another statistic that's of interest is that
59:20
What would that be that's the largest minus the smallest that's called the range
59:43
How can you find the distribution of a difference of two random variables? Well you need to know their Joint distribution
01:00:16
OK, so for those who came in late, I reversed the order.
01:00:21
I got the order wrong. X sub parenthesis N is the largest. X sub parenthesis 1 is the smallest. So that means you have to reverse the labels here and here for the densities.
01:00:43
OK, so here we have the joint density for the largest and smallest if we take what here and here? 1 and N. OK. But if you have a joint density, what's the probability that X minus Y is less than or equal to X if you have a joint density for these two?
01:01:16
It'd be the integral over the set of pairs S, T such that S minus T is less than or equal to X.
01:01:28
F sub X, Y of S, T, dS, dT. The joint density, the probability that the pair X, Y is in a set A is the integral over A of the joint density.
01:01:54
Here our set A is a set here. So we integrate the density over the set described in the event.
01:02:08
About the only thing you have to keep in mind here is you have to set up this integral. So it usually pays to draw a picture. What are the pairs S, T where S is less than or equal to T times X?
01:02:30
I'm sorry, less than or equal to X. S minus T is less than or equal to X. So that's a line. It's a line. Well, the boundary of it is what line?
01:02:40
T equal to S minus X is probably the easiest way. That's a line of slope one. When S is equal to X, T is zero. When S is equal to zero, T is minus X.
01:03:08
So that'd be this line. And what side of that line are we interested in? We're interested in the set where T is bigger or equal to S minus X.
01:03:25
So, right, T bigger or equal to S minus X. What side of that line is the set A then, where S minus T is less than or equal to X? Well, always we always sample one point. It's one side or the other.
01:03:40
How about the point S, T equal to zero, zero? Does that satisfy this inequality? When X is positive, it does, right? So it'd be this side. Okay, so how could we describe that integral? What limits of integration should we have?
01:04:08
Well, it looks like we can let S go from zero to infinity. And then for a particular value of S, what does Y do?
01:04:24
It starts down here and goes to infinity, right? So what would the coordinates of this point be? It's on this line. The first coordinate is S. The second coordinate is T or S minus X.
01:04:45
So Y would go from whatever the Y coordinate is here up to infinity. The Y coordinate there is S minus X. So it goes from S minus X to infinity. And then we just put the density here.
01:05:07
And we integrate Y first and then S.
01:05:31
So we'll stick with this example here. Suppose that we have a uniform sample, independent, identically distributed.
01:05:47
The joint density for the pair, the smallest and the largest, would be?
01:06:02
Well, we use this formula here with i equal to 1 and j equal to n. Now, what is this constant out there?
01:06:24
That's, I'll write it down here, n. i is 1, so I get 0 and then 1. And then j is n, right?
01:06:45
Oh, this is the wrong thing here. It's n minus this, right? It's n minus that. Remember I counted how many we've used. And this would be how many, n minus that would be how many is left. So n minus that or j minus i plus 1.
01:07:01
So here we have n, here we have 1. So n minus 1 plus 1 is, let's see. And so j, oh minus 1, thank you, yeah.
01:07:25
We should have, oh yeah, n minus 2, right. That's good. Why is that good? How many are between the smallest and the largest? n minus 2.
01:07:43
And then 0 above, so this would be n factorial over n minus 2 factorial. And that's n times n minus 1.
01:08:05
Alright, 0 factorial is 1 always. 0 factorial is 1. So we get n factorial over 0 factorial times 1 factorial times this factorial times that factorial times that factorial. And that's n times n minus 1. So we get n, n minus 1.
01:08:21
What else? Here we have the uniform, so capital F is, capital F of x is x to the 1 minus 0, that'd be 1. This density for the uniform is 1. Here we get y minus x, right?
01:08:44
Capital F of y is y, capital F of x is x. Y minus x to the, what's this number here? That's n minus 2, right?
01:09:04
And then we get finally the density at y, that's 1. And here we get this term to the n minus n, or 0. That's 1 again. So there's the density for the range.
01:09:20
And here, what should be the restrictions on x and y? Little x is for capital X sub parenthesis 1, little y is for capital X sub parenthesis n. So this is for the, this random variable, this is for that random variable. What do we know about the order of these two? Which is bigger? This one?
01:09:42
So x should be less than y, and they should be between 0 and 1. So that's the joint density for the pair.
01:10:03
And, um, okay, exercise.
01:10:26
You just use this, applied to capital X equal X sub parenthesis n, capital Y equal X sub parenthesis 1. So time to move on to one of the more important chapters in the book, expectations.
01:11:38
This means integrals.
01:11:46
Okay, so let's start with a discrete random variable. Suppose x is a discrete random variable with probability mass function little p.
01:12:37
Define the expected value of x, e of x.
01:12:40
We read this as the expected value of x. It's the weighted average of the values x could have times the probability it has that value. The value that x could have times the probability it takes on that value. So, simplest example, if the PMF is 1, 6 for values x equal 1, 2, 3, 4, 5, 6,
01:13:21
what kind of random variable would we be looking at? Yeah, roll of a die. What's the expected size of the roll? That's what this is, expected size of the roll. Before we compute this, what do you think it should be?
01:13:41
I heard the right answer somewhere. 3.5. Because that's what? That's the average roll, isn't it? In this case. And what's the sum of the integers from 1 to any particular one?
01:14:24
And the sum of the integers from 1 to n is n times n plus 1 over 2. Here n is 6, so we get 6 times 7 over 2 or 7 halves.
01:14:45
Let's do another one. If x is Poisson with parameter lambda distributed,
01:15:02
well, the values of the Poisson random variable can take on are the non-negative integers or natural numbers, x equals 0 to infinity. And then we get an x here times p of x. p of x for Poisson is lambda to the x over x factorial e to the minus lambda.
01:15:55
And what would that be? Well, we have times x here and we start at 0, so we could just as well start at 1
01:16:06
because the contribution from the term x equals 0 is 0. And then we have this lambda to the x. I'll pull out the e to the minus lambda because it doesn't depend on the index of summation.
01:16:24
And I have x over x factorial. What's that? x minus 1 factorial. Now, you might recall from your calculus days,
01:16:46
e to the lambda is this sum. Unfortunately, this sum is not that sum, but it's not far.
01:17:02
Okay, good idea. Pull out a lambda because that e matches this power to that power, right? If I pull out one power of lambda, I'll have x minus 1 left. So I pull out a lambda.
01:17:21
And then what? Now I shift, put y equal to x minus 1 or j equal to x minus 1, I guess. If x is 1, j is 0. And j has to go to infinity because x does. I get lambda to the j over j factorial.
01:17:43
This is e to the lambda. That's e to the minus lambda, so the answer is lambda. The expected value of Poisson with parameter lambda is lambda.
01:18:10
One more example. Let's do geometric, where the pmf of x, capital x, is this.
01:18:45
Where p is the number between 0 and 1. So here we want to compute this sum.
01:19:05
I'm sorry, here we start at 1, not 0. I compute that.
01:19:23
Well, I can factor out p because every term has p in it.
01:20:02
But each term here is minus this derivative. Is that right? If I differentiate this, I get an x coming down. There it is. I subtract 1 from this and then I multiply by the derivative of 1 minus p.
01:20:23
That's minus 1. That's why this is here. Trust me that you can exchange the sum and derivative here.
01:20:47
But actually in the previous step, does it change anything if I do that? Change the index of summation from 1 to 0. What's the term here when x is 0? I get 1 minus p to the 0 which is 1.
01:21:00
What's the derivative of 1 with respect to p? 0. So it doesn't change anything if I do that. So I'll do that here too. Because now I know how to evaluate this series using a geometric, it's a geometric series. This is 1 over 1 minus, 1 minus p.
01:21:28
This should be p here. The derivative with respect to p. The derivative with respect to p here. Which is, let's see, we get minus p, derivative with respect to p.
01:21:44
1 minus p is p so we get 1 over, derivative with respect to p of 1 over p. That's 1 over, minus 1 over p squared so we get 1 over p. That's the expected value for a geometric. Does that make sense?
01:22:01
Think of rolling a die until you get a 1. The roll on which you get your first 1, that's a geometric random variable. What's p in that case? 1, 6. 1, 6. How long do you expect to wait until you get a 1? 1 over p.
01:22:20
p is 1, 6, 6 times. Or if you're tossing a coin and probably heads is half, tails is half. And you're waiting until you get your first heads. That random variable, the trial on which you get the first heads would be a geometric with parameter p equal half. How long do you expect to wait until you get first heads?
01:22:41
1 over 1 half or 2 tosses. Does that make sense? Or, what's the chance of winning in the lottery? p is about, give me an estimate, 1 in a billion. What do you think? I mean, you have to match 6 numbers.
01:23:04
What's the probability of match? Is that how it works? And the numbers are from what to what? 100? 1 to 100? Anybody do the lottery over here? 1 to 100? You can pick any number from 1 to 100?
01:23:22
1 to 80? Okay, so the probability of matching would be 1 over 80 to the 6th. Probability of winning would be 1 over 80 to the 6th. Suppose you can pick 80 numbers and you have to match all 6. p is 1 over 80 to the 6th. So how many trials do you have to have or expect to have before you win?
01:23:43
80 to the 6th. Okay, well in my lottery you can. Okay. So is that a lot of trials? Do you think you can do that in your lifetime?
01:24:03
Which will come first? The end of your life or winning the lottery probably? Well, I guess, yeah, if you can buy 80 to the 6th tickets, you probably have an even chance of winning.
01:24:21
Okay, so some of the things in this course I hope you take home with you. Like, is it a good idea to play the lottery? Should you gamble in Vegas? Well, if you like gambling, go ahead, but try to lose your money as slowly as possible and have fun.
01:24:40
Okay, so those are a couple of examples of expected values for discrete random variables. If you have a continuous random variable, it's the interval of x times the density over the whole line, provided this is finite.
01:26:19
Now, do we need to integrate over the whole interval from minus infinity to infinity in this case?
01:26:24
No, the density is only non-zero from 0 to 1, so just integrate over the range or the set where the density is different from 0, so that'd be from 0 to 1. Do we need to worry about this condition in this case, or is it satisfied?
01:26:41
It's satisfied because this is a number less than or equal to 1, right? And we're only integrating from 0 to 1. And then we get x here, dx, and what is that equal to? x squared over 2 evaluated between 0 and 1 gives you half. Does that make sense?
01:27:01
The average value of a number picked at random from 0 to 1 would be half.
01:27:47
Let's find the expected value of the smallest, the median, and the largest if you have an i.i.d. sample from the uniform. Now, what was the density for the largest?
01:28:11
Remember here, all of them had to be less than or equal to... The general formula is this, so for uniform, this would be n, this would be x, and this is 1, provided x is between 0 and 1.
01:28:29
So here, what would the limits of integration be for uniform? So, where is this random variable going to be? Between 0 and 1?
01:28:42
We take x, and then we multiply by the density of this guy, which is there. n, x to the n minus 1, and then little f of x, density for uniform is 1 on the interval from 0 to 1, dx. So we get n, integral from 0 to 1, x to the n, dx.
01:29:08
And this would be what? n, x to the n plus 1 over n plus 1, the value of between 0 and 1. Is that right?
01:29:29
And what's that? That's n over n plus 1. The expected value of the largest is n over n plus 1. That's just a little bit less than 1.
01:29:44
That's 1 minus 1 over n plus 1, isn't it? What's it doing as n goes to infinity? Does that make sense? The largest one would be crowding up against 1. What would the expected value of the smallest one be? Do we need to do the computation?
01:30:03
Is there any symmetry in this problem? What if I reflect around a half? If I take a uniformly distributed random variable and then if it wound up being, say, three-fourths, if instead I flipped it and made it a fourth, would that still be uniformly distributed?
01:30:22
Yes, it's still uniform. So what do you think the expected value of x1 would be? No? How far is this one from 1? 1 over n plus 1. How far would the smallest be from 0? 1 over n plus 1.
01:30:42
And now here we integrate from 0 to 1, x. And what was the density for the median?
01:31:06
Remember the ith one had density n factorial over i minus 1 factorial, n minus i factorial, f of x to the i minus 1, little f of x, 1 minus f of x to the n minus i here?
01:31:35
So for the case of uniform, i is n over 2, so we get this.
01:32:16
Capital F is x, little f is 1.
01:32:22
So we'd have to integrate x times that density. Well, there are all these fancy factorials out in front.
01:32:40
Then we have the x and then times the density, x to the n over 2 minus 1, 1 minus x to the n over 2 dx. This is what? The numbers out in front don't change and we have to find out what's this integral, x to the n over 2, 1 minus x to the n over 2 dx.
01:33:21
How do we compute this integral? Well, we know this is a density. I know this is a density.
01:33:52
If I integrate this from 0 to 1, I get 1. So what you have to do is pick i and n so that i minus 1 is n over 2 and n minus i.
01:34:07
Well, maybe I'll call this m here. m minus i is n over 2.
01:34:22
This is a density. So I want to integrate this thing where I have an n over 2 here and an n over 2 there. So I have to pick i and m so that i minus 1 is n over 2 and m minus i is n over 2. So that means that i is n over 2 plus 1 and m is what?
01:34:46
Well, I move this over, I get n plus 1. m is n over 2 plus i. n over 2 plus n over 2 plus 1 is n plus 1.
01:35:06
So what does that tell me? That tells me the integral from 0 to 1, x to the n over 2, 1 minus x to the n over 2, dx is n plus 1 factorial down here.
01:35:28
i minus 1 is n over 2 factorial and m minus i factorial is n over 2 factorial.
01:35:44
So I just evaluated that. Now I have to multiply it by that. What did we get? We get n factorial over n over 2 minus 1 factorial, n over 2 minus 1, n over 2 factorial, and then I divide, I have to multiply by that.
01:36:13
n over 2 factorial, n over 2 factorial, and then n plus 1 factorial.
01:36:22
What's this equal to? Well here I get an n plus 1. These cancel. And up here I get what? n over 2. What's this over this? This number is 1 more than that one.
01:36:43
So I just get the n over 2. The rest of this cancels. And what does this look like? It's a half times n over n plus 1. It's almost a half.
01:37:13
Now for my favorite, the gamma distribution.
01:37:50
There's the density. Let's compute the expected value of x.
01:38:12
Here we only integrate over the place where the density is bigger than zero. The density is zero for x negative. So we integrate from zero to infinity, x times the density.
01:38:23
Well, the density has some constants. I'll put those out in front. Then I get x and then x to the alpha minus 1 times e to the minus lambda x dx. Now, I'm supposed to worry about whether this is finite when I put an absolute value here.
01:38:43
But we're going from zero to infinity so I only have to worry about whether this is finite. I don't have to put absolute value. Is this integral going to be finite, do you think? What do we have? We have something that's growing like a polynomial times something that's decaying like an exponential. What wins in a big way?
01:39:02
Yeah, the exponential wins. So this integral is finite. Don't have to worry about it. So let's combine the powers of x here.
01:39:21
I get x to the alpha. And how do you do this integral? The answer is you don't. Because you don't have to. It's already been done for you. This is a density, right?
01:39:40
So if I integrate this from zero to infinity, I get 1. Now, that means if you integrate the part without the constant from zero to infinity, what do you get? Yeah, you get 1 over that constant. So we know what integrals like this are. They're this.
01:40:04
Now, is this the integral we want to evaluate over there? Not quite. Here it's a power of x minus 1. So over there we want to have a power of x minus 1. Do we have that? Not yet. But wait.
01:40:21
Don't watch. Now we do. Now we have a power of x minus 1. What's the power? Alpha plus 1. I change alpha into alpha plus 1 minus 1.
01:40:40
So now I just use that factor about densities. And over here I get lambda to the alpha plus 1 and then gamma of alpha plus 1. And now, remember the property of gamma functions?
01:41:02
They're like factorials. In fact, the integer is gamma of n is n minus 1 factorial. But generally, gamma of alpha plus 1 is alpha times gamma of alpha. That comes from integration by parts. So this over this would just be alpha.
01:41:23
And this over this is lambda. Now, what is the gamma 1 lambda distribution?
01:41:50
Yeah, it's just exponential. Alpha is 1. You have nothing there, right? That would be the density e to the minus lambda x. And then you have a lambda. And what's gamma of 1? It's 1.
01:42:00
So it's exponential. Exponential is the answer. Well, not exponential factor. Exponential. So if x is exponentially distributed to the parameter lambda, the expected value of x is what? We just take alpha equal 1 here. We get 1 over lambda.
01:42:29
OK. And we'll stop there.