【概率论基础进阶】大数定律和中心极限定理

切比雪夫不等式

切比雪夫不等式：设随机变量$X$的数学期望$E(X)$和方差$D(X)$存在，则对任意的$ϵ>0$，总有
P { ∣ X − E ( X ) ∣ ≥ ϵ } ≤ D ( X ) ϵ 2 P \left\{|X-E(X)|\geq \epsilon \right\}\leq \frac{D(X)}{\epsilon ^{2}} P{∣X−E(X)∣≥ϵ}≤ϵ2D(X)​
这个不等式称为切比雪夫不等式

例1：设随机变量$X$的概率密度 f ( x ) = { 2 e − 2 x x > 0 0 x ≤ 0 f(x)=\left\{

\right. f(x)={​2e−2x0​x>0x≤0​

• 根据切比雪夫不等式估计 P { X ≥ 3 2 } ≤ A , A = ( )
  P{X≥32}≤A,A=()" role="presentation" style="position: relative;">P{X≥32}≤A,A=()$$\begin{array}{r}P\left\{X\ge \frac{3}{2}\right\}\le A,A=()\end{array}$$
  P{X≥23​}≤A,A=()​

$X\sim E(2)$，因此
P { X ≥ 3 2 } = P { X − 1 2 ≥ 1 } = P { X − 1 2 ≥ 1 } + P { X − 1 2 ≤ − 1 } ⏟ 0 = P { ∣ X − 1 2 ∣ ≤ 1 } = P { ∣ X − E X ∣ ≥ 1 } ≤ D X 1 2 = 1 4

P{X≥23​}​=P{X−21​≥1}=P{X−21​≥1}+0 P{X−21​≤−1}​​=P{∣ ∣​X−21​∣ ∣​≤1}=P{∣X−EX∣≥1}≤12DX​=41​​

• 直接计算 P { X ≥ 3 2 } = B , B = ( )
  P{X≥32}=B,B=()" role="presentation" style="position: relative;">P{X≥32}=B,B=()$$\begin{array}{r}P\left\{X\ge \frac{3}{2}\right\}=B,B=()\end{array}$$
  P{X≥23​}=B,B=()​

根据指数分布
P { X > t } = e λ t , t > 0 P \left\{X>t\right\}=e^{\lambda t},t>0 P{X>t}=eλt,t>0
因此
$$P\left\{X\ge \frac{3}{2}\right\}=e^{-2\cdot \frac{3}{2}}=e^{-3}$$

e − 3 ≈ 0.05

e−3≈0.05" role="presentation" style="position: relative;">e−3≈0.05$$\begin{array}{r}{e}^{-3}\approx 0.05\end{array}$$

e−3≈0.05​，而切比雪夫不等式估计的是 0.25

0.25" role="presentation" style="position: relative;">0.25$$\begin{array}{r}0.25\end{array}$$

0.25​，显然是由一定差距的

例2：设$X$的密度为$f(x),DX=1$，而$Y$的密度为$f(-y)$，且$X$与$Y$的相关系数为 − 1 4

−41​​，用切比雪夫不等式估计 P { ∣ X + Y ∣ ≥ 2 } ≤ ( ) P \left\{|X+Y| \geq 2\right\}\leq () P{∣X+Y∣≥2}≤()

设$Z=X+Y$，由切比雪夫不等式
P { ∣ Z − E Z ∣ ≤ ϵ } ≤ D X ϵ 2 P { ∣ ( X + Y ) − E ( X + Y ) ∣ ≤ ϵ } ≤ D ( X + Y ) ϵ 2

P{∣Z−EZ∣≤ϵ}P{∣(X+Y)−E(X+Y)∣≤ϵ}​≤ϵ2DX​≤ϵ2D(X+Y)​​
E Y = ∫ − ∞ + ∞ y f ( − y ) d y = ∫ − ∞ + ∞ ( − y ) f ( − y ) d ( − y ) = − y = t − ∫ − ∞ + ∞ t f ( t ) d t = − E X

EY=∫−∞+∞yf(−y)dy=∫−∞+∞(−y)f(−y)d(−y)=−y=t−∫−∞+∞tf(t)dt=−EX" role="presentation" style="position: relative;">EY=∫+∞−∞yf(−y)dy=∫+∞−∞(−y)f(−y)d(−y)=−y=t−∫+∞−∞tf(t)dt=−EX$$\begin{array}{rl}EY& ={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}yf(-y)dy\\ & ={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}(-y)f(-y)d(-y)\\ & \stackrel{-y=t}{=}-{\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}tf(t)dt\\ & =-EX\end{array}$$

EY​=∫−∞+∞​yf(−y)dy=∫−∞+∞​(−y)f(−y)d(−y)=−y=t−∫−∞+∞​tf(t)dt=−EX​
因此$EZ=E(X+Y)=EX+EY=0$
D Y = E ( Y 2 ) − ( E Y ) 2 = ∫ − ∞ + ∞ y 2 f ( − y ) d y − ( − E X ) 2 = − y = t ∫ + ∞ − ∞ ( − t ) 2 f ( t ) d ( − t ) − ( E X ) 2 = E ( X 2 ) − ( E X ) 2 = D X D ( X + Y ) = D X + D Y + 2 cov ( X , Y ) = D X + D Y + 2 D X D Y ⋅ ρ X Y = 1 + 1 − 1 2 = 3 2

DY=E(Y2)−(EY)2=∫−∞+∞y2f(−y)dy−(−EX)2=−y=t∫+∞−∞(−t)2f(t)d(−t)−(EX)2=E(X2)−(EX)2=DXD(X+Y)=DX+DY+2cov(X,Y)=DX+DY+2DXDY⋅ρXY=1+1−12=32" role="presentation" style="position: relative;">DYD(X+Y)=E(Y2)−(EY)2=∫+∞−∞y2f(−y)dy−(−EX)2=−y=t∫−∞+∞(−t)2f(t)d(−t)−(EX)2=E(X2)−(EX)2=DX=DX+DY+2cov(X,Y)=DX+DY+2DX−−−√DY−−−√⋅ρXY=1+1−12=32$$\begin{array}{rl}DY& =E(Y^2)-(EY{)}^2\\ & ={\int }_{-\mathrm{\infty }}^{+\mathrm{\infty }}{y}^{2}f(-y)dy-(-EX{)}^2\\ & \stackrel{-y=t}{=}{\int }_{+\mathrm{\infty }}^{-\mathrm{\infty }}(-t{)}^{2}f(t)d(-t)-(EX{)}^2\\ & =E(X^2)-(EX{)}^2=DX\\ D(X+Y)& =DX+DY+2\text{cov}(X,Y)\\ & =DX+DY+2\sqrt{DX}\sqrt{DY}\cdot {\rho }_{XY}\\ & =1+1-\frac{1}{2}=\frac{3}{2}\end{array}$$

DYD(X+Y)​=E(Y2)−(EY)2=∫−∞+∞​y2f(−y)dy−(−EX)2=−y=t∫+∞−∞​(−t)2f(t)d(−t)−(EX)2=E(X2)−(EX)2=DX=DX+DY+2cov(X,Y)=DX+DY+2DX ​DY ​⋅ρXY​=1+1−21​=23​​
因此
P { ∣ X + Y ∣ ≤ 2 } = P { ∣ ( X + Y ) − E ( X + Y ) ∣ ≤ 2 } ≤ D ( X + Y ) 2 2 = D ( X + Y ) 4 = 3 8

P{|X+Y|≤2}=P{|(X+Y)−E(X+Y)|≤2}≤D(X+Y)22=D(X+Y)4=38" role="presentation" style="position: relative;">P{|X+Y|≤2}=P{|(X+Y)−E(X+Y)|≤2}≤D(X+Y)22=D(X+Y)4=38$$\begin{array}{rl}P\left\{|X+Y|\le 2\right\}& =P\left\{|(X+Y)-E(X+Y)|\le 2\right\}\\ & \le \frac{D(X+Y)}{2^2}=\frac{D(X+Y)}{4}=\frac{3}{8}\end{array}$$

P{∣X+Y∣≤2}​=P{∣(X+Y)−E(X+Y)∣≤2}≤22D(X+Y)​=4D(X+Y)​=83​​

大数定律

定义：设$X_1,X_2,\cdots ,X_n,\cdots$是一个随机变量序列，$A$是一个常数，如果对任意$ϵ>0$，有
lim ⁡ n → ∞ P { ∣ X n − A ∣ < ϵ } = 1 \lim\limits_{n\to\infty}P \left\{|X_{n}-A|<\epsilon \right\}=1 n→∞lim​P{∣Xn​−A∣<ϵ}=1
则称随机变量序列$X_1,X_2,\cdots ,X_n,\cdots$依概率收敛域常数$A$，记作$X_n\stackrel{P}{\to }A$

随机变量加减一个常数还是随机变量

切比雪夫大数定律：设$X_1,X_2,\cdots ,X_n,\cdots$为两两不相关的随机变量序列，存在常数$C$，使$D(X_1)\le C(i=1,2,\cdots )$，则对任意$ϵ>0$，有
$$\underset{n\to \infty }{\lim }P\left\{\left|\frac{1}{n}\sum _{i=1}^n{X}_i-\frac{1}{n}\sum _{i=1}^{n}E(X_i)\right|<ϵ\right\}=1$$

不太严谨的做题时可以写成 1 n ∑ i = 1 n x i → P 1 n ∑ i = 1 n E ( X i )

1n∑i=1nxi→P1n∑i=1nE(Xi)" role="presentation" style="position: relative;">1n∑i=1nxi→P1n∑i=1nE(Xi)$$\begin{array}{r}\frac{1}{n}\sum _{i=1}^n{x}_i\stackrel{P}{\to }\frac{1}{n}\sum _{i=1}^{n}E(X_i)\end{array}$$

n1​i=1∑n​xi​→Pn1​i=1∑n​E(Xi​)​

伯努利大数定理：设随机变量$X_n\sim B(n,p),n=1,2,\cdots$，则对于任意$ϵ>0$，有
$$\underset{n\to \infty }{\lim }P\left\{\left|\frac{X_n}{n}-p\right|<ϵ\right\}=1$$

即$X_n=Y_1+Y_2+\cdots +Y_n,Y_i\sim B(1,p)$，则有 1 n ∑ i = 1 n Y i = X n n → P p

1n∑i=1nYi=Xnn→Pp" role="presentation" style="position: relative;">1n∑i=1nYi=Xnn→Pp$$\begin{array}{r}\frac{1}{n}\sum _{i=1}^n{Y}_i=\frac{X_n}{n}\stackrel{P}{\to }p\end{array}$$

n1​i=1∑n​Yi​=nXn​​→Pp​

辛钦大数定律：设随机变量$X_1,X_2,\cdots ,X_n,\cdots$独立同分布，具有数学期望$E(X_i)=\mu ,i=1,2,\cdots$，则对于任意$ϵ>0$有
$$\underset{n\to \infty }{\lim }P\left\{\left|\frac{1}{n}\sum _{i=1}^n{X}_i-\mu \right|<ϵ\right\}=1$$

有 1 n ∑ i = 1 n x i → P μ

1n∑i=1nxi→Pμ" role="presentation" style="position: relative;">1n∑i=1nxi→Pμ$$\begin{array}{r}\frac{1}{n}\sum _{i=1}^n{x}_i\stackrel{P}{\to }\mu \end{array}$$

n1​i=1∑n​xi​→Pμ​

切比雪夫大数定律要求随机变量两两不相关，且方差存在
辛钦大数定律要求随机变量独立同分布，且期望存在
做题时看题目给出条件符合哪个，用哪个

例3：设随机变量$X_1,X_2,\cdots ,X_n,\cdots$相互独立，均服从分布函数
F ( x , θ ) = { 1 − e − x 2 θ x ≥ 0 0 x < 0 , θ > 0 F(x,\theta )=\left\{

\right.,\theta >0 F(x,θ)=⎩ ⎨ ⎧​​1−e−θx2​0​x≥0x<0​,θ>0
是否存在实数$a$，使得对任何$ϵ>0$都有 lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0

limn→∞P{|1n∑i=1nXi2−a|≥ϵ}=0" role="presentation" style="position: relative;">limn→∞P{∣∣∣1n∑i=1nX2i−a∣∣∣≥ϵ}=0$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}P\left\{\left|\frac{1}{n}\sum _{i=1}^n{X}_i^2-a\right|\ge ϵ\right\}=0\end{array}$$

n→∞lim​P{∣ ∣​n1​i=1∑n​Xi2​−a∣ ∣​≥ϵ}=0​

由题目可知
lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0 ⇒ lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ < ϵ } = 1

n→∞lim​P{∣ ∣​n1​i=1∑n​Xi2​−a∣ ∣​≥ϵ}=0⇒n→∞lim​P{∣ ∣​n1​i=1∑n​Xi2​−a∣ ∣​<ϵ}=1​
有
f ( x ; θ ) = F ′ ( x ; θ ) = { 2 x θ e − x 2 θ x ≥ 0 0 x < 0 E X i 2 = ∫ − ∞ + ∞ x 2 f ( x ; θ ) d x = ∫ 0 + ∞ x 2 ⋅ 2 x θ e − x 2 θ = x 2 θ = t θ ∫ 0 + ∞ t e − t d t = θ

\begin{aligned} f(x;\theta )&=F'(x;\theta )=\left\{\begin{aligned}& \frac{2x}{\theta }e^{- \frac{x^{2}}{\theta }}&x \geq 0\\&0&x<0\end{aligned}" role="presentation" style="position: relative;">\begin{aligned} f(x;\theta )&=F'(x;\theta )=\left\{\begin{aligned}& \frac{2x}{\theta }e^{- \frac{x^{2}}{\theta }}&x \geq 0\\&0&x<0\end{aligned}$$\text{begin{aligned} f(x;theta )\&=F'(x;theta )=left{begin{aligned}\& frac{2x}{theta }e^{- frac{x^2}{theta }}\&x geq 0\&0\&x<0end{aligned}}$$

\right.\\ EX_{i}^{2}&=\int_{-\infty}^{+\infty}x^{2}f(x;\theta )dx\\ &=\int_{0}^{+\infty}x^{2}\cdot \frac{2x}{\theta }e^{- \frac{x^{2}}{\theta }}\\ &\overset{ \frac{x^{2}}{\theta }=t}{=}\theta \int_{0}^{+\infty}t e^{-t}dt=\theta \end{aligned} f(x;θ)EXi2​​=F′(x;θ)=⎩ ⎨ ⎧​​θ2x​e−θx2​0​x≥0x<0​=∫−∞+∞​x2f(x;θ)dx=∫0+∞​x2⋅θ2x​e−θx2​=θx2​=tθ∫0+∞​te−tdt=θ​
因此期望$E{X}_1^2=\theta$存在，又因为$X_1,X_2,\cdots ,X_n,\cdots$相互独立，均服从同一分布函数，因此$X_1^2,X_2^2,\cdots ,X_n^2,\cdots$独立同分布。根据辛钦大数定律，当$n\to \infty$时，
$$\frac{1}{n}\sum _{i=1}^n{X}_i^2\stackrel{P}{\to }E{X}_i^2=\theta$$
即对任何$ϵ>0$，都有 lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0

limn→∞P{|1n∑i=1nXi2−a|≥ϵ}=0" role="presentation" style="position: relative;">limn→∞P{∣∣∣1n∑i=1nX2i−a∣∣∣≥ϵ}=0$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}P\left\{\left|\frac{1}{n}\sum _{i=1}^n{X}_i^2-a\right|\ge ϵ\right\}=0\end{array}$$

n→∞lim​P{∣ ∣​n1​i=1∑n​Xi2​−a∣ ∣​≥ϵ}=0​

中心极限定理

列维-林德伯格中心极限定理：设随机变量$X_1,X_2,\cdots ,X_n,\cdots$独立同分布，具有数学期望与方差，$E(X_n)=\mu ,D(X_n)={\sigma }^2,n=12,\cdots$，则对于任意实数$x$，有
$$\underset{n\to \infty }{\lim }P\left\{\frac{\sum _{i=1}^n{X}_i-n\mu }{\sqrt{n}\sigma }\le x\right\}=\Phi (x)$$

定理表明当$n$充分大时$\sum _{i=1}^n{X}_i$的标准化 ∑ i = 1 n X i − n μ n σ

∑i=1nXi−nμnσ" role="presentation" style="position: relative;">∑i=1nXi−nμn−−√σ$$\begin{array}{r}\frac{\sum _{i=1}^n{X}_i-n\mu }{\sqrt{n}\sigma }\end{array}$$

n ​σi=1∑n​Xi​−nμ​​近似服从标准正态分布$N(0,1)$，或者说$\sum _{i=1}^n{X}_i$近似的服从$N(n\mu ,n{\sigma }^2)$

棣莫弗-拉普拉斯中心极限定理：设随机变量$X_n\sim B(n,p)(n=1,2,\cdots )$，则对于任意实数$x$，有
$$\underset{n\to \infty }{\lim }P\left\{\frac{X_n-np}{\sqrt{np(1-p)}}\le x\right\}=\Phi (x)$$
其中$\Phi (x)$是标准正态的分布函数

定理表明当$n$充分大时，服从$B(n,p)$的随机变量$X_n$经标准化后得 X n − n p n p ( 1 − p )

Xn−npnp(1−p)" role="presentation" style="position: relative;">Xn−npnp(1−p)−−−−−−−−√$$\begin{array}{r}\frac{X_n-np}{\sqrt{np(1-p)}}\end{array}$$

np(1−p) ​Xn​−np​​近似服从标准正态分布$N(0,1)$，或者说$X_n$近似的服从$N(np,np(1-p))$

例4：设$X_1,X_2,\cdots ,X_n,\cdots$为独立同分布的随机变量列，且均服从参数为$1$的指数分布，则$\underset{n\to \infty }{\lim }P\left\{\sum _{i=1}^n{X}_i\le n\right\}=()$

$X_1,X_2,\cdots ,X_n,\cdots$独立同分布，且$E(X_i)=1,D(X_i)=1$，根据列维-林德伯格中心极限定理，$\sum _{i=1}^n{X}_i$近似的服从 N ( n ⋅ 1 λ , n ⋅ 1 λ 2 )

N(n⋅λ1​,n⋅λ21​)​ 即$N(n,n)$，所以 lim ⁡ n → ∞ P { X n − n p n p ( 1 − p ) ≤ x } = Φ ( x )

limn→∞P{Xn−npnp(1−p)≤x}=Φ(x)" role="presentation" style="position: relative;">limn→∞P{Xn−npnp(1−p)−−−−−−−−√≤x}=Φ(x)$$\begin{array}{r}\underset{n\to \mathrm{\infty }}{lim}P\left\{\frac{X_n-np}{\sqrt{np(1-p)}}\le x\right\}=\mathrm{\Phi }(x)\end{array}$$

n→∞lim​P{np(1−p) ​Xn​−np​≤x}=Φ(x)​，因此
$$\underset{n\to \infty }{\lim }P\left\{\sum _{i=1}^n{X}_i\le n\right\}=\underset{n\to \infty }{\lim }P\left\{\frac{\sum _{i=1}^n{X}_i-n}{\sqrt{n}}\le 0\right\}=\Phi (0)=\frac{1}{2}$$

CSDN话题挑战赛第2期
参赛话题：学习笔记