IS Math-2015-03

题目来源：Problem 3 日期：2024-08-11 题目主题：Math-概率论-矩母函数与分布

解题思路

这组题目主要涉及矩母函数（moment generating function）的应用，以及正态分布和几何分布的性质。解题的关键在于理解矩母函数与随机变量的矩之间的关系，以及如何利用矩母函数来推导随机变量的分布和性质。我们需要逐步解答每个小问，并在过程中建立起前后问题之间的联系。特别注意的是，最后一题需要应用切比雪夫不等式来得到概率的上界。

Solution

1. Moment Generating Function and Moments

Let ${\varphi }_X(t)=E_X[e^{tX}]$ be the moment generating function (MGF) of $X$. We need to express the mean and variance of $X$ in terms of ${\varphi }_X^{\prime }(0)$ and ${\varphi }_X^{\prime \prime }(0)$.

Mean

The mean (expectation) of $X$, denoted by $E_X[X]$, is the first moment of $X$. It can be found by differentiating the MGF with respect to $t$ and evaluating at $t=0$:

$$E_X[X]={\frac{\mathrm{d}{\varphi }_X(t)}{\mathrm{d}t}|}_{t=0}={\varphi }_X^{\prime }(0).$$

Variance

The variance of $X$, denoted by $\mathrm{Var}(X)$, is related to the second central moment of $X$. It can be found using the second derivative of the MGF:

$$\mathrm{Var}(X)={\frac{{\mathrm{d}}^2{\varphi }_X(t)}{\mathrm{d}{t}^2}|}_{t=0}-{\left({\varphi }_X^{\prime }(0)\right)}^2={\varphi }_X^{\prime \prime }(0)-{\left({\varphi }_X^{\prime }(0)\right)}^2.$$

Thus, the MGF provides a direct way to calculate both the mean and variance of the random variable $X$ in terms of ${\varphi }_X^{\prime }(0)$ and ${\varphi }_X^{\prime \prime }(0)$.

2. MGF of Normal Distribution and Sum of Independent Normal Variables

We are given a sequence of mutually independent random variables $X_1,X_2,\dots ,X_N$, each generated according to a 1-dimensional normal distribution with mean $\mu$ and variance ${\sigma }^2$. The probability density function (PDF) for each $X_j$ is

$$p(X_j=x)=\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right).$$

For $X_j\sim N(\mu ,{\sigma }^2)$, we need to calculate ${\varphi }_{X_j}(t)$ and find the distribution of $Y={\sum }_{j=1}^N{X}_j$.

(a) Calculating ${\varphi }_{X_j}(t)$

The MGF ${\varphi }_{X_j}(t)$ of a normal distribution $X_j\sim N(\mu ,{\sigma }^2)$ is given by:

$${\varphi }_{X_j}(t)=E\left[e^{t{X}_j}\right].$$

Substituting the expression for the PDF of a normal distribution:

$${\varphi }_{X_j}(t)={\int }_{-\infty }^{\infty }{e}^{tx}\frac{1}{\sqrt{2\pi {\sigma }^2}}\exp \left(-\frac{(x-\mu {)}^2}{2{\sigma }^2}\right)\mathrm{d}x.$$

This simplifies to:

$${\varphi }_{X_j}(t)=\exp \left(\mu t+\frac{{\sigma }^2{t}^2}{2}\right).$$

(b) Finding the distribution of $Y=X_1+X_2+\cdots +X_N$

Since $X_1,X_2,\dots ,X_N$ are independent and identically distributed (i.i.d.), the MGF of $Y$ is the product of the MGFs of each $X_j$:

$${\varphi }_Y(t)={\left({\varphi }_{X_j}(t)\right)}^N={\left[\exp \left(\mu t+\frac{{\sigma }^2{t}^2}{2}\right)\right]}^N=\exp \left(N\mu t+\frac{N{\sigma }^2{t}^2}{2}\right).$$

This is the MGF of a normal distribution $N(N\mu ,N{\sigma }^2)$. Therefore, $Y$ is generated according to the normal distribution:

$$Y\sim N(N\mu ,N{\sigma }^2).$$

3. MGF of Y with Geometric N

Given that $N$ follows a geometric distribution with parameter $\theta$:

$$P(N=n)=(1-\theta {)}^n\theta ,$$

and $Y=X_1+X_2+\cdots +X_N$, we need to calculate ${\varphi }_Y(t)$ in terms of ${\varphi }_X(t)$.

The MGF ${\varphi }_Y(t)$ is defined as:

$${\varphi }_Y(t)=E_Y\left[e^{tY}\right].$$

Since $Y$ is the sum of $N$ i.i.d. random variables $X_j$, the MGF of $Y$ can be expressed as:

$${\varphi }_Y(t)=E\left[{\varphi }_X(t{)}^N\right].$$

Using the law of total expectation:

$$\begin{array}{rl}{\varphi }_Y(t)& =E_Y[e^{tY}]\\ & =E_N[E_Y[e^{tY}|N]]\\ & =E_N[({\varphi }_X(t){)}^N]\\ & =\sum _{n=1}^{\infty }({\varphi }_X(t){)}^n(1-\theta {)}^{n-1}\theta \\ & =\frac{\theta {\varphi }_X(t)}{1-(1-\theta ){\varphi }_X(t)}\end{array}$$

This is the MGF of $Y$ expressed in terms of ${\varphi }_X(t)$.

4. Mean and Variance of Y

To find the mean and variance of $Y$, we’ll use the derivatives of ${\varphi }_Y(t)$ evaluated at $t=0$.

Mean of Y

$$\begin{array}{rl}E[Y]={\varphi }_Y^{\prime }(0)& =\frac{\theta {\varphi }_X^{\prime }(0)}{(1-(1-\theta ){\varphi }_X(0){)}^2}\\ & =\frac{\theta \mu }{{\theta }^2}=\frac{\mu }{\theta }\end{array}$$

Variance of Y

$$\begin{array}{rl}{\varphi }_Y^{\prime \prime }(0)& =\frac{\theta {\varphi }_X^{\prime \prime }(0)}{(1-(1-\theta ){\varphi }_X(0){)}^2}+\frac{2\theta (1-\theta )({\varphi }_X^{\prime }(0){)}^2}{(1-(1-\theta ){\varphi }_X(0){)}^3}\\ & =\frac{\theta ({\mu }^2+{\sigma }^2)}{{\theta }^2}+\frac{2(1-\theta ){\mu }^2}{{\theta }^2}\\ & =\frac{{\mu }^2+{\sigma }^2}{\theta }+\frac{2(1-\theta ){\mu }^2}{{\theta }^2}\end{array}$$

Therefore,

$$\begin{array}{rl}\text{Var}(Y)& ={\varphi }_Y^{\prime \prime }(0)-({\varphi }_Y^{\prime }(0){)}^2\\ & =\frac{{\mu }^2+{\sigma }^2}{\theta }+\frac{2(1-\theta ){\mu }^2}{{\theta }^2}-\frac{{\mu }^2}{{\theta }^2}\\ & =\frac{{\sigma }^2}{\theta }+\frac{{\mu }^2(1-\theta )}{{\theta }^2}\end{array}$$

5. Upper Bound on Probability using Chebyshev’s Inequality

Given $\xi >E[Y]$, we want to find an upper bound on $P(Y>\xi )$. We can use Chebyshev’s inequality for this purpose.

Chebyshev’s inequality states that for any random variable $X$ with mean $\mu$ and variance ${\sigma }^2$, and for any $k>0$:

$$P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$$

In our case, we want to apply this to $Y$. From the previous questions, we know:

• $E[Y]=\frac{\mu }{\theta }$
• $Var(Y)=\frac{{\sigma }^2}{\theta }+\frac{{\mu }^2(1-\theta )}{{\theta }^2}$

Let’s denote $Var(Y)$ as ${\sigma }_Y^2$ for simplicity.

We want to find $P(Y>\xi )$, which is equivalent to $P(Y-E[Y]>\xi -E[Y])$.

Let $k=\frac{\xi -E[Y]}{{\sigma }_Y}$. Then:

$$P(Y>\xi )=P(Y-E[Y]>\xi -E[Y])=P(|Y-E[Y]|>\xi -E[Y])$$

Applying Chebyshev’s inequality:

$$P(|Y-E[Y]|>\xi -E[Y])\le \frac{{\sigma }_Y^2}{(\xi -E[Y]{)}^2}$$

Substituting the values we know:

$$P(Y>\xi )\le \frac{\frac{{\sigma }^2}{\theta }+\frac{{\mu }^2(1-\theta )}{{\theta }^2}}{(\xi -\frac{\mu }{\theta }{)}^2}=\frac{{\sigma }^2\theta +{\mu }^2(1-\theta )}{(\theta \xi -\mu {)}^2}$$

This gives us an upper bound on $P(Y>\xi )$ in terms of $\mu$, $\sigma$, $\theta$, and $\xi$.

知识点

矩母函数正态分布几何分布独立随机变量和切比雪夫不等式概率上界期望方差

难点思路

1. 理解矩母函数与随机变量矩之间的关系，特别是如何通过矩母函数的导数获取随机变量的矩。
2. 推导复合分布（几何分布的正态分布和）的矩母函数，这需要运用条件期望和全期望公式。
3. 利用矩母函数求解复合分布的期望和方差，这涉及到矩母函数的一阶和二阶导数。
4. 识别出切比雪夫不等式的适用性，并正确应用它来获得概率上界。
5. 在最后一题中，将问题中的条件（$Y>\xi$）转化为适合使用切比雪夫不等式的形式。

解题技巧和信息

1. 矩母函数的导数在 $t=0$ 处的值与随机变量的矩有直接关系：${\phi }^{\prime }(0)$ 给出期望，${\phi }^{\prime \prime }(0)$ 与方差相关。
2. 独立随机变量和的矩母函数等于各个随机变量矩母函数的乘积，这在处理多个独立正态分布的和时非常有用。
3. 对于复合分布，可以使用全期望公式来推导矩母函数，这在处理几何分布的正态分布和时很重要。
4. 切比雪夫不等式提供了一个通用的概率上界，只需要知道随机变量的期望和方差。虽然这个上界可能不是最紧的，但它通常很容易计算和应用。
5. 在应用切比雪夫不等式时，要注意将问题中的条件（这里是 $Y>\xi$）转化为与期望的偏差。
6. 在进行代数运算时，特别是在简化复杂表达式时，要注意保持清晰和准确，避免计算错误。

重点词汇

• Moment Generating Function (MGF) 矩母函数
• Normal Distribution 正态分布
• Geometric Distribution 几何分布
• Law of Total Expectation 全期望公式
• Chebyshev’s Inequality 切比雪夫不等式
• Probability Upper Bound 概率上界
• Expectation 期望
• Variance 方差
• Independent and Identically Distributed (i.i.d.) 独立同分布

参考资料

1. Probability and Statistics, 4th Edition by Morris H. DeGroot and Mark J. Schervish, Chapter 5
2. Introduction to Probability, 2nd Edition by Dimitri P. Bertsekas and John N. Tsitsiklis, Chapter 4
3. G.R. Grimmett, D.R. Stirzaker. Probability and Random Processes, 3rd Edition, Oxford University Press, 2001.
4. S.M. Ross. Introduction to Probability Models, 12th Edition, Academic Press, 2019.