Session 6: Continuous Random Variables

Movement II: One Random Variable

EE 520: Random Processes (Fall 2026). Instructor: John Lipor.

Launch

Your range sensor returns $103.7$ m. Ask for the probability that the true range is exactly $103.7$ m and the answer is zero: no single real number carries any probability when the outcome varies continuously. Yet the reading is obviously informative, and you would happily bet on the target being nearer to $103.7$ m than to $200$ m. Resolving that tension is today’s job. The fix is to stop asking for the probability of a point and start asking for the probability of an interval, and the object that delivers interval probabilities is the density, a rate of probability per unit length rather than a probability itself. By the end you will read a Gaussian noise spec, standardize it, and compute an outage probability from a table; and you will see why two noise models with the same variance can still disagree, sharply, about how alarming a large reading is.

Development

From CDF to density

Session 4 built the CDF $F_X(x) = \mathbb{P}(X \le x)$ for discrete variables, where it was a staircase. The CDF is the object that survives into the continuous world unchanged in meaning: for any random variable, $F_X(x)$ is the probability of landing at or below $x$. What changes is that a continuous variable’s CDF has no jumps; it rises smoothly, and the mass it accumulates over an interval is what we can measure.

Because probability comes only from intervals, a single point carries none: taking $a \to b$ in [eq:intervalprob] gives $\mathbb{P}(X = b) = 0$ for every $b$. This is the resolution of the Launch puzzle. It also means that for continuous variables we need not fuss over $<$ versus $\le$; endpoints have zero probability either way.

A density is a rate, not a probability. The value $f_X(x)$ is not a probability. It is a probability per unit length: the mass in a sliver of width $dx$ at $x$ is $f_X(x)\,dx$. Two consequences that trip people up. First, a density may exceed $1$ (a $\mathrm{Uniform}[0, \tfrac12]$ has $f = 2$ on its support) because it is a rate, and only its integral must be $1$. Second, $f_X(x) = 0.02$ does not mean “a $2\%$ chance”; it means nothing at all until multiplied by a length. This resolves the Session 1 worry about conditioning on a probability-zero event, which we will complete in Session 8: we condition on $\left\{X = x\right\}$ through the density, not through a probability.

Expectation and the tail-integral formula

The mean and LOTUS carry over from the discrete case with sums replaced by integrals.

Session 5 met a discrete tail-sum formula and deferred its proof. Its continuous twin we prove now, because the proof is short and the identity is the workhorse for means of nonnegative quantities.

This is Session 5’s tail sum wearing continuous clothes: there we added $\mathbb{P}(X \ge k)$ over integers, here we integrate $\mathbb{P}(X > x)$ over the half-line. Same idea, same picture, integral instead of sum.

Three models: uniform, exponential, Gaussian

Uniform. $X \sim \mathrm{Uniform}[a, b]$ spreads its mass evenly: $$\begin{equation} f_X(x) = \frac{1}{b - a} \text{ on } [a, b], \qquad F_X(x) = \frac{x - a}{b - a} \text{ on } [a, b]. \end{equation}$$ By symmetry $\mathbb{E}[X] = (a + b)/2$, and a short computation gives $\operatorname{Var}(X) = (b - a)^2 / 12$. It is the model for “somewhere in this interval, no place preferred.”

Exponential. $X \sim \mathrm{Exponential}(\lambda)$, parameterized by rate $\lambda > 0$, is the standard model for a nonnegative waiting time: $$\begin{equation} f_X(x) = \lambda e^{-\lambda x}, \qquad F_X(x) = 1 - e^{-\lambda x}, \qquad x \ge 0. \end{equation}$$ Its survival probability is $\mathbb{P}(X > x) = e^{-\lambda x}$, and the tail-integral formula delivers the mean in one line: $$\begin{equation} \mathbb{E}[X] = \int_0^\infty e^{-\lambda x}\, dx = \frac{1}{\lambda}. \end{equation}$$ The exponential is the continuous echo of Session 5’s geometric, and it inherits the geometric’s signature property.

A component that has already survived $s$ hours is, going forward, as good as new: its remaining lifetime has the same distribution it started with. This is the exact continuous mirror of the geometric argument from Session 5, and it is no accident. A geometric with small per-step success probability $q$, counted on a fine time grid of spacing $\Delta$ with $\lambda = q/\Delta$, converges to an $\mathrm{Exponential}(\lambda)$ as $\Delta \to 0$; the memoryless discrete clock becomes the memoryless continuous one. And the same reconciliation from Session 5 applies: memorylessness is a statement about the waiting time given the mechanism, not a license to ignore evidence about whether the mechanism is even running.

Gaussian. $X \sim {\mathcal N}(\mu, \sigma^2)$, the workhorse noise model, has density $$\begin{equation} f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\!\left(-\frac{(x - \mu)^2}{2\sigma^2}\right), \end{equation}$$ with mean $\mu$ and variance $\sigma^2$. We take the normalizing constant $1/\sqrt{2\pi\sigma^2}$ on faith (verifying $\int f_X = 1$ is a standard polar-coordinates exercise). The Gaussian has no elementary antiderivative, so we never integrate it by hand; instead we standardize to a single reference variable and read a table.

The move that makes every Gaussian computation identical is standardization: if $X \sim {\mathcal N}(\mu, \sigma^2)$ then $Z = (X - \mu)/\sigma \sim {\mathcal N}(0, 1)$, so any probability about $X$ becomes a probability about $Z$: $$\begin{equation} \label{eq:standardize} \mathbb{P}(X \le x) = \mathbb{P}\!\left(Z \le \frac{x - \mu}{\sigma}\right) = \Phi\!\left(\frac{x - \mu}{\sigma}\right). \end{equation}$$ Note the divisor is $\sigma$, the standard deviation, never $\sigma^2$. A few tail values, worth memorizing to one or two figures, anchor all Gaussian estimates:

The “$3\sigma$” folklore is the last row: only about one part in $740$ of a Gaussian’s mass lies beyond three standard deviations on one side.

Why Gaussian noise models are everywhere. The central limit theorem says that if $X_1, X_2, \dots$ are iid with mean $\mu$ and variance $\sigma^2$, then the standardized sum $\frac{1}{\sigma\sqrt{n}}\sum_{i=1}^n (X_i - \mu)$ converges in distribution to ${\mathcal N}(0,1)$ as $n \to \infty$. Because physical noise is so often the accumulation of many small independent disturbances, the Gaussian is its natural limiting shape, which is why it is the default sensor-noise model throughout this course (we state the theorem here and return to sums of Gaussians in Session 14).

Tails are a modeling decision

Variance is not the whole story of a noise model. Two densities can share a mean and a variance and still disagree about how often they produce a large deviation, and that disagreement is exactly what decides whether a surprising reading is “just noise” or a broken sensor. To see it, put the Gaussian beside the Laplace density (a double-sided exponential), used here only as a comparison model: $$\begin{equation} f(z) = \frac{\lambda}{2} e^{-\lambda \left\lvert z \right\rvert}, \qquad \operatorname{Var} = \frac{2}{\lambda^2}. \end{equation}$$ Match its variance to a ${\mathcal N}(0, \sigma^2)$ by setting $2/\lambda^2 = \sigma^2$, i.e. $\lambda = \sqrt{2}/\sigma$. Now both models have the same spread, and we can ask each for the probability of landing more than $k$ standard deviations from the mean. For the Gaussian, $\mathbb{P}(\left\lvert X \right\rvert > k\sigma) = 2Q(k)$; for the variance-matched Laplace, $\mathbb{P}(\left\lvert X \right\rvert > k\sigma) = e^{-\sqrt{2}\,k}$:

Near the center ($k = 1$) the two are comparable and the Gaussian actually puts slightly more mass beyond $1\sigma$. But the ordering has already reversed by $k = 2$, and the reason is structural: the Gaussian tail decays like $e^{-k^2/2}$, quadratically fast in the exponent, while the Laplace decays only like $e^{-\sqrt 2\, k}$, linearly. The heavier (Laplace) tail pulls steadily ahead, and the gap widens as $k$ grows. How much worse it gets at $3\sigma$ and $4\sigma$, and what that means for judging whether a $4\sigma$ reading is noise or a broken sensor, is the subject of HW3.

Same variance $\ne$ same model. Matching variances does not make two noise models interchangeable. The tail behavior, how fast the density decays far from the mean, is a separate modeling choice, and it is the one that governs rare-but-consequential events: outages, false alarms, and the question “is this outlier real?” Choose the tail deliberately.

Worked examples

(a) Outage probability from a Gaussian spec

A receiver’s received power, in decibels, is modeled as $X \sim {\mathcal N}(-70,\, 36)$, so the mean is $-70$ dB and the standard deviation is $\sigma = \sqrt{36} = 6$ dB. The receiver loses lock (an outage) whenever the power drops below $-82$ dB. What is the outage probability?

Standardize with $\sigma = 6$ (not $36$), using [eq:standardize]: $$\begin{equation} \mathbb{P}(\text{outage}) = \mathbb{P}(X < -82) = \Phi\!\left(\frac{-82 - (-70)}{6}\right) = \Phi\!\left(\frac{-12}{6}\right) = \Phi(-2). \end{equation}$$ By symmetry $\Phi(-2) = Q(2)$, and the table gives $$\begin{equation} \mathbb{P}(\text{outage}) = Q(2) \approx 0.0228, \end{equation}$$ about a $2.3\%$ chance. The entire computation was: subtract the mean, divide by the standard deviation, read the table. That is every Gaussian probability you will ever compute.

(b) First alarm among four sensors

$K = 4$ sensors watch a field; once a target enters, each raises an alarm after an independent $\mathrm{Exponential}(\lambda)$ delay. When does the first alarm sound?

The first alarm time is $T = \min(X_1, \dots, X_4)$, and the minimum is the moment for Session 5’s CDF method. The first alarm sounds after $t$ exactly when all four sensors are still silent: $$\begin{equation} \mathbb{P}(T > t) = \mathbb{P}(X_1 > t, \dots, X_4 > t) = \prod_{i=1}^4 \mathbb{P}(X_i > t) = \left(e^{-\lambda t}\right)^4 = e^{-4\lambda t}. \end{equation}$$ But $e^{-4\lambda t}$ is the survival probability of an $\mathrm{Exponential}(4\lambda)$, so $T \sim \mathrm{Exponential}(4\lambda)$: the minimum of independent exponentials is again exponential, with the rates added. Four sensors detect four times as fast, with mean first-alarm time $1/(4\lambda)$. Notice this reused Session 5’s recipe unchanged, only swapping the geometric’s $(1 - q)^k$ for the exponential’s $e^{-\lambda t}$; the min-means-CDF lesson does not care whether the clock is discrete or continuous.

(c) The last alarm, and a new named distribution

Return to the sensor field of example (b), but now ask about the last alarm. A confirmation protocol declares a target only once every one of $m$ sensors has reported, and each sensor’s delay is an independent $\mathrm{Exponential}(\lambda)$. Measure time in units of the mean delay, so $\lambda = 1$ (for general $\lambda$, replace $M$ by $\lambda M$ throughout). How long does confirmation take?

The confirmation time is $M = \max(X_1, \dots, X_m)$, so Session 5’s recipe applies with the maximum in place of the minimum: $$\begin{equation} \label{eq:maxexp} \mathbb{P}(M \le t) = \prod_{i=1}^m \mathbb{P}(X_i \le t) = \left(1 - e^{-t}\right)^m, \qquad t \ge 0. \end{equation}$$ That answers the question exactly, but it hides how $M$ behaves as the field grows. The first alarm got faster, like $1/m$. The last alarm gets slower, and a good guess for the rate is $\ln m$: the waiting time $t$ at which a single sensor is still silent with probability $1/m$ solves $e^{-t} = 1/m$. So center $M$ at $\ln m$ and look at what is left over. For $x \ge -\ln m$, $$\begin{equation} \mathbb{P}(M - \ln m \le x) = \left(1 - e^{-(x + \ln m)}\right)^m = \left(1 - \frac{e^{-x}}{m}\right)^m. \end{equation}$$ This is the same limit that produced the Poisson pmf in Session 5, $(1 + a/m)^m \to e^{a}$, now with $a = -e^{-x}$ held fixed while $m \to \infty$: $$\begin{equation} \label{eq:gumbellimit} \mathbb{P}(M - \ln m \le x) \longrightarrow \exp\!\left(-e^{-x}\right) \qquad \text{for every real } x. \end{equation}$$ The shifted maximum does not spread out or drift away. It settles on a fixed shape, and that shape deserves a name.

Check that $F$ is a legitimate CDF: it is continuous and increasing (because $e^{-x}$ decreases and $\exp(-\,\cdot\,)$ reverses that), $F(x) \to \exp(-\infty) = 0$ as $x \to -\infty$, and $F(x) \to \exp(0) = 1$ as $x \to \infty$. Differentiating gives the density $f(x) = e^{-x} \exp\!\left(-e^{-x}\right)$. Unlike the exponential, the Gumbel lives on the whole real line; it is skewed, with a left tail that dies doubly exponentially and a right tail that dies like $e^{-x}$. In this language, [eq:gumbellimit] says that $M - \ln m$ is approximately standard Gumbel; equivalently, the last alarm $M$ itself is approximately $\mathrm{Gumbel}(\ln m)$.

How fast is the limit reached? At $x = 0$ the exact probability $(1 - 1/m)^m$ is $0.316$ for $m = 4$, $0.358$ for $m = 20$, and $0.366$ for $m = 100$, against the limit $e^{-1} \approx 0.368$ (Figure 1). Twenty sensors are already plenty.

Designing the timeout. With $m = 20$ sensors, how long must the protocol wait so that all twenty have reported with probability $0.95$? The Gumbel approximation asks for $\exp(-e^{-x}) = 0.95$, so $x = -\ln(-\ln 0.95) \approx 2.970$ and $t \approx \ln 20 + 2.970 \approx 5.966$ mean delays. The exact answer, solving $(1 - e^{-t})^{20} = 0.95$, is $t = -\ln\!\left(1 - 0.95^{1/20}\right) \approx 5.967$. The approximation also says how the timeout scales: doubling the field to $40$ sensors adds only $\ln 2 \approx 0.69$ mean delays. Detection got four times faster with four sensors in example (b); confirmation gets only logarithmically slower as sensors are added.

(d) Maxima of Gumbels are Gumbel

The minimum of independent exponentials was again exponential, with rates added. The Gumbel has the mirror-image property for maxima.

The parallel with the exponential is exact: for minima of exponentials the rates $\lambda_k$ add; for maxima of Gumbels the weights $e^{\mu_k}$ add. Session 7 will show this is not a coincidence.

The Proposition also explains why example (c)’s limit had to be Gumbel. Split $mn$ sensors into $n$ blocks of $m$. Each block’s last alarm is approximately $\mathrm{Gumbel}(\ln m)$, so by the Proposition the overall last alarm is approximately $\mathrm{Gumbel}(\ln m + \ln n) = \mathrm{Gumbel}(\ln mn)$, which is exactly what [eq:gumbellimit] predicts for $mn$ sensors directly. A limit for maxima has to survive being maximized again, and the Gumbel does.

Log-sum-exp is a soft maximum. The location $L = \ln \sum_k e^{\mu_k}$ of the maximum satisfies $$\begin{equation} \max_k \mu_k \;\le\; L \;\le\; \max_k \mu_k + \ln n, \end{equation}$$ because the sum $\sum_k e^{\mu_k}$ lies between its largest term and $n$ times its largest term. So $L$ is a smoothed version of $\max_k \mu_k$: the largest location dominates, and close competitors push the maximum a little higher. Session 9 will turn this soft maximum into the softmax.

Consolidation

Work these in class.

1. (Gaussian standardization.) A range estimate is $X \sim {\mathcal N}(50,\, 4)$ (so $\sigma = 2$ m). Find $\mathbb{P}(X > 53)$.
   Answer: standardize with $\sigma = 2$: $\mathbb{P}(X > 53) = Q\!\left((53 - 50)/2\right) = Q(1.5) \approx 0.0668$.

2. (Exponential, memorylessness.) An alarm delay is $\mathrm{Exponential}(\lambda)$ with mean $5$ s, so $\lambda = 0.2$. Compute $\mathbb{P}(\text{delay} > 10)$, and then $\mathbb{P}(\text{delay} > 15 \,|\,\text{delay} > 5)$.
   Answer: $\mathbb{P}(\text{delay} > 10) = e^{-0.2 \cdot 10} = e^{-2} \approx 0.135$; by memorylessness $\mathbb{P}(\text{delay} > 15 \,|\,\text{delay} > 5) = \mathbb{P}(\text{delay} > 10) \approx 0.135$.

3. (Conceptual, density vs probability.) A colleague objects that a $\mathrm{Uniform}[0, \tfrac12]$ density equals $2$, “which is more than $100\%$, so it cannot be right.” In one or two sentences, resolve the objection.
   Answer: $f_X(x) = 2$ is a rate (probability per unit length), not a probability; the total probability is the area $2 \times \tfrac12 = 1$, and densities are unbounded in principle because only their integral must equal one.

Synthesis

The course thesis asks “what is uncertain?” and this session answered it for quantities that vary continuously: the CDF, unchanged in meaning from the discrete case, is the honest object, and the density beneath it is a rate of probability per unit length, which is why a single point carries no probability yet an interval does. From that one shift everything followed. The tail-integral formula gave means of nonnegative quantities directly from survival probabilities, the continuous echo of Session 5’s tail sum. The exponential turned out to be the geometric’s memoryless twin; the last of many exponential alarms, recentered at $\ln m$, produced a new named law, the Gumbel, which is closed under maxima just as the exponential is closed under minima; and the Gaussian reduced every question to “subtract the mean, divide by the standard deviation, read $Q$.” The session’s sharpest lesson was that variance does not determine a model: the Gaussian and the variance-matched Laplace agree on spread yet disagree on tails, and the tail is the modeling decision that governs rare, expensive events. Next session pushes a random variable through a function, deriving the density of $Y = g(X)$, which is how instruments and simulators are actually modeled, and which finally explains what happens each time you call a library sampler.

Self-check questions

1. A continuous $X$ has $f_X(103.7) = 0.02$. Is the probability that $X$ equals $103.7$ therefore $0.02$? If not, what does the number mean?
   Answer: No; $\mathbb{P}(X = 103.7) = 0$ for any continuous $X$. The value $0.02$ is a density (probability per unit length): the mass near $103.7$ in a window of width $dx$ is about $0.02\,dx$.

2. For $X \sim {\mathcal N}(\mu, \sigma^2)$, write $\mathbb{P}(X > c)$ in terms of $Q$. What is the single most common arithmetic slip in this step?
   Answer: $\mathbb{P}(X > c) = Q\!\big((c - \mu)/\sigma\big)$; the common slip is dividing by the variance $\sigma^2$ instead of the standard deviation $\sigma$.

3. Two zero-mean noise models have the same variance, one Gaussian and one Laplace. In one sentence, can you conclude they assign the same probability to a large outlier, and why or why not?
   Answer: No; equal variance fixes the spread but not the tail, and the heavier-tailed Laplace assigns more probability to large deviations, with the gap growing as the deviation grows.

4. With $m = 10$ sensors and $\lambda = 1$, use the Gumbel approximation to estimate the median confirmation time. (The standard Gumbel median is $-\ln \ln 2 \approx 0.367$.) Then find the exact median from [eq:maxexp].
   Answer: approximately $\ln 10 + 0.367 \approx 2.669$; exactly $-\ln\!\left(1 - 0.5^{1/10}\right) \approx 2.704$.

5. Why is the location of the maximum of $n$ independent $\mathrm{Gumbel}(\mu)$ variables $\mu + \ln n$, rather than $\mu$ or $n\mu$?
   Answer: The maximum is at least as large as any one variable, so its location cannot fall below $\mu$; but log-sum-exp exceeds the largest location by at most $\ln n$, and with $n$ equal locations it attains that bound exactly, since $\ln(n e^{\mu}) = \mu + \ln n$. More competitors push the maximum up, but only logarithmically.