Probability

Job: measure-theoretic probability as the language of randomness, limit theorems, and stochastic models used across analysis, statistics, and mathematical physics.

Core

Probability is integration on a measure space \((\Omega,\mathcal{F},\mathbb{P})\) with \(\mathbb{P}(\Omega)=1\). Random variables are measurable maps; laws are pushforward measures; conditional expectation is \(L^2\) projection (and \(L^1\) martingale theory). Limit theorems convert independence + moments into almost-sure and distributional asymptotics.

Layer Content
Kolmogorov setup \((\Omega,\mathcal{F},\mathbb{P})\); r.v.; \(\sigma\)-algebras; independence
Integration \(\mathbb{E}\), \(L^p\); modes of convergence (\(L^p\), a.s., in prob., in law)
Conditioning \(\mathbb{E}[\,\cdot\mid\mathcal{G}]\); regular conditional distributions
Characteristic fns \(\varphi_X(t)=\mathbb{E}e^{itX}\); inversion / continuity theorems
Limit theorems WLLN / SLLN; CLT (Lindeberg–Feller); large deviations (pointer)
Processes (exit) filtrations \(\to\) stochastic processes

Reading map

Page Focus
Foundations Kolmogorov axioms; r.v.; conditional expectation; independence; modes of convergence
Distributions Laws, densities, characteristic functions; named families; CLT/LLN packaging
Stochastic processes Martingales, BM, SDE
SPDE Stochastic PDE, mild solutions, noise regularity
Measure theory \(L^p\), DCT, Fubini, Radon–Nikodym
Ergodic theory Stationary processes; Birkhoff as SLLN generalization

Dependency order

  1. Measure theory (\(L^p\), dominated convergence, product measures).
  2. Foundations \(\to\) Distributions.
  3. Stochastic processes \(\to\) SPDE / rough paths.
  4. Statistics interface: inferential statistics.

Interfaces

Area Contact
Functional analysis \(L^p\) duality; weak convergence of measures; characteristic functions
PDE Fokker–Planck; Feynman–Kac; generators of diffusions
Information theory Entropy, KL as large-deviation rate functions
Mathematical physics Gibbs measures; Euclidean QFT / stochastic quantization

Notation (paper-standard)

  • \(\mathbb{E}\), \(\operatorname{Var}\), \(\operatorname{Cov}\); \(X\sim\mu\); \(\mathcal{L}(X)=\mathbb{P}\circ X^{-1}\)
  • \(\xrightarrow{\mathrm{a.s.}}\), \(\xrightarrow{\mathbb{P}}\), \(\xrightarrow{L^p}\), \(\Rightarrow\) (weak / in distribution)
  • \(\varphi_X\), \(M_X(t)=\mathbb{E}e^{tX}\); \(\mathbb{E}[X\mid\mathcal{G}]\)
  • \(\mathcal{F}_t\), stopping times \(\tau\) (process layer)

Signature theorems (orientation)

  • Kolmogorov extension / Daniell–Kolmogorov: consistent finite-dimensional distributions \(\Rightarrow\) process on \(\mathbb{R}^T\) (Polish-space caveats).
  • Strong LLN (Kolmogorov): i.i.d. integrable \(\Rightarrow\) \(\bar X_n\to\mathbb{E}X_1\) a.s.
  • CLT (Lindeberg–Lévy): i.i.d. finite variance \(\Rightarrow\) \(\sqrt{n}(\bar X_n-\mu)/\sigma\Rightarrow N(0,1)\).
  • Radon–Nikodym \(\Rightarrow\) conditional expectation and likelihood ratios; Girsanov for drift change under equivalence.
  • Portmanteau / Lévy continuity: \(\varphi_{X_n}\to\varphi\) pointwise continuous at \(0\) \(\Rightarrow\) weak convergence.

See it

Conditional expectation as projection. On \(L^2(\Omega,\mathcal{F},\mathbb{P})\), \(\mathbb{E}[\,\cdot\mid\mathcal{G}]\) is the orthogonal projection onto \(L^2(\mathcal{G})\). Tower property, \(\mathbb{E}[YX\mid\mathcal{G}]=Y\mathbb{E}[X\mid\mathcal{G}]\) for \(Y\in L^\infty(\mathcal{G})\), and martingale theory are linear algebra in disguise — the same picture drives Kalman filtering and Doob decomposition.

Check

Does \(X_n\to X\) in probability imply \(\mathbb{E}X_n\to\mathbb{E}X\)?

No without uniform integrability (or \(L^1\) domination). Classic: \(n\cdot\mathbf{1}_{[0,1/n]}\) on \(([0,1],\mathrm{Leb})\) converges to \(0\) in probability (and a.s.) but expectations stay \(1\).

Why is independence of \(\sigma\)-algebras strictly stronger than pairwise independence of a family of r.v.s?

Pairwise \(\mathbb{P}(A_i\cap A_j)=\mathbb{P}(A_i)\mathbb{P}(A_j)\) does not imply \(\mathbb{P}(\bigcap_{k=1}^n A_k)=\prod\mathbb{P}(A_k)\) for \(n\ge3\) (Bernstein’s example). Full independence is a \(\pi\)-system / finite-dimensional product condition.

Pitfalls

  • Almost sure vs in probability: a.s. convergence needs a single null set for the whole sequence; subsequence a.s. + in-probability \(\Leftrightarrow\) in-probability (not a.s.).
  • Characteristic functions determine laws (on \(\mathbb{R}^d\)), but moment problems may be indeterminate (Stieltjes/Hamburger).
  • Conditional probability as a function of \(\omega\) requires regular versions on Polish spaces; naive formulas fail on pathological \(\Omega\).

Refs

  • Durrett, Probability: Theory and Examples
  • Kallenberg, Foundations of Modern Probability
  • Billingsley, Probability and Measure
  • Jacod–Protter, Probability Essentials (compact)

Next

Foundations · Distributions · Stochastic processes · Measure theory · SPDE · Ergodic theory