Descriptive Statistics¶
Job: Summarize finite samples via empirical measures and robust functionals; connect sample moments to population quantities without claiming inferential error rates.
Core¶
Sample \(x_1,\ldots,x_n\in\mathbb{R}^d\). Empirical measure \(\mathbb{P}_n=n^{-1}\sum_{i=1}^n\delta_{x_i}\). Any integrable functional \(T(\mu)\) yields a descriptive statistic \(T(\mathbb{P}_n)\).
| Functional | Population | Sample |
|---|---|---|
| Mean | \(\mathbb{E}_\mu[X]\) | \(\bar x=n^{-1}\sum x_i\) |
| Variance | \(\mathrm{Var}_\mu(X)\) | \(s^2=(n-1)^{-1}\sum(x_i-\bar x)^2\) (unbiased for i.i.d. \(L^2\)) |
| CDF | \(F\) | \(F_n(t)=n^{-1}\sum 1_{x_i\le t}\) |
| Quantile | \(F^{\leftarrow}(p)\) | order-statistic interpolants |
| Covariance | \(\mathrm{Cov}(X,Y)\) | sample covariance matrix |
| Correlation | Pearson \(\rho\) | \(r\); rank: Spearman |
Order statistics \(x_{(1)}\le\cdots\le x_{(n)}\): median, IQR, empirical quantiles. Robustness: median, MAD have high breakdown points; sample mean has breakdown \(1/n\).
Concentration (descriptive geometry): for fixed \(\mu\), Hoeffding/Bernstein bound how \(\bar x\) and \(F_n\) deviate from \(\mu,F\) — still a property of the sample map, used later for inference.
Glivenko–Cantelli: \(\|F_n-F\|_\infty\to0\) a.s. for i.i.d. real samples (uniform Glivenko–Cantelli in VC classes generalizes).
Multivariate: scatter matrices; PCA = spectral decomposition of sample covariance (descriptive dimension reduction). Whitening / Mahalanobis distance \(d_\Sigma(x,m)^2=(x-m)^\top\Sigma^{-1}(x-m)\).
Dependence summaries: mutual information (needs density estimation), rank correlations (copula-based, margin-free).
See it¶
Influence function of the mean: \(\mathrm{IF}(x;T,\mu)=x-\mu\) unbounded \(\Rightarrow\) nonrobust. For the median (at continuous \(F\) with \(f(F^{-1}(1/2))>0\)), \(\mathrm{IF}\) is bounded \(\mathrm{sign}(x-m)/(2f(m))\).
Check¶
Why divide by \(n-1\) in \(s^2\)?
For i.i.d. \(L^2\) samples, \(\mathbb{E}\sum(X_i-\bar X)^2=(n-1)\sigma^2\) (one degree of freedom lost to estimating the mean). MLE under Gaussian uses \(n\) and is biased for \(\sigma^2\).
Does \(r=0\) imply independence?
No. Pearson \(r\) detects linear correlation. \(Y=X^2\) with symmetric \(X\) can have \(\mathrm{Cov}(X,Y)=0\) while dependent. Use distance correlation / HSIC for stronger dependence measures.
Pitfalls¶
- Histograms and KDE depend on bandwidth; they are estimators, not unique "the" density of the sample.
- Sample covariance is singular for \(n\le d\); need regularization / sparsity in high dimension.
- Descriptive success (nice plots) \(\neq\) inferential validity (selection bias, multiple comparisons).
Refs¶
van der Vaart, Asymptotic Statistics (empirical processes); Huber–Ronchetti (robust); Tukey (EDA historically).