##  [Cramér–Rao Bound](/cramer-rao-bound-1) 

 Definition

Under standard regularity conditions for a parametric statistical model with likelihood p(x; θ), the variance of any unbiased estimator (θ̂) of a scalar parameter θ is lower-bounded by the reciprocal of the Fisher information: Var(θ̂) ≥ 1 / I(θ), where I(θ)=E[(∂/∂θ log p(X;θ))^2]. In the multivariate case Cov(θ̂) ≥ I(θ)^{-1} in the matrix sense. The bound quantifies a fundamental information–precision trade-off for unbiased estimators.

 

 

 

 

 

 





## Principle

Principle

Fisher information measures the sensitivity of the model's likelihood to parameter changes; greater Fisher information implies a smaller lower bound on achievable unbiased estimator variance—information bounds precision.

 

 

 

 

 





## Demonstration

Demonstration

Illustrative scenario → Estimating the mean μ of i.i.d. Gaussian samples with known variance σ^2: the sample mean is unbiased with Var(μ̂)=σ^2/n and Fisher information I(μ)=n/σ^2, so Var(μ̂)=1/I(μ), achieving the Cramér–Rao bound and demonstrating attainability under regularity and sufficiency.

 

 

 

 

## Misapplication

Misapplication

Using the bound for biased estimators without accounting for bias (ignoring bias contribution to mean-squared error), applying it when regularity conditions fail (non-differentiable likelihood, parameter on the boundary), or treating it as necessarily tight for small samples; the error is misremembering scope and required conditions.

 

 

 

 

 





## Consequence

Consequence

Provides a benchmark for estimator performance and guides the design of efficient estimators (e.g., maximum likelihood achieves the bound asymptotically under regularity); it also formalizes how model information limits achievable precision for unbiased estimation.

 

 

 

 

## Reversal

Reversal

When estimators are biased, the mean-squared error can be below the CR bound for variance alone; in models with irregular likelihoods, discrete parameters, or failure of regularity conditions, the CR bound may not hold or require modification (e.g., van Trees, Hammersley–Chapman–Robbins bounds).

 

 

 

 

 





## Boundary

Boundary

Clearly within → parametric models with differentiable likelihoods, interior parameter values, and finite Fisher information where unbiased estimators are considered. Boundary case → small-sample regimes where asymptotic efficiency concepts may mislead. Clearly outside → nonparametric settings lacking finite-dimensional Fisher information or problems with nonregular parameters.

 

 

 

 

 





## Semantic Tension

Semantic Tension

Bias–variance trade-off ↔ Cramér–Rao focus on unbiased variance: minimizing variance subject to unbiasedness may be suboptimal for overall MSE, so practical estimator design often balances bias to reduce MSE despite the CR bound's variance limit.

 

 

 

 

 





## Synthesis

Synthesis

The Cramér–Rao bound links differential information in the likelihood to a formal lower limit on the variance of unbiased estimators and serves as both a diagnostic (how informative is the model) and a target (efficient estimators aim to attain it asymptotically). Its practical use requires checking regularity, considering bias, and recognizing alternatives when assumptions fail.