MCRB (modified CRB)

Modified CRB (Cramér-Rao Bound) is a theoretical concept in statistics that quantifies the best possible accuracy of parameter estimation for a given statistical model. The Cramér-Rao bound was first introduced by the Swedish mathematician Harald Cramér in 1946, and it states that the variance of any unbiased estimator of a parameter must be at least as large as the reciprocal of the Fisher information. The Fisher information measures the amount of information that an observable random variable contains about an unknown parameter, and it is a fundamental quantity in statistical inference.

The modified CRB is an extension of the classical CRB that takes into account certain additional constraints that may be present in some estimation problems. For example, in many practical situations, the true value of the parameter to be estimated may be restricted to a certain range, or the estimator may be subject to certain measurement errors or noise. In such cases, the modified CRB can provide a more accurate estimate of the lower bound on the variance of the estimator.

To understand the concept of modified CRB, let us first revisit the classical CRB. Consider a statistical model with an unknown parameter θ that we wish to estimate based on a random sample of data X = {X1, X2, ..., Xn}. Let f(X | θ) be the probability density function (pdf) of the data, and let T(X) be a function of the data that serves as an estimator of θ. The estimator T(X) is said to be unbiased if E[T(X)] = θ for all possible values of θ, and it is said to be efficient if its variance is as small as possible among all unbiased estimators.

The Cramér-Rao bound states that for any unbiased estimator T(X) of θ, the variance of T(X) must satisfy:

Var(T(X)) ≥ [1 / I(θ)],

where I(θ) is the Fisher information of the statistical model, given by:

I(θ) = E[∂ log f(X | θ) / ∂θ]2,

where the expectation is taken with respect to the pdf f(X | θ). The Fisher information quantifies the amount of information that the data X contains about the unknown parameter θ, and it depends on the specific form of the pdf f(X | θ).

The Cramér-Rao bound is a fundamental result in statistical inference, as it provides a lower bound on the variance of any unbiased estimator of θ. In other words, it tells us that no unbiased estimator can have a smaller variance than the one given by the bound. Moreover, the bound is achieved when the estimator T(X) is chosen to be the maximum likelihood estimator (MLE) of θ, which is defined as the value of θ that maximizes the likelihood function L(θ | X) = f(X | θ).

However, in many practical situations, the true value of the parameter θ may be subject to certain constraints or limitations. For example, in a medical trial, the efficacy of a drug may be restricted to a certain range, or in a signal processing application, the estimator may be subject to certain measurement errors or noise. In such cases, the classical CRB may not provide an accurate estimate of the lower bound on the variance of the estimator, as it does not take into account the additional constraints or limitations.

To address this issue, the modified CRB was introduced, which is a generalization of the classical CRB that takes into account certain constraints or limitations on the parameter estimation problem. The modified CRB provides a more accurate estimate of the lower bound on the variance of the estimator, as it incorporates the additional constraints or limitations into the calculation of the Fisher information.

The general formulation The modified CRB can be formulated in a variety of ways, depending on the specific constraints or limitations of the problem. In general, the modified CRB involves modifying the Fisher information by adding a term that accounts for the constraints or limitations on the parameter estimation.

For example, consider a statistical model where the true value of the parameter θ is known to lie in the interval [a, b], where a and b are known constants. In this case, the modified CRB can be formulated as:

Var(T(X)) ≥ [1 / I*(θ)],

where I*(θ) is the modified Fisher information of the statistical model, given by:

I*(θ) = E[(∂ log f(X | θ) / ∂θ)2 / w(X | θ)],

where w(X | θ) is a weight function that takes into account the constraints or limitations on the parameter estimation. In this case, the weight function is defined as:

w(X | θ) = (1 / (b - a))^2 if a ≤ θ ≤ b, and 0 otherwise.

This weight function ensures that the Fisher information is zero outside the interval [a, b], and it is inversely proportional to the width of the interval inside the interval. Intuitively, this means that the Fisher information is highest at the center of the interval, and it decreases as we move away from the center towards the boundaries of the interval.

Another example of a modified CRB is the robust CRB, which takes into account the presence of outliers or data points that do not follow the assumed statistical model. The robust CRB modifies the Fisher information by adding a term that penalizes the influence of the outliers on the parameter estimation. This term can be expressed as a function of the data itself, and it is typically chosen to be a function that is robust to outliers, such as the Huber function or the Tukey biweight function.

The modified CRB has applications in a variety of fields, such as signal processing, communications, finance, and medical imaging. In signal processing, the modified CRB is used to design efficient estimators for time-delay estimation, frequency estimation, and phase estimation, which are important tasks in wireless communication systems. In finance, the modified CRB is used to estimate the volatility of financial assets, which is a key parameter in many financial models. In medical imaging, the modified CRB is used to estimate the diffusion tensor parameters from diffusion-weighted magnetic resonance imaging (DW-MRI) data, which is an important tool for studying the microstructure of biological tissues.

In conclusion, the modified CRB is a powerful tool in statistical inference that provides a more accurate estimate of the lower bound on the variance of parameter estimators, by taking into account certain constraints or limitations on the parameter estimation problem. The modified CRB can be formulated in a variety of ways, depending on the specific constraints or limitations of the problem, and it has applications in a wide range of fields, from signal processing to finance to medical imaging.