Probability Densities and Normal Distribution

Probability Densities

Probability densities are fundamental to describing the likelihood of outcomes for continuous random variables. While a discrete random variable (like the roll of a die) has a countable set of outcomes, a continuous random variable (like a robot's exact position $h$, or time, or temperature) can take on an infinite, non-countable number of values within a range.

For continuous variables, we cannot assign a probability to a single specific value (e.g., the probability that a robot's position is exactly $2.000000000...$). This probability would be zero. Instead, we use a Probability Density Function (PDF), often denoted $p(x)$.

The Role of Density

The PDF describes the relative likelihood of the random variable $x$ taking on a value near a specific point.

• Probability is Area: The probability that the variable falls within a certain interval (e.g., between position $a$ and $b$) is given by the area under the PDF curve (the integral) over that interval. $$P(a \le x \le b) = \int_a^b p(x) dx$$
• Total Area is One: Like all probability distributions, the total area under the entire PDF curve must equal one, representing 100% certainty that the value exists somewhere on the range. $$\int_{-\infty}^{\infty} p(x) dx = 1$$

Application in Estimation

In systems like the Kalman Filter, the state of the system ($\mathbf{x}_k$ or $h_t$) is a continuous random variable that is always described by a PDF, typically a Normal (Gaussian) distribution.

• The mean of the PDF is the filter's best estimate of the state.
• The variance (or covariance matrix $P_k$) of the PDF is the filter's measure of uncertainty or error.

By maintaining and updating this density function recursively, the filter can track the mean (estimate) and the variance (uncertainty) over time.

The Normal (Gaussian) Distribution (1D)

The Normal distribution, often called the Gaussian distribution or the bell curve, is arguably the most important probability distribution in statistics, science, and engineering. It is the fundamental distribution used in the Kalman Filter to model both the system's state and its associated uncertainties.

Definition and Parameters

In one dimension (1D), the Normal distribution $\mathcal{N}$ is defined entirely by just two parameters:

• The Mean ($\mu$): This represents the center or the expected value of the distribution. In the context of filtering, $\mu$ is the best estimate of the robot's position ($h_t$) or the value of the state vector ($\hat{x}_k$)

• The Variance ($\sigma^2$): This measures the spread or dispersion of the data around the mean. A larger $\sigma^2$ means higher uncertainty. In filtering, the variance is the measure of the estimation error. The square root of the variance, $\sigma$, is the standard deviation.

  The Normal distribution's Probability Density Function (PDF) $p(x)$ is given by the formula:$$p(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x - \mu}{\sigma}\right)^2}$$

from ipynb.fs.defs.plot_probability_density_and_normal_distribution  import plot_normal_dist
plot_normal_dist()

The area under the Normal distribution curve within one standard deviation ($\pm 1\sigma$) of the mean encompasses approximately $68.2\%$ of the total probability, within two standard deviations ($\pm 2\sigma$) it covers about $95.4\%$, and within three standard deviations ($\pm 3\sigma$) it includes nearly $99.7\%$ of the total probability.

Significance in the Kalman Filter

The Gaussian assumption is essential to the Kalman Filter's mathematical tractability and efficiency:

• Closure Property: When the system is linear and the noise is Gaussian, the distribution remains Gaussian after both the Prediction and Measurement Update steps.
• Moment Tracking: Because the state distribution is Gaussian, the filter only needs to track the first two moments (the mean and the covariance) to completely characterize the distribution. This avoids the need to store or compute complex, high-dimensional probability tables.