Maximum A Posteriori (MAP) estimation can be viewed as an extension of Maximum Likelihood Estimation (MLE). In statistical inference, this distinction is about how each method incorporates prior knowledge.

• MLE is primarily concerned with the hidden factors that best explain the observed data, focusing exclusively on maximizing the likelihood of observing the data given different parameter values. It poses the question: 'Given the parameters, how probable is the observed data?' However, MLE does not account for any pre-existing beliefs or information about the parameter values before observing the data.
• In contrast, MAP takes a more comprehensive approach. It not only considers the likelihood of the observed data but also integrates our prior beliefs or knowledge about the parameters. This integration is achieved by multiplying the likelihood by a prior distribution, a crucial step that embeds our preconceived notions or assumptions into the estimation process. This dual consideration of both belief and observation allows MAP to better infer the hidden factors, providing a more nuanced estimate, especially valuable when handling limited data or when prior information is substantial and credible.

Essentially, while MLE offers a data-centric view, MAP delivers a balanced perspective, amalgamating both data and prior knowledge to deduce parameter values.

MAP Formal

The MAP estimation is about finding the parameter values $\theta$ that maximize the posterior probability given the data. The posterior probability is indeed a product of the likelihood and the prior probability, as you've described. However, the way it's typically represented in the formula might need some refinement for clarity and accuracy.

Standard Notation:

The standard way to express the posterior probability in MAP estimation is:

$$ P(\theta | X = x) = \frac{P(X = x | \theta) \cdot P(\theta)}{P(X = x)} $$

Where:

• $P(\theta | X = x)$ is the posterior probability of $\theta$ given the observed data ($X = x$) (what we aim to maximize in MAP).
• $P(X = x | \theta)$ is the likelihood of observing $X = x$ given the parameters $\theta$.
• $P(\theta)$ is the prior probability of $\theta$ (reflecting our prior belief about the parameter).
• $P(X = x)$ is the evidence, or the probability of observing the data. It acts as a normalizing constant.

It is essential to recognize that although the evidence (denominator in the Bayesian formula) plays a crucial role in determining the posterior probability, it does not influence the specific value of the parameter $\theta$ that maximizes this probability. This independence arises because the evidence is a constant with respect to $\theta$. As a result, in the context of optimization for MAP estimation, we can focus solely on the numerator, which is the product of the likelihood and the prior. This approach allows us to simplify the formula without impacting the determination of the optimal $\theta$. For convenience, we can define a new term, such as $\mathcal{B}$, to represent this product of the likelihood and the prior, keeping in mind that this term is essentially the unnormalized version of the posterior probability.

$$ \mathcal{B}(\theta) = P(X = x|\theta) \cdot P(\theta) $$

Where:

• $X$ represents the random variable associated with the observed data.
• $x$ specific values that the data takes.
• $\theta$ denotes the model parameters.