Probability

Importance Sampling

According to wiki, importance sampling is a Monte Carlo method for evaluating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest.

$$\begin{align*} E_p[f(X)] & = \int{}f(x)p(x)dx \\ & = \int{}f(x)\frac{p(x)}{q(x)}q(x)dx \\ & = E_q[\frac{p(X)}{q(X)}f(X)] \end{align*}$$

The term $\frac{p(x)}{q(x)}$is a weight factor. The left side of the equation is the expectation of $f(X)$ under distribution $p(x)$. Suppose we have samples of $X$under distribution $q(x)$, we can reconstruct the expectation of $f(X)$ under distribution $p(x)$ by weighting the samples (i.e. multiplying $f(x)$by $\frac{p(x)}{q(x)}$.

The distribution $p(x)$is sometimes called the target distribution and they are often not directly accessible.