Probability

Importance Sampling

According to wiki, importance sampling is a Monte Carlo method for evaluating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest.

\begin{align*} E_p[f(X)] & = \int{}f(x)p(x)dx \\ & = \int{}f(x)\frac{p(x)}{q(x)}q(x)dx \\ & = E_q[\frac{p(X)}{q(X)}f(X)] \end{align*}

The term $\frac{p(x)}{q(x)}$ is a weight factor. The left side of the equation is the expectation of $f(X)$ under distribution $p(x)$ . Suppose we have samples of $X$ under distribution $q(x)$ , we can reconstruct the expectation of $f(X)$ under distribution $p(x)$ by weighting the samples (i.e. multiplying $f(x)$ by $\frac{p(x)}{q(x)}$ .

The distribution $p(x)$ is sometimes called the target distribution and they are often not directly accessible.

PreviousMatrix Properties NextExpectation-Maximization Algorithm

Last updated 1 year ago

Importance Sampling

\begin{align*} E_p[f(X)] & = \int{}f(x)p(x)dx \\ & = \int{}f(x)\frac{p(x)}{q(x)}q(x)dx \\ & = E_q[\frac{p(X)}{q(X)}f(X)] \end{align*}

The term

\frac{p(x)}{q(x)}

is a weight factor. The left side of the equation is the expectation of

f(X)

under distribution

p(x)

. Suppose we have samples of

X

under distribution

q(x)

, we can reconstruct the expectation of

f(X)

under distribution

p(x)

by weighting the samples (i.e. multiplying

f(x)

\frac{p(x)}{q(x)}

The distribution

p(x)

is sometimes called the target distribution and they are often not directly accessible.