Analytic Marginalizing For Posteriors

Consider the model

\[ X_{t+1} = X_t r e^{-\beta X_t + \sigma Z_t } \]

with \(Z_t\) a unit normal random variable. The likelihood of the sequence of \(T\) observations of \(X\) under this model is thus

\[P(X | r, \beta, \sigma) = \frac{1}{\sqrt{2 \pi \sigma^2}^{T-1}} \exp\left(\frac{\sum_t^{T-1} \left(\log X_{t+1} - \log X_t - \log r + \beta X_t\right)^2 }{2 \sigma^2}\right) \]

To integrate out \(r\), \(P(X | \beta, \sigma) = \int P(X | r, \beta, \sigma ) P(r) dr\), we’ll make this look like a Gaussian in \(\log r\) by completing the square; getting the square on the outside of the sum. First we collect all the other terms as the factor, \(M_t\);

\[M_t := \log X_{t+1} - \log X_t + \beta X_t\]

Also define \(a = \log r\), then after expanding the square inside the sum we have

\[\sum_t^{T-1} \left(\log r - M_t\right)^2 = \sum_t^{T-1} a^2 - 2 \sum_t^{T-1} a M_t + \sum_t^{T-1} M_t^2 \]

(using the linearity of the summation operator). Use the trick of adding and subtracting \(\left( \sum M_t \right)^2/(T-1)\), to get:

\[ = -\sum_t^{T-1} M_t^2 -\frac{\left(\sum_t^{T-1} M_t\right)^2}{T-1} + \frac{\left(\sum_t^{T-1} M_t\right)^2}{T-1} - 2 a \sum M_t + (T-1)a^2\]

\[ = -\sum_t^{T-1} M_t^2 -\frac{\left(\sum_t^{T-1} M_t\right)^2}{T-1} +(T-1)\left( \left(\frac{\sum_t^{T-1} M_t}{T-1}\right)^2 - \frac{2 a \sum M_t}{T-1} + a^2\right)\]

\[ = -\sum_t^{T-1} M_t^2 -\frac{\left(\sum_t^{T-1} M_t\right)^2}{T-1} +(T-1)\left( \frac{\sum_t^{T-1} M_t}{T-1} - a\right)^2\]

Returning this expression into our exponential in place of the sum of squares, we have

\[P(r, \beta, \sigma | X) = \frac{1}{\sqrt{2 \pi \sigma^2}^{T-1}} \exp\left(-\frac{(T-1)\left( \frac{\sum_t^{T-1} M_t}{T-1} - a\right)^2}{2 \sigma^2}\right) \exp\left(\frac{-\sum_t^{T-1} M_t^2 -\frac{\left(\sum_t^{T-1} M_t\right)^2}{T-1} }{2 \sigma^2}\right) \]

Note that the second \(\exp\) term does not depend on \(a\). The remaining argument has Gaussian form in \(da\), so after pulling out the constant terms we can easily integrate this over \(da\). (Note that we have an implicit uniform prior on \(a\) here).

\[ \int \exp\left(\frac{-\left(\frac{\sum_t^{T-1} M_t}{T-1} - a \right)^2}{2 \sigma^1 (T-1)^{-1} } \right) d a = \sqrt{\frac{2\pi\sigma^2}{T-1}}\]

which we can combine with the remaining terms to recover

\[ \frac{1}{(T-1)\left(2 \pi \sigma^2 \right)^{T-2}} \exp\left(\frac{-\sum M_t^2 + \frac{\left(\sum M_t\right)^2}{T-1}}{2\sigma^2} \right)\]

marginalizing over \(\sigma\)

Now that we have effectively eliminated the parameter \(r\) from our posterior calculation, we wish to also integrate out the second parameter, \(\sigma\). Once again we can “integrate by analogy;” the expression above in the variable \(\sigma^2\) looks like a Gamma distribution,

\[ \int x^{\alpha - 1} e^{-\beta x} dx = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \]

Where we take

\[ \alpha = T/2\]


\[ \beta = \frac{1}{2} \left( \sum M_t^2 - \frac{ \left( \sum M_t \right)^2}{T - 1}\right) \],

leaving us with

\[\frac{1}{(T-1)\sqrt{2 \pi}^{T-2} } \frac{\tfrac{1}{2}^{T/2} \left( \sum M_t^2 - \frac{ \left( \sum M_t \right)^2}{T - 1}\right)^{T/2}}{\Gamma(T/2)} \]

Additional recruitment functions

The above derivation can be followed identically for the three-parameter recruitment functions I refer to as the Allen and Myers models after an appropriate choice of \(M_t\). In both the Ricker and Allen models we must first reparamaterize the models to isolate the \(\alpha\) term correctly.


The original parameterization

\[ X_{t+1} = X_t e^{r \left( 1 - \frac{X_t}{K}\right)}\] does not partition into the form above. Taking \(\beta = \tfrac{r}{K}\) and \(a = r\), we can write \(M_t\) as above,

\[M_t := \log X_{t+1} - \log X_t + \beta X_t\]


\[ X_{t+1} = \frac{ r X_t^{\theta} }{1 + \frac{X_t^{\theta}}{K} } Z_t\]

For \(Z_t\) lognormal with log-mean zero and log-standard-deviation \(\sigma\), The log-likelihood takes the form

and thus we can write \(M_t\) as

\[M_t := \log X_{t+1} - \theta \log X_t + \log\left(1 + \frac{X_t^{\theta}}{K} \right)\]


The original parameterization

\[ X_{t+1} = Z_t X_t e^{r \left( 1 - \frac{X_t}{K} \right) \frac{\left(X_t - \theta\right)}{K}} \]

does not let us isolate an additive constant (log-mean term) as we did in the example above. Writing the argument of the exponent in standard quadratic form,

\[ X_{t+1} = Z_t X_t e^{c + b X+t + a X_t^2} \]


\[c = \tfrac{-rC}{K}\] \[b = \tfrac{r}{K}\left(\tfrac{C}{K} + 1\right)\] \[a = \tfrac{r}{K^2}\]


\[M_t := \log X_{t+1} - \log X_t - b X_t + a X_t^2 \]