C.4 The covariance matrix

The terms of the covariance matrix $$V_ij$ are defined by: 

\begin{equation}  V_{ij} = \left< \left(u_ i - u^0_ i\right) \left(u_ j - u^0_ j\right) \right> \label{eqn:def_ covar} \end{equation}

\noindent Its leading diagonal terms may be recognised as equalling the variances of each of our 

$n_u$ variables; its cross terms measure the correlation between the variables. If a component $V_ij > 0$, it implies that higher estimates of the coefficient $u_i$ make higher estimates of $u_j$ more favourable also; if $V_ij < 0$, the converse is true. 

It is a standard statistical result that 

$V = (-A)^-1$. In the remainder of this section we prove this; readers who are willing to accept this may skip onto Section~ \ref{sec:correlation_ matrix}. 

Using 

$Δu_i$ to denote $u_i - u^0_i$, we may proceed by rewriting Equation~ (\ref{eqn:def_ covar}) as: 

\begin{eqnarray}  V_{ij} &  = &  \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \mathrm{P}\left( \mathbf{u} | \left\{  \mathbf{x}_ i, f_ i, \sigma _ i \right\}  \right) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} \\ &  = &  \frac{ \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} }{ \idotsint _{u_ i=-\infty }^{\infty } \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} } \nonumber \end{eqnarray}

The normalisation factor in the denominator of this expression, which we denote as 

$Z$, the \textit{partition function}, may be evaluated by $n_u$-dimensional Gaussian integration, and is a standard result: 

\begin{eqnarray}  Z &  = &  \idotsint _{u_ i=-\infty }^{\infty } \exp \left(\frac{1}{2} \Delta \mathbf{u}^\mathbf {T} \mathbf{A} \Delta \mathbf{u} \right) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} \\ &  = &  \frac{(2\pi )^{n_\mathrm {u}/2}}{\mathrm{Det}(\mathbf{-A})} \nonumber \end{eqnarray}

Differentiating 

$_e(Z)$ with respect of any given component of the Hessian matrix $A_ij$ yields: 

\begin{equation}  -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(Z) \right] = \frac{1}{Z} \idotsint _{u_ i=-\infty }^{\infty } \Delta u_ i \Delta u_ j \exp (-Q) \, \mathrm{d}^{n_\mathrm {u}}\mathbf{u} \end{equation}

\noindent which we may identify as equalling 

$V_ij$: 

\begin{eqnarray}  \label{eqa:v_ zrelate} V_{ij} &  = &  -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(Z) \right] \\ &  = &  -2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e((2\pi )^{n_\mathrm {u}/2}) - \log _ e(\mathrm{Det}(\mathbf{-A})) \right] \nonumber \\ &  = &  2 \frac{\partial }{\partial A_{ij}} \left[ \log _ e(\mathrm{Det}(\mathbf{-A})) \right] \nonumber \end{eqnarray}

\noindent This expression may be simplified by recalling that the determinant of a matrix is equal to the scalar product of any of its rows with its cofactors, yielding the result: 

\begin{equation}  \frac{\partial }{\partial A_{ij}} \left[\mathrm{Det}(\mathbf{-A})\right] = -a_{ij} \end{equation}

\noindent where 

$a_ij$ is the cofactor of $A_ij$. Substituting this into Equation~ (\ref{eqa:v_ zrelate}) yields: 

\begin{equation}  V_{ij} = \frac{-a_{ij}}{\mathrm{Det}(\mathbf{-A})} \end{equation}

Recalling that the adjoint 

$A^†$ of the Hessian matrix is the matrix of cofactors of its transpose, and that $A$ is symmetric, we may write: 

\begin{equation}  V_{ij} = \frac{-\mathbf{A}^\dagger }{\mathrm{Det}(\mathbf{-A})} \equiv (-\mathbf{A})^{-1} \end{equation}

\noindent which proves the result stated earlier. 

$