On each iteration we will compute a diagonal preconditioning matrix $\mathbf{D}$ (we omit the subscript $n$ ). $\mathbf{D}$ is expected to be a rough approximation to the Hessian. In our experiments, following Martens (2010), we set D to the diagonal of the Fisher matrix computed over $\mathcal{A}_{n} .$ To precondition, we define a normalized parameter vector $\tilde{\boldsymbol{\theta}}=\mathbf{D}^{1 / 2} \boldsymbol{\theta}$, compute the Krylov subspace in terms of $\tilde{\boldsymbol{\theta}}$, and convert back to the "canonical" coordinates. The result is the subspace spanned by the ve…