---

gnn_hessian.h File Reference

#include "gnn_node.h"
#include "gnn_dataset.h"
#include "gnn_criterion.h"

Include dependency graph for gnn_hessian.h:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Functions
int	gnn_hessian_vector_mult (gnn_line *line, const gsl_vector *v, gsl_vector *vH, double eps)
	Fast multiplication by the Hessian matrix.
int	gnn_hessian_diagonal (gnn_line *line, gsl_vector *H, double eps)
	Finite differences diagonal Hessian matrix approximation.
int	gnn_hessian_levenberg_marquadt_init (gnn_grad *grad, gsl_matrix *H)
	Initializes the Levenberg-Marquadt Hessian approximation.
int	gnn_hessian_levenberg_marquadt (gnn_grad *grad, gsl_matrix *H, size_t k)
	Levenberg-Marquadt Hessian approximation.
int	gnn_hessian_inverse_init (gnn_grad *grad, gsl_matrix *H, double alpha)
	Levenberg-Marquadt Hessian Inverse approximation initialization.
int	gnn_hessian_inverse (gnn_grad *grad, gsl_matrix *H, size_t k)
	Levenberg-Marquadt Hessian Inverse approximation.
int	gnn_hessian_finite_differences (gnn_line *line, gsl_matrix *H, double eps)
	Finite differences Hessian matrix approximation.

---

Function Documentation

int gnn_hessian_diagonal (  gnn_line *    line, gsl_vector *    H, double    eps )

This numerical efficient method computes the diagonal values of the Hessian matrix using the central differentes method, given by where is a small value that gives the required precision, since the error resulting from this approximation is . The same as above written in matrix notation is: where is the Hessian matrix and . Note that this requires a single forward and backward step for evaluation, since the assumption of the diagonal Hessian is equivalent to an uncorrelated parameter space. Parameters: grad  A pointer to a Evaluations. buffer. H  A pointer to a vector of size , where is the number of parameters involved in the gnn_node. The resulting diagonal terms of the Hessian will be stored in this vector. eps  The required precision . Returns: Returns 0 if succeeded. Definition at line 184 of file gnn_hessian.c.

int gnn_hessian_finite_differences (  gnn_line *    line, gsl_matrix *    H, double    eps )

This numerical efficient method computes the values of the Hessian matrix using the central differentes method, given by where is a small value that gives the required precision, since the error resulting from this approximation is . Parameters: line  A pointer to a Evaluations. buffer. H  A pointer to a matrix of size , where is the number of parameters involved in the gnn_node. The resulting Hessian will be stored in this matrix. eps  The required precision . Returns: Returns 0 if succeeded. Definition at line 534 of file gnn_hessian.c.

int gnn_hessian_inverse (  gnn_grad *    grad, gsl_matrix *    H, size_t    k )

This function computes the k-th iteration of the outer-product inverse Hessian approximation, using the k-th pattern. The outer product approximation used in gnn_hessian_levenberg_marquadt can be used to develop an efficient procedure for computing the inverse of the Hessian matrix. Recalling the formula for the outer product approximation where , it is easy to see that this leads to a sequential form for computing the Hessian: Using the following matrix identity where is the identity matrix, and setting , and , then the previous formula can be rewritten in order to obtain an iterative procedure for building the inverse of the Hessian: The procedure can be initialized by , where is a small value (because the procedure actually seeks to approximate ). It is important to note that, as in the case for gnn_hessian_levenberg_marquadt, this approximation is only valid for a sum-of-squares error function. Warning: Evaluation??? Shouldn't it be handled completely by a gnn_eval or gnn_grad?? Parameters: grad  A pointer to a Evaluations. buffer. H  A pointer to a matrix of size , where is the number of parameters involved in the gnn_node. The resulting inverse Hessian will be stored in this matrix. k  The index of the pattern to be used. Returns: Returns 0 if succeeded. Definition at line 450 of file gnn_hessian.c.

int gnn_hessian_inverse_init (  gnn_grad *    grad, gsl_matrix *    H, double    alpha )

This function initializes the outer product inverse approximation routine. For further details, please refer to gnn_hessian_inverse. Parameters: grad  A pointer to a Evaluations. buffer. H  A pointer to a matrix of size , where is the number of parameters involved in the gnn_node. The resulting inverse Hessian will be stored in this matrix. alpha  A small value for . Returns: Returns 0 if succeeded. Definition at line 376 of file gnn_hessian.c.

int gnn_hessian_levenberg_marquadt (  gnn_grad *    grad, gsl_matrix *    H, size_t    k )

This function computes the k-th iteration of the Levenberg-Marquadt Hessian approximation, using the k-th pattern. The Levenberg-Marquadt approximation (also known as the outer product approximation) of the Hessian matrix is given by where the sum runs over all pattern outputs , . It is very important to note that this approximation is only valid for an error function of the form that is, for an sum-of-squares error function (and its equivalent forms). The previous formula can be written in matrix form, giving the sequential procedure for building the Hessian: which is the way used by libgnn. Warning: Evaluation??? Shouldn't it be handled completely by a gnn_eval or gnn_grad?? Parameters: grad  A pointer to a Evaluations. buffer. H  A pointer to a matrix of size , where is the number of parameters involved in the gnn_node. The resulting Hessian will be stored in this matrix. k  The index of the pattern to be used. Returns: Returns 0 if succeeded. Definition at line 309 of file gnn_hessian.c.

int gnn_hessian_levenberg_marquadt_init (  gnn_grad *    grad, gsl_matrix *    H )

This function initializes the Hessian for the Levenberg-Marquadt approximation. For further details, see gnn_hessian_levenberg_marquadt. Parameters: grad  A pointer to a Evaluations. buffer. H  A pointer to a matrix of size , where is the number of parameters involved in the gnn_node. The resulting Hessian will be stored in this matrix. Returns: Returns 0 if succeeded. Definition at line 250 of file gnn_hessian.c.

int gnn_hessian_vector_mult (  gnn_line *    line, const gsl_vector *    v, gsl_vector *    vH, double    eps )

Some applications do not require the Hessian explicitally. Instead, what really is needed is a fast method for computing , where is the Hessian matrix (which is assumed to be symmetric, so that ) and is a vector. This numerical method computes this multiplication using central differences: where is a small value that gives the required precision, since the error resulting from this approximation is . Parameters: grad  A pointer to a Evaluations. buffer. v  A pointer to the vector . vH  A pointer to a vector of size , where is the number of parameters involved in the gnn_node. The resulting multiplication will be stored in this vector. eps  The required precision . Returns: Returns 0 if succeeded. Definition at line 97 of file gnn_hessian.c.

---