Main Page   Modules   Data Structures   File List   Data Fields   Globals   Related Pages  

gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm.
[gnn_trainer : Trainers for Models.]

Detailed Description

This trainer implements the Conjugate Gradient minimization procedure. The conjugate gradients method takes one step after another in special conjugate directions, minimizing the error function along the current line.

The conjugate directions are built using a sequential procedure, given by the formula:

where is the new search direction, which is conjugate to the old , and are the new and the previous computed gradients with respect to the function's parameters. The is a coefficient which can be computed in different ways. There are three forms:

Informally, two directions are said to be conjugate, if they are "ortogonal" in a sense considering the second-order Taylor approximation of the error surface.

After choosing the direction, the algorithm seeks the minimum along this line. This is performed, in practice, using a line-search procedure. You can install any of the available line-search procedures for the training algoritm. Please refer to (gnn_line_search) for further details.

Defines

#define GNN_CONJUGATE_GRADIENT_TOL   0.0001
 Default precision for line search procedures.

#define GNN_CONJUGATE_GRADIENT_STEP   0.1
 Default bracketing step taken by the line search procedures.

#define GNN_CONJUGATE_GRADIENT_RESTART   100
 Default restart.

#define GNN_CONJUGATE_GRADIENT_ALPHA   gnn_line_search_brent
 Default line search procedure.

#define GNN_CONJUGATE_GRADIENT_BETA   gnn_conjugate_gradient_polak_ribiere
 Default .


Typedefs

typedef double(* gnn_conjugate_gradient_beta )(gnn_trainer *trainer)
 Type for evaluation procedures.

typedef _gnn_conjugate_gradient gnn_conjugate_gradient
 Conjugate gradients trainer structure.


Functions

int gnn_conjugate_gradient_reset (gnn_trainer *trainer)
 The trainer's "reset" implementation.

void gnn_conjugate_gradient_destroy (gnn_trainer *trainer)
 The trainers "destroy" implementation.

gnn_trainer * gnn_conjugate_gradient_new (gnn_node *node, gnn_criterion *crit, gnn_dataset *data)
 Creates a new conjugate gradient descent trainer.

double gnn_conjugate_gradient_polak_ribiere (gnn_trainer *trainer)
 The Polak-Ribière form for the coefficient.

double gnn_conjugate_gradient_hestenes_stiefel (gnn_trainer *trainer)
 The Hestenes-Stiefel form for the coefficient.

double gnn_conjugate_gradient_fletcher_reeves (gnn_trainer *trainer)
 The Fletcher-Reeves form for the coefficient.

int gnn_conjugate_gradient_set_tol (gnn_trainer *trainer, double tol)
 Sets the precision tolerance for the line search procedure.

double gnn_conjugate_gradient_get_tol (gnn_trainer *trainer)
 Gets the tolerance for the line search procedure.

int gnn_conjugate_gradient_set_step (gnn_trainer *trainer, double step)
 Sets the initial step for the interval bracketing procedure.

double gnn_conjugate_gradient_get_step (gnn_trainer *trainer)
 Gets the initial step for the interval bracketing procedure.

int gnn_conjugate_gradient_set_restart (gnn_trainer *trainer, size_t restart)
 Sets the number of iterations before restarting.

size_t gnn_conjugate_gradient_get_restart (gnn_trainer *trainer)
 Gets the number of iterations before reinitializing the direction.

int gnn_conjugate_gradient_set_line_search (gnn_trainer *trainer, gnn_line_search_type lsearch)
 Sets the line search procedure.

gnn_line_search_type gnn_conjugate_gradient_get_alpha (gnn_trainer *trainer)
 Gets the installed line search procedure.

int gnn_conjugate_gradient_set_beta (gnn_trainer *trainer, gnn_conjugate_gradient_beta beta)
 Sets the form of the coefficient.

gnn_conjugate_gradient_beta gnn_conjugate_gradient_get_beta (gnn_trainer *trainer)
 Gets the evaluation function.

Define Documentation

#define GNN_CONJUGATE_GRADIENT_ALPHA   gnn_line_search_brent
 

This is the default line search procedure used by the conjugate gradients algorithm.

Definition at line 77 of file gnn_conjugate_gradient.h.

#define GNN_CONJUGATE_GRADIENT_BETA   gnn_conjugate_gradient_polak_ribiere
 

This defines the default method used for the parameter in the conjugate gradients method.

Definition at line 86 of file gnn_conjugate_gradient.h.

#define GNN_CONJUGATE_GRADIENT_RESTART   100
 

This constant defines the default number of iterations that the conjugate gradients algorithm takes before restarting the algorithm.

Definition at line 68 of file gnn_conjugate_gradient.h.

#define GNN_CONJUGATE_GRADIENT_STEP   0.1
 

This constant defines the initial step taken by the bracketing algorithm to search the minimum along the line.

Definition at line 59 of file gnn_conjugate_gradient.h.

#define GNN_CONJUGATE_GRADIENT_TOL   0.0001
 

This constant defines the tolerance for the line search procedure used by the conjugate gradients algorithm.

Definition at line 50 of file gnn_conjugate_gradient.h.

Typedef Documentation

typedef struct _gnn_conjugate_gradient gnn_conjugate_gradient
 

This type extends the basic gnn_trainer : Trainers for Models. structure for the conjugate gradients trainer.

Definition at line 104 of file gnn_conjugate_gradient.h.

typedef double(* gnn_conjugate_gradient_beta)(gnn_trainer *trainer)
 

This type defines the form of the procedures for computing the coefficient used by the conjugate gradients method.

Definition at line 95 of file gnn_conjugate_gradient.h.

Function Documentation

void gnn_conjugate_gradient_destroy (  gnn_trainer *    trainer )  [static]
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..

Definition at line 209 of file gnn_conjugate_gradient.c.

double gnn_conjugate_gradient_fletcher_reeves (  gnn_trainer *    trainer ) 
 

This function returns the Fletcher-Reeves form of the coefficient for the evaluation of the new conjugate direction:

where

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns the coefficient.

Definition at line 404 of file gnn_conjugate_gradient.c.

gnn_line_search_type gnn_conjugate_gradient_get_alpha (  gnn_trainer *    trainer ) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns a pointer to the installed line-search procedure.

Definition at line 613 of file gnn_conjugate_gradient.c.

gnn_conjugate_gradient_beta gnn_conjugate_gradient_get_beta (  gnn_trainer *    trainer ) 
 

This function returns a pointer to the installed evaluation procedure.

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
A pointer to a function.

Definition at line 671 of file gnn_conjugate_gradient.c.

size_t gnn_conjugate_gradient_get_restart (  gnn_trainer *    trainer ) 
 

This function returns the number of iterations executed by the conjugate gradients trainer before reinitializing the search direction.

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns the number of iterations.

Definition at line 556 of file gnn_conjugate_gradient.c.

double gnn_conjugate_gradient_get_step (  gnn_trainer *    trainer ) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
The trainer's internal step.

Definition at line 505 of file gnn_conjugate_gradient.c.

double gnn_conjugate_gradient_get_tol (  gnn_trainer *    trainer ) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns the tolerance's value.

Definition at line 457 of file gnn_conjugate_gradient.c.

double gnn_conjugate_gradient_hestenes_stiefel (  gnn_trainer *    trainer ) 
 

This function returns the Hestenes-Stiefel form of the coefficient for the evaluation of the new conjugate direction:

where

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns the coefficient.

Definition at line 368 of file gnn_conjugate_gradient.c.

gnn_trainer* gnn_conjugate_gradient_new (  gnn_node *    node,
gnn_criterion *    crit,
gnn_dataset *    data
) 
 

This function creates a new conjugate gradients trainer (gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm.).

Parameters:
node  A pointer to a gnn_node.
crit  A pointer to a gnn_criterion : Basic Criterion Function..
data  A pointer to a gnn_dataset : Datasets for Training..
Returns:
Returns a pointer to a new gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm. trainer.

Definition at line 251 of file gnn_conjugate_gradient.c.

double gnn_conjugate_gradient_polak_ribiere (  gnn_trainer *    trainer ) 
 

This function returns the Polak-Ribière form of the coefficient for the evaluation of the new conjugate direction:

where

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns the coefficient.

Definition at line 333 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_reset (  gnn_trainer *    trainer )  [static]
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
Returns:
Returns 0 if suceeded.

Definition at line 103 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_set_beta (  gnn_trainer *    trainer,
gnn_conjugate_gradient_beta    beta
) 
 

This function sets a new form for evaluating the coefficient used by the conjugate gradients method to build the new search direction. 

 gnn_trainer *trainer;
 trainer = gnn_conjugate_gradient_new (node, crit, data);

 // use the Fletcher-Reeves form
 gnn_conjugate_gradient_set_beta (trainer,
                                  gnn_conjugate_gradient_fletcher_reeves);
Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
beta  A pointer to the a evaluation procedure.
Returns:
Returns 0 if succeeded.

Definition at line 645 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_set_line_search (  gnn_trainer *    trainer,
gnn_line_search_type    lsearch
) 
 

This function sets a new line search procedure used by the conjugate gradients trainer. 

 gnn_trainer *trainer;
 trainer = gnn_conjugate_gradient_new (node, crit, data);

 // use the Golden-Section line search
 gnn_conjugate_gradient_set_line_search (trainer, gnn_line_search_golden);

Please refer to (Line Search Procedures for Nodes.) for the available line search procedures.

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
lsearch  A pointer to a line search procedure.
Returns:
Returns 0 if succeeded.

Definition at line 590 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_set_restart (  gnn_trainer *    trainer,
size_t    restart
) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
restart  A stricly positive integer.
Returns:
Returns 0 if succeeded.

Definition at line 525 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_set_step (  gnn_trainer *    trainer,
double    step
) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
step  A stricly positive real value.
Returns:
Returns 0 if succeeded.

Definition at line 477 of file gnn_conjugate_gradient.c.

int gnn_conjugate_gradient_set_tol (  gnn_trainer *    trainer,
double    tol
) 
 

Parameters:
trainer  A pointer to a gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm..
tol  A stricly positive real value.
Returns:
Returns 0 if succeeded.

Definition at line 429 of file gnn_conjugate_gradient.c.

Generated on Sun Jun 13 20:51:44 2004 for libgnn Gradient Retropropagation Machine Library by doxygen1.2.18