---

gnn_criterion : Basic Criterion Function.
[Criterions]

---

Detailed Description

Todo:

1) Document the datatypes... gnn_criterion and its function types... 2) Implement "evaluate" and "eval"

A criterion is a special structure used for measuring the quality of a given model. It implements a function that takes two input vectors and as arguments, and computes a cost value that should be minimized during training.

Normally, measures the error or distance between the vectors and . This is important, because when modelling, the aim is to minimize the error of the Model, that is, the estimation

should be similar to the desired output .

In order to use the criterion during training, which is almost always an optimization process which uses the gradient , the function's gradient with respect to , that is, , should be available.

libgnn provides a common structure for criterions with the gnn_criterion : Basic Criterion Function. and its associated functions. The gnn_criterion : Basic Criterion Function. data type is a generic structure, which needs to be extended in order to be usable. Basically, the structure should be equipped with an evaluation function and a gradient evaluation function.

To create a new criterion, you have two options:

• use one of the ready-to-use criterions
• implement your own criterion

To use one of the available criterions, just create it using its constructor function. In example, to create a gnn_mean_square criterion, invoke

   gnn_criterion *crit;
   
   crit = gnn_mean_square_new (10);

which creates a new criterion of size 10.

All criterion conform to a common interface, which is:

• gnn_criterion_evaluate_e : Evaluates the criterion given a vector and a target .
• gnn_criterion_evaluate_dy : Evaluates the gradient for the last evaluated and vectors.
• gnn_criterion_destroy : Destroys a criterion (by freeing its memory). Each of these functions behave different for the specific type of criterion.

To implement a new criterion, you have to provide 3 specific functions to be called by those mentioned above and install them into the gnn_criterion : Basic Criterion Function. structure using the gnn_criterion_init function.

The 3 specific functions are of the following types:

• gnn_criterion : Basic Criterion Function. : A function that computes the value of the criterion or cost using and .
• gnn_criterion : Basic Criterion Function. : A function that computes the gradient
• gnn_criterion : Basic Criterion Function. : A destructor for the specific extension's data.

If your own criterion need additional resources, such as additional parameters, then you have to extend the gnn_criterion : Basic Criterion Function. with the additional fields that are needed. Remember to deallocate the specific memory in your destructor, like in a gnn_node.

Another different form of a gnn_criterion : Basic Criterion Function. is a regularizator. It performs a regularization of the error function, in the form:

where is an error criterion, called the "subcriterion" in libgnn, and is a regularization term. The scalar is called the regularization coefficient, which determines the extent in which the regularization term affects the overall resulting error . Regularization do limit the gnn_node's function complexity in order to perform better in generalization. A regularization term usually penalizes the curvature of the output mapping. Some of them (e.g. Weight Decay) depend directly on the values of the function's parameters. They do coexist with a gnn_criterion : Basic Criterion Function., and need an error function upon construction.

Modules
gnn_cross_entropy : Cross-Entropy Error Criterion.
	Cross Entropy Error function.
gnn_mse : Mean-Square-Error Criterion.
	Mean Square Error function.
gnn_msre : Mean-Square-Relative-Error Criterion.
	Mean Square Relative Error function.
gnn_weight_decay : Weight Decay Regularization.
	Weight Decay Regularization.
gnn_weight_elimination : Weight Elimination Regularization.
	Weight Elimination Regularization.
Typedefs
typedef _gnn_criterion	gnn_criterion
	The datatype for criterions.
typedef double(*	gnn_criterion_e )(gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t)
	The datatype for criterions evaluation functions.
typedef int(*	gnn_criterion_dy )(gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t, gsl_vector *dy)
	The datatype for .
typedef void(*	gnn_criterion_destructor )(gnn_criterion *crit)
	The datatype for destructors for criterion extensions.
Functions
int	gnn_criterion_init (gnn_criterion *crit, const char *type, size_t size, gnn_criterion_e e, gnn_criterion_dy dy, gnn_criterion_destructor destroy)
	Initializes a gnn_criterion : Basic Criterion Function..
size_t	gnn_criterion_get_size (gnn_criterion *crit)
	Returns a criterion's size.
const char *	gnn_criterion_get_type (gnn_criterion *crit)
	Returns the name of the criterion.
double	gnn_criterion_evaluate_e (gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t)
	Evaluates the criterion.
int	gnn_criterion_evaluate_dy (gnn_criterion *crit, gsl_vector *dy)
	Evaluates the criterion's gradient.
int	gnn_criterion_eval_dy (gnn_criterion *crit, gsl_vector *dy)
	Evaluates the criterion's gradient.
void	gnn_criterion_destroy (gnn_criterion *crit)
	Destroys the criterion.

---

Typedef Documentation

typedef struct _gnn_criterion gnn_criterion

This is the datatype on which all implementations of criterions build on. It basically contains buffers for intermediate values and three pointers for the criterion's methods. Definition at line 48 of file gnn_criterion.h.

typedef void(* gnn_criterion_destructor)(gnn_criterion *crit)

This datatype defines the form for gnn_criterion : Basic Criterion Function.'s destructors. If your specific implementation of a gnn_criterion : Basic Criterion Function. has additional resources, then they (and nothing else) should be freed by this function, and then return. Definition at line 89 of file gnn_criterion.h.

typedef int(* gnn_criterion_dy)(gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t, gsl_vector * dy)

This datatype defines the form for gnn_criterion : Basic Criterion Function.'s gradient evaluation functions. It takes a pointer to the current criterion, the vectors (model output/estimation) and (desired target value). The function should ADD the resulting gradient onto the values stored in the buffer pointed by "dy". Definition at line 75 of file gnn_criterion.h.

typedef double(* gnn_criterion_e)(gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t)

This datatype defines the form for gnn_criterion : Basic Criterion Function.'s evaluation functions. It takes a pointer to the current criterion, the vectors (model output/estimation) and (desired target value). The function should return a real value (double) that provides an adecuate optimization criterion. Remember that gnn_trainer : Trainers for Models.'s minimize the criterions. Definition at line 60 of file gnn_criterion.h.

---

Function Documentation

void gnn_criterion_destroy (  gnn_criterion *    crit )

This function destroys a given criterion structure by freeing its memory. It will call the specific destroy function (if any) for the criterion, which was previosly installed. Then it frees the fields of gnn_criterion : Basic Criterion Function. and the structure itself. Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. Definition at line 365 of file gnn_criterion.c.

int gnn_criterion_eval_dy (  gnn_criterion *    crit, gsl_vector *    dy )

This function evaluates the criterion's gradient with respect to the last evaluated estimation and target and ADDS the result to the vector pointed by "dy". It is important to note the following rules: This function is called by gnn_criterion : Basic Criterion Function.. Composed criterions, which depend on inner subcriterions often need to compute their gradient as the result of summing up their subgradients. This is why the evaluation of the gradient has been split up into the base function gnn_criterion : Basic Criterion Function. (which clears the buffer "dy" first) and this function. This function calls the criterion's specific gradient function. Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. dy  A pointer to a buffer vector of the correct size where the gradient is added up. Returns: Returns 0 if succeeded. Definition at line 345 of file gnn_criterion.c.

int gnn_criterion_evaluate_dy (  gnn_criterion *    crit, gsl_vector *    dy )

This function evaluates the criterion's gradient with respect to the last evaluated estimation and target . The gradient always is of the same size. Internally, it clears the buffer "dy" (where the resulting gradient should be placed in) and then calls gnn_criterion_evaluate. Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. dy  A pointer to a buffer vector of the correct size for storing the gradient. Returns: Returns 0 if succeeded. Definition at line 309 of file gnn_criterion.c.

double gnn_criterion_evaluate_e (  gnn_criterion *    crit, const gsl_vector *    y, const gsl_vector *    t )

This function evaluates the criterion for the given vectors "y" and "t", which are (usually) the Models output and the desired target vector . It will call the specific evaluation function for the criterion, which was previosly installed. Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. y  A pointer to a vector (the estimation). t  A pointer to a vector (the target). Returns: The value of the criterion. Definition at line 279 of file gnn_criterion.c.

size_t gnn_criterion_get_size (  gnn_criterion *    crit )

This functions returns the size of the vectors and . Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. Returns: Returns the size. Definition at line 237 of file gnn_criterion.c.

const char* gnn_criterion_get_type (  gnn_criterion *    crit )

This functions returns a string which identifies uniquely the current criterion's type. Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. Returns: Returns a non-modifiable string. Definition at line 255 of file gnn_criterion.c.

int gnn_criterion_init (  gnn_criterion *    crit, const char *    type, size_t    size, gnn_criterion_e    e, gnn_criterion_dy    dy, gnn_criterion_destructor    destroy )

This function initializes a gnn_criterion : Basic Criterion Function. by setting its size and installing the specific functions for , and the destructor. It is mandatory to provide: A size that is stricly positive. The evaluation function. The gradient evaluation function. If the destructor is ommitted (by giving NULL as argument), then it will be assumed that no special deallocation is necessary for the extension. In example, suppose you want to implement a criterion called "my_criterion" and that you have already coded the evaluation function and the gradient function, which you called "my_criterion_e" and "my_criterion_dy" respectivelly. Your criterion doesn't need any additional data to compute them, so you didn't extend the basic gnn_criterion : Basic Criterion Function.. Then, the code would be a template for the constructor: gnn_criterion * my_criterion_new (size_t size) { int status; gnn_criterion *crit; // allocate memory for the criterion crit = (gnn_criterion *) malloc (sizeof (gnn_criterion)); // initialize status = gnn_criterion_init (crit, "my criterion", size, my_criterion_e, my_criterion_dy, my_criterion_destroy); if (status) print_an_error_message_and_return_null (); // return return crit; } Parameters: crit  A pointer to a gnn_criterion : Basic Criterion Function.. type  A string containing the name of the criterion. size  The size of the input vectors and . e  A pointer to the evaluation function. dy  A pointer to the gradient evaluation function. destroy  A pointer to the destructor function. Returns: Returns 0 if the initialization could be performed with success. Definition at line 181 of file gnn_criterion.c.

---