---

gnn_weight_elimination : Weight Elimination Regularization.
[gnn_criterion : Basic Criterion Function.]

---

Detailed Description

This datatype implements the Weight Elimination Regularization, given by the error penalty term:

Where the are parameters that have been chosen from a gnn_node by hand. They are usually those of a gnn_weight : Affine / Linear Transform linear transform node. The term is a scale parameter that usually is taken of order unity. The weight elimination regularizer will favour few large terms rather than many small ones, like in weight decay (gnn_weight_decay).

Although the weights never fall to zero in practice, after training the associated gnn_node a while, you can call the gnn_weight_elimination_prun function, which deletes the weights below a given threshold.

As in general with all regularizers, the gnn_weight_elimination doesn't work by itself, and needs a gnn_criterion : Basic Criterion Function. to build upon.

The resulting error function is:

where is the regularization coeffcient. The corresponding gradient is:

Functions
double	gnn_weight_elimination_e (gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t)
	The evaluation function.
int	gnn_weight_elimination_dy (gnn_criterion *crit, const gsl_vector *y, const gsl_vector *t, gsl_vector *dy)
	The gradient evaluation function.
void	gnn_weight_elimination_destroy (gnn_criterion *crit)
	The destroy function.
gnn_criterion *	gnn_weight_elimination_new (gnn_criterion *crit, double nu, double wp, gnn_node *node)
	Creates a Weight Elimination regularization for gnn_weight : Affine / Linear Transform.
gnn_criterion *	gnn_weight_elimination_new_with_type (gnn_criterion *crit, double nu, double wp, gnn_node *node, const char *type)
	Creates a Weight Elimination regularization for a given type.
gnn_criterion *	gnn_weight_elimination_new_with_pbundle (gnn_criterion *crit, double nu, double wp, gnn_pbundle *pb)
	Creates a Weight Elimination regularization for a given pbundle.
int	gnn_weight_elimination_set_nu (gnn_criterion *crit, double nu)
	Sets the regularization coefficient.
double	gnn_weight_elimination_get_nu (gnn_criterion *crit)
	Gets the regularization coefficient.
int	gnn_weight_elimination_set_wp (gnn_criterion *crit, double wp)
	Sets the scale parameter .
double	gnn_weight_elimination_get_wp (gnn_criterion *crit)
	Gets the current scale parameter .
int	gnn_weight_elimination_prun (gnn_criterion *crit, double th)
	Perform parameter pruning.

---

Function Documentation

void gnn_weight_elimination_destroy (  gnn_criterion *    crit )  [static]

This function implements destroy function. Parameters: crit  A pointer to a gnn_weight_elimination criterion. Definition at line 232 of file gnn_weight_elimination.c.

int gnn_weight_elimination_dy (  gnn_criterion *    crit, const gsl_vector *    y, const gsl_vector *    t, gsl_vector *    dy )

This function implements criterion's gradient evaluation function. Parameters: crit  A pointer to a gnn_weight_elimination criterion. y  A pointer to an estimation vector . t  A pointer to the desired target vector . dy  A pointer to a buffer vector where the result should be placed. Returns: Returns 0 if succeeded. Definition at line 177 of file gnn_weight_elimination.c.

double gnn_weight_elimination_e (  gnn_criterion *    crit, const gsl_vector *    y, const gsl_vector *    t )

This function corresponds to the evaluation function. Parameters: crit  A pointer to a gnn_weight_elimination criterion. y  A pointer to an estimation vector . t  A pointer to the desired target vector . Returns: A real number corresponding to the value of the criterion. Definition at line 121 of file gnn_weight_elimination.c.

double gnn_weight_elimination_get_nu (  gnn_criterion *    crit )

This function gets the currently used regularization coefficient of the gnn_weight_elimination regularizer. Parameters: crit  A pointer to a gnn_weight_elimination criterion. Returns: Returns the regularization coefficient . Definition at line 478 of file gnn_weight_elimination.c.

double gnn_weight_elimination_get_wp (  gnn_criterion *    crit )

This function gets the currently used scale parameter by the gnn_weight_elimination regularizer. Parameters: crit  A pointer to a gnn_weight_elimination criterion. Returns: Returns the scale parameter . Definition at line 529 of file gnn_weight_elimination.c.

gnn_criterion* gnn_weight_elimination_new (  gnn_criterion *    crit, double    nu, double    wp, gnn_node *    node )

This function builds a new gnn_weight_elimination regularizer for all gnn_weight : Affine / Linear Transform nodes contained in the given node node. Parameters: crit  A pointer to a gnn_weight_elimination criterion. nu  The regularization coefficient . wp  The scale factor . node  A pointer to a gnn_node. Returns: Returns a pointer to a gnn_weight_elimination if succeeded or NULL. Definition at line 273 of file gnn_weight_elimination.c.

gnn_criterion* gnn_weight_elimination_new_with_pbundle (  gnn_criterion *    crit, double    nu, double    wp, gnn_pbundle *    pb )

This function builds a new gnn_weight_elimination regularizer for all parameters contained in the given parameter bundle pb. Parameters: crit  A pointer to a gnn_weight_elimination criterion. nu  The regularization coefficient . wp  The scale parameter . pb  A gnn_pbundle. Returns: Returns a pointer to a gnn_weight_elimination if succeeded or NULL. Definition at line 361 of file gnn_weight_elimination.c.

gnn_criterion* gnn_weight_elimination_new_with_type (  gnn_criterion *    crit, double    nu, double    wp, gnn_node *    node, const char *    type )

This function builds a new gnn_weight_elimination regularizer for all parameters that pertain to nodes of type type contained in the given node node. Parameters: crit  A pointer to a gnn_weight_elimination criterion. nu  The regularization coefficient . wp  The scale parameter . node  A pointer to a gnn_node. type  A string containing the type. Returns: Returns a pointer to a gnn_weight_elimination if succeeded or NULL. Definition at line 316 of file gnn_weight_elimination.c.

int gnn_weight_elimination_prun (  gnn_criterion *    crit, double    th )

This function pruns the penalized parameters that fall below the given threshold th in magnitude. These are frozen to zero. Parameters: crit  A pointer to a gnn_weight_elimination criterion. th  The threshold for parameter pruning. Returns: Returns 0 if succeeded. Definition at line 551 of file gnn_weight_elimination.c.

int gnn_weight_elimination_set_nu (  gnn_criterion *    crit, double    nu )

This function sets a new regularization coefficient for the gnn_weight_elimination regularizer. It should be positive. Parameters: crit  A pointer to a gnn_weight_elimination criterion. nu  The new regularization coefficient . Returns: Returns 0 if succeeded. Definition at line 450 of file gnn_weight_elimination.c.

int gnn_weight_elimination_set_wp (  gnn_criterion *    crit, double    wp )

This function sets a new scale paramter for the gnn_weight_elimination regularizer. It should be stricly positive. Parameters: crit  A pointer to a gnn_weight_elimination criterion. wp  The new scale parameter . Returns: Returns 0 if succeeded. Definition at line 500 of file gnn_weight_elimination.c.

---