, where 
denotes the 
-th input pattern, and 
is the order of the dataset,
for 
,The gnn_trainer : Trainers for Models. type defines a common interface for handling training algorithms. It provides the basic structure on which all training algorithms are based upon. Like all basic structures in libgnn, the gnn_trainer : Trainers for Models. structure cannot be used by itself. Instead, implementations of trainers fill in the necessary parts to become a fully-functional gnn_trainer : Trainers for Models..
What is a gnn_trainer : Trainers for Models.? A gnn_trainer : Trainers for Models. is an object that, given a Model (in form of a gnn_node), a criterion (gnn_criterion : Basic Criterion Function.) and a dataset (gnn_dataset : Datasets for Training.):
, where
denotes the
-th input pattern, and
is the order of the dataset,
for
,
Although one pattern is presented at a time, the needed gradient 
is estimated by perfoming a weighted sum over the particular gradients 
for each pattern
:
where 
is the weight of the pattern
. How many patterns are considered in this sum? Well, in the literature, when only one pattern is considered, then the procedure is called on-line training, and in a batch training, all patterns are taken. In libgnn, this sum is taken over so-called minibatches, which are simply groups of patterns of the same size. E.g. if there a 10 patterns, and the minibatches are of size 4, then the patterns are presented in the following groups:
(Please note that the last minibatch has been cut down to fit.) This operation is executed automatically by calling gnn_trainer_train on a gnn_trainer : Trainers for Models.. gnn_trainer_train returns the mean cost of all patterns presented to the model during the current epoch and adjusts the patterns of the model.
To reset a trainer and start over again (the parameters are kept), call gnn_trainer_reset.
How to extend the gnn_trainer : Trainers for Models. structure to implement your own trainer? In the same spirit as in all other basic structures, you can use the gnn_trainer : Trainers for Models. structure itself or extend it like the following hypothetical "my_trainer" structure:
typedef struct _my_trainer { gnn_trainer trainer; // the specific data for "my_trainer" ... } my_trainer;
Secondly, you have to provide at least four functions: a constructor, which allocates and initializes a new "my_trainer" structure; a "reset" function to reset the training procedure, which should be of type gnn_trainer : Trainers for Models.; a "train" function to train a minibatch and update the parameters, which should be of type gnn_trainer : Trainers for Models.; and a "destroy" function, which destroys all additional data used by the "my_trainer" structure, of type gnn_trainer : Trainers for Models. (alternatively, you can use the default implementations for the "reset" and "destroy" functions).
Please read the specific documentation for the function types that you should provide: gnn_trainer : Trainers for Models. and gnn_trainer : Trainers for Models..
Modules | |
| gnn_bfgs : Broyden-Fletcher-Goldfarb-Shanno Algorithm. | |
| gnn_conjugate_gradient : Conjugate Gradient Descent Algorithm. | |
| gnn_gradient_descent : Gradient Descent Algorithm. | |
| gnn_lmbfgs : Limited Memory BFGS Algorithm. | |
| gnn_momentum : Gradient Descent with Momentum Term. | |
| gnn_rprop : Resilient Error Backpropagation Trainer. | |
| Resilient Backpropagation. | |
Typedefs | |
| typedef _gnn_trainer | gnn_trainer |
| The datatype for trainer reset functions. | |
| typedef int(* | gnn_trainer_reset_type )(gnn_trainer *trainer) |
| The datatype for trainer reset functions. | |
| typedef int(* | gnn_trainer_train_type )(gnn_trainer *trainer) |
| The datatype for trainer train functions. | |
| typedef void(* | gnn_trainer_destroy_type )(gnn_trainer *trainer) |
| The datatype for trainer destroy functions. | |
Functions | |
| int | gnn_trainer_default_reset (gnn_trainer *trainer) |
| Default "reset" function for a trainer. | |
| void | gnn_trainer_default_destroy (gnn_trainer *trainer) |
| Default "destroy" function for a trainer. | |
| const char * | gnn_trainer_get_type (gnn_trainer *trainer) |
| Get the trainer's type. | |
| int | gnn_trainer_init (gnn_trainer *trainer, const char *type, gnn_node *node, gnn_criterion *crit, gnn_dataset *data, gnn_trainer_reset_type reset, gnn_trainer_train_type train, gnn_trainer_destroy_type destroy) |
| Initializes a gnn_trainer : Trainers for Models. structure. | |
| void | gnn_trainer_destroy (gnn_trainer *trainer) |
| Destroys the trainer. | |
| int | gnn_trainer_reset (gnn_trainer *trainer) |
| Resets the trainer. | |
| double | gnn_trainer_train (gnn_trainer *trainer) |
| Trains the node. | |
| gnn_dataset * | gnn_trainer_get_dataset (gnn_trainer *trainer) |
| Get the dataset. | |
| gnn_node * | gnn_trainer_get_node (gnn_trainer *trainer) |
| Get the model. | |
| gnn_criterion * | gnn_trainer_get_criterion (gnn_trainer *trainer) |
| Get the criterion. | |
| int | gnn_trainer_batch_set_size (gnn_trainer *trainer, size_t size) |
| Set the size for minibatches. | |
| size_t | gnn_trainer_batch_get_size (gnn_trainer *trainer) |
| Get the size of the minibatches. | |
| size_t | gnn_trainer_get_pattern_index (gnn_trainer *trainer) |
| Returns the index of the next minibatch. | |
| size_t | gnn_trainer_get_epoch (gnn_trainer *trainer) |
| Get the number of the current epoch. | |
| double | gnn_trainer_get_epoch_cost (gnn_trainer *trainer) |
| Get the mean cost. | |
| gnn_grad * | gnn_trainer_batch_get_grad (gnn_trainer *trainer) |
| Returns a pointer to the internal Evaluations. buffer. | |
| double | gnn_trainer_batch_get_e (gnn_trainer *trainer) |
| Returns the last evaluated batch's mean error. | |
| gsl_vector * | gnn_trainer_batch_get_dx (gnn_trainer *trainer) |
| Returns the last batch's evaluated gradient dx. | |
| gsl_vector * | gnn_trainer_batch_get_dw (gnn_trainer *trainer) |
| Returns the last batch's evaluated gradient dw. | |
| int | gnn_trainer_batch_process (gnn_trainer *trainer) |
| Processes a batch. | |
| int | gnn_trainer_batch_next (gnn_trainer *trainer) |
| Moves onto the next batch. | |
|
|
This is the datatype that contains the basic elements for the implementation of a trainer. Every new type of trainer should extend this structure. Definition at line 49 of file gnn_trainer.h. |
|
|
This function is called by the gnn_trainer_destroy function, and should free all extra data that was allocated for the specific trainer type. The gnn_trainer : Trainers for Models. data is handled automatically (and freed) by gnn_trainer_destroy, so only additional data should be freed by implementations of this function type. Definition at line 93 of file gnn_trainer.h. |
|
|
This is the datatype for the gnn_trainer : Trainers for Models.'s "reset" function. These functions are called by the gnn_trainer_reset function. They should reset the trainer's internal state, so that a new training procedure can begin. Definition at line 60 of file gnn_trainer.h. |
|
|
This function returns the sum of the patterns' costs. This is the datatype for the gnn_trainer : Trainers for Models.'s "train" function. This function is called by the gnn_trainer_train function. Basically, it should update the model's parameters in order to minimize the criterion, using a minibatch of patterns, starting at "currpat". In order to simplify the implementation, libgnn provides the gnn_trainer_batch_process function, which processes a minibatch: it returns mean cost of the so far evaluated patterns and the estimation
of the gradient, where the sums are taken for the patterns in the minibatch. You can then use the returned gradient to update the model's parameters. After that, don't forget to return the sum of the costs (which is returned by gnn_trainer_batch_process). Definition at line 81 of file gnn_trainer.h. |
|
|
This function returns a the last evaluated batch's mean gradient with respect to its parameters The returned vector can be freely accessed and its values modified, but it should not be freed.
Definition at line 671 of file gnn_trainer.c. |
|
|
This function returns a the last evaluated batch's mean gradient with respect to its inputs The returned vector can be freely accessed and its values modified, but it should not be freed.
Definition at line 652 of file gnn_trainer.c. |
|
|
This function returns a the last evaluated batch's mean error
Definition at line 633 of file gnn_trainer.c. |
|
|
This function returns a pointer to the internal error and gradients evaluation buffer.
Definition at line 618 of file gnn_trainer.c. |
|
|
This function returns a strictly positive number corresponding to the size of the minibatches that are processed upon a call of gnn_trainer : Trainers for Models..
Definition at line 534 of file gnn_trainer.c. |
|
|
This function moves to the next batch. That is, if sets the trainers internal state to prepare to process the next batch. If the end of the dataset is reached, then the next epoch is initiated, the mean cost info cleared, the dataset reset. This function is very handy when implementing your own trainers. It should be used in conjunction with gnn_trainer : Trainers for Models..
Definition at line 730 of file gnn_trainer.c. |
|
|
This function processes the current minibatch of patterns, computing the batch's mean cost and its gradients
Several calls of this function will process the same minibatch. Only a call of gnn_trainer : Trainers for Models. moves onto the next one. This function is very handy when implementing your own trainers. It provides an easy way to obtain the gradient and handles the necessary field updates on the gnn_trainer : Trainers for Models.. Please refer to gnn_trainer : Trainers for Models. for details.
Definition at line 704 of file gnn_trainer.c. |
|
||||||||||||
|
This function sets the size of the minibatches that should be processed upon a call of gnn_trainer : Trainers for Models.. This size should be stricly positive. Although it can be greather than the number of available patterns in the dataset, the effective minibatch will be smaller.
Definition at line 505 of file gnn_trainer.c. |
|
|
This function is the default "destroy" function for a trainer. It assumes that there isn't any additional data for the specific trainer type, so it actually just returns.
Definition at line 165 of file gnn_trainer.c. |
|
|
This function is the default "reset" function for a trainer. It does nothing.
Definition at line 147 of file gnn_trainer.c. |
|
|
This function destroys the gnn_trainer : Trainers for Models. by calling the installed destructor function for the trainer type and by freeing the gnn_trainer : Trainers for Models.'s structure. It doesn't destroy the node, nor the criterion, nor the dataset. They should be destroyed independently.
Definition at line 365 of file gnn_trainer.c. |
|
|
This function returns a pointer to the criterion that the trainer uses for training.
Definition at line 484 of file gnn_trainer.c. |
|
|
This function returns a pointer to the dataset that the trainer uses for training.
Definition at line 449 of file gnn_trainer.c. |
|
|
This function returns the number of the current epoch.
Definition at line 569 of file gnn_trainer.c. |
|
|
This function returns the mean cost of the patterns presented in the current epoch. That is, if the patterns 
where It is important to note that this value considers also the last evaluated minibatch's cost.
Definition at line 593 of file gnn_trainer.c. |
|
|
This function returns a pointer to the model that the trainer trains.
Definition at line 466 of file gnn_trainer.c. |
|
|
This function returns the index of the first pattern in the minibatch to be processed.
Definition at line 552 of file gnn_trainer.c. |
|
|
This function returns a string which contains the name of the trainer's type. The string should not be modified.
Definition at line 187 of file gnn_trainer.c. |
|
||||||||||||||||||||||||||||||||||||
|
This function initializes a gnn_trainer : Trainers for Models. structure by setting its internal fields adecuately. Every extension to the gnn_trainer : Trainers for Models. structure should have a constructor that calls this function to:
The associated sizes should match. That is, the input pattern's sizes contained in the dataset should corresponde to the node's input size. Likewise, the output pattern's sizes should match with the node's output size and the criterion's size. The following example should clarify its use. Suppose you want to create a new trainer type, called "my_trainer". You have already implemented the two of the needed functions, "my_trainer_reset", "my_trainer_train". Actually, your special training function does not need any additional data, and so the destructor isn't needed. Then, the following code could be a possible implementation of the constructor: gnn_trainer * my_trainer_new (gnn_node *model, gnn_criterion *crit, gnn_dataset *data) { int status; my_trainer *t; // this is just an extension of gnn_trainer gnn_trainer *tview; // points to the same structure, but uses another // view // alloc space t = (my_trainer *) malloc (sizeof (*t)); if (t == NULL) print_error_message_and_return_error_code (); // cast to a gnn_trainer tview = (gnn_trainer *) t; // initialize status = gnn_trainer_init (tview, "my_trainer", model, crit, data, my_trainer_reset, my_trainer_reset, NULL); if (status) print_error_message_and_return_error_code (); // finish initialization ... return tview; }
Definition at line 271 of file gnn_trainer.c. |
|
|
This function resets the trainer, i.e. the error is cleared, the epoch reset, the pattern iterator set to the first pattern, etc.
Definition at line 391 of file gnn_trainer.c. |
|
|
This function executes a full iteration of the training algorithm on the node. In this context, an iteration consists of presenting a batch of patterns to the node and adjust its parameters accordingly. Every call to this function presents the next patterns until the end is reached. The last batch will be of smaller if the number of total patterns is not a multiple of the size of the batches. If the end is reached, then the trainer goes to the next epoch and the dataset is automatically reset.
Definition at line 426 of file gnn_trainer.c. |
1.2.18
.
.
.
and 
have been evaluated in the current epoch, then the value 
.