previous,
up,
next
Background
In many existing neural network models,
e.g. multi-layer-perceptrons, radial basis function networks or
self-organizing feature maps the topology of the network has to be
chosen in advance (before training). This can be a difficult step
since neural networks are often used for problems, where the
statistical properties of the training data are not well known.
Choosing a too large network may result in overfitting the training
data and poor generalization to new data. A too small network may not
be able to capture the relations underlying the training data and will
show poor results on new data, too.
For this reason, methods to adapt the network topology (or in general
to change the number of free parameters) have been investigated
extensively. Examples include Optimal Brain Damage (Le Cun et al.,
1990), Cascade Correlation (Fahlman & Lebiere, 1990), the upstart
algorithm (Frean, 1990), weight decay (Krogh et al, 1992) and weight
elimination (Weigend, Rumelhart and Huberman,1991).
(Last updated: Feb. 7, 1997)