A FUNCTION ESTIMATION APPROACH TO SEQUENTIAL LEARNING WITH NEURAL NETWORKS
Visakan Kadirkamanathan and Mahesan Niranjan
September 1992
In this paper, we investigate the problem of optimal sequential learning, viewed as a problem of estimating an underlying function sequentially rather than estimating a set of parameters of the neural network.
Firstly, we arrive at a sub-optimal solution to the sequential estimate which can be mapped by a growing Gaussian radial basis function (GaRBF) network. This network adds hidden units for each observation. The function space approach in which the estimates are represented as vectors in a function space, is used in developing a growth criteria to limit its growth. A simplification of the criterion leads to two joint criteria on the distance of the present pattern and the existing unit centres in the input space and on the approximation error of the network for the given observation to be satisfied together. This network is similar to the resource allocating network (RAN) and hence RAN can be interpreted from a function space approach to sequential learning.
Secondly, we present an enhancement to the RAN. The RAN either allocates a new unit based on the novelty of an observation or adapt the network parameters by the LMS algorithm. The function space interpretation of the RAN lends itself to an enhancement of the RAN in which the extended Kalman filter (EKF) algorithm is used in place of the LMS algorithm. The performance of the RAN and the enhanced network are compared in the experimental tasks of function approximation and time-series prediction demonstrating the superior performance of the enhanced network with fewer number of hidden units. The approach adopted here has led us towards the minimal network required for a sequential learning problem.
If you have difficulty viewing files that end '.gz',
which are gzip compressed, then you may be able to find
tools to uncompress them at the gzip
web site.
If you have difficulty viewing files that are in PostScript, (ending
'.ps' or '.ps.gz'), then you may be able to
find tools to view them at
the gsview
web site.
We have attempted to provide automatically generated PDF copies of documents for which only PostScript versions have previously been available. These are clearly marked in the database - due to the nature of the automatic conversion process, they are likely to be badly aliased when viewed at default resolution on screen by acroread.