Next: Gradient Computation Up: Training the RNN Previous: Training the RNN

RNN Objective Function

As discussed in earlier sections, the recurrent network is used to estimate the posterior probability of a phone given the input acoustic data. For this to be valid, it is necessary to use an appropriate objective function for estimating the network weights. An appropriate criterion for the softmax output of (5) is the cross-entropy objective function. For the case of Viterbi training, this objective function is equivalent to the log posterior probability of the aligned phone sequence and is given by

It has been shown in [9] that maximisation of (11) with respect to the weights is achieved when .