LSTM-CRF-和-CTC算法详解-(纯英版)

1、Speech recognition and NER problems with recurrent neural networksOctober 9, 2018RNNMemoryMemoryRNN and CNNLSTM Comparing the fully connected layers, the parameters in each LSTM neuron will be more than at lease 4 times more. So there is GRU, and you can consider using GRU when LSTM is facing a risk

2、 of overfitting In other articles, sigmoid activation is sometimes used instead of tanh activation. It overcomes the gradient vanish problem, but still cant solve the problem of gradient explosionFurther Like CNN, simple RNN and LSTM can be deep. In speech recognition, the device can choose to give

3、the final result of the recognition after obtaining the entire speech. Conventional RNNS are only able to make use of previous context, and there is no reason not to exploit future context as well.BRNNsStill it can be further, just try to combine BRNNs with LSTM, and that becomes bidirectional LSTM.

4、Deep bidirectional RNNsStructure learning HMMsX: Y:start:Johnsawthetree.endThe task: given X, find YFrom HMMs to CRFsLSTMs-CRFsP(x, y) for CRFsW WCRF-gradient ascentConnectionist Temporal Classification (CTC) The CTC network predicts only the sequence of phonemes (typically as a series of spikes, se

5、parated by blanks). The choice of labelling can be read directly from the CTC outputs (follow the spikes).And the predictions of the framewise network must be post-processed before use.Prefix search decodingEach node either ends or extends the prefix at its parent node. The number above an extending

6、 node is the total probability beginning with that prefix. At every iteration the extensions of the most probable remaining prefix are explored.This strategy includes two aspects, one is that the same path does not repeat the calculation, and the other is that the state is no longer searched.The CTC

7、 Forward-Backward Algorithmresult1Speech recognition with deep recurrent neural networks. Author: Alex Graves, Abdel-Rahman Mohamed and Geoffrey Hinton2 Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks. Author: Alex Graves, Santiago Fernandez3 Neura