Can I read Learning Various Length Dependence by Dual Recurrent Neural Networks on EtoBox?

Learning Various Length Dependence by Dual Recurrent Neural Networks by Zhang, Chenpeng; Li, Shuai; Ye, Mao; Zhu, Ce; Li, Xue is a scholarly article available to read on EtoBox.

What is Learning Various Length Dependence by Dual Recurrent Neural Networks about?

Recurrent neural networks (RNNs) are widely used as a memory model for sequence-related problems. Many variants of RNN have been proposed to solve the gradient problems of training RNNs and process long sequences. Although some classical models have been proposed, capturing long-term dependence while responding to short-term changes remains a challenge. To this problem, we propose a new model named Dual Recurrent Neural Networks (DuRNN). The DuRNN consists of two parts to learn the short-term dependence and progressively learn the long-term dependence. The first part is a recurrent neural network with constrained full recurrent connections to deal with short-term dependence in sequence and generate short-term memory. Another part is a recurrent neural network with independent recurrent connections which helps to learn long-term dependence and generate long-term memory. A selection mechanism is added between two parts to help the needed long-term information transfer to the independent neurons. Multiple modules can be stacked to form a multi-layer model for better performance. Our contributions are: 1) a new recurrent model developed based on the divide-and-conquer strategy to learn

Author

Zhang, Chenpeng; Li, Shuai; Ye, Mao; Zhu, Ce; Li, Xue

Published

2020

Language

EN