PlumX Metrics
Embed PlumX Metrics

Dynamic Feed-Forward LSTM

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN: 1611-3349, Vol: 14117 LNAI, Page: 191-202
2023
  • 0
    Citations
  • 0
    Usage
  • 0
    Captures
  • 0
    Mentions
  • 0
    Social Media
Metric Options:   Counts1 Year3 Year

Conference Paper Description

We address the insufficient hidden states capabilities and single-direction feeding flaws of existing LSTM caused by its horizontal recurrent steps. To this end, we propose the Dynamic Feed-Forward LSTM (D-LSTM). Specifically, our D-LSTM first expands the capabilities of hidden states by assigning an exclusive state vector to each word. Then, the Dynamic Additive Attention (DAA) method is utilized to adaptively compress local context words into a fixed size vector. Last, a vertical feed-forward process is proposed to search context relations by filtering informative features in the compressed context vector and updating hidden states. With the help of exclusive hidden states, each word can preserve its most correlated context features and hidden states do not interfere with each other. By setting an appropriate context window size for DAA and stacking multiple such layers, the context scope can be gradually expanded from a central word to both sides and achieve the whole sentence at the top layer. Furthermore, the D-LSTM module is compatible with parallel computing and amenable to training via back-propagation for its vertical prorogation. Experimental results on both classification and sequence tagging datasets insist that our models achieve competitive performance compared to existing LSTMs.

Provide Feedback

Have ideas for a new metric? Would you like to see something else here?Let us know