Department or Administrative Unit
Computer Science
Document Type
Article
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Author Copyright
© 2020 by the authors.
Publication Date
1-16-2020
Journal
Entropy
Abstract
Current neural networks architectures are many times harder to train because of the increasing size and complexity of the used datasets. Our objective is to design more efficient training algorithms utilizing causal relationships inferred from neural networks. The transfer entropy (TE) was initially introduced as an information transfer measure used to quantify the statistical coherence between events (time series). Later, it was related to causality, even if they are not the same. There are only few papers reporting applications of causality or TE in neural networks. Our contribution is an information-theoretical method for analyzing information transfer between the nodes of feedforward neural networks. The information transfer is measured by the TE of feedback neural connections. Intuitively, TE measures the relevance of a connection in the network and the feedback amplifies this connection. We introduce a backpropagation type training algorithm that uses TE feedback connections to improve its performance.
Recommended Citation
Moldovan, Adrian; Caţaron, Angel; and Andonie, Rǎzvan, "Learning in Feedforward Neural Networks Accelerated by Transfer Entropy" (2020). All Faculty Scholarship for the College of the Sciences. 97.
https://digitalcommons.cwu.edu/cotsfac/97
Comments
This article was originally published in Entropy. The full-text article from the publisher can be found here.