An efficient concurrent implementation of a neural network algorithm
Document Type
Article
Department or Administrative Unit
Computer Science
Publication Date
10-25-2005
Abstract
The focus of this study is how we can efficiently implement the neural network backpropagation algorithm on a network of computers (NOC) for concurrent execution. We assume a distributed system with heterogeneous computers and that the neural network is replicated on each computer. We propose an architecture model with efficient pattern allocation that takes into account the speed of processors and overlaps the communication with computation. The training pattern set is distributed among the heterogeneous processors with the mapping being fixed during the learning process. We provide a heuristic pattern allocation algorithm minimizing the execution time of backpropagation learning. The computations are overlapped with communications. Under the condition that each processor has to perform a task directly proportional to its speed, this allocation algorithm has polynomial‐time complexity. We have implemented our model on a dedicated network of heterogeneous computers using Sejnowski's NetTalk benchmark for testing.
Recommended Citation
Andonie, R., Chronopoulos, A. T., Grosu, D., & Galmeanu, H. (2006). An efficient concurrent implementation of a neural network algorithm. Concurrency and Computation: Practice and Experience, 18(12), 1559–1573. https://doi.org/10.1002/cpe.987
Journal
Concurrency and Computation: Practice and Experience
Rights
Copyright © 2005 John Wiley & Sons, Ltd.
Comments
This article was originally published in Concurrency and Computation: Practice and Experience. The full-text article from the publisher can be found here.
Due to copyright restrictions, this article is not available for free download from ScholarWorks @ CWU.