An Implementation of Distributed Framework of Artificial Neural Network for Big Data Analysis

處理巨量資料分析之分散式類神經網路框架設計－以金融時間序列資料為例

10.6245/JLIS.2016.422/656

張景堯(Jiing-Yao Chang)；劉文卿(Wen-Ching Liou)；何善豪(Shan Hao Ho)

Artificial Neural Network ； Big Data Analysis ； Data Mining ； Distributed Computing ； Multilayer Perceptron ； 巨量資料分析 ； 資料採礦 ； 類神經網路 ； 多層感知器 ； 分散式運算

圖書館學與資訊科學

42卷2期（2016 / 10 / 01）

45 - 64

英文

In this research, we introduce a distributed framework of artificial neural network (ANN) to deal with the big data real‐time analysis and return proper outcomes in very short delay. The result of our experiment shows that training the distributed ANN model could be converged in 17 seconds on 24‐core clustering platform and learns that multi‐model with stratification strategy would obtain most true positive predictions with nearly 70% precision at voting threshold value equal to 0.7. In our system, ANNs are used in the data mining process for identifying patterns in financial time series. We implement a framework for training ANNs on a distributed computing platform. We adopt Apache Spark to build the base computing cluster because it is capable of high performance in‐memory computing. We investigate a number of distributed back propagation algorithms and techniques, especially ones for time series prediction, and incorporate them into our framework with some modifications. With various options for the details, we provide the user with flexibility in neural network modeling.

本研究設計一個分散式類神經網路框架以處理巨量資料之即時分析並能在極短的時間內得到不錯的結果。我們的實驗結果顯示在24 核心叢集平台上訓練分散式類神經網路模型可於17 秒收斂，進行預測時在0.7 投票閥值（voting threshold）設定下採用分層多重模型（multi-model with stratification）可獲得最多的真陽性結果且準確率達70%左右。在我們所建構的系統裡，類神經網路是用在資料採礦階段來發掘金融時間序列資料之模式。我們將訓練類神經網路的框架建置在分散式運算平台上，該平台我們採用具高效能記憶體內運算（in-memory computing）的Apache Spark 來建造底層基礎的運算叢集環境。我們評估了一些特別適用於預測金融時間序列資料的分散式後向傳導演算法，加以調整並整合進我們所設計的框架。同時，我們也提供了許多細部的選項，讓使用者在進行類神經網路建模時能有很高的客製化彈性。

1. Feng, A. (2013). Spark and Hadoop at Yahoo: Brought to you by YARN. Retrieved from http://ampcamp.berkeley.edu/wp-content/uploads/2013/07/andy-feng-ampcamp-3-presentation-Spark_on_YARN.pdf
2. Andonie, R.,Chronopoulos, A.,Grosu, D.,Galmeanu, H.(1998).Distributed backpropagation neural networks on a PVM heterogeneous system.Parallel and Distributed Computing and Systems Conference (PDCS'98)
3. Chen, M.,Mao, S.,Zhang, Y.,Leung, V. C.(2014).Big data: Related technologies, challenges and future prospects.Springer.
4. Dahl, G.,McAvinney, A.,Newhall, T.(2008).Parallelizing neural network training for cluster systems.Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Networks
5. Dahl, G.,McAvinney, A.,Newhall, T.(2008).Parallelizing neural network training for cluster systems.Paper presented at the Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Networks
6. Ganeshamoorthy, K.,Ranasinghe, D.(2008).On the performance of parallel neural network implementations on distributed memory architectures.Cluster Computing and the Grid, 2008. CCGRID'08. 8th IEEE International Symposium
7. Gu, R.,Shen, F.,Huang, Y.(2013).A parallel computing platform for training large scale neural networks.Big Data, 2013 IEEE International Conference
8. Ho, S. H.(2014).National Chengchi University.
9. Jones, R. D.,Lee, Y.,Barnes, C.,Flake, G.,Lee, K.,Lewis, P.,Qian, S.(1990).Function approximation and time series prediction with neural networks.Neural Networks, 1990., 1990 IJCNN International Joint Conference
10. Kaastra, I.,Boyd, M.(1996).Designing a neural network for forecasting financial and economic time series.Neurocomputing,10(3),215-236.
11. Kimoto, T.,Asakawa, K.,Yoda, M.,Takeoka, M.(1990).Stock market prediction system with modular neural networks.Neural Networks, 1990., 1990 IJCNN International Joint Conference on
12. Liu, Z.,Li, H.,Miao, G.(2010).MapReduce-based backpropagation neural network over large scale mobile data.Natural Computation (ICNC), 2010 Sixth International Conference on
13. Padgavankar, M.,Gupta, S.(2014).Big Data Storage and Challenges.International Journal of Computer Science & Information Technologies,5(2)
14. Pethick, M.,Liddle, M.,Werstein, P.,Huang, Z.(2003).Parallelization of a backpropagation neural network on a cluster computer.International conference on parallel and distributed computing and systems (PDCS 2003).
15. Sudhakar, V.,Murthy, C. S. R.(1998).Efficient mapping of backpropagation algorithm onto a network of workstations.Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on,28(6),841-848.
16. Suresh, S.,Omkar, S.,Mani, V.(2005).Parallel implementation of back-propagation algorithm in networks of workstations.Parallel and Distributed Systems, IEEE Transactions on,16(1),24-34.
17. White, H.(1988).Economic prediction using neural networks: the case of IBM daily stock returns.Neural Networks, 1988., IEEE International Conference
18. Xin, R. S.,Rosen, J.,Zaharia, M.,Franklin, M. J.,Shenker, S.,Stoica, I.(2013).Shark: SQL and rich analytics at scale.Proceedings of the 2013 ACM SIGMOD International Conference on Management of data
19. Yoon, H.,Nang, J. H.,Maeng, S.(1990).A distributed backpropagation algorithm of neural networks on distributed-memory multiprocessors.Proceedings of the 1990 Frontiers of Massively Parallel Computation
20. Zaharia, M.,Chowdhury, M.,Das, T.,Dave, A.,Ma, J.,McCauley, M.,Stoica, I.(2012).Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing.Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation