運行於雲端運算平台KDCM及MPI應用程式的效能評估

10.29428/9789860544169.201801.0151

邱日清；趙至勛；郭紹群

雲端運算 ； 平行程式 ； Communication-Intensive ； Task mapping ； MPI ； 核心驅動程式 ； Cloud Computing ； parallel program ； Communication-Intensive ； Task mapping ； MPI ； Kernel Driver

NCS 2017 全國計算機會議

2017（2018 / 01 / 01）

802 - 807

繁體中文

因應運算雲端化的使用需求，如何提升平行程式在雲端平台上的執行效率是一個熱門的話題，運行於雲端平台的應用程式分成兩種類型：計算密集型和通訊密集型。在運行計算密集型的程式中，節點間的效能差異會影響工作運作的並行度，而在運行通訊密集型的程式中，工作之間通訊速度會影響資料交換的效率，為了減少通訊延遲，將具有高資料量交換的工作們分配到高網路效能的處理單元，此技術稱為通訊感知實現工作映射(communication-aware task mapping)。本論文以探討通訊感知實現工作映射的雲端運算為出發點，以減少通訊延遲為目標，由於網際網路的速度充滿了不確定性，而實際分析底層網路拓樸與硬體效能是相當耗費資源的，於是我們提出一個針對建構雲端連線類型實現工作映射的方法，透過使用者以分離的方式調度雲端運算資源以集中具有高資料量交換的程序於相同的節點，最後根據運算節點異質性定義負載指標與提出節點過載的解決方案。為了有效分析雲端工作之間的複雜通訊與統一管理及分配運算資源，我們參考Chiu 和Guo提出的MPI Kernel Cluster裝載於Kernel Distributed Computing Management，借助KDCM分析運算工作並分派工作給節點的能力，和我們改良MPIKC分散式運算叢集的Communication System以建構足以應付複雜資料傳遞拓樸的通訊管道。MPIKC與KDCM緊密的配合，提供使用者以管線化考量而設計的應用程式的適切性，大幅優化通訊密集類型程式的執行效能。最後，我們以兩個通訊密集型程式實驗，分別是訓練類神經網路與進階加密標準的加密程式，比較程序在各種分配狀況下的運行時間，結果顯示本論文的方法最多可省下12倍的運行時間，證實本論文所提出的設計可有效減少平行程式的運行時間。

In order to meet the operational requirements of cloud computing, how to improve the performance of parallel applications on a cloud platform is a hot topic. There are two types of applications running on the cloud platform: computation-intensive and communication-intensive. In running a computation-intensive program, the difference in performance of hardware between nodes can affect the parallelism of the work; and in the communication-intensive program, the communication speed within tasks can affect the efficiency of data exchange. To optimize communication, tasks that exchange lots of data should be mapped to processing units that have a high network performance. This technique is called communication communication-aware task mapping. In this paper, we explore the communication-aware task mapping as the starting point to reduce the communication latency as the goal. Because the speed of the Internet is full of uncertainty, and it is impractical to actually analyze the underlying network topology and hardware performance, so we propose a method to implement task mapping based on building a cloud connection type. The user can dispatch the cloud computing resources as the separated way to collect processes with high data communication rates at the same node. Finally, we also define the load status of different node according to the heterogeneity of node. In order to analyze the complex communication within tasks effectively and allocate computing resources more easily, we adopt MPI Kernel Cluster on Kernel Distributed Computing Management, proposed by Chiu and Guo. By its ability to analyze and dispatch tasks, and we improved MPIKC's Communication System to build a communication channel that is sufficient to handle complex data transfer topology. As MPIKC and KDCM fit tightly, they provide the feasibility of the program to be designed in a pipeline and significantly optimize the performance of communication-intensive types of programs. At the end, we experimented with two communication-intensive programs, respectively, training Neural Network and Advanced Encryption Standard encryption program. We compare the running time of the process under various distribution conditions and the results show that the method of our purpose can save up to 12 times running time. We confirm that our purpose can effectively reduce the running time of parallel program.