NUMA系統的負載平衡機制與減少遠端記憶體存取的任務挑選策略的設計與實作

10.29428/9789860544169.201801.0140

林振瑋；姜美玲；蘇暐倫

NUMA ； 多核心處理器 ； 負載平衡 ； 遠端記憶體存取 ； 資源競爭 ； NUMA ； Multicore Processor ； Load Balancing ； Remote Memory Access ； Resource Contention

NCS 2017 全國計算機會議

2017（2018 / 01 / 01）

742 - 747

繁體中文

Non-Uniform Memory Access (NUMA)架構的多核心系統解決了Symmetric Multi-Processing (SMP)架構的擴展性問題，但是，在NUMA系統中，Task的記憶體Page可能會被配置在不同的NUMA節點上，並且當Task在進行跨節點的遠端記憶體存取時，所花費的時間是比存取Task所在節點的記憶體時來得久，因此，遠端記憶體的存取和競爭Interconnect Link會造成NUMA系統的效能下降。雖然Linux核心已經實作了Automatic Page Migration機制，透過Page Table Invalidation和Page Fault Handling的處理，會將Task存取到的遠端記憶體Page搬移至當前執行的節點上，以避免去存取遠端記憶體，但是這些額外的處理卻會增加Task存取記憶體時的Overhead。當多處理器系統中各處理器的負載不平衡時，Linux核心會執行負載平衡機制將Task搬移至不同的處理器佇列來等候執行。而在NUMA系統中，Task有可能會被搬移至不同的節點，Task存取在原先節點上的記憶體Page時，即發生遠端記憶體存取。為了降低Task在搬移過後遠端記憶體存取的發生，本篇論文修改了Linux核心的負載平衡機制，並在跨節點的Task搬移時，加入了Task挑選策略，而我們所提的Task挑選策略主要是利用Task的記憶體使用量來進行決策，有優先挑選最少共享記憶體Page以及優先挑選最小記憶體配置量等二種，會挑選合適的Task來進行跨節點的搬移。實驗的結果顯示，我們所提出的Task挑選策略相較於Linux核心內預設的First-Fit挑選策略，因為能夠降低遠端記憶體存取以及資源競爭，因此有效地提高了NUMA系統的效能。

Multi-core systems with the Non-Uniform Memory Access (NUMA) architecture solve the scalability problem of the Symmetric Multi-Processing (SMP) architecture. However, in a NUMA system, a task and its memories may be allocated on different NUMA nodes. Processor cores accessing memories on remote NUMA node take longer time than on local node. Remote memory access and contention for inter-node interconnect degrade system performance. Although in the Linux kernel, the automatic page migration mechanism is implemented, it migrates only referenced pages to the current node where the task is running on. This mechanism avoids remote memory access, but it is completed by page table invalidation and page fault handling, which incurs additional overhead for memory access. When load unbalance occurs in the multiprocessor systems, the original Linux kernel scheduler will migrate the first movable task from the busiest CPU core to the idlest one. In the NUMA system, the task may be migrated across nodes, which incurs remote memory access when the migrated task accesses its previously allocated memory on the original node. In order to reduce remote memory access incurred from inter-node task migration, we modify Linux kernel's load balancing mechanism and propose two task selection policies that are incorporated in the load balancing mechanism. These policies will choose suitable task for inter-node migration according to task's memory usage information. The experimental results show that our proposed task selection policies can reduce more remote memory access and resource contention than the default Linux kernel. As a result, the performance of NUMA system is effectively increased.