Rule Generation for Detecting Web Robots using Genetic Algorithm

10.29428/9789860544169.201801.0050

Eva Marpaung；Po-Ching Lin；Hao-Wei Chang；Shin-Ping Huang

Web robots ； Genetic Algorithm ； Rule Generation

NCS 2017 全國計算機會議

2017（2018 / 01 / 01）

264 - 269

英文

The growth of the Internet has led to an unavoidable growth in the volume of web robots traffic. Web robots, also known as Internet bots, are software applications that run automated tasks over the Internet. It is difficult to ignore the impact of web robots. Web robots can be the cause of many security problems on the Internet, as they send spam, steal information from infected machines, and perform distributed denial-of-service attacks. They also consume valuable bandwidth and web server resources. In this paper, we present a method of web robot detection using Genetic Algorithm (GA). Our method involves two main stages: session labeling and inferring GA-based rules. The detection relies on behavioral differences between web robots and human. Applying to the HTTP logs from the National Chung Cheng University campus, we demonstrate that our method obtains a good result with high detection rates, 0.8 for true-positive rate (TPR) and 0.03 for false-positive rate (FPR). These results show the effectiveness of our method.