基於多模態融合之跟隨機器人

A Human-Following Robot Based on a Multi-Modal Fusion Strategy

10.6342/NTU201703926

鄭良健

多模態融合辨識 ； 藍牙 ； 模糊積分 ； 跟隨機器人 ； Multimodal Recognition System ； Bluetooth ； Fuzzy Integral ； Following Robot

國立臺灣大學機械工程學系學位論文

2017年

碩士

黃漢邦

英文

隨著科技的進步，機器人領域的蓬勃發展，我們可以察覺到機器人已經慢慢的融入在這個社會上。其中最引人關注的莫過於服務型機器人，主要是因為與工業下的機器人相比，服務型機器人在與人互動這一方面考慮的因素要比工業下的機器人來得多。此外，人們對於服務型機器人更具有好奇心。為了達到機器人與人類之間的自然互動與和諧的共存，人機互動便是此發展重點。本論文致力於結合多模態融合辨識、手機藍牙以及雷射點所提供的距離資訊，來發展一個跟隨機器人，並達到當跟丟目標者時，服務型機器人能靠距離資訊去尋找回目標者，並從新進行辨識及跟隨。

With the advance of science and technology, robotics is becoming increasingly important. We can observe the integration of robots into our society. One of the most interesting aspects in the field of robotics is service robots. Compared with other robots used in the industry, service robots are required to be much more interactive with humans. People are therefore more intrigued by service robots. To achieve natural interaction between service robots and humans, human-computer interaction is one of the most important aspects of robotics. This thesis is focused on the development of a following robot based on a multimodal recognition system, distance information provided by Bluetooth from Android smartphones, and a laser rangefinder. When the following robot loses track of the target person, it has the ability to find the target person by using the distance information. After the target person is found by the following robot, the following robot can recognize the target person by using the multimodal recognition system.

1. [1] A. Ball, D. Rye, F. Ramos, and M. Velonaki, "Unsupervised Clustering of People from 'Skeleton' Data," Proc. of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction, NY, USA, pp. 225-226, 2012.
   連結：
2. [3] F. Bertacchini, E. Bilotta, and P. Pantano, "Shopping with a Robotic Companion," Computers in Human Behavior, Vol. 76, pp. 1-14, 2017.
   連結：
3. [4] Breazeal and Cynthia, "Social Interactions in Hri: The Robot View," IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), Vol. 34, No. 2, pp. 181-186, 2004.
   連結：
4. [6] A. Carballo, A. Ohya, and S. i. Yuta, "Multiple People Detection from a Mobile Robot Using Double Layered Laser Range Finders," Proc. of the IEEE ICRA Workshop on People Detection and Tracking, Kobe, Japan, 2009.
   連結：
5. [7] B. Chen, S. Chen, and J. Feng, "A Study of Multisensor Information Fusion in Welding Process by Using Fuzzy Integral Method," International Journal of Advanced Manufacturing Technology, Vol. 74, No. 1-4, pp. 413-422, 2014.
   連結：
6. [8] J. Y. C. Chen, "Uav-Guided Navigation for Ground Robot Tele-Operation in a Military Reconnaissance Environment," Ergonomics, Vol. 53, No. 8, pp. 940-950, 2010.
   連結：
7. [9] M.-N. Chen, "Development of Shopping Assistant Robots," Master Thesis, Department of Mechanical Engineering, National Taiwan University, 2013.
   連結：
8. [10] S.-B. Cho and J. H. Kim, "Combining Multiple Neural Networks by Fuzzy Integral for Robust Classification," IEEE Transactions on Systems, Man, and Cybernetics, Vol. 25, No. 2, pp. 380-384, 1995.
   連結：
9. [13] S.-Y. Chung and H.-P. Huang, "Robot Motion Planning in Dynamic Uncertain Environments," Advanced Robotics, Vol. 25, No. 6-7, pp. 849-870, 2011.
   連結：
10. [14] W. Chung, H. Kim, Y. Yoo, C.-B. Moon, and J. Park, "The Detection and Following of Human Legs through Inductive Approaches for a Mobile Robot with a Single Laser Range Finder," IEEE transactions on industrial electronics, Vol. 59, No. 8, pp. 3156-3166, 2012.
    連結：
11. [15] A. Corradini, M. Mehta, N. O. Bernsen, J. Martin, and S. Abrilian, "Multimodal Input Fusion in Human-Computer Interaction," NATO Science Series Sub Series III Computer and Systems Sciences, Vol. 198, pp. 1-15, 2005.
    連結：
12. [17] B.-G. Erin-Ee-Lin Lau and S.-C. L. Lee, "Enhanced Rssi-Based High Accuracy Real-Time User Location Tracking System for Indoor and Outdoor Environments," International Journal on Smart Sensing and Intelligent Systems, Vol. 1, No. 2, pp. 534-548, 2008.
    連結：
13. [21] G.Johansson, "Visual Motion Perception," Scientific American, Vol. 232, No. 6, pp. 76-88, 1975.
    連結：
14. [22] A. Ghasemi and E. S. Sousa, "Asymptotic Performance of Collaborative Spectrum Sensing under Correlated Log-Normal Shadowing," IEEE Communications letters, Vol. 11, No. 1, pp. 34-36, 2007.
    連結：
15. [23] H.-M. Gross, H. Boehme, C. Schroeter, S. Mueller, A. Koenig, E. Einhorn, C. Martin, M. Merten, and A. Bley, "Toomas: Interactive Shopping Guide Robots in Everyday Use - Final Implementation and Experiences from Long-Term Field Trials," Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, St. Louis, USA, pp. 2005-2012, 2009.
    連結：
16. [24] E. Guizzo and E. Ackerman, "The Rise of the Robot Worker," IEEE Spectrum, Vol. 49, No. 10, pp. 34-41, 2012.
    連結：
17. [25] M. Heerink, M. Díaz, J. Albo-Canals, C. Angulo, A. Barco, J. Casacuberta, and C. Garriga, "A Field Study with Primary School Children on Perception of Social Presence and Interactive Behavior with a Pet Robot," Proc. of the 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, France, pp. 1045-1050, 2012.
    連結：
18. [26] C.-W. Hsu, C.-C. Chang, and C.-J. Lin, "A Practical Guide to Support Vector Classification," 2003, pp. 1-16.
    連結：
19. [28] G. Johansson, "Visual Perception of Biological Motion and a Model for Its Analysis," Perception and Psychophysics, Vol. 14, No. 2, pp. 201-211, 1973.
    連結：
20. [32] J.-G. Lee, B.-K. Kim, S.-B. Jang, S.-H. Yeon, and Y. W. Ko, "Accuracy Enhancement of Rssi-Based Distance Estimation by Applying Gaussian Filter," Indian Journal of Science and Technology, Vol. 9, No. 20, pp. 1-5, 2016.
    連結：
21. [33] L. Lewis, T. Metzler, and L. Cook, "Evaluating Human-Robot Interaction Using a Robot Exercise Instructor at a Senior Living Community," Proc. of International Conference on Intelligent Robotics and Applications, Tokyo, Japan, pp. 15-25, 2016.
    連結：
22. [34] LIBSVM -- A Library for Support Vector Machines. Retrieved Jan 15, 2017, from https://www.csie.ntu.edu.tw/~cjlin/libsvm/
    連結：
23. [36] R. C. Luo, N.-W. Chang, S.-C. Lin, and S.-C. Wu, "Human Tracking and Following Using Sensor Fusion Approach for Mobile Assistive Companion Robot," Proc. of Industrial Electronics, 2009. IECON'09. 35th Annual Conference of IEEE, Porto, Portugal, pp. 2235-2240, 2009.
    連結：
24. [37] S.-q. Ma, F.-c. Chen, Q. Wang, and Z.-q. Zhao, "Sugeno Type Fuzzy Complex-Value Integral and Its Application in Classification," Procedia Engineering, Vol. 29, pp. 4140-4151, 2012.
    連結：
25. [38] S. Naskar, A. K. Seth, and A. Nath, "Application of Radio Frequency Controlled Intelligent Military Robot in Defense," Proc. of International Conference on Communication Systems and Network Technologies, Jammu, India, pp. 396-401, 2011.
    連結：
26. [43] R. Rojas, "The Backpropagation Algorithm," in Neural Networks, 1st Ed, Berlin: Springer, 1996 pp. 149-182.
    連結：
27. [44] N. K. M. S and R. V. Babu, "Human Gait Recognition Using Depth Camera," Proc. of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing, NY, USA, 2012.
    連結：
28. [47] K. K. Sharma and S. D. Joshi, "Signal Separation Using Linear Canonical and Fractional Fourier Transforms," Optics communications, Vol. 265, No. 2, pp. 454-460, 2006.
    連結：
29. [48] T. B. Sheridan, "Human–Robot Interaction Status and Challenges," The Journal of the Human Factors and Ergonomics Society, Vol. 58, No. 4, pp. 525-532, 2016.
    連結：
30. [49] R. Sibson, "Slink: An Optimally Efficient Algorithm for the Single-Link Cluster Method," The computer journal, Vol. 16, No. 1, pp. 30-34, 1973.
    連結：
31. [52] M. Sugeno, "Theory of Fuzzy Integrals and Its Applications," Doctorial Thesis, Tokyo Institute of Technology, 1974.
    連結：
32. [53] S. Sugiyama, K. Baba, and T. Yoshikawa, "Guide Robot with Personal Identification Method Using Dress Color Information Via Kinect," Proc. of the 2012 IEEE International Conference on Robotics and Biomimetics, Guangzhou, China, pp. 2195-2200, 2012.
    連結：
33. [54] S. Sugiyama and T. Wada, "Improved Personal Identification Method for Guide Robots Using Dress Color Information Via Kinect," Proc. of International Conference of Soft Computing and Pattern Recognition, Hanoi, Vietnam, pp. 111-116, 2013.
    連結：
34. [55] Summary: SLINK: An optimally efficient algorithm for the single-link cluster method. Retrieved Jan 10, 2017, from http://www.cs.ucsb.edu/~veronika/MAE/summary_SLINK_Sibson72.pdf
    連結：
35. [56] D. M. J. Tax and R. P. W. Duin, "Support Vector Domain Description," Pattern Recognition Letters, Vol. 20, No. 11-13, pp. 1191-1199, 1999.
    連結：
36. [58] K. Werner, J. Oberzaucher, and F. Werner, "Evaluation of Human Robot Interaction Factors of a Socially Assistive Robot Together with Older People," Proc. of Sixth International Conference on Complex, Intelligent, and Software Intensive Systems, Palermo, Italy, 2012.
    連結：
37. [59] J. Xavier, M. Pacheco, D. Castro, and A. Ruano, "Fast Line, Arc/Circle and Leg Detection from Laser Scan Data in a Player Driver," Proc. of the IEEE International Conference on Robotics and Automation, Barcelona, Spain, pp. 3930-3935, 2005.
    連結：
38. [60] Q.-h. Xing, L. Xia, F.-x. Liu, and C.-a. Shang, "Emerging Research in Artificial Intelligence and Computational Intelligence," Proc. of International Conference, AICI, Chengdu, China, pp. 463-470, 2012.
    連結：
39. [2] U. Bekcibasi and M. Tenruh, "Increasing Rssi Localization Accuracy with Distance Reference Anchor in Wireless Sensor Networks," Acta Polytechnica Hungarica, Vol. 11, No. 8, pp. 103-120, 2014.
40. [5] C. Breazeal, A. Takanishi, and T. Kobayashi, Social Robots That Interact with People, 1st Ed, Berlin Heidelberg: Springer, 2008.
41. [11] H.-C. Chu, M.-F. Chien, T.-H. Lin, and Z.-J. Zhang, "Design and Implementation of an Auto-Following Robot-Car System for the Elderly," Proc. of International Conference on System Science and Engineering, Puli, Taiwan, pp. 1-4, 2016.
42. [12] S.-Y. Chung and H.-P. Huang, "Predictive Navigation by Understanding Human Motion Patterns," International Journal of Advanced Robotic Systems, Vol. 8, No. 1, pp. 52-64, 2011.
43. [16] G. Eason, B. Noble, and I. Sneddon, "On Certain Integrals of Lipschitz-Hankel Type Involving Products of Bessel Functions," Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, Vol. 247, No. 935, pp. 529-551, 1955.
44. [18] Face Detection using Haar Cascades. Retrieved March 15, 2017, from http://docs.opencv.org/trunk/d7/d8b/tutorial_py_face_detection.html
45. [19] Face Recognition with OpenCV. Retrieved March 15, 2017, from http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html#id27
46. [20] Y.-w. Fu, X. Li, and Z. Zhuang, "A Decision Level Fusion Algorithm for Target Recognition with Fuzzy Densities Determined Adaptively," Acta Electronica Sinica, Vol. 32, No. 9, pp. 1433-1435, 2004.
47. [27] S. Jan, W. Verdonck, and J. D. Schutter, "Dynamic Model Identification for Industrail Robots," IEEE Control Systems, Vol. 27, No. 5, pp. 58-71, 2007.
48. [29] T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro, "Person Identification and Interaction of Social Robots by Using Wireless Tags," Proc. of International Conference on Intelligent Robots and Systems, Las Vegas, Nevada, pp. 1657-1664, 2003.
49. [30] Y. Kuroki, M. Fujita, T. Ishida, K. i. Nagasaka, and J. i. Yamaguchi, "A Small Biped Entertainment Robot Exploring Attractive Applications," Proc. of the IEEE lnternatiooal Conference on Robotics & Automation, Taipei, Taiwan, pp. 471-476, 2003.
50. [31] I. Lai, "The Following Robot with Searching and Obstacle-Avoiding," Master Thesis Department of Electrical Engineering, National Central University, 2009.
51. [35] Linear discriminant analysis. Retrieved March 15, 2017, from https://en.wikipedia.org/wiki/Linear_discriminant_analysis
52. [39] M. Ota, H. Hisahara, H. Takemura, and H. Mizoguchi, "Recovery Function of Target Disappearance for Human Following Robot," Proc. of Innovative Engineering Systems (ICIES), 2012 First International Conference on, Alexandria, Egypt, pp. 125-128, 2012.
53. [40] R. Pinillos, S. Marcos, R. Feliz, E. Zalama, and J. Gómez-García-Bermejo, "Long-Term Assessment of a Service Robot in a Hotel Environment," Robotics and Autonomous Systems, Vol. 79, pp. 40-57, 2016.
54. [41] J. Preis, M. Werner, C. Linnhoff-Popien, and C. Linnhoff-Popien, "Gait Recognition with Kinect," Proc. of 1st International Workshop on Kinect in Pervasive Computing, New Castle, UK, pp. 1-4, 2012.
55. [42] D. Punetha, N. Kumar, and V. Mehta, "Development and Applications of Line Following Robot Based Health Care Management System," International Journal of Advanced Research in Computer Engineering & Technology, Vol. 2, No. 8, pp. 2446-2450, 2013.
56. [45] J. Sales, J. V. Martí, R. Marín, E. Cervera, and P. J. Sanz, "Comparob: The Shopping Cart Assistance Robot," International Journal of Distributed Sensor Networks, Vol. 12, No. 2, pp. 1-15, 2016.
57. [46] Self-propelled luggage US 20140107868 A1. Retrieved May, 10, 2017, from https://www.google.com/patents/US20140107868
58. [50] A. Sinha and K. Chakravarty, "Pose Based Person Identification Using Kinect," Proc. of IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK, pp. 498-503, 2013.
59. [51] A. Sinha, K. Chakravarty, and B. Bhowmick, "Person Identification Using Skeleton Information from Kinect," Proc. of the Sixth International Conference on Advances in Computer-Human Interactions, Nice, France, pp. 101-108, 2013.
60. [57] M. T. Vo and A. Waibel, "Multimodal Human-Computer Interaction," Proceedings of ISSD, Vol. 108, No. 1-2, pp. 116-134, 1993.
61. [61] Y. Xu, F. Shi, F. Yang, and W. Wang, "A Submarine Operational Capability Evaluation Approach Based on Fuzzy Integral," Computer Simulation, Vol. 26, No. 9, pp. 19-23, 2009.