基於深度學習與雙目視覺水果採摘機械手臂控制之應用

Application of Robot Arm Control System for Fruit Harvesting Based on Deep Learning and Binocular Vision

黃登淵(DENG-YUNG HUANG)；許景貿(JING-MAU SHIU)

人工智慧 ； 雙目視覺 ； 深度學習 ； 機械手臂 ； Artificial intelligence ； binocular vision ； deep learning ； robotic arm

科學與工程技術期刊

19卷1期（2023 / 03 / 01）

47 - 53

繁體中文;英文

電腦視覺結合深度學習控制機械手臂在工業界是一個重要研究領域。目前，大部分的機械手臂控制系統需要使用昂貴的距離感測器來估測物體位置。然而，這些感測器的價格高昂，限制了機械手臂系統的應用。因此，本文提出一個結合深度神經網路模型和雙目視覺的控制系統，用於水果採摘應用。這個系統包括了使用Yolov5目標偵測模型判斷物體座標位置，並使用OAK-D雙目相機進行雙目深度估測來估測物體距離。實驗結果證實，本文所提出的方法可以有效地控制機械手臂，並且可以減少使用昂貴感測器的成本，進而提高了機械手臂系統的泛用性。

The use of computer vision and deep learning techniques is essential for effective robotic arm control systems. Current robotic arm control systems rely on expensive distance sensors to determine the location of objects. The high cost of these sensors is a barrier to the widespread application of robotic arm control systems. Therefore, this paper proposes a control system that combines a deep neural network model and binocular vision for fruit-picking robots. In this system, theYOLOv5 object detection model is used to determine object coordinates, and an OAK-D stereo camera is used for depth estimation. The experimental results indicated that the proposed system was successful in effectively controlling a robotic arm by using low-cost sensors, making it a suitable option for robotic arm control systems.

1. Hespeler, S. C.,Nemati, H.,Dehghan-Niri, E.(2021).Non-destructive thermal imaging for object detection via advanced deep learning for robotic inspection and harvesting of chili peppers.Artificial Intelligence in Agriculture,5,102-117.
2. Horn, B.,Klaus, B.,Horn, P.(1986).Robot vision.Cambridge, MA:MIT press.
3. Krizhevsky, A.,Sutskever, I.,Hinton, G. E.(2017).Imagenet classification with deep convolutional neural networks.Communicaitons of the ACM,60(6),84-90.
4. Kuhnert, K. D.,Stommel, M.(2006).Fusion of stereo-camera and pmd-camera data for real-time suited precise 3d environment reconstruction.2006 IEEE/RSJ International Conference on Intelligent Robots and Systems,Beijing:
5. Li, D.,Wang, X.,Kong, D.(2018).Deeprebirth: Accelerating deep neural network execution on mobile devices.Proceedings of the AAAI Conference on Artificial Intelligence,32(1),2322-2330.
6. Ota, K.,Dao, M. S.,Mezaris, V.,Natale, F. G. D.(2017).Deep learning for mobile multimedia: A survey.ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM),13(3s),1-22.
7. Rai, N.,Rai, B.,Rai, P.(2014).Computer vision approach for controlling educational robotic arm based on object properties.2nd International Conference on Emerging Technology Trends in Electronics, Communication and Networking,Surat, India:
8. Redmon, J.,Divvala, S.,Girshick, R.,Farhadi, A.(2016).You only look once: Unified, real-time object detection.Proceedings of the IEEE conference on computer vision and pattern recognition,Las Vegas, NV:
9. Ren, S.,He, K.,Girshick, R.,Sun, J.(2015).Faster r-cnn: Towards real-time object detection with region proposal networks.Advances in neural information processing systems,28
10. Ribler, R. L.,Vetter, J. S.,Simitci, H.,Reed, D. A.(1998).Autopilot: Adaptive control of distributed applications.Proceedings of the Seventh International Symposium on High Performance Distributed Computing (Cat. No. 98TB100244),Chicago, IL:
11. Sermanet, P.,Eigen, D.,Zhang, X.,Mathieu, M.,Fergus, R.,LeCun, Y.(2013).,未出版
12. Simonyan, K.,Zisserman, A.(2014).,未出版