Smart Advertising Robot with Image Recognition for Data Analytics


  • Nik Ahmad Faisal Mohd Kamarolzaman Universiti Teknologi Malaysia
  • Che Fai Yeong UTM
  • Eileen Lee Ming Su UTM
  • Sheng Fei Chik UTM
  • Feng Duan 3Department of Automation, College of Computer and Control Engineering, Nankai University, China
  • Jeffery Too Chuan Tan Institute of Industrial Science, The University of Tokyo
  • Ping Hua Tan DF Automation and Robotics Sdn Bhd, Johor, Malaysia
  • Patrick Jun Hua Chin DF Automation and Robotics Sdn Bhd, Johor, Malaysia



Advertising, Autonomous Navigation, Artificial Intelligence, Face Recognition


Conventional advertisement methods has the drawback of targeting inappropriate customer segments. A Smart Advertising Robot (SMADBOT) is proposed, aims to improve the effectiveness of advertising through targeted advertising. SMADBOT is a robot implemented with digital display which provides targeted advertisements through data analytics using data obtained from machine vision. Microsoft Cognitive Services is used to predict emotion, gender and age group of people without the need of a self-trained model. Microsoft Power BI is used to classified consumers into various groups to achieve effective targeted advertising. Robot Operating System (ROS) is used as a framework to integrate data from different sensors to perform autonomous navigation. Several metrics were used to evaluate the performance of SMADBOT including age, emotion prediction errors and stopping accuracy. Emotion prediction achieved a mean accuracy of 94% and had a navigation accuracy of 5.51cm in error. SMADBOT was deployed in real environment to further validate and test proposed system, where 184 face counts were collected after a 3-hour autonomous navigation. Data collected were successfully classified into various customer segments for effective targeted advertising.


C. Yoon, H. Lee, S. H. Jeon and H. Lee, "Mobile digital signage system based on service delivery platform location based targeted advertisement service," ICTC 2011, Seoul, 2011, pp. 582-586.

B. Lopes and R. L. Pereira, "ShopAssist — A unified location-aware system for shopping," 2016 Global Information Infrastructure and Networking Symposium (GIIS), Porto, 2016, pp. 1-6.

Harry Zhang, Claudia Guerrero, David Wheatley, and Y. S. Lee, “Privacy Issues and User Attitudes towards Targeted Advertising: A Focus Group Study," 2010 Proceedings of the Human Factors and Ergonomics Society Annual Meeting September 2010 54(19):1416.

L. W. Chen and Y. F. Ho, "A face-based signage interacting system for mobile users using smartphones," 2016 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), San Francisco, CA, 2016, pp. 1033-1034.

K. C. Yin, H. C. Wang, D. L. Yang and J. Wu, "A Study on the Effectiveness of Digital Signage Advertisement," 2012 International Symposium on Computer, Consumer and Control, Taichung, 2012, pp. 169-172.

J. M. Santos, D. Portugal and R. P. Rocha, "An evaluation of 2D SLAM techniques available in Robot Operating System," 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Linkoping, 2013, pp. 1-6..

W. Hess, D. Kohler, H. Rapp and D. Andor, "Real-time loop closure in 2D LIDAR SLAM," 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, 2016, pp. 1271-1278..

Herring, S. (2013). Telepresence robots for academics. Proceedings Of The American Society For Information Science And Technology, 50(1), 1-4

Robert Holmberg, Oussama Khatib(2000, November). Development and Control of a Holonomic Mobile Robot for Mobile Manipulation Tasks.




How to Cite

Mohd Kamarolzaman, N. A. F., Yeong, C. F., Lee Ming Su, E., Fei Chik, S., Duan, F., Chuan Tan, J. T., … Hua Chin, P. J. (2017). Smart Advertising Robot with Image Recognition for Data Analytics. ELEKTRIKA- Journal of Electrical Engineering, 16(3), 11–16.