Embodied Intelligence for Human-Robot Collaboration
August 22 (Friday) 9:00-11:30 AM
Location: Shenyang Hall, 3rd Floor

Introduction:
       Researcher at the Institute of Software, Chinese Academy of Sciences, and former Chief Engineer of the Institute of Software, Chinese Academy of Sciences. Director of the Human-Computer Interaction Professional Committee of the Chinese Computer Society, Director of the Computer Graphics and Human-Computer Interaction Professional Committee of the Chinese Automation Society, Honorary Chairman of the ACM SIGCHI China Chapter, and an expert in the Information Field Advisory Expert Group of the National Key Basic Research and Development Program (973). He has engaged in research in software engineering, computer graphics, and human-computer interaction, and is one of the pioneers in the field of computer user interfaces in my country. He has published more than 200 academic papers in high-level international journals and conferences at home and abroad, and has written an academic monograph entitled "Pen-Based User Interfaces". He has presided over and participated in many National Natural Science Foundation projects, National High Technology Research and Development Program projects (863 Program), and National Key Basic Research and Development Program projects (973 Program), with outstanding results and significant contributions to the country. In 1992, he was awarded a special government allowance by the State Council and has received numerous National Science and Technology Progress Awards, Beijing Science and Technology Progress Awards, and Chinese Academy of Sciences Science and Technology Progress Awards.

Report Title: The Relationship between Human-Computer Interaction and Artificial Intelligence

Report Abstract: 

       As titled

Introduction:
       The first recipient of the National Outstanding Young Scientist Fund in the field of Industrial Internet, and Chief Scientist of the National Key R&D Program for Industrial Software. Second-level professor and Lantian Distinguished Professor at Beihang University, professor at the School of Automation and the School of Software, and deputy director of the National Key Laboratory of Intelligent Manufacturing of Complex Products. His research areas include Industrial Internet and Industrial Software, Industrial AI and Industrial Big Models. He proposed the first theoretical and technical system for industrial big models and established the first national standard system for industrial big models. He presided over more than 30 national and provincial and ministerial projects, including national major science and technology projects, national key R&D programs, and major research programs of the Natural Science Foundation of China, including one project worth hundreds of millions of yuan and three projects worth tens of millions of yuan. He has published more than 100 papers in internationally renowned journals such as IEEE Transactions, with over 10,000 citations. He presided over or participated in the formulation of 15 international/national standards. He has obtained more than 70 patents and software copyrights, and his core technologies serve hundreds of thousands of enterprises. He has won three first prizes at the provincial and ministerial levels. He serves on more than 10 domestic and international committees, including those of IEEE, CCF, CAAI, and CAA. He is Vice Chairman of the Intelligent IoT Committee of the China Simulation Society and Vice Chairman of the Cloud Control and Decision-making Committee of the China Command and Control Society. He is also an Executive Director of the China Simulation Society and an editorial board member of international journals such as IEEE TNNLS and TMECH. He serves as Vice Chairman of the Talent Group of the China Industrial Internet Industry Alliance and was the first university in China to offer an "Industrial Internet" course. He has chaired dozens of IEEE and domestic and international academic conferences and has been invited to deliver over 100 plenary presentations.

Report Title: Industrial Big Models + Embodied Intelligence: Driving the Future Industrial World

Report Abstract: 

        This report will summarize the hot technologies for the integration and innovation of the Industrial Internet and AI 2.0, and explore the theoretical system of industrial embodied intelligence and the industrial metaverse for the AI 3.0 era. Based on this technology, we propose the definition and connotation of industrial large-scale models, as well as their system architecture, construction methods, and core key technologies. Focusing on the full lifecycle of industrial manufacturing products, we discuss typical application scenarios for industrial large-scale models. We will also discuss future challenges facing industrial large-scale models and look forward to future technological and industrial development directions.

Introduction:
        Professor at Beijing University of Posts and Telecommunications, a "Top Talent" professor, formerly taught in the Department of Computer Science at Tsinghua University. His main research areas include embodied intelligence, dexterous manipulation, and robotic large-scale models. He was selected as a Beijing Zhiyuan Scholar and serves concurrently as a Council Member of the China Artificial Intelligence Society, Secretary-General of the Cognitive Systems and Information Processing Committee, Distinguished Member of the China Artificial Intelligence Society, Standing Committee Member of the Intelligent Robotics Committee of the China Computer Federation, and Senior Member of the IEEE. He has published over a hundred papers in leading journals such as Nature Communications and IEEE TRO, as well as conference papers at conferences such as ICRA. He has also published one book in both Chinese and English, and has received eight Best Paper Awards from international conferences and journals. His "tactile dexterous hand" was selected for the National 13th Five-Year Plan Science and Technology Innovation Achievement Exhibition. His work on decision-making consulting for humanoid robots and embodied intelligence has been adopted by the State Council and has been commended by central leaders. He has also won First Prize in the Natural Science Award of the Chinese Society of Automation and a Special Prize in the World Robot Competition.

Report Title: Interactive Grasping Operations with a Human-Robot Co-integrated Dexterous Prosthetic Hand

Report Summary:

        Human-Robot Co-integrated Dexterous Prosthetic Hands have become a research hotspot in recent years, with enhancing the interactive experience for patients with limb amputations being a key issue. This study, based on a developed tactile dexterous prosthetic hand, proposes a multi-stage grasping method and an active grip force adjustment strategy, achieving for the first time the deformable manipulation tasks of multiple grasps and toothpaste squeezing with one hand. Furthermore, a soft myoelectric prosthetic hand system for physical mouse operation is introduced, enabling natural interactive mouse operation for patients with limb amputations for the first time. Finally, a summary and outlook on research on interactive manipulation of complex tools are presented.

Introduction:
        Assistant Professor at the School of Computing and Information Systems, Singapore Management University. His research focuses on human-agent interaction, including human-robot collaboration, and enhancing individual user productivity and learning through AI agents. His research has been published in numerous leading international conferences, including CHI, UIST, HRI, and CSCW, and he has received two CHI Best Paper nominations and one HRI Best Paper Award. Served on the Technical Program Committee for CHI 24 and 25, and participated in the organizing committee for DIS 26.

Report Title: Supporting More Effective Human-Embodied Agent Interaction through the Visual Modality

Report Summary:

        With the development of large language models, language has become the primary interface for communication between humans and embodied intelligent agents. In this report, I will discuss the limitations of language as a communication medium, namely its often inability to accurately capture specific task contexts. Through two projects, I will demonstrate how the visual modality can provide contextual information that language lacks, and how the complementary use of language and visual modalities can promote more effective human-robot interaction. First, I will introduce a robot programming system called ImageInThat (HRI '25), which allows users to program robots by directly manipulating images of the task environment. Experimental results show that users using ImageInThat completed indoor tidying tasks more efficiently and accurately than using verbal instructions alone. I will also present an experiment (CHI '25) that demonstrated that spatial highlighting in a virtual environment editing task helps users more effectively identify and correct errors made by the agent in understanding verbal instructions. These studies demonstrate that mixed-modal interaction not only enhances humans' ability to fine-tune control over AI agents but also enables agents to communicate with humans in a more contextual and effective manner, thereby fostering more productive human-machine collaborations.

Introduction:
        Assistant Professor and doctoral supervisor in the Robotics and Autonomous Systems field at the Hong Kong University of Science and Technology, Guangzhou. He currently focuses on research related to AI for Robotics, aiming to integrate AI and robotics technologies to promote the development of next-generation robotics. The Robotic Intelligence and Learning Laboratory, which he leads, has conducted research in various areas of embodied intelligence, including humanoid robot design and control, intelligent robot manipulation, multimodal perception, human-robot interaction, and biomedical robotics. Dr. Qiang Nie graduated from the Department of Mechanical and Automation Engineering at the Chinese University of Hong Kong in 2020. He subsequently held a dual-appointed postdoctoral position at the Tianshi Robotics Institute and the Hong Kong Logistics Robotics Research Center. During his postdoctoral period, he participated in multiple RGC and Hong Kong government ITF research projects and co-founded Asuka Tech Limited in Hong Kong. He later joined Tencent Youtu Lab as a senior researcher, leading cutting-edge research in computer vision and artificial intelligence. Dr. Qiang Nie has published nearly 30 papers in top journals and conferences, including IJCV, TIP, ECCV, CVPR, ICCV, NeurIPS, AAAI, IROS, and ICRA, and has served as a reviewer for these conferences and journals. He has extensive research experience in both academia and industry. He has over 20 patents granted and pending in China and abroad.

Report Title: Robot Behavioral Intelligence: Building a Bridge for Human-Robot Synchrony

Report Summary:

        Human-robot collaboration is a time-honored topic and a core focus of embodied intelligence research. However, current research primarily focuses on how robots can autonomously complete tasks described by humans in unstructured scenarios using multimodal perception capabilities such as language and vision. These technologies often rely on collecting robot data or human videos, focusing more on imitating motion trajectories and lacking in-depth exploration of human-robot behavior itself. This report will explore the concept of robot behavioral intelligence, exploring how robots understand their own behavior when interacting with objects and how they understand human behavior when interacting with humans. We will examine the perspectives of action and behavior, aiming to enable robots to truly understand both their own behavior and human behavior. By achieving resonance with humans based on behavioral intelligence, robots will be able to achieve truly autonomous behavior and intelligent generalization, thereby enabling natural human-machine interaction and collaboration.

Introduction:
        Professor, Doctoral Supervisor, and Dean Assistant at the School of Instrument Science and Engineering, Southeast University. He is Deputy Secretary-General of the Force and Tactile Perception and Interaction Committee of the Chinese Instrument Society and a member of the Robotics Committee of the Jiangsu Artificial Intelligence Society. His research focuses on surgical robots, virtual reality, and human-machine interaction. He has presided over and completed numerous national and provincial projects, including those funded by the National Natural Science Foundation of China and national key R&D projects. He has published over 60 academic papers as first or corresponding author in journals and conferences such as IEEE TVCG, IEEE TRO, IEEE ICRA, ACM CHI, and ACM UIST. He has been granted over 20 national invention patents and has received First Prize in the Ministry of Education Science and Technology Progress Award, First Prize in the Jiangsu Provincial Science and Technology Award, First Prize in the Jiangsu Institute of Engineers Science and Technology Award, and the Best Paper Award at the Asian Graphics Conference. As the lead instructor, he mentored students who won the National Special Prize in the main track of the Challenge Cup Undergraduate Extracurricular Academic Science and Technology Works Competition and Second Prize in the National Undergraduate Biomedical Engineering Innovation Design Competition.

Report Title: Force Feedback and Human-Robot Collaboration in Surgical Robots

Report Introduction:

        Force feedback is a key technology for restoring a rich tactile feel in human-robot interaction. This report will introduce force feedback technologies developed by the presenters in recent years to address diverse human-robot interaction needs, as well as human-robot collaboration and shared intelligent control of surgical robots using force sensing in offline environments. These technologies include force rendering software technology for expressing data in general scenarios, force reproduction hardware technology based on deformable intelligent structures, and human-robot collaboration methods for robot-assisted traction manipulation in head and neck surgery. The report will also explore the relationship between force feedback and embodied intelligent control and interaction with robots.