Preliminary Results of the Optimized Network Interface for Long Distance Haptic Teleoperation ^†

Abstract

Bilateral haptic teleoperation (BHT) has been the center of interest for researchers for over half a century. It is a type of cutting-edge technology that enables the operator to transmit touch sensations over the internet to any part of the globe. The BHT suffers from issues such as stability and transparency due the presence of network latency, jitters, and device impedance. In this paper, we designed an optimized network solution for bilateral haptic teleoperation. In this regard, successful long-distance haptic teleoperation experiments were performed with a pair of haptic devices, i.e., a Phantom Desktop (TouchX) and a Novint Falcon device, to test the robustness and versatility of the framework.

1. Introduction

Over the past 50 years, we have witnessed an increased use of bilateral haptic teleoperation systems in numerous fields, e.g., underwater exploration, space investigation, medical surgeries, military demining, etc., by enabling human users to perform complex, remote, or even unsafe tasks at a certain distance [1,2]. The bilateral haptic teleoperation system consists of a master and slave device connected over the network via a communication channel. In bilateral teleoperation, both the master and the slave aim to perform motion synchronization. The environment force is indirectly reflected by deploying a firm coupling between the master and the slave (using a virtual spring, damper, etc.), thereby reflecting the slave dynamics to the operator by means of the master [3].

Haptic data generally comprise of two interactive parameters, i.e., the position (including the cartesian position, velocity, acceleration, etc.) and the force (including the torque, momentum, damping force, etc.). The positional data is used to exchange the orientation and state of the haptic device operation, and are also utilized to calculate the force and velocity via differentiation methods [4]. The instances of the haptic data packets are shown in Figure 1

The performance of the haptic teleoperation systems is directly related to the performance of the network. Haptic data transmitted over the network/internet is highly susceptible to the network latency, delay, delay jitter, and the bandwidth of the network [5]. In this paper, we designed a control buffer, and deployed it using the user datagram protocol (UDP), along with a high-level controller. To analyze the performance of the designed framework, significantly long-distance haptic teleoperation experiments were performed from HHRCM-Lab (NCRA-NEDUET), Karachi, Pakistan, including a 7000 km experiment with an average round-trip time (RTT) delay of 280 ms.

This paper is structured as follows. In Section 1, the abstract and introduction are discussed, Section 2 describes the teleoperation and network framework, Section 3 describes the experiment, and Section 4 contains the results and conclusions.

2. Teleoperation and Network Framework

Haptic teleoperation systems need a robust communication interface between human operators and robotic systems.

Such teleoperation systems enable the human operator to experience the feeling of a real environment, i.e., to remotely control and operate devices and objects with their own hands by having the real-time feedback of these manipulations and the environment [6]. Figure 1 and Figure 2 show the control design architecture and block diagram of bilateral haptic teleoperation systems, respectively.

The equations below show the master and slave dynamic model of the haptic teleoperation system [7]. Equations (1) and (2) describe the master and slave dynamics, respectively.

$${\mathrm{M}}_m{\stackrel{..}{x}}_m+B_m{\stackrel{.}{x}}_m=f_m+f_h$$

(1)

$${\mathrm{M}}_s{\stackrel{..}{x}}_s+B_s{\stackrel{.}{x}}_s=f_s-f_e$$

(2)

Haptic data need to be transmitted bilaterally over the network, maintaining the stability and transparency of the system. To achieve this, connection-oriented and connection-less internet protocols, i.e., the Transmission-Controlled Protocol (TCP) and the User Datagram Protocol (UDP), are used. In the designed framework, the UDP is deployed in order to ensure the dissolute transmission of haptic data.

2.1. User Datagram Protocol (UDP) and Control Buffer

The UDP, or the user datagram protocol, is a transport layer protocol according to the TCP/IP model. It is a connection-less protocol, in which the server and client communicate with each other using a dedicated IP and port number [8]. Haptic data are sent over the network in the form of data packets (datagrams), as demonstrated in Figure 3. These packets are sampled significantly and sequenced using the designed buffer, which ensures the swift and secure transmission of packets in the haptic teleoperation.

2.2. High-Level Controller

In bilateral haptic teleoperation, there is a trade-off between the transparency and stability of a system [1,3]. To maintain the stability of a system, a high-level passivity-based controller is deployed at the server end during haptic teleoperation. In order to improve the transparency and quality of the feedback, the controller is tuned at certain specific values of damping and stiffness, i.e., $c_1$, $K_p$, etc.

3. Experimental Setup

The experiment was carried out using the serial structured haptic device, a Phantom Desktop (TouchX) connected with Intel core i7 11th Gen PC at the master end, and the parallel-structured haptic device, a Novint Falcon device connected with Intel core i7 10th Gen Laptop. The network adopted for the experiment was a wired wide area network (WAN) using dedicated IP and a port number, and the devices were at a long distance (7000 km) at the server end and a wireless WAN (eduroam) was used at the client end. The experiment consisted of performing long-distance stable teleoperation for multi-degree of freedom tasks using haptic devices and ensuring significant stability, viable feedback, and the efficient depiction of force position parameters over the interactive GUI. Figure 4 shows the hardware setup of the experiment.

4. Results

Long-distance haptic teleoperation was performed using the force–position architecture. The deployed framework enhanced the robustness and transparency of the system. An average RTT delay of 280 ms was observed, while the maximum delay was 620 ms during an uninterrupted 30 min experiment. Results in terms of position coordinates before and after contact with the environment were recorded with stable and improved performance. Figure 5 shows position data graph (x, y, z) coordinates, before and after the contact point with the environment (client-side human operator).

5. Conclusions

Long-distance haptic teleoperation was performed with improved robustness and stability, including hardware and software experiments (emulated synchronous and asynchronous delays) based on the versatility of the framework with several haptic devices over different networks. Further work is being performed to enhance the adaptive control of haptic data packets.