Fig 3: Proposed hybrid network framework
The algorithm for predicting teleoperator behaviour using LSTM and DDPG
to control a telepresence robot during delayed communication is as
follows:
ALGORITHM 1: A hybrid approach of integrating LSTM and DDPG to
predict teleoperator behaviour
1: define and declare the prediction function of teleoperator
behaviour
2: // preprocess data
3: processed_data = preprocess_data(data)
4: // train LSTM network
5: lstm_model = train_lstm(processed_data)
6: // make predictions using LSTM network
7: teleoperator_actions = lstm_model.predict(processed_data)
8: // train DDPG algorithm
9: ddpg_model = train_ddpg(processed_data)
10: // use DDPG algorithm to choose actions for telepresence
robot
11: telepresenceRobot_actions= ddpg_model.choose_actions(teleoperator
12: // update LSTM and DDPG models with new data DDPG algorithm to
choose actions for telepresence robot
13: lstm_model.update(new_data)
14: ddpg_model.update(new_data)
15: // use reinforcement learning to reward DDPG algorithm for
positive outcomes
16: ddpg_model.reward(positive_outcomes)
17: // continuously update telepresence robot actions in real-time
based on predicted teleoperator actions and current state
18: while True:
19: teleoperator _actions = lstm_model.predict(current_data)
20: telepresenceRobot_actions =ddpg_model.choose_actions(teleoperator
21: execute_actions(telepresenceRobot_actions)
22: end while
The above algorithm outlines the basic steps for predicting teleoperator
behaviour using LSTM and DDPG to control an autonomous car. It includes
preprocessing the data, training the LSTM and DDPG models, making
predictions using the LSTM model, choosing actions for the telepresence
robot using the DDPG model, continuously updating the models with new
data, and using reinforcement learning to reward the DDPG model for
positive outcomes. The telepresence robot’s actions are continuously
updated in a loop based on the predicted teleoperator actions and the
current state of the robot.
Experiment Setup and Result Discussion: In this section, the
experimental setup and its results are both explained with the proposed
approach of controlling the telepresence robot during the delayed
communication with the teleoperator.
The setup consists of a custom-manufactured telepresence robot and a
remote-controlled setup of a teleoperator, as shown in Figure 4. The
telepresence robot is powered by two DC-geared motors of 200 watts. The
telepresence robot was equipped in the actual experiments with two Lidar
sensors at various heights with a maximum measurement range of ten
meters.