The construction industry has been plagued with long-standing issues, including an aging workforce, as well as safety and health problems. There is great but unconsolidated potential for robotic construction to improve work productivity, safety, and workers’ occupational health.
Construction robots can
help human reduce the labor
avoid hazard zone
Improve the tasks expandability
Our solution divides into three parts.
First, a teleoperation interface has been developed to collect and visualize the data from the on-site camera and operator input.
Second, a keyframe extraction method was proposed to filter the redundant noisy data in the human demonstrations.
Third, latent space exploration is introduced for the reinforcement learning model to explore.
First, extract and publish the VIVE controller transformation and transition data regarding its bases to the Robot Operating System (ROS) with Steam VR beta version and HTC VIVE SDK.
Second, the obtained controller's relative position and transformation variation is fed to the inverse kinematics algorithm to determine the goal position of each robotic arm joint.
Third, after receiving the joint data from the corresponding ROS node, the associated robotic arm trajectory is shown in ROS Rviz.
The last module incorporates a UDP protocol to establish communication between ROS nodes and the actual robotic arm.
Hierarchical reinforcement learning structure
A hierarchical reinforcement learning structure to train model-free policies to accomplish the trajectory tasks is proposed. There are three components in it: keyframe policy πk, primitive policy πp, and keyframe classifier Cψ.
Latent Space Exploration Framework
To utilize information from latent space, we proposed a modified beta-variational autoencoder (VAE). R represents the robot states, a represents the robot action.
In this way, the output of beta-VAE binds with the robot action which can extract information that is more related to the tasks. Then the previous state and goal were replaced with the latent space state and goal.
To keep the system sustainably running, we also designed a goal state generation model which learns from the demonstrations.
Excavation task system
At the end effector, a bucket was attached to simulate the excavator shovel. Sand was put in a box that is 6 inches thick to simulate soft soil. The robot can be controlled using a VR controller (HTC VIVE).
For excavation task, our method has higher success rates than the state-of-the-art methods.
Latent space validation: robot end-effector position estimation
The top figure is the loss in regular Cartesian coordinate and the bottom figure is in the log axis coordinate. The bottom figure is to further distinguish the losses. The training loss of each method stays similar while the validate loss goes up and down dramatically.
To study the influence of the latent dimension, an experiment with different latent dimensions was performed. The result shows that the experiment with our proposed binding model outperforms the experiment using the pure auto-encoder method.
Our impacts lie in three aspects.
First, most of the time the construction industry is reluctant to change, our solution addresses significant challenges to collect data from expert demonstrations in construction sites.
Second, as demonstrated by the keyframe extraction results, the proposed method can eliminate over 80% of the redundant frames in the expert demonstration.
Third, to enhance the extensibility to a wider range of tasks and reduce the computational loads of RL, an integrated approach that combines vision-based trajectory generation with latent space exploration was proposed.