Augmented reality visualization of scene depth for aiding ROV pilots in underwater manipulation

Underwater manipulation is a key technology for marine industries and exploration that can be efficiently adopted in other application fields, such as underwater archaeology, biological manipulation, scientific expedition, as well as offshore construction in the Oil and Gas industry. It is performed remotely by expert pilots thanks to the visual feedbacks provided by one or more cameras but without any information about the distance between the end-effector and the target. To this end, the paper presents a novel system based on a sensorized robotic arm, stereoscopic 3D perception and augmented reality visualization to support ROV's pilots in underwater manipulation tasks. The system, thanks to the adoption of an optical-stereo camera, provides a visual feedback of the underwater scene on which a depth map of the underwater workspace is augmented on. In particular, combining the kinematics of the robotic arm and the standard photogrammetric model of the stereo camera, it is possible to generate a depth map that shows to the pilots the distances of the surface of the scene objects from the end-effector's pose. Experimental tests carried out in the context of the CoMAS (In-situ conservation planning of Underwater Archaeological Artefacts) project have demonstrated the effectiveness of the proposed system.