EgoReveal: Learning Active Vision and Object Manipulation under Partial Occlusions from Egocentric Demonstrations
Abstract
Learning from Demonstrations (LfD) enables robots to learn object manipulation directly from human demonstrations. In contrast to resource-intensive and labor-intensive teleoperation, egocentric demonstrations are easy to collect, enabling scalable data collection. Manipulated objects are often partially occluded in everyday environments, yet prior egocentric demonstration-based methods that require no robots or specialized teaching devices do not explicitly address occlusions. To fill this gap, we propose EgoReveal, a framework that learns viewpoint-repositioning together with manipulation actions from egocentric human demonstrations to reposition the viewpoint and obtain target visibility under partial occlusions. EgoReveal instantiates this as two imitation policies: one predicts the future camera-arm pose sequence from the current view and camera-arm pose, and the other predicts the future control commands from object keypoints and a history of control commands. EgoReveal switches between the two based on target visibility. In experiments where the manipulated object is partially occluded initially, EgoReveal repositions the camera to obtain visibility and completes the task more reliably than fixed-camera baselines.
EgoReveal includes two imitation policies. The first is the Active Vision policy that predicts a camera-arm pose trajectory capable of avoiding occlusions, given the current visual observation and the current state of the camera-arm. The second is the Manipulation policy that predicts control commands from 3D keypoints of the object to be grasped, extracted from the camera view established at the end of the Active Vision policy.
These policies are trained to imitate human demonstrations collected through smart glasses, enabling learning from natural egocentric human demonstrations acquired without teleoperation or specialized teaching devices.
In addition, to determine when to switch between the two policies, namely, when occlusion has been sufficiently resolved, and the system can transition to object manipulation, EgoReveal introduces an Occlusion Resolution Assessment (ORA) module. This module evaluates whether the target object is adequately visible and decides the appropriate timing for switching from the Active Vision policy to the Manipulation policy.
Evaluation
Video Presentation
BibTeX
@article{egoreveal2026,
title={EgoReveal: Learning Active Vision and Object Manipulation under Partial Occlusions from Egocentric Demonstrations},
author={Hiromu Kurihara and Kazutoshi Tanaka and Masashi Hatano and Tadashi Kozuno and Hideo Saito and Mariko Isogawa},
journal={IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2026}
}