Soham Patil

A transformer fusing RGB-D observations with proprioception estimates binary hand-object contact states, replacing hardware tactile sensors with a learned pseudo-tactile signal. The predicted contacts drive downstream RL for in-hand object reorientation, generalize zero-shot to novel objects, and are validated on a real robot.

Visuomotor dexterous policies are notoriously brittle to camera viewpoint shifts; regularizing a 2D monocular backbone with privileged 3D geometric supervision from simulation resolves this without depth sensors at test time. Zero-shot sim-to-real transfer on an xArm7 with a 16-DoF LEAP Hand achieves 76.7% grasping success across eight unseen objects and six uncalibrated viewpoints.

An end-to-end aerial grasping pipeline for cluttered environments: a language instruction selects the target, the platform actively repositions for better views, a grasp network proposes 6-DoF candidates per view, and collision-aware feasibility checking selects the one to execute. Validated on a real aerial manipulator in cluttered scenes.