Chris Yu-liang Liu

Computer Vision: Three-dimensional Hand Tracking and Surface-geometry Measurement for a Robot-vision System Chris Yu-liang Liu

This book describes the design, implementation, calibration and testing of two vision systems of a robot-vision system for a service robot to recognize objects and perform object-related tasks from task guidance and demonstration provided by a general user. First, real-time tracking of a human hand is achieved using images acquired from three calibrated synchronized cameras. Hand pose is determined from positions of physical markers and input to the robot system in real-time. Second, a multi-line laser camera range sensor is mounted on a robot end-effector to provide 3D geometry information about objects in the robot environment. The laser-camera sensor includes two cameras to provide stereo vision. For hand tracking, a novel score-based hand tracking scheme is presented employing dynamic multi-threshold marker detection, stereo camera-pair utilization scheme, marker matching and labeling using epipolar geometry and hand pose axis analysis, to enable real-time hand tracking under occlusion and non-uniform lighting environments. For surface-geometry measurement, two different approaches are analyzed for 2D to 3D coordinate mapping, using Bezier surface fitting and neural networks.