Adapting to the Body
This figure shows three snapshots of data from a sequence of
activity monitored by the system and visualized using a hand-tuned
kinematic model. The host reaches for a cup and drinks from it. The
top row consists of images from the head-mounted camera. The bottom
row shows depictions of the corresponding data provided by three of
the four the Intersense devices.
The following is an excerpt from my thesis, which will soon be
published in its entirety:
We would like to interpret the images from the camera and the
orientations from the orientation sensors in a consistent way across
different wearers and across sessions of use by the same wearer. A
three dimensional kinematic model that represents the major axes of
the limbs and the relative position and orientation of the camera
serves as a well-grounded representation for the body of the wearer
and its relationship to the sensors. As with standard motion capture
systems, it can also serve as a useful representation for visualizing
the output of the sensors. We have developed methods that use the
streams of orientations and images to autonomously estimate a
kinematic model that describes the wearer's body, the camera's
relative orientation and position, and the relationship between the
kinematic model and images captured from the camera. In addition, the
system autonomously estimates the projected direction of gravity in
the image for each pixel by using image motion to estimate the
orientation of the head-mounted camera with respect to the
head-mounted orientation sensor.
Ideally, we would like the wearer to don the equipment without
worrying about the details of sensor placement, or
calibration. Similarly, we would like to give the designer of the
equipment flexibility in sensor placement, so that he can put more
emphasis on factors such as comfort and ease of use. The more the
system is able to autonomously adapt to changes in sensor position,
sensor alignment, and the body of the wearer, the more easily the
system will be used without error. The methods we describe impose few
constraints on the wearer and designer since they only require the use
of a camera with known intrinsic parameters, sensors that are in rigid
alignment with the body parts, and the visibility of a portion of the
hand's workspace from the camera. Moreover, in practice the constraint
for rigid alignment with the body parts can be weakened.