not logged in. (login)


Computer-Human Interaction and Assistive Technology


The ``Camera Mouse'' system has been developed to provide computer access for people with severe disabilities. The system tracks the computer user's movements with a video camera and translates them into the movements of the mouse pointer on the screen. Body features such as the tip of the user's nose or finger can be tracked. The visual tracking algorithm is based on cropping an online template of the tracked feature from the current image frame and testing where this template correlates in the subsequent frame. The location of the highest correlation is interpreted as the new location of the feature in the subsequent frame. Various body features are examined for tracking robustness and user convenience. A group of 20 people without disabilities tested the Camera Mouse and quickly learned how to use it to spell out messages or play games. Twelve people with severe cerebral palsy or traumatic brain injury have tried the system, nine of whom have shown success. They interacted with their environment by spelling out messages and exploring the internet.

Tracking methods are evaluated in a real-time feature tracking system used for human-computer interaction (HCI). The Camera Mouse, a HCI system for people with severe disabilities that interprets video input to manipulate the mouse pointer was improved and used as the test platform for this study.
Tracking methods tested are the Lucas-Kanade tracker and a tracker based on normalized correlation. Both methods are evaluated with and without multidimensional Kalman filters. Two-, four-, and six-dimensional filters are tested to model feature location, velocity, and acceleration. The various tracker and filter combinations are evaluated for accuracy, computational efficiency, and practicality. The normalized correlation coefficient tracker without Kalman filtering is found to be the tracker best suited for a variety of human-computer interaction tasks.

A human-computer interface (HCI) system called "The Camera Mouse" is evaluated. It tracks a user's movements with a video camera and translates them to movements of the mouse pointer on the screen. The main objectives for the experimentation were to quantitatively define the performance of the system for different users, features, and applications, to determine the optimal settings for different kinds of users, and to compare measurements over all users in order to suggest enhancements to a future system of this type. The experiments were conducted with 11 participants including a subject with severe physical disabilities. Each subject repeatedly performed a number of tasks. During each trial, a different feature was tracked and the elapsed time and mouse movement trajectories were measured. These measurements were used to quantify the system's performance.


Personnel:
  • Wajeeha Akram
  • Omar Al-Hinai
  • Margrit Betke
  • Sarah Dubauskas
  • Mikhail Gorman
  • John Magee
  • Kenda Stewart
  • Laura Tiberii
  • Benjamin Waber

  • For more information, see this project's webpage.

    Related Publications

    title year

    Margrit Betke, Oleg Gusyatin and Mikhail Urinson , "Symbol design: a user-centered method to design pen-based interfaces and extend the functionality of pointer input devices," Universal Access in the Information Society, Vol. 4, No. 3, pp 223-236, 2006.
    2006

    Oleg Gusyatin, Mikhail Urinson and Margrit Betke , "A Method to Extend Functionality of Pointer Input Devices," User-Centered Interaction Paradigms for Universal Access in the Information Society, pp 426-439, 2004.
    2004

    Kristen Grauman, Margrit Betke, Jonathan Lombardi, James Gips and Gary Bradski , "Communication via Eye Blinks and Eyebrow Raises: Video-Based Human-Computer Interfaces," Universal Access in the Information Society, Vol. 2, No. 4, pp 359-373, 2003.
    2003

    Margrit Betke, James Gips and Peter Fleming , "The Camera Mouse: Visual Tracking of Body Features to Provide Computer Access For People with Severe Disabilities," IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 10, No. 1, pp 1-10, 2002.
    2002