The following is a list of publications produced by HSI related faculty and students. The link for each paper leads you to the citation, keyword, and abstract. For copies of the full papers, please use standard methods for obtaining academic publications, such as online library searches.
Multimodal feedback: An Assessment of Performance and Mental Workload
Authors: H. S. Vitense, J. Jacko, V. K. Emery
Abstract: Multimodal interfaces offer great potential to humanize interactions with computers by employing a multitude of perceptual channels. This paper reports on a novel multimodal interface using auditory, haptic and visual feedback in a direct manipulation task to establish new recommendations for multimodal feedabck, in particular uni-, bi-, and trimodal feedback. A close examination of combinations of uni-, bi-, and trimodal feedback is necessary to determine which enhances performance without increasing workload. Thirty-two participants were asked to complete a task consisting of a series of 'drag-and-drops' while the type of feedback was manipulated. Each participant was exposed to three unimodal feedback conditions, three bimodal feedback conditions and one trimodal feedback condition that used auditory, visual and haptic feedback alone, and in combination. Performance under the different conditions was assessed with measures of trial completion time, target highlight time and a self-reported workload assessment captured by the NASA Task Load Index (NASA-TLX). The findings suggest that certain types of bimodal feedback can enhance performance while lowering self-perceived mental demand.
Citation: Vitense, H. S., Jacko, J. A., & Emery, V. K. (2003) Multimodal feedback: an assessment of performance and mental workload. Ergonomics, 46(1-3), 68-87.