Search CALO Publications:

Author and Title
Edward C. Kaiser. Dynamic New Vocabulary Enrollment through Handwriting and Speech in a Multimodal Scheduling Application. Making Pen-Based Interaction Intelligent and Natural. Papers from the 2004 AAAI Symposium, Technical Report FS-04-06, Arlington, VA., USA, October 21-24, 2004, pgs. 85-91.

Abstract
Our goal is to automatically recognize and enroll new vocabulary in a multimodal interface. To accomplish this our technique aims to leverage the mutually disambiguating aspects of co-referenced, co-temporal handwriting and speech. The co-referenced semantics are spatially and
temporally determined by our multimodal interface for schedule chart creation. This paper motivates and describes our technique for recognizing out-of-vocabulary (OOV) terms and enrolling them dynamically in the system. We report results for the detection and segmentation of OOV words within a small multimodal test set. On the same test set we also report utterance, word and pronunciation level error rates both over individual input modes and multimodally. We show that combining information from handwriting and speech yields better results than achievable by either mode alone.

Download