Treffer: Pictures from Words, Pictures from Text: Constructing Pictorial Representations of Meaning from Text

This report describes a design study carried out at the Computing Research Laboratory, New Mexico State University. The goal was to investigate the feasibility of creating a visual interpretation of the spoken messages passed among the crew of an Unmanned Aerial Vehicle (UAV). Because of the limited domain of UAV crew dialogs, it seemed possible to produce visual representations of their messages. These visualized messages could be used for communication among UAV crew members who do not speak the same language, and they could also visually verify any spoken communications. The following tasks were carried out by the research team: (1) a general analysis of how images could be used to represent dialogs; (2) a more specific analysis of dialogs and parse trees produced from experimental UAV dialogs; (3) participation in discussions and in a simulated flight as crew members while visiting researchers at AFRL, Mesa; (4) development of a set of icons and conventions to represent mission status and the form of crew member dialogs; (5) design of a prototype visualization system -- Chat2Pix; (6) implementation of the prototype using Visual Basic to run on Windows systems; (7) redesign of the prototype to display more structured information; and (8) implementation and delivery of a second version of the visualization system -- Chat2Pix2.