Abstract

Facial expressions and speech are means to convey information. They can be used to reinforce speech or even complementary to speech. The main goal of our research is to investigate how facial expressions can be associated to text-based speech in an automated way. As a first step we studied how people attach smileys to text in chat sessions and facial expressions to text balloons in cartoons. We developed an expert system with a set of rules that describe dependencies between text and facial expressions. The specific facial expressions are stored in a nonverbal dictionary and we developed a search engine for that dictionary. Finally we present a tool to generate 3D facial animations.