Sign of the Times

From: San Jose Mercury News, CA
Oct. 10, 2002

DEPAUL UNIVERSITY CREATING ANIMATED INTERPRETER FOR THE DEAF
By Jon Van
Chicago Tribune

CHICAGO - The first step toward developing a computing program that enables the deaf to ``hear'' has been created by computer scientists at DePaul University.

The project's urgency was underscored after a deaf computer scientist was nearly arrested by an airport security guard because of a failure to communicate. The frustrating episode led to a system where a hearing person speaks into a microphone attached to a computer running software that turns words into American Sign Language performed by an animated figure on the computer's screen.

Hence, a security guard could ask the system ``show me your boarding pass'' and a deaf person will see the message in sign language.

It's an important development because many people who are born deaf understand sign language, but they cannot read or write English with high proficiency, said Rosalee Wolfe, a DePaul computer science professor. Signs or notes written on paper are meaningless to many deaf people who are quite articulate in sign language.

Creating an automated signing interpreter is an immensely complex undertaking, Wolfe said.

``American Sign Language isn't a signed version of English, but a completely separate language with its own grammar and rules,'' she said.

Besides the hand signs, ASL uses facial expressions and body language to clarify meaning, so it requires the animated signer to achieve a level of detail that surpasses even the most sophisticated animation Hollywood churns out, she said.

``It's a huge project requiring skills that range from linguistics to knowledge of how the body works,'' Wolfe said.

More than a dozen faculty members and students at DePaul have worked on the project since 1998, when it started with broad communication objectives. They estimate they have worked about 25,000 hours on the project.

Devising a sign-language interpreter is the brainchild of Karen Alkoby, a deaf computer scientist graduate student, who did her thesis on the project.

Alkoby said that a small percentage of sign-language interpreters achieve true fluency in ASL and that misunderstandings are common because facial expressions and body movement are critical to ASL grammar.

``Deaf people have always wished for a small and mobile device that can translate into ASL while they are having family dinner conversation,'' Alkoby said in an e-mail interview.

An incident in 1999 when Alkoby walked past a security area toward an airline gate also played a role in her interest. A furious security guard yelled at Alkoby after she walked by the guard. She heard nothing and kept walking.

``She would not let me go until all the other security guards came to see what the problem was,'' Alkoby said. ``I have heard stories from many deaf people who are frustrated with the airport security because they do not understand the security guard's commands.''

While some off-the-shelf software has been used in the sign-language project, most has been created from scratch because nothing commercially available could do the job.

Jorge Toro, a doctoral student working on the project, said that in the early days they began with just an animation of a hand making signs before they expanded the hand to a full person's figure, which they call ``Paula'' after DePaul.

Paula is now able to interpret a menu of commands and statements that airport security guards would likely communicate, and the computer scientists seek an opportunity to field-test their technology.

Alkoby said the manual signing achieved by Paula is good, but it's still necessary to add all the non-manual signals needed for full communication.

Researchers are pushing toward a software program that could be loaded into a handheld computer to provide a broad range of communication, said McDonald.

Copyright 2002 Knight Ridder.