Chinese lab hopes to commercialize sign-language recognition platform

From: EE Times Online - 07 Mar 2003

By Mike Clendenin, EE Times
March 9, 2003 (12:40 a.m. EST)

BEIJING — A group of researchers at China's Institute of Computing Technology (ICT) is looking to commercialize a sign-language recognition program that could help more than 20 million hearing-impaired people in China more easily communicate.

The program allows written and verbal input of Chinese characters, or other languages, which are then translated into sign language gestures by a virtual person. The hearing-impaired can use the program, along with a set of “digital gloves,” to input sign language that is then spoken or converted to text.

ICT has spent years working on the technology, building up a database of words, hand gestures and facial expressions. Researchers reckon the program's accuracy is about 85 to 90 percent, based upon trials done with the hearing-impaired. “It will not get to 100 percent because even two people using sign language would not have that. The gestures can sometimes be difficult o understand,” said Wang Zhaoqi, deputy director or ICT's Digital Laboratory.

Nevertheless, there are still parts of the program that need work. One challenge has been fine-tuning facial expressions, and the timing of those expressions with the hand gestures. Indeed, the dapperly dressed virtual interpreter still appears a bit stony faced when communicating.

Researchers are trying to overcome this hurdle by building a database of facial expressions based on extensive modeling of volunteer subjects. Using sensors pasted to more than a dozen points on the human face, Wang and his team have gathered expressions made by the test subjects while reading or expressing feelings ranging from sadness and anger to joy or frustration.

“So far, you can see that the lips just move. It is very difficult to get the face expressions but it is important because if you do not use the expression, you cannot always understand the meaning,” Wang said. “So we have to work harder to make them more realistic.”

Another bottleneck to mass acceptance is the program platform. So far, it is limited to desktop PCs and notebooks with more powerful processors. “Realistically, if you can only use a desktop, than commercially there is no future because you can't ask everyone to sit in front of their computer to communicate,” said Huang Shu, an ICT project manager that is trying to find high-volume applications for the technology.

“But if you could put it into a handheld, then that would be more realistic. We have tried to put it into a PDA but there is not enough computing power in the display graphics accelerator,” Huang said.

Wang and his researchers are looking into ways to use software-based accelerators, but so far no breakthroughs have occurred. A custom ASIC is another possibility, but it's likely such a solution would be too expensive, Huang said. Until then, the program will likely be limited to a few low-volume markets, such as broadcast television and education.

Copyright © 2002 CMP Media LLC. All rights reserved.