Home

Overview

A sign language is a language which, instead of acoustically conveyed sound patterns,uses visually transmitted sign patterns to fluidly express a speaker's thoughts. With recent advances in Computer Vision , Sign Language Recognition offers great potential as a unique form of interaction between computer and a human user. This project is an effort to build an assistive tool to make computers more accessible to people.

Usability

Visually Impaired can use sign language to operate their machines using sign based commands as a keyboard/input replacement.

Speech impaired people can use to communicate by adding a Text to Speech Feature to communicate over VOIP.

Elderly people for convenience.

Demo

http://www.anuragbiyani.info/SLRVideo.swf

Please see a high resolution video here.

Features

Maintaining database of features ( synching features).

Training / Incremental.

Framework to add Multiple Techniques / Voting / Support.

Well Documented/Structured to encourage Open Source Development

Existing SLR Database

Gesture Symbol ONE TWO THREE FOUR ZERO

Challenges Faced

Hardware/ Software Compatibility Issues

Illuminance Changes in real-time stream input.

2-D transformations/calculation on inherently 3-D objects.

Non availability of robust open source implementation of ML Techniques.

Gesture Recognition in real-time is hard !!

Future Work

All type of gesture recognition, esp, moving gestures. - Difficult

Combinations of Facial & Hand Expressions. – Moderately Difficult.

Our system in current form can be adapted to it, with appropriate dataset.

Using sign language as shortcuts to various applications & functionality. – Very Easy!

Integrate with Mobile Devices to help disabled to communicate in normal/emergency situations. – Easy but very relevant.

Architecture & Design

References

Grover D., "Real-Time Tracking and analysis of Drosophilia behavior and gene expression". University of Southern California 2009.

OpenCV (Open Source Computer Vision) is a library of programming functions for real time computer vision.

http://opencv.willowgarage.com/wiki American Sign Language Linguistic Research Project

http://www.bu.edu/asllrp/ Team Members

Mentor: Ojas Mulay: ojasmulay at gmail dot com

Anurag Biyani: biyani at usc dot edu

Parth Patel: parthpat at usc dot edu

Prakhar Garg: gargp at usc dot edu

Mayank Misra: mmisra at usc dot edu

Ankit Sharma: ankit at usc dot edu

Umang Jain: ujain at usc dot edu

Nishkam Agrawal: nishkama at usc dot edu