Total views : 358

Monocular Vision-based Signer-Independent Pakistani Sign Language Recognition System using Supervised Learning


  • School of Mechanical and Manufacturing Engineering (SMME), National University of Sciences and Technology (NUST), H-12 Main Campus, Islamabad, Pakistan


Background/Objectives: To construct a Pakistani sign language learning-based gesture recognition system with a reasonable rate of accuracy. Methods/Statistical Analysis: It should be <70 words. Include the method adapted to study the objectives/sampling details or simulation or statistical analysis of data; technique employed; mention unique/ important points of modification of methodology in the current study. Mention about test samples the control employed or approach used for comparing the test sample. Findings: The proposed system uses static images to extract local and global, region and boundary-based descriptors for acquiring gesture information, which is provided as input for supervised learning method known as Support Vector Machine (SVM). The purpose of this research is to formally introduce a practical learning-based PSL recognition system, which can lay the groundwork for future research pertaining to PSL. The proposed system was developed and the ten class supervised learning based system was able to achieve an accuracy of 83%. Application/Improvements: It is a preliminary work, which will be further improved to construct a real-time static and dynamic gesture based PSL system that is able to recognize words and sentences information.


Fourier Descriptors, Gesture recognition, Hu Moments, Sign Language Recognition, Support Vector Machines.

Full Text:

 |  (PDF views: 242)


  • Lahamy H. and Litchi, D. Real-time hand gesture recognition using range cameras. In: Proceedings of the 2010 Canadian Geomatics Conference and Symposium of Commission I, Calgary, Canada. 2010. p. 38–40.
  • Lambrecht J, Kruger J. Spatial Programming for Industrial Robots based on Gestures and Augmented Reality. Presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems. 2012. p. 466–72.
  • Correa M, Ruz-del-Solar J, Verchae Lee-Ferng J, Castillo N. Real-Time Hand Gesture Recognition for Human-Robot Interaction. Springer-Verlag: Berlin. 2009.
  • Hashimoto M, Takahashi K, Shimada M. Wheelchair control using an EOG- and EMG-based gesture interface. In: Proceedings of IEEE/ASME International Conference on Advanced Mechatronics. 2009. p. 1212–7.
  • Sohn K, Kim Y, Oh P. 2-Tier Control of a Humanoid Robot and Use of Sign Language Learned by Monte Carlo Method. Presented at The 22nd IEEE International Symposium on Robot and Human Interactive Communication. 2013; 547–52.
  • Ding J, Yu L, Wang Y, Pan Z. EasyHouse-I: A virtual house presentation system based on Internet. Presented at the 11th International Conference on Human-Computer Interaction. Las Vegas, Nevada, 2005.
  • Kela J, Korpipaa P, Mantyjarvi J, Kallio S, Savino G, Jozzo L, Marca D. Accelerometer-based gesture control for a design environment. Personal and Ubiquitous Computing. 2006; 10(5):285–99.
  • Abid MR, Melo LBS, Petriu EM. Dynamic Sign Language and Voice Recognition for Smart Home Interactive Application. Presented at the IEEE International Symposium on Medical Measurements and Applications (MeMeA). 2013; 139–44.
  • Lee S-H, Sohn M-K, Kim D-J, Kim H. Smart TV Interaction System using Face and Hand Gesture Recognition. Presented at the IEEE International Conference on Consumer Electronics. 2013; 173–4.
  • Pu Q, Gupta S, Gollakota S, Patel S. Whole-Home Gesture Recognition using Wireless Signals. Presented at the MobiCom’13. Miami, FL. 2013.
  • Hsu C-W, Lin C-J. A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks. 2002; 13:415–25.
  • Padden C, Humphries T. Deaf in America: Voices from a culture. Cambridge, MA: Harvard University Press, 1988.
  • Li Y, Chen X, Zhang X, Wang K, Wang ZJ. A Sign-Component-Based Framework for Chinese Sign Language Recognition using Accelerometer and sEMG Data. IEEE Transactions on Biomedical Engineering. 2012; 59(10):2695–704.
  • Gweth Y, Plahl C, Ney H. Enhanced continuous sign language recognition using PCA and neural network features. CVPR Workshop on Gesture Recognition. 2012.
  • Starner T, Weaver J, Pentland A. Real-time American sign language recognition using desk- and wearable computer-based video. Proceedings of IEEE Transactions of Patterns Analysis and Machine Intelligence. 1998. p. 1371–5.
  • Oszust M, Wysocki M. Recognition of Signed Expression Observed by Kinect Sensor. In: Proceedings of the 10th IEEE International Conference on Advanced Video and Signal Based Surveillance. 2013. p. 220–5.
  • Sun C, Zhang T, Bao B-K, Xu C, Mei T. Discriminative Exemplar Coding for Sign Language Recognition with Kinect. In: IEEE Transactions on Cybernetics. 2013; 43(5):1418–28.
  • Haralick RM, Shapiro LG. Image Segmentation Techniques. Computer Vision, Graphics, and Image Processing, 1985; 29(1):100–32.
  • Zarit BD, Super BJ, Quck FKH. Comparison of five color models in skin pixel classification. In: Proceedings of International Conference on Computer Vision, 1999; 58–63.
  • Yao H, Gao W. Face detection and location based on skin chrominance and lip chrominance transformation from color images. Pattern Recognition. 2001; 34:1555–64.
  • Jones MJ, Rehg JM. Statistical Color Models with Application to Skin Detection. International Journal of Computer Vision. 2002; 46(1):81–96.
  • Gonzales RC, Woods RE. Digital Image Processing, 3rd ed. New York, NY: Prentice-Hall, 2008.
  • Zhang D, Lu G. A Comparative Study of Fourier Descriptors for Shape Representation and Retrieval. Presented at The 5th Asian Conference on Computer Vision. 2002. p. 646–51.
  • Kunttu I, Lepisto L, Rauhamaa J, Visa A. Multiscale Fourier descriptor for shape-based image retrieval. Presented at the IEEE International Conference on Pattern Recognition. 2004. p. 765–8.
  • Nixon MS, Aguando A. Feature Extraction and Image Processing. Philadelphia, PA: Elsevier. 2007.
  • Hu M-K. Visual pattern recognition by moment invariants. IRE Transactions on Information Theory. 1962; 8(2):179–87.
  • Wilson AD, Bobick AF. Parametric hidden markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1999; 21(9):884–900.
  • Araga Y, Shirabayashi M, Kaido K, Hikawa H. Real Time Gesture Recognition using Posture Classifier and Jordan Recurrent Neural Network. Presented at the IEEE World Congress on Computational Intelligence. Brisbane, Australia, 2012.
  • Cortes C, Vapnik V. The Nature of Statistical Learning Theory. New York, NY: Springer, 1995.
  • Burges CJC. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery. 1998; 2(2):955–74.
  • Wang X, Paliwal KK. Feature extraction and dimensionality reduction algorithms and their applications in vowel recognition. Pattern Recognition. 2003; 36:2429–39.
  • Crammer K, Singer Y. On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines. Journal of Machine Learning Research. 2001; 2:265–93.
  • Dietterich TG, Bakiri G. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research. 1995; 2:263–86.


  • There are currently no refbacks.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.