Portal de Periódicos da CAPES

SIBI Sign Language Recognition Using Convolutional Neural Network Combined with Transfer Learning and non-trainable Parameters

2021; Elsevier BV; Volume: 179; Linguagem: Inglês

10.1016/j.procs.2020.12.011

1877-0509

Suharjito Suharjito, Narada Thiracitta, Gunawan Herman,

Video Surveillance and Tracking Methods

Sign Language Recognition (SLR) is a complex classification problem to be solved. Every language has their own syntax and grammar. New model combination and dataset is required to developed new SLR system for different syntax and grammar. In this study, we implemented a Convolutional Neural Network (CNN) model which is Inflated 3D model combined with transfer learning method from ImageNet and Kinectic dataset to overcome small dataset problems. There is no public dataset available for SIBI Dataset. Therefore, we collected the dataset by our-self using a mobile phone camera with the following specification. Samsung Galaxy S6 Camera, 16-megapixel Sony Exmore RS IMX240 sensor. The camera is in static position. We used 200 videos as our dataset with 10 words (classes) and 2 signers. We split the dataset into 3 parts (training, validation, testing). After several training and testing with 5 different froze layer combination. The highest validation accuracy was 100% and the highest testing accuracy is 97.50%. The best result was obtained by using a model with the most froze inception module.