relation: https://eprints.untirta.ac.id/53405/
title: PENGEMBANGAN MODEL CONVOLUTIONAL NEURAL NETWORK DAN SEQUENCE-TO-SEQUENCE UNTUK PENERJEMAH BAHASA ISYARAT SIBI DUA ARAH
creator: ANGGRAINI, HARA
subject: QA75 Electronic computers. Computer science
subject: T Technology (General)
description: Individuals with hearing and speech impairments often face communication barriers due to the general public’s limited understanding of sign language. One of the official sign language systems in Indonesia is the Sistem Isyarat Bahasa Indonesia (SIBI). This research develops two machine learning models to build a bidirectional sign language translation system. The first model utilizes a Convolutional Neural Network (CNN) to recognize static hand gesture images representing SIBI letters and convert them into text. The second model applies a Sequence-to-Sequence (Seq2seq) approach based on Long Short-Term Memory (LSTM) to translate Indonesian sentences into gesture sequences, which are then visualized through concatenated gesture video clips.  The dataset consists of static images of SIBI letters and paired sentence–gesture video data. The CNN model was evaluated using manually collected external test data and achieved an accuracy of 94.96%. Meanwhile, the Seq2seq model obtained an average Bleu Score of 0.9854, indicating a high level of similarity between the predicted output and reference gestures. The inference process is supported by automatic text correction using RapidFuzz and an interactive user interface built with ipywidgets, enhancing the usability of the system.  The results demonstrate that the integration of CNN and Seq2seq models is effective in developing a prototype of a data-driven bidirectional sign language translator system, which has great potential for application in inclusive communication and education contexts.
date: 2025-08-01
type: Thesis
type: NonPeerReviewed
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/1/Hara%20Anggraini_3337210014_Fulltext.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/2/Hara%20Anggraini_3337210014_01.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/3/Hara%20Anggraini_3337210014_02.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/4/Hara%20Anggraini_3337210014_03.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/5/Hara%20Anggraini_3337210014_04.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/6/Hara%20Anggraini_3337210014_05.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/7/Hara%20Anggraini_3337210014_Ref.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/8/Hara%20Anggraini_3337210014_Lamp.pdf
format: text
language: id
identifier: https://eprints.untirta.ac.id/53405/9/Hara%20Anggraini_3337210014_CP.pdf
identifier:   ANGGRAINI, HARA  (2025) PENGEMBANGAN MODEL CONVOLUTIONAL NEURAL NETWORK DAN SEQUENCE-TO-SEQUENCE UNTUK PENERJEMAH BAHASA ISYARAT SIBI DUA ARAH.  S1 thesis, Fakultas Teknik Universitas Sultan Ageng Tirtayasa.