ANGGRAINI, HARA (2025) PENGEMBANGAN MODEL CONVOLUTIONAL NEURAL NETWORK DAN SEQUENCE-TO-SEQUENCE UNTUK PENERJEMAH BAHASA ISYARAT SIBI DUA ARAH. S1 thesis, Fakultas Teknik Universitas Sultan Ageng Tirtayasa.
![[img]](https://eprints.untirta.ac.id/style/images/fileicons/text.png) |
Text (Fulltext)
Hara Anggraini_3337210014_Fulltext.pdf Restricted to Registered users only Download (2MB) |
|
Text (Bab 1)
Hara Anggraini_3337210014_01.pdf Restricted to Registered users only Download (1MB) |
|
Text (Bab 2)
Hara Anggraini_3337210014_02.pdf Restricted to Registered users only Download (3MB) |
|
Text (Bab 3)
Hara Anggraini_3337210014_03.pdf Restricted to Registered users only Download (764kB) |
|
Text (Bab 4)
Hara Anggraini_3337210014_04.pdf Restricted to Registered users only Download (1MB) |
|
Text (Bab 5)
Hara Anggraini_3337210014_05.pdf Restricted to Registered users only Download (438kB) |
|
Text (Daftar Pustaka)
Hara Anggraini_3337210014_Ref.pdf Restricted to Registered users only Download (341kB) |
|
Text (Lampiran)
Hara Anggraini_3337210014_Lamp.pdf Restricted to Registered users only Download (763kB) |
|
Text (Cek Plagiasi)
Hara Anggraini_3337210014_CP.pdf Restricted to Registered users only Download (18MB) |
Abstract
Individuals with hearing and speech impairments often face communication barriers due to the general public’s limited understanding of sign language. One of the official sign language systems in Indonesia is the Sistem Isyarat Bahasa Indonesia (SIBI). This research develops two machine learning models to build a bidirectional sign language translation system. The first model utilizes a Convolutional Neural Network (CNN) to recognize static hand gesture images representing SIBI letters and convert them into text. The second model applies a Sequence-to-Sequence (Seq2seq) approach based on Long Short-Term Memory (LSTM) to translate Indonesian sentences into gesture sequences, which are then visualized through concatenated gesture video clips. The dataset consists of static images of SIBI letters and paired sentence–gesture video data. The CNN model was evaluated using manually collected external test data and achieved an accuracy of 94.96%. Meanwhile, the Seq2seq model obtained an average Bleu Score of 0.9854, indicating a high level of similarity between the predicted output and reference gestures. The inference process is supported by automatic text correction using RapidFuzz and an interactive user interface built with ipywidgets, enhancing the usability of the system. The results demonstrate that the integration of CNN and Seq2seq models is effective in developing a prototype of a data-driven bidirectional sign language translator system, which has great potential for application in inclusive communication and education contexts.
| Item Type: | Thesis (S1) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Contributors: |
|
|||||||||
| Additional Information: | Penyandang tunarungu dan tunawicara sering menghadapi hambatan komunikasi karena sebagian besar masyarakat belum memahami bahasa isyarat. Salah satu sistem bahasa isyarat resmi di Indonesia adalah Sistem Isyarat Bahasa Indonesia (SIBI). Penelitian ini mengembangkan dua model machine learning untuk membangun sistem penerjemah bahasa isyarat dua arah. Model pertama menggunakan arsitektur Convolutional Neural Network (CNN) untuk mengenali gambar gestur huruf SIBI dan menerjemahkannya ke dalam teks. Model kedua menggunakan pendekatan Sequence-to-Sequence (Seq2seq) berbasis Long Short-Term Memory (LSTM) untuk menerjemahkan kalimat bahasa Indonesia menjadi urutan gesture, yang divisualisasikan melalui penyusunan video gerakan per kata. Dataset yang digunakan mencakup gambar statis huruf-huruf SIBI dan pasangan kalimat teks dengan video gesture tiap kata. Evaluasi model CNN dilakukan menggunakan data uji manual (eksternal) dan menghasilkan akurasi sebesar 94,96%. Sementara itu, model Seq2seq dievaluasi menggunakan Bleu Score dengan rata-rata 0,9854, yang menunjukkan tingkat kesesuaian tinggi antara output model dan referensi. Proses inferensi dilengkapi dengan koreksi input menggunakan RapidFuzz dan antarmuka interaktif berbasis ipywidgets, yang memudahkan pengguna dalam mengoperasikan sistem. Hasil penelitian menunjukkan bahwa kombinasi model CNN dan Seq2seq efektif dalam membangun prototipe sistem penerjemah bahasa isyarat dua arah berbasis data, dengan potensi besar untuk diterapkan dalam konteks pendidikan, komunikasi inklusif, dan teknologi asistif. | |||||||||
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science T Technology > T Technology (General) |
|||||||||
| Divisions: | 03-Fakultas Teknik 03-Fakultas Teknik > 55201-Jurusan Teknik Informatika |
|||||||||
| Depositing User: | Mrs Hara Anggraini | |||||||||
| Date Deposited: | 01 Aug 2025 07:44 | |||||||||
| Last Modified: | 01 Aug 2025 07:44 | |||||||||
| URI: | http://eprints.untirta.ac.id/id/eprint/53405 |
Actions (login required)
 |
View Item |