Samudro, Ahmad Arif (2019) Normalisasi Teks Bahasa Indonesia pada Media Sosial Berdasarkan FastText Embeddings. Undergraduate thesis, Institut Teknologi Sepuluh Nopember.
Preview |
Text
05211440000118-Undergraduate_Theses.pdf Download (1MB) | Preview |
Abstract
Menurut survei Asosiasi Penyelenggara Jasa Internet Indonesia (APJII) pengguna internet di Indonesia pada tahun 2017 adalah 143,76 jiwa, dan 87,13 persen diantara mereka menggunakannya untuk kebutuhan media sosial sebagai wadah bertukar informasi[1]. Namun seiring berjalannya waktu, komunikasi tertulis pada media sosial cenderung menggunakan ejaan dan tata bahasa yang tidak formal, dengan alasan agar komunikasi terjadi lebih natural, santai, dan terasa akrab[2]. Hal ini tentunya akan sedikit mengganggu penyebaran informasi karena banyaknya perbedaan ejaan kata pada media sosial Twitter dan Facebook. Banyaknya informasi yang simpang siur di platform media sosial merupakan suatu sumber data yang sangat berharga bagi para pegiat NLP. Namun masih banyak kendala yang ditemui dalam praktik NLP itu sendiri, dimana komputer sulit memahami bahasa manusia yang tidak baku baik secara ejaan maupun semantiknya. Dan untuk meminimalisir masalah tersebut, diperlukan serangkaian aktivitas untuk menormalisasi teks, diantaranya dengan model word embedding FastText.
Penggabungan model word embedding dengan algoritma Levensthein’s distance, dan Jaro-Winkler distance dapat menghasilkan sistem normalisasi yang cukup baik.
Hasil terbaik dari training model yang didapatkan adalah model FastText ke-3 dengan akurasi pengujian awal sebesar 45,6%. Dimana parameter penting yang perlu diperhatikan adalah learning algorithm dan context window. Pengujian data sampel pada model ini menghasilkan akurasi terbaik sebesar 84,65% pada threshold 55% dan 60%.
===============================================================================================
According to a survey of the Indonesian Internet Service Providers (APJII) internet users in Indonesia in 2017 were 143.76 people, and 87.13 percent of them used it for social media needs as a place to exchange information [1]. But over time, written communication on social media tends to use informal spelling and grammar, arguing that communication occurs more naturally, relaxed, and feels familiar [2]. This of course will slightly disrupt the spread of information because of the many differences in the spelling of words on Twitter and Facebook social media. The amount of confusing information on social media platforms is a very valuable data source for NLP activists. But there are still many obstacles encountered in the practice of NLP itself, where computers find it difficult to understand human language that is not standard both in spelling and semantics. And to minimize these problems, a series of activities is needed to normalize text, including the word embedding model FastText.
Combining the word embedding model with the Levensthein’s distance algorithm, and Jaro-Winkler distance can produce a fairly good text normalization system.
The best results from the training model obtained were the 3rd FastText model with initial testing accuracy of 45.6%. Where the important parameters to consider are learning algorithm and context window. The testing of sample data in this model produces the best accuracy of 84.65% on the threshold of 55% and 60%.
| Item Type: | Thesis (Undergraduate) |
|---|---|
| Additional Information: | RSSI 004.678 Sam n-1 2019 |
| Uncontrolled Keywords: | Natural Language Processing, Word Embedding, Word2Vec, Fasttext, Social Media. |
| Subjects: | H Social Sciences > HM Sociology > HM742 Online social networks. T Technology > T Technology (General) > T57.5 Data Processing |
| Divisions: | Faculty of Information Technology > Information System > 57201-(S1) Undergraduate Thesis |
| Depositing User: | Ahmad Arif Samudro |
| Date Deposited: | 16 Jun 2021 07:32 |
| Last Modified: | 16 Jun 2021 07:32 |
| URI: | http://repository.its.ac.id/id/eprint/60580 |
Actions (login required)
 |
View Item |