Studi Perbandingan Metode Deteksi Perilaku Menebak Terhadap Pemilihan Soal Dan Penilaian Kemampuan Pada Sistem Cat (Computerized Adaptive Testing)

Bimantoro, Akbar Noto Ponco (2022) Studi Perbandingan Metode Deteksi Perilaku Menebak Terhadap Pemilihan Soal Dan Penilaian Kemampuan Pada Sistem Cat (Computerized Adaptive Testing). Masters thesis, Institut Teknologi Sepuluh Nopember.

	Text 6025201035-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (2MB) | Request a copy
	Text 6025201035-Master_Thesis.pdf - Accepted Version Restricted to Repository staff only until 1 April 2024. Download (3MB) | Request a copy

Abstract

Computerized Adaptive Testing (CAT) merupakan penilaian adaptif berdasarkan konteks berbasis komputer. Namun, hasil penilaian bisa jadi tidak valid apabila peserta tes tidak mengerjakan soal dengan semestinya. Misalnya, guessing behaviour atau perilaku menebak dimana peserta tes menjawab tanpa mencoba mengerjakan soal dengan serius. Dengan kata lain, menebak jawaban dari pilihan yang ada. Terdapat beberapa penelitian terkait deteksi perilaku ini, contohnya menggunakan treshold waktu, logit kemampuan siswa, dan indeks kesulitan soal. Namun, metode tersebut belum banyak diimplementasikan pada CAT. Sehingga belum banyak studi tentang pengaruh penambahan metode deteksi perlak menebak pada kinerja pemilihan soal dan penilaian kemampuan CAT.

Oleh karena itu, penelitian ini melakukan studi perbandingan metode deteksi menebak K-Seconds, Surface Features, Distribusi RT, dan Pseudo-guessing berdasarkan logit kemampuan pada performa pemilihan soal dan penilaian kemampuan sistem CAT 4PL Fuzzy Mamdani Inference. Penambahan metode tersebut dibandingkan dengan native CAT dan diujikan kepada 48 siswa sekolah dasar. Hasil penilaian kemudian divalidasi oleh pakar atau guru masing-masing murid.

Berdasarkan hasil uji coba, penambahan deteksi perilaku menebak meningkatkan performa pemilihan soal sehingga siswa / peserta penilaian merasa lebih percaya diri. Selain itu, sistem dapat memilihkan soal berdasarkan pengetahuan, pemahaman materi, dan kemampuan dengan lebih baik. Dengan menggunakan Surface features, akurasi penilaian kemampuan mengalami peningkatan dari 92,5% menjadi 97.5%. Selain itu, jumlah responden yang menyatakan bahwa soal yang dipilihkan telah sesuai kemampuannya meningkat sebesar 2,43%. Akan tetapi, metode lainnya tidak mengalami kenaikan akurasi penilaian kemampuan, bahkan pada Distribusi RT dan K-Seconds mengalami penurunan sebesar 5% dan 7.5%. Meskipun demikian, kedua metode tersebut tetap membuat peserta lebih percaya diri. Temuan ini menjadikan Surface features
sebagai opsi metode terbaik yang dapat dimasukkan kedalam sistem CAT.
==============================================================================================
Computerized Adaptive Testing (CAT) is a context-based and computerized adaptive assessment. However, the result of the assessment may be invalid if the test taker does not take the test properly. For example, guessing behaviour where test taker answers the question without understanding it. In other words, guessing the answer from the given choices. There are studies related to the detection of this behaviour, e.g., by using response time threshold, student ability’s logit, and the question difficulty index. However, there aren’t many studies explaining how guessing behaviour detection method affects CAT system.
Therefore, this research will carry out a comparative study of K-Seconds, Surface Features, RT Distribution, and Pseudo Guessing of Logit Ability on CAT performance, especially in selecting question and assessing test taker’s abilities. The detection method addition will be compared to native CAT and will be tested on elementary students. The results of the system will be validated by experts or
their teacher. Based on the test results, the addition of guessing detection method improve the question selection performance. Thus, the student / test taker feel more confident. Moreover, this addition can select question based on test taker’s knowledge and understanding better. By using Surface features, the accuracy of the ability assessment increased from 92,5% to 97,5%. In addition, the number of respondents who stated that the selected questions were matching their abilities increased by 2.43%. Meanwhile, the other methods did not increase the accuracy of the ability assessment, even the RT Distribution and K-Seconds performance
decreased by 5% and 7,5% respectively. Nevertheless, both methods increase the
number of confident respondents. This finding makes Surface features the best
method that can be included in the CAT system.

Item Type:	Thesis (Masters)
Uncontrolled Keywords:	computerized adaptive test, fuzzy, guessing behavior, mamdani inference, penilaian adaptif , adaptive assessment
Subjects:	L Education > L Education (General) T Technology > T Technology (General) > T58.5 Information technology. IT--Auditing T Technology > T Technology (General) > T58.62 Decision support systems
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Informatics Engineering > 55101-(S2) Master Thesis
Depositing User:	AKBAR NOTO PONCO BIMANTORO
Date Deposited:	04 Feb 2022 03:49
Last Modified:	31 Oct 2022 02:24
URI:	http://repository.its.ac.id/id/eprint/92736

Actions (login required)

View Item