Pengembangan Multiple Imputation Framework dengan Pendekatan Accuracy-Level Metric dan Sequential Multivariate Outlier Detection for Mixed Data (SMODMD)-Based Clustering

Agustini, Mety (2026) Pengembangan Multiple Imputation Framework dengan Pendekatan Accuracy-Level Metric dan Sequential Multivariate Outlier Detection for Mixed Data (SMODMD)-Based Clustering. Doctoral thesis, Institut Teknologi Sepuluh Nopember.

Text 7003211008-Doctoral.pdf - Accepted Version Restricted to Repository staff only Download (5MB) | Request a copy

Abstract

Analisis statistik yang mengabaikan missing data dan outlier berpotensi menurunkan kualitas inferensi melalui hilangnya informasi, penurunan kekuatan statistik, serta peningkatan kesalahan baku. Penelitian ini mengembangkan suatu kerangka kerja multiple imputation berbasis clustering untuk menangani data campuran yang mengandung missing data dan outlier secara simultan. Kerangka yang diusulkan dirancang sebagai pendekatan yang modular, mudah diimplementasikan, dan efisien secara komputasi sehingga tetap efektif tanpa memerlukan model yang kompleks. Kontribusi metodologis pertama adalah penghitungan jarak Mahalanobis pada variabel kategorik ordinal menggunakan Cumulative Indicator-based Mahalanobis Distance (CIMD), sehingga diperoleh jarak Mahalanobis pada data campuran. Jarak Mahalanobis pada data campuran dihitung secara terpisah untuk variabel kontinu dan ordinal, kemudian digabungkan secara aditif untuk memperoleh ukuran jarak multivariat campuran yang komprehensif. Kontribusi kedua, prosedur Sequential Multivariate Outlier Detection for Mixed dData (SMODMD) yang memanfaatkan jarak Mahalanobis untuk mendeteksi outlier pada data campuran. Dua pendekatan cutoff digunakan untuk menentukan batas outlier, yaitu cutoff chi-square dan cutoff interquartile range (IQR). Kemudian hasil SMODMD digunakan untuk pengelompokan pada data campuran atau dinamakan metode SMODMD-Based Clustering. Evaluasi melalui studi simulasi pada berbagai struktur data memperlihatkan SMODMD-based clustering dengan cutoff IQR menghasilkan kualitas pengelompokan paling baik pada data extreme outlier dibandingkan ketiga metode pengelompokan lainnya. Kontribusi metodologis ketiga adalah pengembangan Accuracy-Level Metric (AL_k), yaitu metrik evaluasi berbasis terhitung (counted-based metric) dengan k tingkat (level) yang dirancang untuk melakukan evaluasi model pada data campuran. Untuk variabel kontinu, metrik ini mencakup Counted Squared Error (CSE_k), Counted Absolute Error (CAE_k), Counted Absolute Percentage Error (CAPE_k), dan Symmetric Counted Absolute Percentage Error (SCAPE_k). Sementara itu, untuk variabel ordinal digunakan ukuran accuracy k tingkat (Acc_k). Metrik ini memungkinkan evaluasi kinerja imputasi secara interpretatif serta memfasilitasi pemilihan dataset imputasi yang paling optimal dari beberapa dataset hasil multiple imputation (MI) sebelum proses clustering dilakukan. Suatu aspek yang umumnya tidak diperhatikan dalam prosedur MI konvensional. Kontribusi ke empat, penelitian ini mengintegrasikan accuracy-level metric dan SMODMD-based clustering untuk pengembangan kerangka kerja multiple imputation berbasis clustering yang diterapkan pada data Sensus Ekonomi 2016 Provinsi Jawa Timur. Hasil penelitian menunjukkan bahwa pendekatan yang diusulkan memberikan solusi yang praktis, robust, dan interpretatif dalam menangani data campuran yang mengandung missing data dan extreme outlier secara bersamaan.
=======================================================================================================================================
Statistical analyses that ignore missing data and outliers may compromise the quality of inference by causing information loss, reducing statistical power, and increasing standard errors. This study develops a clustering-based multiple imputation framework to simultaneously handle mixed data containing both missing values and outliers. The proposed framework is designed as a modular, easily implementable, and computationally efficient approach, allowing it to remain effective without requiring complex modeling structures. The first methodological contribution is the computation of Mahalanobis distance for ordinal categorical variables using the Cumulative Indicator-based Mahalanobis Distance (CIMD), thereby enabling the extension of Mahalanobis distance to mixed data. The Mahalanobis distance for mixed data is calculated separately for continuous and ordinal variables and then combined additively to obtain a comprehensive multivariate mixed-distance measure. The second contribution is the development of a sequential multivariate outlier detection for mixed data (SMODMD) procedure, which utilizes the Mahalanobis distance to detect outliers in mixed datasets. Two cutoff approaches are employed to determine outlier thresholds, namely the chisquare cutoff and the interquartile range (IQR) cutoff. The SMODMD results are subsequently used for clustering mixed data, which is known as SMODMD-based clustering method. According to evaluation through simulation experiments on different data structures, SMODMD-based clustering with an IQR cutoff achieved the best clustering quality on extreme outlier data when compared to the other three clustering techniques. The third methodological contribution is the development of an Accuracy-Level Metric (ALk), a count-based metric with k level, which is designed to assess model performance in mixed data. For continuous variables, these metrics include Counted Squared Error (CSEk), Counted Absolute Error (CAEk), Counted Absolute Percentage Error (CAPEk), and Symmetric Counted Absolute Percentage Error (SCAPEk). Meanwhile, for ordinal variables, the accuracy with k level (Acck) metric is employed. This metric enables an interpretable evaluation of imputation performance and facilitates the selection of the most optimal imputed dataset from multiple datasets generated by multiple imputation (MI) prior to the clustering process, an aspect that is generally overlooked in conventional MI procedures. The fourth contribution is the integration of the accuracy-level metric and SMODMD-based clustering into a unified clustering-based multiple imputation framework, which is applied to the 2016 Economic Census data of East Java Province. The outcomes indicate that the proposed approach provides a practical, robust, and interpretable solution for handling mixed data containing missing data and extreme outliers simultaneously.

Item Type:	Thesis (Doctoral)
Uncontrolled Keywords:	SMODMD-based clustering, accuracy-level metric, cumulative indicator, multiple imputation, outlier detection
Subjects:	Q Science Q Science > QA Mathematics Q Science > QA Mathematics > QA278.55 Cluster analysis
Divisions:	Faculty of Science and Data Analytics (SCIENTICS) > Statistics > 49001-(S3) PhD Thesis
Depositing User:	Mety Agustini
Date Deposited:	14 Aug 2026 07:37
Last Modified:	14 Aug 2026 07:37
URI:	http://repository.its.ac.id/id/eprint/144107

Actions (login required)

View Item