Consumer Engagement Databricks Data Lake For Advanced Analytics

Haikal, Muhammad (2026) Consumer Engagement Databricks Data Lake For Advanced Analytics. Other thesis, Institut Teknologi Sepuluh Nopember.

Text 5026211101-Undergraduate_Thesis.pdf - Accepted Version Restricted to Repository staff only Download (1MB) | Request a copy

Abstract

Data Consumer Care dalam tim Consumer Engagement di Versuni tersebar di berbagai sistem yang tidak terhubung, sehingga membutuhkan upaya manual yang signifikan sebelum analisis dapat dilakukan. Proyek ini merancang dan mengimplementasikan data lake terpusat di Databricks untuk mengatasi permasalahan tersebut. Mengikuti arsitektur medallion, pipeline data otomatis mengubah data mentah dari sistem sumber termasuk SAP, Genesys, dan Medallia menjadi dataset gold layer yang siap digunakan untuk kebutuhan bisnis. Kerangka tata kelola data berdasarkan standar COBIT 2019 APO14 ditetapkan untuk memastikan workspace dikelola secara konsisten. Kualitas data divalidasi pada 24 tabel gold layer menggunakan pemeriksaan berbasis SQL di lima dimensi: kelengkapan, akurasi, konsistensi, ketiadaan duplikasi, dan ketepatan waktu. Kemampuan analitik platform didemonstrasikan melalui dashboard Power BI yang terhubung via direct query, proof of concept machine learning untuk prediksi NPS menggunakan regresi linear berganda, dan analisis data eksploratif melalui SQL editor Unity Catalog. Hasil penelitian menunjukkan bahwa data lake Databricks yang terkelola dan terotomatisasi mengurangi penanganan data secara manual, memberikan akses yang andal kepada stakeholder terhadap data performa Consumer Care yang selalu diperbarui, serta membuka fondasi untuk analitik lanjutan dan pemodelan prediktif ke depannya.
=======================================================================================================================================
Consumer care data within the Consumer Engagement team at Versuni is spread across disconnected systems, requiring significant manual effort before any analysis can begin. This project designs and implements a consolidated data lake in Databricks to address that problem. Following the medallion architecture, automated data pipelines transform raw data from source systems including SAP, Genesys, and Medallia into business-ready gold layer datasets. A data governance framework based on the COBIT 2019 APO14 standard is established to ensure the workspace is managed consistently. Data quality is validated across 24 gold layer tables using SQL-based checks across five dimensions: completeness, accuracy, consistency, absence of duplication, and timeliness. The platform's analytical capabilities are demonstrated through Power BI dashboards connected via direct query, a machine learning proof of concept for NPS prediction using multiple linear regression, and exploratory data analysis through the Unity Catalog SQL editor. The results show that a governed, automated Databricks data lake reduces manual data handling, gives stakeholders reliable access to up-to-date consumer care performance data, and opens the foundation for advanced analytics and predictive modelling going forward.

Item Type:	Thesis (Other)
Uncontrolled Keywords:	Databricks, Data Lake, Arsitektur Medallion, Tata Kelola Data, COBIT 2019, Consumer Analytics, Medallion Architecture, Data Governance,
Subjects:	Q Science > QA Mathematics > QA76.9.D37 Data warehousing.
Divisions:	Faculty of Intelligent Electrical and Informatics Technology (ELECTICS) > Information System > 57201-(S1) Undergraduate Thesis
Depositing User:	Muhammad Haikal
Date Deposited:	06 Aug 2026 13:52
Last Modified:	06 Aug 2026 13:52
URI:	http://repository.its.ac.id/id/eprint/144124

Actions (login required)

View Item