Interpretasi model Stacking Ensemble untuk analisis sentimen ulasan aplikasi pinjaman online menggunakan LIME

Aliyatul Munna; Eri Zuliarso

doi:10.24246/aiti.v21i2.183-196

Authors

Aliyatul Munna Universitas Stikubank https://orcid.org/0009-0008-3061-1300
Eri Zuliarso Universitas Stikubank https://orcid.org/0000-0002-8800-4748

DOI:

https://doi.org/10.24246/aiti.v21i2.183-196

Keywords:

online loan, stacking ensemble, LIME, blackbox, classification

Abstract

Local Interpretable Model-agnostic Explanations (LIME) can be used to overcome black box problems in the results of sentiment analysis classification models. This research uses reviews of online loan applications on the Play Store as a dataset. Each classification model has weaknesses and its performance can be improved by using stacking ensembles, especially to overcome the problem of imbalanced data classes. The dataset that has been obtained will be cleaned, pre-processed and converted into a numerical vector using TF-IDF. Classification is carried out using three basic models, namely random forest, naïve Bayes and support vector machine (SVM). The output of the basic classification model is used as an input for stacking ensemble logistic regression. Based on the comparison of the four models, stacking ensemble has the best performance with an accuracy of 87.05%. The application of LIME for interpreting classification models with sample data succeeded in explaining the factors that influence model decisions with a prediction probability of 95% and in accordance with manual observations. The results of this research can be used as insight and education to the public about the ease of online loan and its dangers, which are reflected in the positive and negative sentiments in a review.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

M. Arnani, “Daftar Pinjol Resmi Berizin OJK Desember 2023,” Kompas.com. Accessed: Dec. 24, 2023. [Online]. Available: https://money.kompas.com/read/2023/12/18/202154826/daftar-pinjol-resmi-berizin-ojk-desember-2023

P. H. Untari, “OJK: Outstanding Pembiayaan Pinjol Naik 17,66% jadi Rp58 Triliun per Oktober 2023 Artikel ini telah OJK: Outstanding Pembiayaan Pinjol Naik 17,66% jadi Rp58 Triliun per Oktober 2023,” https://finansial.bisnis.com. Accessed: Dec. 25, 2023. [Online]. Available: https://finansial.bisnis.com/read/20231204/563/1720599/ojk-outstanding-pembiayaan-pinjol-naik-1766-jadi-rp58-triliun-per-oktober-2023

A. A. Faisal, A. Wiradimadja, D. A. Ajra, M. D. Adhitama, R. Ramadhan, and S. Mardianto Albertus, “Attack On Pinjol: Siapa Yang Salah Antara Pinjol Dan Debitur,” Cinematology: Journal Anthology of Film and Television Studies, vol. 2, pp. 88–95, 2022.

R. Sugangga and E. H. Sentoso, “Perlindungan Hukum Terhadap Pengguna Pinjaman Online (Pinjol) Ilegal,” Pakuan Justice Journal of Law (PAJOUL), vol. 1, no. 1, pp. 47–61, 2020.

A. Kusno, M. B. Arifin, W. G. Mulawarman, and others, “Pengungkapan Pemerasan dan Pengancaman pada Alat Bukti Kasus Pinjaman Online (Kajian Linguistik Forensik),” Diglosia: Jurnal Kajian Bahasa, Sastra, dan Pengajarannya, vol. 5, no. 3, pp. 555–570, 2022. DOI: https://doi.org/10.30872/diglosia.v5i3.423

N. Muhamad, “OJK Terima 39 Ribu Aduan 2022-2024, Terbanyak soal Bank dan Pinjol.” Accessed: Feb. 18, 2024. [Online]. Available: https://databoks.katadata.co.id/datapublish/2024/02/05/ojk-terima-39-ribu-aduan-2022-2024-terbanyak-soal-bank-dan-pinjol

F. Novika, N. Septivani, and others, “Pinjaman Online Ilegal Menjadi Bencana Sosial Bagi Generasi Milenial,” Management Studies and Entrepreneurship Journal (MSEJ), vol. 3, no. 3, pp. 1174–1192, 2022.

D. A. K. Putra, “Karakteristik Verba dan Adjektiva Dalam Iklan Aplikasi Pinjaman Online,” Adabiyyāt: Jurnal Bahasa dan Sastra, vol. 6, no. 1, pp. 42–65, 2022. DOI: https://doi.org/10.14421/ajbs.2022.06103

Aldinata, A. M. Soesanto, V. C. Chandra, and D. Suhartono, “Sentiments comparison on Twitter about LGBT,” Procedia Comput Sci, vol. 216, pp. 765–773, 2023, doi: 10.1016/j.procs.2022.12.194. DOI: https://doi.org/10.1016/j.procs.2022.12.194

N. Habbat, H. Nouri, H. Anoun, and L. Hassouni, “Sentiment analysis of imbalanced datasets using BERT and ensemble stacking for deep learning,” Eng Appl Artif Intell, vol. 126, p. 106999, Nov. 2023, doi: 10.1016/j.engappai.2023.106999. DOI: https://doi.org/10.1016/j.engappai.2023.106999

A. Daza Vergaray, J. C. H. Miranda, J. B. Cornelio, A. R. López Carranza, and C. F. Ponce Sánchez, “Predicting the depression in university students using stacking ensemble techniques over oversampling method,” Inform Med Unlocked, vol. 41, p. 101295, 2023, doi: 10.1016/j.imu.2023.101295. DOI: https://doi.org/10.1016/j.imu.2023.101295

M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?,’” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA: ACM, Aug. 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778. DOI: https://doi.org/10.1145/2939672.2939778

X. Zhu, Q. Chu, X. Song, P. Hu, and L. Peng, “Explainable prediction of loan default based on machine learning models,” Data Science and Management, vol. 6, no. 3, pp. 123–133, Sep. 2023, doi: 10.1016/j.dsm.2023.04.003.

F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.

Q. Liu, J. Wang, D. Zhang, Y. Yang, and N. Wang, “Text Features Extraction based on TF-IDF Associating Semantic,” in 2018 IEEE 4th International Conference on Computer and Communications (ICCC), IEEE, Dec. 2018, pp. 2338–2343. doi: 10.1109/CompComm.2018.8780663. DOI: https://doi.org/10.1109/CompComm.2018.8780663

X. Ma et al., “Predicting the utilization factor of blasthole in rock roadways by random forest,” Underground Space (China), vol. 11, 2023, doi: 10.1016/j.undsp.2023.01.006. DOI: https://doi.org/10.1016/j.undsp.2023.01.006

A. Anggrawan, H. Hairani, and C. Satria, “Improving SVM Classification Performance on Unbalanced Student Graduation Time Data Using SMOTE,” International Journal of Information and Education Technology, vol. 13, no. 2, 2023, doi: 10.18178/ijiet.2023.13.2.1806. DOI: https://doi.org/10.18178/ijiet.2023.13.2.1806

X. Zhu, Q. Chu, X. Song, P. Hu, and L. Peng, “Explainable prediction of loan default based on machine learning models,” Data Science and Management, vol. 6, no. 3, 2023, doi: 10.1016/j.dsm.2023.04.003. DOI: https://doi.org/10.1016/j.dsm.2023.04.003