Penerapan Prinsip Psikometri untuk Analisis dan Pengembangan Butir Soal TKA yang Berkualitas di Ma'had An-Nahla Al-Islamy

Tritiyatma Hadinugrahaningsih, Hayyun Lisdiana, Rika Siti Sya'adah, Vida Zenitha Sudariasri, Achmad Ridwan, Lia Marsya Fazliana, Rifky Maulana Dimyati, Alfati Ridho Saputra, Almanda Nazwa Putri Suryana, Rizky Putri Sumpari

Abstract


The quality of the Academic Competency Test (TKA) is determined not only by content relevance and item construction but also by its psychometric characteristics. This community service program aimed to improve teachers' understanding and skills in applying psychometric principles to analyze and develop TKA items at Ma'had An-Nahl Al-Islamy. The program employed a participatory approach through socialization, training, item analysis practice, mentoring, and evaluation. Evaluation was conducted using pretests, posttests, practical observations, and participant feedback questionnaires. The results indicated an increase in the mean score from 54.55 in the pretest to 70.45 in the posttest, representing an improvement of 15.91 points and a normalized gain of 0.41. Participant feedback also indicated positive responses regarding the relevance of the training materials and implementation. The program provides a foundation for teachers to develop more systematic and data-driven assessment practices.

Keywords


Psychometrics, Item Analysis, Academic Competency Test, Item Quality, Community Service.

Full Text:

PDF

References


Ali, S. H., Carr, P. A., & Ruit, K. G. (2016). Validity and reliability of scores obtained on multiple-choice questions: Why functioning distractors matter. *Journal of the Scholarship of Teaching and Learning, 16*(1), 1–14. https://doi.org/10.14434/josotl.v16i1.19106

Ali, S. H., & Ruit, K. G. (2015). The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. *BMC Medical Education, 15*, Article 40. https://doi.org/10.1186/s12909-015-0319-0

Bhat, S. K., & Prasad, K. H. L. (2021). Item analysis and optimizing multiple-choice questions for a viable question bank in ophthalmology: A cross-sectional study. *Indian Journal of Ophthalmology, 69*(2), 343–346. https://doi.org/10.4103/ijo.IJO_1610_20

Bhattacherjee, S., Mukherjee, A., Bhandari, K., & Rout, A. J. (2022). Evaluation of multiple-choice questions by item analysis, from an online internal assessment of 6th semester medical students in a rural medical college, West Bengal. Indian Journal of Community Medicine, 47 (1), 92–95. https://doi.org/10.4103/ijcm.ijcm_1156_21

Brown, G. T. L., & Abdulnabi, H. H. A. (2017). Evaluating the quality of higher education instructor-constructed multiple-choice tests: Impact on student grades. Frontiers in Education, 2, Article 24. https://doi.org/10.3389/feduc.2017.00024

DeCarlo, L. T. (2023). Classical item analysis from a signal detection perspective. Journal of Educational Measurement. https://doi.org/10.1111/jedm.12358

Hayat, B. (2021). Klasika: Program analisis item dan tes dengan pendekatan klasik. JP3I (Jurnal Pengukuran Psikologi dan Pendidikan Indonesia), 10 (1), 1–11. https://doi.org/10.15408/jp3i.v10i1.20551

Jannah, R., Hidayat, D. N., Husna, N., & Khasbani, I. (2021). An item analysis on multiple-choice questions: A case of a junior high school English try-out test in Indonesia. Leksika: Jurnal Bahasa, Sastra dan Pengajarannya, 15 (1), 9–17. https://doi.org/10.30595/lks.v15i1.8768

Kheyami, D., Jaradat, A., Al-Shibani, T., & Ali, F. A. (2018). Item analysis of multiple choice questions at the Department of Paediatrics, Arabian Gulf University, Manama, Bahrain. Sultan Qaboos University Medical Journal, 18 (1), e68–e74. https://doi.org/10.18295/squmj.2018.18.01.011

Kulshreshtha, S., Gupta, G., Goyal, G., Gupta, K., & Davda, K. (2024). Item analysis of single best response type multiple choice questions for formative assessment in obstetrics and gynaecology. The Journal of Obstetrics and Gynecology of India, 74 (3), 256–264. https://doi.org/10.1007/s13224-023-01904-2

Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G., & Srinivas, V. (2021). Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Medical Journal Armed Forces India, 77 (Suppl. 1), S85–S89. https://doi.org/10.1016/j.mjafi.2020.11.007

Marsevani, M. (2022). Item analysis of multiple-choice questions: An assessment of young learners. English Review: Journal of English Education, 10 (2). https://doi.org/10.25134/erjee.v10i2.6241

Namdeo, S. K., & Sahoo, B. (2016). Item analysis of multiple choice questions from an assessment of medical students in Bhubaneswar, India. International Journal of Research in Medical Sciences, 4 (5), 1716–1719. https://doi.org/10.18203/2320-6012.ijrms20161256

Pratama, D. (2020). Analysis of Classical Test Theory (CTT) approach on academic ability test instrument. JISAE: Journal of Indonesian Student Assessment and Evaluation, 5 (2), 43–54. https://doi.org/10.21009/jisae.v5i2.12701

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4 (1), Article 1301013. https://doi.org/10.1080/2331186X.2017.1301013

Rezigalla, A. A., Eleragi, A. M. E. S. A., Elhussein, A. B., Alfaifi, J., Alghamdi, M. A., Al Ameer, A. Y., Yahia, A. I. O., Mohammed, O. A., & Adam, M. I. E. (2024). Item analysis: The impact of distractor efficiency on the difficulty index and discrimination power of multiple-choice items. BMC Medical Education, 24, Article 445. https://doi.org/10.1186/s12909-024-05433-y

Yahia, A. I. O. (2021). Post-validation item analysis to assess the validity and reliability of multiple-choice questions at a medical college with an innovative curriculum. The National Medical Journal of India, 34 (6), 359–362. https://doi.org/10.25259/NMJI_414_20

Zaini, Z., Hardika, R. C., Abrori, M. A. M., & Anggara, F. (2025). Pelatihan analisis tingkat kesukaran soal dan daya pembeda soal untuk identifikasi kualitas butir soal bagi guru di SMP Bahrul Ulum. Journal of Empowerment Community, 7(2). https://doi.org/10.36423/jec.v7i2.2449

Permendikdasmen Republik Indonesia Nomor 9 Tahun 2025 tentang Tes Kemampuan Akademik. (2025).




DOI: https://doi.org/10.31004/abdira.v6i4.2447

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Tritiyatma Hadinugrahaningsih, Hayyun - Lisdiana, Rika Siti Sya'adah, Vida Zenitha Sudariasri, Achmad Ridwan, Lia Marsya Fazliana, Rifky Maulana Dimyati, Alfati Ridho Saputra, Almanda Nazwa Putri Suryana, Rizky Putri Sumpari

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.