Item Analysis of Final Examination Questions for Social Studies in Junior High Schools through the ITEMAN Program

Abdul  Wahab; Annim  Hasibuan; Roswani Siregar; Risnawaty; Tri Zahra Ningsih

doi:10.23887/jere.v7i3.59239

Authors

Abdul Wahab Universitas Muslim Indonesia, Makassar, Indonesia
Annim Hasibuan Universitas Islam Labuhan Batu, Labuhanbatu, Indonesia
Roswani Siregar Universitas Al-Azhar Medan, Medan, Indonesia
Risnawaty UMN Al-Washiyah, Medan, Indonesia
Tri Zahra Ningsih SMP Negeri 46 Kerinci, Kerinci, Indonesia

DOI:

https://doi.org/10.23887/jere.v7i3.59239

Keywords:

Item Analysis, Social Learning, ITEMAN

Abstract

The analysis of the quality of the test items is very important because it will affect the results of the test itself. The problem is that teachers often do not analyze test items, so the test results do not accurately describe the potential of students. For this reason, this study aims to analyze the items on the semester final exam for social studies class VII. This type of research is a quantitative description. The research sample was the answer sheets of Class VII students from public middle schools, totaling 119 sheets. The research instrument consisted of end-of-semester exam question sheets, student answer sheets, and a scoring rubric. The research data was obtained from student answer sheets, which were collected after the semester final exams for social studies took place. The data analysis technique uses statistical analysis through the ITEMAN program. The results of the study place the validity and reliability of the instrument in the high category. The results of the analysis of the difficulty level of the items and the differential power showed that there was an imbalance in the composition of the test items. So it was concluded that the test questions for the end of semester exams for social studies subjects in junior high school needed to be reviewed. The results of this study are expected to have implications for teachers in making social studies test questions in the future as well as input for the government in making policies, especially in the field of education.

References

Adiga, M. N. S., Acharya, S., & Holla, R. (2021). Item Analysis of Multiple-Choice Questions in Pharmacology in an Indian Medical School. Journal of Health and Allied Sciences NU, 11(3), 130–135. https://doi.org/10.1055/s-0041-1722822.

Aulia, I. F., Sukirlan, M., & Sudirman. (2014). Analysis of the Quality of Teacher-Made Reading Comprehension Test Items Using ITEMAN. Unila Journal of English Teaching, 3(4). https://www.neliti.com/publications/194356/analysis-of-the-quality-of-teacher-made-reading-comprehension-test-items-using-i.

Baran-Łucarz, M. (2019). Formative assessment in the English as a foreign language classroom in secondary schools in Poland. Report on a mixed-method study. Journal of Education Culture and Society, 10(2), 309–327. https://doi.org/10.15503/jecs20192.309.327.

Bichi, A. A. (2016). Classical test theory : An introduction to linear modeling approach to test and item analysis. International Journal for Social Studies, 2(9). https://edupediapublications.org/journals

Black, P., & Wiliam, D. (2018). Classroom Assessment and Pedagogy. Assessment in Education: Principles, Policy and Practice, 1–25. https://doi.org/https://doi.org/10.1080/0969594X.2018.1441807.

Boopathiraj, C., & Chellamani, K. (2013). Analysis of Test Items on Difficulty Level and Discrimination Index in the Test for Research in Education. International Journal of Social Science & Interdisciplinary Research, 2(2). https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=a9e08c03848e95760275e36f75cae88e49bc6c65.

Cahyadi, W., Aswita, D., & Ningsih, T. Z. (2022). Analysis of The Development of Non-Cognitive Assessment Instrument to Support Online History Learning in Jambi City High School. AL-ISHLAH: Jurnal Pendidikan, 14(3), 3265–3274. https://doi.org/10.35445/alishlah.v14i3.2044.

Clark, L. A., & Watson, D. (2019). Constructing Validity: New Developments in Creating Objective Measuring Instruments. Psychological Assessment, 176(3), 1412. https://doi.org/10.1037/pas0000626.Constructing.

Clauser, J. C., & Hambleton, R. K. (2017). Item Analysis for Classroom Assessments in Higher Education. In C. Secolsky & D. B. Denison (Eds.), Handbook on Measurement, Assessment, and Evaluation in Higher Education (pp. 355–369). Routledge.

Dewi, S. S., Hariastuti, R. M., & Utami, A. U. (2019). Analisis Tingkat Kesukaran Dan Daya Pembeda Soal Olimpiade Matematika (Omi) Tingkat Smp Tahun 2018. Transformasi : Jurnal Pendidikan Matematika Dan Matematika, 3(1), 15–26. https://doi.org/10.36526/tr.v3i1.388.

Din, M. (2020). Evaluating university students’ critical thinking ability as reflected in their critical reading skill: A study at bachelor level in Pakistan. Thinking Skills and Creativity, 35(September 2019), 100627. https://doi.org/10.1016/j.tsc.2020.100627.

Doley, S. K. (2023). Stimulus Appraisal-Based L2 Attitude and Motivation among Indian ESL Learners. International Journal of Instruction, 16(2), 603–622. https://doi.org/10.29333/iji.2023.16232a.

Elgadal, A. H., & Mariod, A. A. (2021). Item Analysis of Multiple-choice Questions (MCQs): Assessment Tool For Quality Assurance Measures. Sudan Journal of Medical Sciences, 16(3), 334–346. https://doi.org/10.18502/sjms.v16i3.9695.

Gao, X., Li, P., Shen, J., & Sun, H. (2020). Reviewing assessment of student learning in interdisciplinary STEM education. International Journal of STEM Education, 7(1), 1–14. https://doi.org/10.1186/s40594-020-00225-4.

Gazi, F., Atan, T., & Kılıç, M. (2022). The Assessment of Internal Indicators on The Balanced Scorecard Measures of Sustainability. Sustainability (Switzerland), 14(14), 1–19. https://doi.org/10.3390/su14148595.

Gibta, Melani, M., Susanti, I., & Dharma, U. S. (2020). Item Analysis of Force Material Problem in Elementary School. Jurnal Pendidikan Sekolah Dasar, 3(April), 23–32. https://repository.usd.ac.id/37825/1/6315_31431-89165-1-PB.pdf.

Hairida, H., & Junanto, T. (2018). The Effectiveness of Performance Assessment in Project-Based Learning by Utilizing Local Potential to Increase the Science Literacy. International Journal of Pedagogy and Teacher Education, 2(July), 17. https://doi.org/10.20961/ijpte.v2i0.25722.

Herbert, J. S., Mitchell, A., Brentnall, S. J., & Bird, A. L. (2020). Identifying Rewards Over Difficulties Buffers the Impact of Time in COVID-19 Lockdown for Parents in Australia. Frontiers in Psychology, 11(December), 1–11. https://doi.org/10.3389/fpsyg.2020.606507.

Ibrahim, A., & Yahia, O. (2021). Post-validation item analysis to assess the validity and reliability of multiple- choice questions at a medical college with an innovative curriculum. Medical Education, 34(6), 359–362. https://www.researchgate.net/profile/Amar-Yahia-2/publication/361839573_Post-validation_item_analysis_to_assess_the_validity_and_reliability_of_multiple-choice_questions_at_a_medical_college_with_an_innovative_curriculum/links/62de9f57aa5823729ee0bce6/Post-validation-item-analysis-to-assess-the-validity-and-reliability-of-multiple-choice-questions-at-a-medical-college-with-an-innovative-curriculum.pdf.

Ivars-Baidal, J. A., Celdrán-Bernabeu, M. A., Femenia-Serra, F., Perles-Ribes, J. F., & Giner-Sánchez, D. (2021). Measuring the progress of smart destinations: The use of indicators as a management tool. Journal of Destination Marketing and Management, 19, 100531. https://doi.org/10.1016/j.jdmm.2020.100531.

Jin, K. Y., Reichert, F., Cagasan, L. P., de la Torre, J., & Law, N. (2020). Measuring digital literacy across three age cohorts: Exploring test dimensionality and performance differences. Computers and Education, 157(June), 103968. https://doi.org/10.1016/j.compedu.2020.103968.

Kim, Y., Dykema, J., Stevenson, J., Black, P., & Moberg, D. P. (2019). Straightlining: Overview of Measurement, Comparison of Indicators, and Effects in Mail–Web Mixed-Mode Surveys. Social Science Computer Review, 37(2), 214–233. https://doi.org/10.1177/0894439317752406.

Kolovou, D., Naumann, A., Hochweber, J., & Praetorius, A. K. (2021). Content-specificity of teachers’ judgment accuracy regarding students’ academic achievement. Teaching and Teacher Education, 100(April), 103298. https://doi.org/10.1016/j.tate.2021.103298.

Leber, J., Renkl, A., Nückles, M., & Wäschle, K. (2018). When the type of assessment counteracts teaching for understanding. Learning: Research and Practice, 4(2), 161–179. https://doi.org/10.1080/23735082.2017.1285422.

Ma’rifah, U., Algiovan, N., & Sutarsyah, C. (2021). An Item Analysis of English Test During Online Learning. International Journal of Multicultural and Multireligious Understanding, 8(12), 647–654. https://doi.org/10.18415/ijmmu.v8i12.3396.

Mai, R., Niemand, T., & Kraus, S. (2021). A tailored-fit model evaluation strategy for better decisions about structural equation models. Technological Forecasting and Social Change, 173(August), 121142. https://doi.org/10.1016/j.techfore.2021.121142.

Moses, D. A., Leonard, M. K., Makin, J. G., & Chang, E. F. (2019). Real-time decoding of question-and-answer speech dialogue using human cortical activity. Nature Communications, 10(1), 1–14. https://doi.org/10.1038/s41467-019-10994-4.

Mrkva, K., Posner, N. A., Reeck, C., & Johnson, E. J. (2021). Do Nudges Reduce Disparities? Choice Architecture Compensates for Low Consumer Knowledge. Journal of Marketing, 85(4), 67–84. https://doi.org/10.1177/0022242921993186.

Nengsi, A. R., & Efrina, G. (2019). Optimalisasi Tes Prestasi Buatan Guru Mata Pelajaran Ips Sd Untuk Evaluasi Pembelajaran Yang Presisi. Ta’dib, 22(2), 121. https://doi.org/10.31958/jt.v22i2.1498.

Ningsih, T. Z., Sariyatun, & Sutimin, L. A. (2019). Development of portfolio assessment to measure the student’s skill of using primary source evidence. New Educational Review, 56(2), 101–113. https://doi.org/10.15804/tner.2019.56.2.08.

Özer, M., & Bilgisi Öz, M. (2020). What Does PISA Tell Us About Performance of Education Systems? PISA Eğitim Sistemlerinin Performansı Hakkında Bize Ne Söylüyor? Bartın University Journal of Faculty of Education, 9(2), 217–228. https://doi.org/10.14686/buefad.697153.

Perdana, R., Riwayani, R., Jumadi, J., & Rosana, D. (2019). Development, Reliability, and Validity of Open-ended Testto Measure Student’s Digital Literacy Skil. International Journal of Educational Research Review, 4(4), 504–516. https://doi.org/10.24331/ijere.628309.

Permatasari, B. D., Gunarhadi, & Riyadi. (2019). The influence of problem based learning towards social science learning outcomes viewed from learning interest. International Journal of Evaluation and Research in Education, 8(1), 39–46. https://doi.org/10.11591/ijere.v8i1.15594.

Pittman, R. T., Zhang, S., Binks-Cantrell, E., Hudson, A., & Joshi, R. M. (2020). Teachers’ knowledge about language constructs related to literacy skills and student achievement in low socio-economic status schools. Dyslexia, 26(2), 200–219. https://doi.org/10.1002/dys.1628.

Prasetyono, H., Abdillah, A., Djuhartono, T., Ramdayana, I. P., & Desnaranti, L. (2021). Improvement of teacher’s professional competency in strengthening learning methods to maximize curriculum implementation. International Journal of Evaluation and Research in Education, 10(2), 720–727. https://doi.org/10.11591/ijere.v10i2.21010.

Quaigrain, K., & Arhin, A. K. (2017). Using Reliability and Item Analysis to Evaluate A Teacher-Developed Test in Educational Measurement and Evaluation. Cogent Education, 12(1). https://doi.org/10.1080/2331186X.2017.1301013.

Qurrota, A. A. S., Siskawati, F. S., & Irawati, T. N. (2022). Analisis Kelayakan Butir Soal pada Media INTERMATHLY (Interesting Mathematic Monopoly). Jurnal Cendekia : Jurnal Pendidikan Matematika, 6(1), 634–654. https://doi.org/10.31004/cendekia.v6i1.1181.

Reynolds, C. R., Altmann, R. A., & Allen, D. N. (2021). The Problem of Bias in Psychological Assessment. In C. R. Reynolds, R. A. Altmann, & D. N. Allen (Eds.), Mastering Modern Psycological Testing (pp. 573–613). Springer US.

Rintayati, P., Lukitasari, H., & Syawaludin, A. (2020). Development of Two-Tier Multiple Choice Test to Assess Indonesian Elementary Students’ Higher-Order Thinking Skills. International Journal of Instruction, 14(1), 555–566. https://doi.org/10.29333/IJI.2021.14133A.

Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., Furmark, T., & Carlbring, P. (2019). The Negative Effects Questionnaire: Psychometric properties of an instrument for assessing negative effects in psychological treatments. Behavioural and Cognitive Psychotherapy, 47(5), 559–572. https://doi.org/10.1017/S1352465819000018.

Said, A., & Muslimah, M. (2021). Evaluation of Learning Outcomes of Moral Faith Subjects during Covid-19 Pandemic at MIN East Kotawaringin. Bulletin of Science Education, 1(1), 13–26. https://doi.org/10.51278/bse.v1i1.99.

Sarwanto, Fajari, L. E. W., & Chumdari. (2020). Open-Ended Questions to Assess Critical-Thinking Skills in Indonesian Elementary School. International Journal of Instruction, 14(1), 615–630. https://doi.org/10.29333/IJI.2021.14137A.

Schaeffer, N. C., & Dykema, J. (2020). Advances in the Science of Asking Questions. Annual Review of Sociology, 46(March), 37–60. https://doi.org/10.1146/annurev-soc-121919-054544.

Shakir, M. A. (2021). Assessment of Learning Achievement of Visually Impaired Children at Primary Level. Pakistan Journal of Educational Research and Evaluation, 9(2), 44–52. http://journals.pu.edu.pk/journals/index.php/PJERE/article/view/5311.

Shirazi, M. A., Alavi, S. M., & Salarian, H. (2019). An Investigation into Item Types and Text Types of Reading Comprehension Section of Iranian Ph.D. Entrance Exams Using G-theory. Journal of Modern Research in English Language Studies, 6(1), 1–29. https://doi.org/10.30479/jmrels.2019.10591.1326.

Surucu, L., & Maslakci, A. (2020). Business & Management Studies : Business & Management Studies: An International Journal, 8(3), 2694–2726. https://doi.org/10.15295/bmij.v8i3.1540.

Suwarto, S. (2021). The Characteristics of Indonesia Second-semester Final Test for Eighth-grade Students Endonezya İ kinci Yar ı y ı l Sekizinci S ı n ı f Ö ğ rencileri İ çin Final S ı nav ı n ı n Özellikleri. Turkish Online Journal of Qualitative Inquiry (TOJQI), 12(9), 356–370. https://search.ebscohost.com/login.aspx?direct=true&profile=ehost&scope=site&authtype=crawler&jrnl=13096591&AN=160604451&h=K2DbfvLAoTNBnejP68bkGfn5XmFXzQcBX55uayj0k6Qa8zBEUzB8aJR83OL1kTIwX0GlzBBmysM4r1uFMZ1dGA%3D%3D&crl=c.

Suzana, A. (2018). Analisis Tingkat Kesukaran dan Daya Beda Butir-Butir Soal Penilaian Akhir Tahun Matematika Kelas X di SMA Negeri 1 Purbalingga. MathGram Matematika, 2(2), 1–8. https://ejournal.unugha.ac.id/index.php/mthg/article/view/172.

Taherdoost, H. (2016). Validity and Reliability of the Research Instrument ; How to Test the Validation of a Questionnaire / Survey in a Research Hamed Taherdoost To cite this version : HAL Id : hal-02546799 Validity and Reliability of the Research Instrument ; How to Test the. International Journal of Academic Research in Management, 5(3), 28–36. https://doi.org/10.2139/ssrn.3205040.

Utama, C., Sajidan, Nurkamto, J., & Wiranto. (2020). The instrument development to measure higher-order thinking skills for pre-service biology teacher. International Journal of Instruction, 13(4), 833–848. https://doi.org/10.29333/iji.2020.13451a.

Warju, W., Ariyanto, S. R., Soeryanto, S., & Trisna, R. A. (2020). Analisis Kualitas Butir Soal Tipe Hots Pada Kompetensi Sistem Rem Di Sekolah Menengah Kejuruan. Jurnal Pendidikan Teknologi Dan Kejuruan, 17(1), 95. https://doi.org/10.23887/jptk-undiksha.v17i1.22914.

Weller, S. C., Vickers, B., Russell Bernard, H., Blackburn, A. M., Borgatti, S., Gravlee, C. C., & Johnson, J. C. (2018). Open-ended interview questions and saturation. PLoS ONE, 13(6), 1–18. https://doi.org/10.1371/journal.pone.0198606.

Wibawa, E. A. (2019). Karakteristik Butir Soal Tes Ujian Akhir Semester Hukum Bisnis. Jurnal Pendidikan Akuntansi Indonesia, 17(1), 86–96. https://doi.org/10.21831/jpai.v17i1.26339.

Wibowo, T. E., & Faizah, S. (2021). Pengembangan Soal Tes Untuk Mengukur Kemampuan Pemecahan Masalah Siswa Pada Materi Bentuk Aljabar. Alifmatika: Jurnal Pendidikan Dan Pembelajaran Matematika, 3(2), 145–158. https://doi.org/10.35316/alifmatika.2021.v3i2.145-158.

Wijaya, I. M. K. (2013). Pengetahuan, sikap dan motivasi terhadap keaktifan kader Dalam pengendalian tuberkulosis. Jurnal Kesehatan Masyarakat, 8(2), 137–144. https://doi.org/10.15294/kemas.v8i2.2637.

Wijayanti, P. S. (2020). Item Quality Analysis For Measuring Mathematical Problem-Solving Skills. AKSIOMA: Jurnal Program Studi Pendidikan Matematika, 9(4), 1223–1234. https://doi.org/10.24127/ajpm.v9i4.3036.

Yunida, H., & Arthur, R. (2023). Bloom ’ s Taxonomy Approach to Cognitive Space Using Classic Test Theory and Modern Theory. East Asian Journal of Multidisciplinary Research (EAJMR), 2(1), 95–108. https://doi.org/10.55927/eajmr.v2i1.2331.

Zhampeissova, K., Alena, G., Ekaterina, V., & Zhanna, E. (2020). “Academic Performance and Cognitive Load in Mobile Learning.” International Journal of Interactive Mobile Technologies, 14(21), 78–91,. https://doi.org/10.3991/ijim.v14i21.18439.

Zohar, A., & Alboher Agmon, V. (2018). Raising test scores vs. teaching higher order thinking (HOT): senior science teachers’ views on how several concurrent policies affect classroom practices. Research in Science and Technological Education, 36(2), 243–260. https://doi.org/10.1080/02635143.2017.1395332.