Analytical Thinking Skill and Problem Solving Skills Instruments in Biology  Learning Using Rasch Model

Riska Ardianti; Abdul  Razak; Heffi  Alberida; Muhyiatul Fadilah; Mutia Sari

doi:10.23887/jere.v8i4.84026

Authors

Riska Ardianti Universitas Negeri Padang, Padang, Indonesia
Abdul Razak Universitas Negeri Padang, Padang, Indonesia
Heffi Alberida Universitas Negeri Padang, Padang, Indonesia
Muhyiatul Fadilah Universitas Negeri Padang, Padang, Indonesia
Mutia Sari SMA Adabiah 2 Padang, Padang, Indonesia

DOI:

https://doi.org/10.23887/jere.v8i4.84026

Keywords:

Instrument analysis, Rasch Model, Biologi

Abstract

The low level of analytical thinking and problem-solving skills of students is often caused by the lack of teacher evaluation of these skill levels and the inadequate use of assessment instruments. To overcome this problem, it is necessary to prepare and analyze quality instruments in order to accurately describe students' abilities. This study aims to analyze students' analytical thinking skills and problem-solving skills using Rasch modeling. This study is a quantitative descriptive research with a sample of 30 grade XI students of SMA who were selected using random sampling techniques. The research instrument consisted of eight essay questions, namely four questions to measure analytical thinking skills and four questions for problem-solving skills. Data analysis was carried out using Rasch modeling to evaluate the validity, reliability, level of difficulty, and differentiation of the questions. The results of the study show that the instrument has good to excellent validity. All items meet the fit criteria, with item reliability in the good to special category, although the reliability of the person is classified as low to sufficient. The difficulty level of the questions is ideally distributed, reflecting the balance between easy and difficult questions, as well as the excellent differentiation of items. This instrument is feasible to use, but it is recommended to increase the number of question items in order to increase the reliability of the person in the separate measurement. The implication of this study is the importance of developing valid and reliable assessment instruments to support the skill-based learning and evaluation process.

References

Adi, N. R. M., Amaruddin, H., Adi, H. M. M., & A’yun, I. L. Q. (2022). Validity and Reliability Analysis Using the Rasch Model to Measure the Quality of Mathematics Test Items of Vocational High Schools. Journal of Research and Educational Research Evaluation, 11(2), 103–113. https://doi.org/10.15294/jere.v11i2.58835.

Agus, M., Penna, M. P., Peró-Cebollero, M., Guàrdia-Olmos, J., & Pessa, E. (2016). Investigating the probabilistic reasoning in verbal–numerical and graphical–pictorial formats in relation to cognitive and non-cognitive dimensions: The proposal of a model. Personality and Individual Differences, 94, 44–53. https://doi.org/10.1016/j.paid.2016.01.003.

Anggraini, T., Nugroho, S. E., & Subali, B. (2019). Analysis of the Analytical Junior High School Deaf Students Thinking Ability in SMPLBN Ungaran. Journal of Innovative Science Education, 8(2), 209–214. https://doi.org/10.15294/jise.v0i0.28232.

Arnold, J. C., Boone, W. J., Kremer, K., & Mayer, J. (2018). Assessment of competencies in scientific inquiry through the application of Rasch measurement techniques. Education Sciences, 8(4), 184. https://doi.org/10.3390/educsci8040184.

Astriani, D., Susilo, H., Suwono, H., & Lukiati, B. (2017). Profil keterampilan berpikir analitis mahasiswa calon guru IPA dalam perkuliahan biologi umum. JPPIPA (Jurnal Penelitian Pendidikan IPA), 2(2), 66–70. https://doi.org/10.26740/jppipa.v2n2.p66-70.

Azizah, U., & Nasrudin, H. (2022). Problem Solving Thinking Skills : Effectiveness of Problem- Solving Model in Teaching Chemistry College Students. Jurnal Penelitian Pendidikan IPA, 8(3), 1462–1469. https://doi.org/10.29303/jppipa.v8i3.1700.

Azzarkasyi, M., Rizal, S., & Kasmawati, K. (2019). The identification of student misconceptions on the concept of electricity using the cri decision matrix three level test. Asian Journal of Science Education, 1(1), 10–15. https://doi.org/10.24815/ajse.v1i1.14614.

Barendse, M. T., Oort, F. J., & Timmerman, M. E. (2015). Using Exploratory Factor Analysis to Determine the Dimensionality of Discrete Responses. Structural Equation Modeling: A Multidisciplinary Journal, 22(1), 87–101. https://doi.org/10.1080/10705511.2014.934850.

Boone, W., Staver, J., & Yale, M. (2013). Rasch analysis in the human sciences. Springer Science & Business Media. https://doi.org/10.1007/978-94-007-6857-4.

Choi, Y. J., & Asilkalkan, A. (2019). R Packages for Item Response Theory Analysis: Descriptions and Features. Measurement: Interdisciplinary Research and Perspectives, 17(3), 168–175. https://doi.org/10.1080/15366367.2019.1586404.

Chong, J., Mokshein, S. E., & Mustapha, R. (2022). Applying the Rasch Rating Scale Model (RSM) to investigate the rating scales function in survey research instrument. Jurnal Cakrawala Pendidikan, 41(1), 97–111. https://doi.org/10.21831/cp.v41i1.39130.

Darmana, A., Sutiani, A., Nasution, H. A., Ismanisa, I., & Nurhaswinda, N. (2021). Analysis of RASCH model for the validation of chemistry national exam instruments. Jurnal Pendidikan Sains Indonesia (Indonesian Journal of Science Education), 9(3), 329–345. https://doi.org/10.24815/jpsi.v9i3.19618.

Demir, E. (2022). An examination of high school students ’ critical thinking dispositions and analytical thinking skills. Journal of Pedagogical Research, 6(4), 190–200. https://doi.org/10.33902/JPR.202217357.

Fahmi, R., Muslem, A., & Usman, B. (2021). The use of Problem Based Learning to improve students’ speaking ability. English Education Journal, 12(2), 260–281. https://doi.org/10.24815/eej.v12i2.17920.

Fajaruddin, S., Retnawati, H., Yusron, E., & Sofyaningsih, V. (2021). Exploring the final examination test item characteristics of Pancasila and civic education. International Journal of Education and Learning, 3(3), 232–240. https://doi.org/10.31763/ijele.v3i3.435.

Fitriani, F., Fadly, W., & Faizah, U. N. (2021). Analisis Keterampilan Berpikir Analitis Siswa pada Tema Pewarisan Sifat. Jurnal Tadris IPA Indonesia, 1(1), 55–67. https://doi.org/10.21154/jtii.v1i1.64.

García-Pérez, L., García-Garnica, M., & Olmedo-Moreno, E. M. (2021). Skills for a working future: How to bring about professional success from the educational setting. Education Sciences, 11(1), 1–25. https://doi.org/10.3390/educsci11010027.

Gazi, F., Atan, T., & Kılıç, M. (2022). The Assessment of Internal Indicators on The Balanced Scorecard Measures of Sustainability. Sustainability (Switzerland), 14(14), 1–19. https://doi.org/10.3390/su14148595.

Gumala, Y., Yohamintin, Y., Farhana, H., Syaodih, E., & Samsudin, A. (2024). Development of an Instrument to Asses Critical Thinking Skills for Elementary School Students on Energy Topics. Edukatif: Jurnal Ilmu Pendidikan, 6(2), 1725–1734. https://doi.org/10.31004/edukatif.v6i2.5442.

Hagell, P. (2014). Testing rating scale unidimensionality using the principal component analysis (PCA)/t-test protocol with the Rasch model: the primacy of theory over statistics. Open Journal of Statistics, 4(6), 456–465. https://doi.org/10.4236/ojs.2014.46044.

Harris, R., & Clayton, B. (2019). The current emphasis on learning outcomes. International Journal of Training Research, 17(2), 93–97. https://doi.org/10.1080/14480220.2019.1644777.

Hasanah, D., & Purwanto, J. (2023). Rasch model analysis of physics test of HOTS on the topic of elasticity and hooke’s law. Thabiea : Journal of Natural Science Teaching, 6(1), 25. https://doi.org/10.21043/thabiea.v6i1.20422.

Hayati, S., & Lailatussaadah, L. (2016). Validitas dan Reliabilitas Instrumen Pengetahuan Pembelajaran Aktif, Kreatif, dan Menyenangkan (PAKEM) Menggunakan Model Rasch. Jurnal Ilmiah Didaktika, 16(2), 169–179. https://doi.org/10.22373/jid.v16i2.593.

Holmes, S. D., Meadows, M., Stockford, I., & He, Q. (2018). Investigating the Comparability of Examination Difficulty Using Comparative Judgement and Rasch Modelling. International Journal of Testing, 18(4), 366–391. https://doi.org/10.1080/15305058.2018.1486316.

Jacobs, M., Mhakure, D., Fray, R. L., Holtman, L., & Julie, C. (2014). Item difficulty analysis of a high-stakes mathematics examination using rasch analysis. Pythagoras, 35(1), 1–7. https://doi.org/10.4102/pythagoras.v35i1.220.

Jin, K. Y., & Wang, W. C. (2014). Item Response Theory Models for Performance Decline During Testing. Journal of Educational Measurement, 51(2), 178–200. https://doi.org/10.1111/jedm.12041.

Kania, N., Fitriani, C., & Bonyah, E. (2023). Analysis of Students’ Critical Thinking Skills Based on Prior Knowledge Mathematics. International Journal of Contemporary Studies in Education (IJ-CSE), 2(1), 49–58. https://doi.org/10.56855/ijcse.v2i1.248.

Karla, D., Pandey, V. K., Rastogi, P., & Kumar, S. (2022). A Comprehensive Review on Significance of Problem-Solving Abilities in Workplace. World Journal of English Language, 12(3), 88–95. https://doi.org/10.5430/wjel.v12n3p88.

Kim, S., Lu, Z., & Cohen, A. S. (2020). Reliability for tests with items having different numbers of ordered categories. Applied Psychological Measurement, 44(2), 137–149. https://doi.org/10.1177/0146621619835498.

Kniffin, K. M., & Baert, H. (2015). Maximizing learning through assessment in middle and high school physical education. Journal of Physical Education, Recreation and Dance, 86(4), 7–16. https://doi.org/10.1080/07303084.2015.1009203.

Kusumawati, M., & Hadi, S. (2018). An analysis of multiple choice questions (MCQs): Item and test statistics from mathematics assessments in senior high school. REiD (Research and Evaluation in Education), 4(1), 70–78. https://doi.org/10.21831/reid.v4i1.20202.

Maat, S. M. (2015). Psychometric Evaluation on Mathematics Beliefs Instrument Using Rasch Model. Creative Education, 6(16), 1797–1801. https://doi.org/10.4236/ce.2015.616183.

Mahjabeen, W., Alam, S., Hassan, U., Zafar, T., Butt, R., Konain, S., & Rizvi, M. (2017). Difficulty Index, Discrimination Index and Distractor Efficiency in Multiple Choice Questions. Annals of PIMS-Shaheed Zulfiqar Ali Bhutto Medical University, 13(4), 310–315. https://doi.org/10.48036/apims.v13i4.9.

Mamat, M. N., Maidin, P., & Mokhtar, F. (2014). Simplified Reliable Procedure for Producing Accurate Student’s Ability Grade Using Rasch Model. Procedia - Social and Behavioral Sciences, 112(Iceepsy 2013), 1077–1082. https://doi.org/10.1016/j.sbspro.2014.01.1272.

McCamey, R. (2014). A Primer on the One-Parameter Rasch Model. American Journal of Economics and Business Administration, 6(4), 159–163. https://doi.org/10.3844/ajebasp.2014.159.163.

Mio, C., Costantini, A., & Panfilo, S. (2022). Performance measurement tools for sustainable business: A systematic literature review on the sustainability balanced scorecard use. Corporate Social Responsibility and Environmental Management, 29(2), 367–384. https://doi.org/10.1002/csr.2206.

Mokshein, S. E., Ishak, H., & Ahmad, H. (2019). The use of rasch measurement model in English testing. Cakrawala Pendidikan, 38(1), 16–32. https://doi.org/10.21831/cp.v38i1.22750.

Mruk-Tomczak, D., & Jerzyk, E. (2024). Employees of the Future: Expected Competences at the Higher Education Level. Marketing of Scientific and Research Organizations, 51(1), 117–147. https://doi.org/10.2478/minib-2024-0006.

Ocak, G., & EĞMiR, E. (2016). The Relationship Between Pre-Service Teachers’ Critical Thinking Tendencies and Problem Solving Skills. Participatory Educational Research, 3(5), 33–44. https://doi.org/10.17275/per.16.spi.2.4.

Oon, P. T., Spencer, B., & Kam, C. C. S. (2017). Psychometric quality of a student evaluation of teaching survey in higher education. Assessment and Evaluation in Higher Education, 42(5), 788–800. https://doi.org/10.1080/02602938.2016.1193119.

Peeters, M. J., & Augustine, J. M. (2023). Using Rasch measurement for instrument rating scale refinement. Currents in Pharmacy Teaching and Learning, 15(1), 110–118. https://doi.org/10.1016/j.cptl.2023.02.015.

Permasutha, M. B., Rahayu, G. R., Giri, M. K. W., & Pradiptha, D. A. G. F. (2024). Multiple-Choice Questions in Basic Biomedical Science Module. Jurnal Pendidikan Dan Pengajaran, 57(1), 47–56. https://doi.org/10.23887/jpp.v57i1.63314.

Polya, G. (2004). How to Solve it; A New Aspect of Mathematical Method. In Princeton university press. https://doi.org/10.1017/cbo9780511616747.007.

Primi, C., Morsanyi, K., Donati, M. A., Galli, S., & Chiesi, F. (2017). Measuring Probabilistic Reasoning: The Construction of a New Scale Applying Item Response Theory. Journal of Behavioral Decision Making, 30(4), 933–950. https://doi.org/10.1002/bdm.2011.

Rafikasari, F., Ibrahim, M., Amin, S. M., & Djazilan, S. (2021). Keefektifan Pembelajaran Agama Islam melalui Pendekatan Pembelajaran Aktif, Kreatif, Efektif, dan Menyenangkan (Pakem) di Sekolah Dasar. Jurnal Basicedu, 5(5), 3232–3241. https://doi.org/10.31004/basicedu.v5i5.1314.

Rahim, A., & Haryanto, H. (2021). Implementation of Item Response Theory (IRT) Rasch Model in Quality Analysis of Final Exam Tests in Mathematics. Journal of Research and Educational Research Evaluation, 10(2), 57–65. https://doi.org/10.15294/jere.v10i2.51802.

Ramadhani, R., & Fitri, Y. (2020). Validitas E-Modul Matematika Berbasis EPUB3 Menggunakan Analisis Rasch Model. Jurnal Gantang, 5(2), 95–111. https://doi.org/10.31629/jg.v5i2.2535.

Ramos, J. J. R. (2018). Critical thinking skills among senior high school students and its effect in their academic performance. International Journal of Social Sciences & Humanities (IJSSH), 3(2), 60–72. https://doi.org/10.58885/ijssh.v3i2.60.jr.

Rozental, A., Kottorp, A., Forsström, D., Månsson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2019). The Negative Effects Questionnaire: psychometric properties of an instrument for assessing negative effects in psychological treatments. Behavioural and Cognitive Psychotherapy, 47(5), 559–572. https://doi.org/10.1017/S1352465819000018.

Sari, I. P. M., Jatmiko, B., & Suprapto, N. (2023). Students Physics Problem-Solving Skills in Daily Life Context: Between Confession and Fact. Jurnal Penelitian Pendidikan IPA, 9(1), 231–241. https://doi.org/10.29303/jppipa.v9i1.2561.

Sari, R. A., Misdalina, M., & Kesumawati, N. (2021). PMRI-based e-module development on sequences and series materials. Desimal: Jurnal Matematika, 4(3), 325–338. https://doi.org/10.24042/djm.v4i3.9545.

Shafira, N. N. A. (2015). Peran MCQ Sebagai Instrumen Evaluasi Dalam Pendidikan Kedokteran. Jambi Medical Journal: Jurnal Kedokteran Dan Kesehatan, 3(2), 132–139. https://doi.org/10.22437/jmj.v3i2.3089.

Stark, E., Kintz, S., Pestorious, C., & Teriba, A. (2018). Assessment for learning: using programmatic assessment requirements as an opportunity to develop information literacy and data skills in undergraduate students. Assessment & Evaluation in Higher Education, 43(7), 1061–1068. https://doi.org/10.1080/02602938.2018.1432029.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan Rasch pada Assessment Pendidikan. Trim Komunikata.

Susilawati, E., Hamidah, I., Rustaman, N., & Liliawati, W. (2024). Problem Solving Learning in Science Education: A Systematic Literature Review. Jurnal Penelitian Pendidikan IPA, 10(8), 548–558. https://doi.org/10.29303/jppipa.v10i8.5033.

Sutama, S., Fuadi, D., Narimo, S., Hafida, S. H. N., Novitasari, M., Anif, S., Prayitno, H. J., Sunanih, S., & Adnan, M. (2022). Collaborative mathematics learning management: Critical thinking skills in problem solving. International Journal of Evaluation and Research in Education, 11(3), 1015–1027. https://doi.org/10.11591/ijere.v11i3.22193.

Talebian-darzi, M., Fallah, V., & Salehi, M. (2021). Identifying the components of social responsibility in the curriculum of the skill university. Journal of Educational Sciences, 28(1), 208–228. https://doi.org/10.22055/edus.2021.33681.3048.

Tanty, H., Fernando, C., Valencia, J., & Justin, V. (2022). Critical Thinking and Problem Solving Among Students. Business Economic, Communication, and Social Sciences Journal (BECOSS), 4(3), 173–180. https://doi.org/10.21512/becossjournal.v4i3.8633.

Tesio, L., Caronni, A., Kumbhare, D., & Scarano, S. (2024). Interpreting results from Rasch analysis 1. The “most likely” measures coming from the model. Disability and Rehabilitation, 46(3), 591–603. https://doi.org/10.1080/09638288.2023.2169771.

Tishana, A., Alvendri, D., Pratama, A. J., Jalinus, N., & Abdullah, R. (2023). Filsafat Konstruktivisme dalam Mengembangkan Calon Pendidik pada Implementasi Merdeka Belajar di Sekolah Kejuruan. Journal on Education, 5(2), 1855–1867. https://doi.org/10.31004/joe.v5i2.826.

Van Zile-Tamsen, C. (2017). Using Rasch analysis to inform rating scale development. Research in Higher Education, 58(8), 922–933. https://doi.org/10.1007/s11162-017-9448-0.

Wahab, A., Hasibuan, A., Siregar, R., Risnawaty, & Ningsih, T. Z. (2023). Item Analysis of Final Examination Questions for Social Studies in Junior High Schools through the ITEMAN Program. Journal of Education Research and Evaluation, 7(3), 526–536. https://doi.org/10.23887/jere.v7i3.59239.

Walters, W., MacLaughlin, V., & Deakin, A. (2023). Perspectives and reflections on assessment in physical education: A narrative inquiry of a pre-service, in-service and physical education teacher educator. Curriculum Studies in Health and Physical Education, 14(1), 73–91. https://doi.org/10.1080/25742981.2022.2053334.

Weideman, A. (2019). Assessment literacy and the good language teacher: Insights and applications. Journal for Language Teaching, 53(1), 103–121. https://doi.org/10.4314/jlt.v53i1.5.

Weideman, A. J. (2020). Complementary evidence in the early-stage validation of language tests: Classical Test Theory and Rasch analyses. Per Linguam, 36(2), 57–75. https://doi.org/10.5785/36-2-970.

Wibisono, S. (2018). Aplikasi Model Rasch Untuk Validasi Instrumen Pengukuran Fundamentalisme Agama Bagi Responden Muslim. Jurnal Pengukuran Psikologi Dan Pendidikan Indonesia (JP3I), 5(1), 1–29. https://doi.org/10.15408/jp3i.v5i1.9239.

Wicaksono, D. A., Roebianto, A., & Sumintono, B. (2021). Internal Validation of the Warwick-Edinburgh Mental Wellbeing Scale:Rasch Analysis in the Indonesian Context. Journal of Educational, Health and Community Psychology, 10(2), 229–248. https://doi.org/10.12928/jehcp.v10i2.20260.

Widarta, F. O., Ikhsan, I., Dekar, M., & Rahman, A. (2023). Analysis of Science Subject Evaluation Instruments in Elementary Schools based on Stimulus Types and Cognitive Process Dimensions. Jurnal Eksakta Pendidikan (JEP), 7(1), 135–144. https://doi.org/10.24036/jep/vol7-iss1/742.

Wright, B., & Stone, M. (1999). Measurement Essentials (2nd editio). Wide Range, Inc.

Yorulmaz, A., Altintas, S., & Sidekli, S. (2017). Investigation of the Effects of Mathematical Thinking States of Form Teachers on Their Mathematics Teaching Anxieties. European Journal of Educational Research, 6(4), 485–493. https://doi.org/10.12973/eu-jer.6.4.485.

Yurt, E. (2022). Teachers’ Views and Experiences Regarding Acquiring Analytical Thinking Skills in the Middle School Mathematics Curriculum. International Journal on Social and Education Sciences, 4(4), 599–619. https://doi.org/10.46328/ijonses.475.

Yustiandi, Y., & Saepuzaman, D. (2021). Analysis of model fit and item parameter of work and energy test using item response theory. Gravity: Jurnal Ilmiah Penelitian Dan Pembelajaran Fisika, 7(2), 21–28. https://doi.org/10.30870/gravity.v7i2.10563.

Zeng, W., Huang, F., Yu, L., & Chen, S. (2018). Towards a learning-oriented assessment to improve students’ learning—a critical review of literature. Educational Assessment, Evaluation and Accountability, 30, 211–250. https://doi.org/10.1007/s11092-018-9281-9.

Zlatkin-Troitschanskaia, O., Pant, H. A., & Coates, H. (2016). Assessing student learning outcomes in higher education: challenges and international perspectives. Assessment & Evaluation in Higher Education, 41(5), 655–661. https://doi.org/10.1080/02602938.2016.1169501.