Measuring Changes of Students Conceptual Understanding of Literacy and Numeracy in Natural Science by Using Rasch Model

low literacy and numeracy abilities of Indonesian students based on the PISA encourage assessment policies based on literacy and numeracy. This study aims to analyze the feasibility of the items developed and changes in students' understanding of concepts after implementing literacy and numeracy-based learning processes on science subject matter topics related to education for sustainable development. The research method used in this research is the one-group pretest-posttest-design experimental method. Concerning the one-group pretest-posttest-design experimental method, the sample in this study was one whole class. Data was collected through a test instrument for students' conceptual understanding of science subjects. Raw data from students' grades were processed using the Rasch Model analysis. Through Rasch's analysis, the pattern of changes in students' conceptual understanding abilities was identified by comparing the response patterns of problem difficulty and students' abilities. The research results show that the items developed in the test instrument are appropriate for measuring students' abilities. Literacy and numeracy-based learning can improve students' understanding of concepts. Seven out of 53 students were identified as having decreased abilities. Student misconceptions cause it, the types of questions developed, student internal factors


A B S T R A C T
The low literacy and numeracy abilities of Indonesian students based on the PISA encourage assessment policies based on literacy and numeracy.This study aims to analyze the feasibility of the items developed and changes in students' understanding of concepts after implementing literacy and numeracy-based learning processes on science subject matter topics related to education for sustainable development.The research method used in this research is the one-group pretestposttest-design experimental method.Concerning the one-group pretest-posttest-design experimental method, the sample in this study was one whole class.Data was collected through a test instrument for students' conceptual understanding of science subjects.Raw data from students' grades were processed using the Rasch Model analysis.Through Rasch's analysis, the pattern of changes in students' conceptual understanding abilities was identified by comparing the response patterns of problem difficulty and students' abilities.The research results show that the items developed in the test instrument are appropriate for measuring students' abilities.Literacy and numeracy-based learning can improve students' understanding of concepts.Seven out of 53 students were identified as having decreased abilities.Student misconceptions cause it, the types of questions developed, student internal factors, and the teaching materials used.This analysis estimates the effectiveness of educational innovations and student ability in detail.

INTRODUCTION
PISA international survey participated by various countries aims to assess the proficiency of 15year-old students in literacy, numeracy, and science, as well as skills in applying what students have learned in school to real life.Indonesia's PISA score in 2018 was relatively dropped in all fields, with a score of 371 on literacy skills, 379 on numeracy skills, and 396 on science skills.Based on this score, Indonesia is ranked 76 out of 77 countries in literacy ability, 62 out of 71 countries in scientific literacy ability, and 72 out of 78 countries in numeracy ability (Argina et al., 2017;Rachman et al., 2021).Currently, the nation is also faced with an era of disruption that initiates the birth of the newest, more innovative, and massive interaction model with two essential choices: change or become extinct (Chamisah, 2017;Zahara & Sani, 2019).In this era of disruption, the younger generation's growth is increasing and is believed to experience a demographic bonus between 2030-2045.Therefore, innovation in preparing superior human resources to increase Indonesian competitiveness in the eyes of the world is needed, even though the literacy and numeracy culture of the Indonesian population is said to be very weak (Khofifah & Ramadan, 2021;Rachman et al., 2021).In the current context, literacy and numeracy do not only cover the ability to read and write but also count.Literacy and numeracy are also interpreted as life skills that cover many aspects of human life (Gal et al., 2020;Perdana & Suswandari, 2021).One of the indicators used in measuring countries' development is by looking at the people's lives and literacy level.Serious efforts are needed to improve abilities in literacy and numeracy to support the achievement of the collective intelligence of the peoplein the global context of competition, mainly in education, literacy and numeracy are necessary needs controlled by all stakeholders, including teachers, students, parents, and the school ecosystem (Limpo et al., 2018;Rand & Morrow, 2021).Children will develop literacy and numeracy skills through experience and support from the surrounding environment, including parents, teachers, and peers (Browne et al., 2014;Johnson et al., 2020).
One of the literacies that students must master is scientific literacy.Scientific literacy is a person's ability to understand science, communicate science (oral and written), and apply scientific knowledge to solve problems so that they have a high attitude and sensitivity towards themselves and their environment in making decisions based on scientific considerations (Le et al., 2022;Lucy Avraamidou & Bryan, 2018).Meanwhile, according to PISA, scientific literacy can be characterized as consisting of four aspects that will be obtained, namely: (1) realizing life situations involving science and technology, (2) understanding the natural world, including technology, based on scientific knowledge including knowledge of nature itself, (3) competence includes identifying scientific questions, explaining scientific phenomena and using scientific evidence as a basis for making decisions and conclusions (Narut & Supradi, 2019;Radovanović et al., 2015).
However, it is undeniable that there are always several difficulties in implementing learning innovations.There were several obstacles to implementing the literacy and numeracy learning process, such as the need for more supporting facilities.The teacher also felt difficulty compiling HOTS-based questions because the cognitive processes achieved in assessing literacy and numeracy lead to HOTS abilities (Hadi et al., 2018;Harta et al., 2020).Another challenge is that the students still need to be used to being faced with problems with a stimulus in the form of discourse texts.This challenge means that students need help to solve literacy and numeracy-based problems.In short, the literacy and numeracy learning process has yet to run optimally.Previous study stated that the problems in implementing literacy and numeracy-based assessments include the lack of references owned by teachers, lack of facilities such as gadgets, low interest in reading students, and students' difficulty in interpreting texts (Nasamu, 2021;Sehuwaky & Mastuti, 2021).This condition becomes an obstacle in the habitation of literacy and numeracy-based learning.
Nevertheless, teachers must innovate to create learning situations that attract students' interest in learning so that students can achieve the expected learning outcomes (Donovan et al., 2013;Lin et al., 2017).Teachers must develop their competencies to help the teaching and learning process.It is suggested that in teaching and learning activities there are three primary abilities required for a teacher, namely: (1) the ability to plan teaching and learning materials and activities, (2) the ability to implement and manage teaching and learning activities, and (3) the ability to assess learning outcomes.Previous study emphasized that measuring student learning outcomes can be done by using assessment activities (Murtonen et al., 2017).Therefore, the researcher intends to develop a literacy and numeracy-based test instrument to determine students' literacy and numeracy abilities by comparing students' conceptual understanding during the pre-test and post-test.The items of the instrument were developed by including the context of natural sciences (IPA) in the fifth grade of elementary schools related to education for sustainable development.Students' literacy and numeracy skills in supporting education for sustainable development in real life are expected to improve.Based on the mentioned literature and research, this study focuses on developing items to measure students' conceptual understanding of natural science related to education for sustainable development.Another novelty of this research is in using the Rasch model to analyze students' ability and the items of the instruments, precisely the level of item difficulty.By using the Rasch model analysis, the feasibility of the items of the instruments was analyzed.Previous study stated that the advantages of Rasch model analysis include identifying incorrect answers and judgments as well as predicting missing data based on systematic response patterns (Hamdu et al., 2020).A research using the Rasch Model has been done to analyze the quality of the items of the instruments and students' abilities (Hamdu et al., 2020;Utami et al., 2019).In this research, the size of the change in students' conceptual understanding is also carried out through the pre-post-test design using the stacking and racking analysis of the Rasch Model.By this analysis, it is hoped that (1) teachers will be able to improve the items of the instruments and (2) the size of changes in students' conceptual understanding can be well predicted to help teachers evaluate the existing learning process.Therefore, this research aims to analyze (1) the change in students' conceptual understanding of literacy and numeracy in natural science related to education for sustainable development and (2) the items of the instruments to measure students' literacy and numeracy in the subject of natural science by using Rasch model.

METHOD
The method used in this study is quantitative.In the quantitative method, experimental research is carried out in a pre-experimental design.The pre-experimental designs used in this study were singlegroup designs (one-group pretest-posttest design).Previous study stated that the pre-experimental design did not have a control group to compare with the experimental group (Walliman, 2017).Some did not even test the experimental group before the experiment, and random sample selection could also be omitted.Single-group design (one group pretest-posttest design) in this research involved three steps: (1) conducting a pre-test that measures the dependent variable; (2) applying experimental treatment (O) to the subject; and (3) performing a post-test, re-measure the dependent variable.
Research data is collected by following the mentioned steps: (1) Pre-test: using items of the instruments that measure students' conceptual understanding of literacy and numeracy; (2) Treatment: learning activity to strengthen students' conceptual understanding of literacy and numeracy in natural science related to education for sustainable development.The subject's environmental aspects include air pollution, the amount of contaminant substances in water pollution, and the calculation of oxygen in water.(3) Post-test: using items of the instruments that measure students' conceptual understanding of literacy and numeracy.Both instruments used in the pre-test and post-test data are similar.The instrument was validated through a content-validity test by expert judgment to see items' appropriateness from the content view, fundamental level of the item, language, and test construction.Items' validity is also tested through modern test theory using the Rasch model.Test instrument of students' conceptual understanding in natural science is show in Table 1.The pre-test and post-test data were analyzed using the Rasch model.According to previous study the Rasch model estimates items' difficulty levels and students' abilities before and after the pre-test and post-test (B Sumintono, 2018).Through the Rasch model analysis, test items can show 'good' tests and students' abilities can also be estimated accurately.Moreover, using the Rasch model analytical technique allows researchers to precisely interpret changes due to applying a learning innovation in the pretest-posttest design.The differences associated with applying the experimental treatment were eventually evaluated by comparing the pre-test and post-test scores.The overall design for the singlegroup pre-post-test design of this research is illustrated as show in Figure 1.The research subject in this study was one group of fifth-grade students from two elementary schools.The total number of students is 53.All students participated in the pretest and posttest regarding the focus relevance of the pre-experimental design and the psychometry tools used to analyze the data.The data is analyzed by using the Rasch model.Rasch model is an approach of data analysis that separate (1) items of the instruments based on the difficulty level and (2) the abilities of the test-takers.Rasch model analysis is independent of the number of research samples.In the current curriculum in Indonesia, the fifth grade of elementary schools includes a topic of education for sustainable development in natural science.

Result
Data on the nature and pattern of the items' difficulty level change was obtained through the respondents' items entries.The pre-test items are marked with a letter code (A), and (B) is the code for the post-test items.The entry number column shows the order of the items, while the item column is the items' code.It implies there are 15 items, and items 1 and 16 are the same.The total score column shows the number of students correctly answering the questions.Before knowing the changes in the difficulty level of the items, the researcher first analyzed the differences in the difficulty levels of the items during the pre-test and post-test by paying attention to the standard deviation value (1, 10) and the measured value for each item.The difficulty level of the items was classified into four categories, namely: (1) the group of 'difficult' items, namely > + 1 SD (1, 10), (2) the group of 'difficult' items 0, 0 logit to + 1 SD (1, 10), (3) 'easy' items 0, 0 logit to -1SD (1, 10), and (4) very-easy questions are < -1SD (1, 10).Table 2 shows the difficulty level of the items during the pre-test and post-test on multiple-choice questions.Table 2 shows a change in the level of difficulty of the varied items, some are negative, and some are positive.Items with a negative value (-) become positive (+), as indicated by item number 8. The item becomes a question with a problematic category previously included in the easy category.Moreover, Treatment Postest conversely, item 5 shows that when the pre-test is positive (+) and the post-test is negative (-), the item becomes an 'easy' category.Then on items 14 and 15, when the pre-test showed a negative value (-), then at the post-test, the value was also negative, which was more significant than the standard deviation (-1, 10).The item became a 'very-easy' category.The pattern of changes in the size of the difficulty level of the pretest-postest items that show significant changes can be seen through the difference between the following pretest-posttest measure values.Differences in racking analysis of pre-posttest multiple choice questions are show in Table 3.
The following graph as show in Figure 2 shows the change in the level of difficulty of the items during the pre-posttest.2 shows that the items that experienced a decrease in difficulty (become easier) with significant changes were items 3, 5, 9, 14, and 15.The item with an insignificant decrease was number 11.Then the items with the change in the increase in the difficulty of the items (becomes more complex) with significant change are items number 12 and 8.The increase in items' difficulty is insignificant in items 1, 4, and 13.Items 2, 6, 7, and 10 do not change the items' difficulty level.Items 6 and 10 are still in the 'easy' category, item 2 is still in the 'difficult' category both in the pre-test and post-test, and item 7 is still in the 'difficult' category.This finding shows the possibility of students needing help understanding the concept of natural science related to education for sustainable development if it is associated with everyday problems.The students have difficulty interpreting the theory they have learned into contextual problems.Likewise, for item number 8, which at the time of the post-test became a question with a 'problematic' category, previously (during the pre-test), it was included in the 'easy' category.The question has the same cognitive process as item 2, namely "interpretation and integrity," and has the same context.Table 4, and Table 5 shows the percentage of items' difficulty level along with the cognitive processes that are expected to be achieved by students on each item developed.1 2 3 4 5 6 7 8 9 10 1 12 13 14 15 Pre-test -0 1,1 -0 -1 1,4 -0 3 -0 -1 -0 0,6 0,2 -1 -0 -1  5 are shows that most items with the cognitive 'interpretation and integration' level are challenging for the students.This finding shows that some students still struggle with items at this cognitive level.Moreover,item number 7 is a question of the 'difficult' category during the pretestposttest with the cognitive level category of 'finding information.'

Discussion
If the data difference is more significant than +/-0.5, the decrease or increase can be significant.Changes in items' difficulty levels at the post-test were varied.Some items were positive and negative.Some were fixed (unchanged).When the post-test was identified, ten items had negative scores.Two items had the same score (unchanged) during the pre-test and post-test, with a score of (-0.03) for item 6 and item 10.Then there were five post-test systems.The post-test has a positive score, two of which have the same score as the pre-test and the post-test: item 2 with a logit value (1.06) and item 7 (2.96).Items with a positive score during the post-test indicate the item was intricate.Predictably, only a tiny percentage of the students can correctly answer the question.
The nature of change in the items' difficulty level in the pretest-posttest is shown two factors (Laliyo et al., 2021;Bambang Sumintono & Widhiarso, 2015).Firstly, it is positive if the level of Secondly, the nature of the change in the difficulty level of the pre-test and post-test items with a negative value if, at the time of the pre-test, the difficulty level of certain items changed to be higher (difficult) at the time of the post-test (pre-test value < post-test), or the size of the pretest-posttest items changes from small to larger.One of the factors causing the increasing difficulty of the items is the presence of distractors and the opportunity for students to answer questions by guessing.In fact, according to the result percentage of items' difficulty level along with the cognitive processes, the first cognitive level in literacy and numeracybased questions is 'finding information'.Students' ability to find information includes seeking, accessing, and finding explicit information from discourse (Novitasari et al., 2022;Zhu et al., 2020).In addition to the distractors, another factor that causes students to have difficulty in solving problems is the students' misconceptions about the concepts they have learned.Therefore, students need help with applying what they have learned in everyday life.Students' misconceptions are caused by the complexity of the subject concept when they follow the material (Erman, 2016;Shine & Heath, 2020).Students need to understand the concepts the teacher conveys, so the concepts students understand differ from those scientists define.
Understanding the material students study, but misconceptions can also occur in several forms of the developed items.Most students understood the order.However, if it was applied as story items, some students needed clarification, causing errors in answering them.Since the questions developed by the researchers are literacy and numeracy-based, many questions use various stimuli.One is an item with a stimulus in the form of reading text or discourse, which causes students' misconceptions about the question.It can be caused by students not being used to answering questions with a relatively long text stimulus.Misconceptions arise due to students' learning difficulties (Schoenfeld et al., 2019;Tümay, 2016).Internal factors that cause student learning difficulties, as stated by previous studyinclude physiology, students' five senses, and student interest in learning (Randall, 2019).Meanwhile, external factors that cause students' learning difficulties include the social environment, facilities, and infrastructure, such as learning media.
The issue of misconception is a problem that is difficult to solve.The students' thinking framework is already strong enough, so it is difficult to change.Likewise, with the problem of student learning difficulties, it is not easy to overcome.The teacher's task is to help overcome students' misconceptions and learning difficulties by continuing to innovate in implementing learning that can construct knowledge independently (Li & Schoenfeld, 2019;Wartono et al., 2018).Another thing is to use tools such as learning media to make it easier for students to understand abstract concepts.By selecting relevant learning strategies or media, students can obtain meaningful learning and get an excellent conceptual change to develop their cognitive processes (Bulkani et al., 2022;Dafouz & Smit, 2021).
Furthermore, the decline in students' conceptual understanding based on the research finding is caused by (1) students' misconceptions, (2) types of the items of the instrument, including the use of stimuli, distractors, and stages of cognitive development, (3) students' internal factors, such as the case of students who do not follow the learning process, and (4) teaching materials used by the teacher, precisely the material that has not been appropriately conveyed to cause students' ability improve in solving questions properly and correctly.Therefore, the teacher considers using the Rasch analysis model in evaluating the learning strategies that have been carried out.This analysis can obtain valuable information related to the level of students' ability, the level of items' difficulty, and the quality of the items of the instruments.Teachers can consider the best way to help improve students' conceptual understanding and develop students' cognitive processes.The stacking and Racking Rasch analysis model can be used to (1) analyze changes in students' conceptual understanding abilities, (2) analyze changes in the level of items' difficulty, and (3) analyze the changes in students' ability and items' difficulty by comparing the students' pre-test and post-test results.

CONCLUSION
Based on the pre-test results of students' initial abilities in literacy and numeracy in natural science, there is an increase in students' conceptual understanding of literacy and numeracy related to education for sustainable development.Students with low abilities dominate more than students with high abilities.Applying literacy and numeracy learning can change students' conceptual understanding abilities.This research also shows that students' literacy and numeracy skills tend to increase after the literacy and numeracy-based learning process.This study also shows that the learning innovation process (literacy and numeracy-based) can improve students' conceptual understanding.Based on Rasch model

Figure 1 .
Figure 1.The Single Group Pretest-Posttest Design

Table 1 .
Test Instrument of Students Conceptual Understanding in Natural Science

Table 2 .
Comparison of Difficulty Levels of Multiple-Choice Questions

Table 3 .
Differences in Racking Analysis of Pre-Posttest Multiple Choice Questions

Table 4 .
Percentage, Level of Difficulty of The Items Along with the Cognitive Processes (Pre-Test Multiple Choice Questions)

Table 5 .
Percentage Level of Items' Difficulty Along with the Cognitive Processes (Post-Test Multiple Choice Questions)

Table 4 ,
and Table Ghullam Hamdu / Measuring Changes of Students Conceptual Understanding of Literacy and Numeracy in Natural Science by Using Rasch Model difficulty of certain items at the time of the pre-test changed to lower (easy) at the time of the post-test (pre-test score > post-test), or the logit size of pretest-posttest items changed from large to more minor.