Design of an Electronic Test for Geometry Material Based on Students' Numeracy Literacy and Metacognitive Knowledge

recommendation for a digital-based evaluation instrument that adapts to the scope of the latest curriculum in Indonesia.


INTRODUCTION
"Merdeka Belajar" is a new policy launched by the Ministry of Education, Culture, Research and Technology of the Republic of Indonesia to evaluate the previous curriculum.The independent learning curriculum expects that students can utilize technology in the learning process to produce millennial generations who understand the material quickly by changing textual learning products to digital (Indarta et al., 2022;Xaverius Wartoyo, 2022).One form of digital learning product is an electronic test.Electronic tests are a modern method for evaluating students as an ideal alternative to paper-based tests (Alyahya & Almutairi, 2019;Marasulova, 2019).One of the advantages of the electronic test is that it can be a catalyst for schools to help students get to know and develop their professional abilities related to technology in learning (Harrison et al., 2017).According to teachers, electronic exams are more efficient by modifying conventional exam techniques, and assisting in a more assertive assessment process (Aladl, 2020;Zahedi et al., 2021).Meanwhile, according to the students, the electronic test was valuable and exciting because it included elements of text, video, photos, and illustrations appropriate to the questions' context.Student interest in digital-based evaluation systems does have a good effect on their learning motivation.The reason to learn has indeed increased along with the increasing sophistication of digital equipment in learning (Mada & Anharudin, 2019;Panagiotidis et al., 2018).
Digitalization in learning in Indonesia is essential, given the relationship between mathematical literacy skills and the PISA framework.In the PISA 2021 framework, literacy skills describe an individual's capacity to reason mathematically and use mathematical concepts, procedures, facts, and tools to explain and predict a phenomenon (Qodr et al., 2021).For this reason, the teacher, as one of the determinants of the quality of education, encourages students to have literacy skills.The concept of literacy recognizes the importance of students developing an understanding of various mathematical concepts and processes in exploring the benefits of mathematics in the real world (Asrizal et al., 2023;Umbara & Suryadi, 2019).Students must be aware of their mathematical literacy skills, contained in one of their numeracy skills.Mastery of specific numeracy skills is necessary for students.Terms must become an essential standard because numeracy skills are a way of managing various data and numbers to evaluate a statement based on specific contextual limitations.In addition, students' different numeracy skills can be the basis for teachers to design a lesson that directs students to get used to finding solutions to math problems.
A student with good numeracy skills and a broad conceptual understanding can explain a specific reason for solving mathematical problems (Awofala & Blessing, 2014;Via et al., 2021).A student with good numeracy skills and a broad conceptual understanding can explain a specific reason for solving mathematical problems.There are several stages in solving mathematical problems: understanding the problem, developing a problemsolving plan, implementing a problem-solving strategy, and re-examining or evaluating (Ortiz, 2016;Riyadi et al., 2021;Zhang & Seepho, 2013).Stages in problem-solving are also related to metacognitive knowledge, defined as students' ability to know, control, and evaluate their thinking processes.For this reason, it is essential to first analyze students' metacognitive knowledge before compiling various learning instruments.The final result leads to the achievement of learning objectives according to the abilities of each student.
Metacognitive knowledge is the various components that we know about ourselves.The subcomponents of metacognitive knowledge are declarative, procedural, and conditional knowledge.In learning mathematics, metacognitive knowledge contributes, primarily referring to students' learning techniques and processes and their ideas about the nature of mathematics (Stephanou & Mpiontini, 2017;Tian et al., 2018).In addition, the categories of people, tasks, and strategies from metacognitive knowledge are essential in mathematics performance.For this reason, it is significant to make metacognitive knowledge the basis of teaching, especially in teaching mathematics.Several studies have shown that when working on math practice questions, students have not explored their metacognitive knowledge (Chytry, 2020;Schneider & Artelt, 2010;William & Maat, 2020).They simply obtain the test results and are unconcerned with the reasons for their errors.
Several researchers have studied electronic test design, students' numeracy literacy skills, and metacognitive knowledge.Research shows that students feel optimal benefits and are more comfortable working on evaluation questions using the electronic test form.Furthermore, the use of electronic tests can reduce material costs for purchasing paper and ink (Abdul et al., 2022;Aldalalah, 2019).In addition, the level of student anxiety when evaluating the digital form was also low.However, the numeracy skills of Indonesian students so far still show poor quality (Simarmata, 2023;Widodo, 2021).
Meanwhile, the ability to count is an important factor in solving problems that form the basis of mathematics.However, the literacy skills of Indonesian students are also still at the lowest rank.In addition, according to several studies, the minimum competency assessment for students is not based on the ability to understand primary material but on numeracy skills (Susongko, 2021;Tensa Novela & Widyastono, 2022).In fact, literacy skills are competencies to meet the demands of modern society (Bolstad, 2023).In other studies regarding metacognitive knowledge, one of the components to improve students' thinking skills is the development of metacognitive knowledge (Mas'ud & Malik, 2022;Parlan. et al., 2017).For this reason, when the teacher wants to achieve learning goals, the teacher needs to encourage students to use their metacognitive knowledge systematically (Bolstad, 2023;Putra et al., 2021).
Based on various descriptions of the previous background, it is necessary to study the creation of electronic tests for students' numeracy skills based on metacognitive knowledge, which other researchers have never done before.Therefore, this study aims to design an electronic test for the math skills of junior high school students based on valid and practical metacognitive knowledge.The design of this electronic test is very helpful as a form of evaluation of curriculum implementation that wants to realize digitization in learning.In addition, the ability to count is essential in solving students' mathematical problems, where all problems in advanced mathematics prioritize mathematical problem-solving.One way to improve students' numeracy skills is to make metacognitive knowledge the basis for preparing practice questions.

METHOD
This study is a development-research using the Research and Development (R&D) method.The development model used is the ADDIE model.The reason for choosing the ADDIE model is that many educational designers use this instructional design model to develop teaching, and the results of each instructional step support each other for the next stage (Spatioti et al., 2022).ADDIE's research stages consist of five instructional step processes: the Analysis, Design, Development, Implementation, and Evaluation phases (Yeh & Tseng, 2019).The five instructional process steps allow teachers to design instructions to include overall feedback (Kathryn et al., 2016;Yeh & Tseng, 2019).This study took four development steps due to the limited research time: Analysis, Design, Development, and Implementation.The electronic test design in this study focuses on test questions that aim to maximize students' numeracy skills in terms of metacognitive knowledge.This research instrument consisted of a test device based on numeracy literacy skills, an interview sheet, and a test device validation sheet.Using test instruments helps determine students' numeracy literacy abilities so that teachers can provide appropriate treatment in learning by learning objectives and students' proficiency in understanding mathematical problems related to numeracy literacy.Meanwhile, the preparation of interview sheets aims to determine students' metacognitive knowledge so that teachers can carry out learning activities by understanding students' characteristics and their comfort in learning.For validation instruments, it is helpful to determine the suitability of test equipment that has gone through an analysis process from experts in the field.Accurate use of the three research instruments positively impacts research results regarding numeracy literacy abilities based on students' metacognitive knowledge.
The subjects in this study were 26 students of class IX junior high school at one of the junior high schools in Cirebon City.The subject selection technique used purposive sampling, considering three indicators of metacognitive ability.The researchers obtained metacognitive abilities based on the results of initial interviews and a preliminary test in the form of a numeracy ability test.The initial interview results can provide information regarding the dominance of student characteristics according to each metacognitive indicator.Finally, students focused on three groups that characterized metacognitive indicators.After obtaining information regarding grouping students based on the supremacy of their metacognitive knowledge indicators, each group received a pretest based on numeracy literacy abilities.The pre-test results provide information about students' basic numeracy literacy abilities so that they can provide considerations for preparing e-test questions by the characteristics of students' metacognitive knowledge and numeracy literacy abilities.Then, after going through the pre-test phase, the research continued preparing e-test questions, then continued to the validation phase of the e-test instrument and analysis of the results.

Result
The researchers began the study by analyzing students' numeracy literacy skills based on metacognitive knowledge in solving geometry material problems.The first stage in this study was to determine the nine research participants by conducting initial interviews with the subjects.From the results of metacognitive knowledge interviews, there were nine students with three dominant students on planning indicators (planning), three dominant students on comprehension monitoring indicators, and three dominant students on evaluation indicators (evaluation).The findings regarding students' metacognitive knowledge are shown in Table 1.A student with dominance on evaluation indicators, namely S7, only completes four planning and monitoring sub-indicators of understanding, while S8 and S9 only fulfil six planning sub-indicators and five monitoring sub-indicators.In the next stage, nine subjects took the geometry material test, which consisted of three description questions.The researchers used these three questions to determine students' ability to use symbols in spatial and form material, to analyze information from graphs, tables and diagrams, as well as skills in space, form, and measurement.Researchers analyzed the results of student answers, then conducted interviews with each research subject.Based on the description and analysis of data for numeracy literacy skills in subjects S1, S2, and S3 on questions 1, 2, and 3, they dominate planning indicators in solving HOTS questions as show in Table 2.

Table 2. Numeracy Literacy Ability S1, S2, and S3
In Table 2, students who are dominant on planning indicators are able to analyze information from pictures, graphs, tables and diagrams, are adequate in using symbols in space and material form, and are hampered in spatial concept skills, shapes and measurements.Meanwhile, the conclusion from the description and data analysis of questions 1, 2, and 3 from the answers to subjects S4, S5, and S6 is that students' numeracy skills dominate indicators of monitoring comprehension in solving questions as show in Table 3.

Table 3. Numeracy Literacy Ability S4, S5, And S6
In Table 3, students who are dominant on the monitoring indicator on comprehension monitoring are quite capable of analyzing information from pictures, graphs, tables, and diagrams.They are exceptionally talented in using symbols in material space and form, and are hindered in terms of spatial, shape, and measurement concept skills.Meanwhile, the conclusion from the description and analysis of the data on questions 1, 2, and 3 from the answers of subjects S7, S8, and S9 is that students' numeracy skills dominate indicators of monitoring comprehension in solving questions as show in Table 4. Students with numeracy literacy skills, namely those who can analyze information in various forms, can use multiple numbers and mathematical symbols to solve a problem and interpret problem analysis result to predict and make decisions appropriately.Based on the results of written answers and interviews with all subjects, the researchers concluded that students could analyze information from the questions.Of all these answers, students can mention any information in the question.Then, they are also exceptionally skilled at solving test questions using various numbers and mathematical symbols.Based on the results of written answers, students used several mathematical symbols to solve test questions.However, students still could not mention these symbols at the interview stage.
In addition, students are adequately able to interpret the results of problem analysis to make decisions and be able to solve spatial and form material problems appropriately.Based on the results of the interviews as a whole, students were able to explain the strategies they applied, but at the time of writing the answers were still not quite right.Therefore, students with metacognitive knowledge, namely the aspects of planning, understanding monitoring, and evaluation are adequately capable of numeracy literacy based on the results of solving geometry material test questions.After knowing the students' numeracy literacy skills at the preliminary analysis stage, the research then entered the initial investigation, design, and limited realization stages with the following results: First, Analysis Phase.The results the analysis of the syllabus and lesson plans for Geometry material conclude that, learning is still one way because the discussion is not yet active.This learning passivity causes students to be hampered in representing their skills in solving abstract practice questions.In addition, students had difficulty finding initial ideas for solving problems; learning does not get adequate teaching material support.The available teaching materials are still not able to reduce students' difficulties in understanding the material; and the average score of the completeness of student learning in geometry material in mathematics, especially the understanding of the concept of fields and space, is still low.The following analysis results collect references to developing teaching materials that can overcome students' difficulties in finding initial ideas for solving problems and understanding abstract material.The pre-test results show the classes that will become the next phases below.
Second, Design Phase.In this phase, the researchers analyzed the material contained in the subject matter of geometry mathematics.Then, field and space materials are selected as prerequisite materials, and students must master them.The researchers then designed an e-test based on HOTS questions by paying attention to the stages of students' metacognitive knowledge which students can access via online.Figure 1 shows the electronic test design using Moodle.In this phase, the results show that the e-test instrument has excellent criteria.In the e-test, the revision includes the display design of the e-test and the presentation of images.A focused explanation of the modification of teaching materials is shown in Table 6.

Before Revision
After Revision The cover design is less attractive The cover design has improved according to validators' input and students' needs.Pictures of field views and geometric spaces are less clear.
According to validator input and student needs validators' input and students' needs, the field view images and geometry spaces have gone through design improvements.
Base on Table 6 student response data via questionnaires were submitted after the implementation of learning was completed in 10 meetings.Then, calculations and data analysis are continued based on percentages.Then, the calculation results show the positive percentage of all questionnaires.Based on the calculations, the average percentage of students' positive responses was 82.63%, while the negative responses were 17.37%.Because the average percentage of students' positive responses was ≥ 80%, the final conclusion showed that students gave positive and good responses to the use of the e-test on geometry material.The results of a good and positive student response questionnaire show that the e-test meets practical criteria.

Discussion
The design of an electronic test for numeracy literacy skills based on students' metacognitive knowledge focuses on applying technology in learning, compiling evaluation questions for mastery of basic math skills, and the efficiency of using paper during exams.These three focuses of this study provide space for the learning process to develop teachers' abilities, improve students' mathematical skills, and adapt to technological developments in teaching.Changes in the teaching process by integrating technology, one of which is digital evaluation, generate benefits in learning (Alyahya & Almutairi, 2019;Firmasari & Santi, 2017).This advantage allows students to understand and develop digital knowledge that they can do at home and school because technology opens spaces for learning anywhere and anytime (Dhawan, 2020;Odede & Jiyane, 2019).
In the new Indonesian curriculum, which is fully implemented in Indonesia starting in 2014, learning must adapt to developments in technological developments.In this study, the preparation of the e-test as one of the technology-based learning tools aims to produce an e-test instrument design that meets valid and practical criteria (Amir et al., 2020;Puspitawangi et al., 2016).The broader goal is to obtain a pattern of implementing mathematics learning in the long term.The importance of the meaning of the validity of the test instrument lies in the high quality of the instrument and which avoids failure from a more comprehensive implementation in learning (Hernández-Rodríguez et al., 2021;Johnson et al., 2019).Then, devices become more suitable and researchers can be held accountable for instrument design.One of the aims of expert validation of research instruments is to support adequate instrument design and support research conclusions.
The general assessment of the validators emphasizes the web cover design for the e-test and more explicit geometric images.Completeness in designing the e-test provides an interesting e-test presentation.This provides an understanding to students in completing the e-test according to the purpose of preparing the instrument.The revision of the results from the input of the validators provides compatibility between the application of the applicable curriculum, especially in learning mathematics, and the teachers' desire to design learning that focuses on digital-based evaluation instruments.The teachers' goal to develop this digital-based evaluation instrument refers to appropriate technology to meet learning objectives (Haleem et al., 2022;Hwang et al., 2022).
Validation shows the e-test instrument's feasibility to enter the next phase, namely practicality.In the practicality phase, the students' positive responses showed that the e-test instrument was following the classroom's evaluation needs and met students' digital learning standards.Students have responded positively, which means that the quality and quantity of the e-test instrument is appropriate to represent students' understanding of geometry material and mastery of technology in learning.Even though digital-based evaluation requires a higher student focus and better expertise in using technology, students feel that using technology in education is more exciting and flexible.
Technological sophistication still has a position that must be integrated with students' numeracy skills.Numeracy literacy has the highest quality scale and direct recognition within the PISA 2021 framework.This recognition is central because numeracy literacy is very important for dealing with problems related to mathematics and its application in everyday life (Megawati & Sutarto, 2021;Situmorang & Sinaga, 2022).In fact, according to previous study students with solid numeracy skills will improve their abilities in other fields (Barham et al., 2019).In addition, other studies state that increasing students' numeracy literacy can positively affect selfconfidence and ability to handle daily tasks and provide longer learning motivation (Md-Ali et al., 2016).Knowledge of numeracy literacy in learning mathematics has a good effect on students.Therefore, these skills must be integrated into teaching materials and evaluation questions.
The introduction of technology in learning, especially digital-based, challenges teachers and students.Meanwhile, the learning environment has undergone a remarkable transition in recent years as a result of technological growth (Alnasib, 2023;Jacob & Karn, 2003).Previous study show show that digital learning tools in mathematics education must pay attention to the details of the classroom situation (Viberg et al., 2023).Providing technology to students does not only go in one direction, but requires collaboration between students and teachers to encourage situational learning practices that are conducive to achieving learning goals.Meanwhile, other research shows that integrating technology into learning mathematics can increase motivation and attract students' interest in learning (Çerkini & Zejnullahu, 2022).Besides that, the study by other study concludes that metacognitive knowledge contributes specifically to improving students' intrinsic motivation in learning mathematics (Tian et al., 2018).Studies by several experts show that the design of instruments in digital-based mathematics learning with an emphasis on students' metacognitive knowledge is essential to be integrated with teaching and learning activities in class.
The strength of this research is digitization in the format of a learning management system that focuses on assessment based on numeracy literacy and metacognitive knowledge, indicating that education for the future must have a strong foundation in terms of technology, basic knowledge, and student psychology.Meanwhile, this strength can have implications for aligning curriculum developments that want students to learn independently and reduce paper consumption as a mandatory tool in conducting school evaluations.However, in its implementation, the electronic design of this test still has limitations, namely on the material side, and the new implementation location is in the city of Cirebon.For future research, designing the electron test based on students' numeracy, literacy, and metacognitive knowledge can positively affect more comprehensive users and additions to the mathematics material section.

CONCLUSION
The design of an electronic test for students' numeracy skills based on valuable and practical metacognitive knowledge provides an alternative digital-based evaluation instrument that is attractive to students and follows the latest curriculum developments.The test design focuses on displaying web-based evaluation questions to be more effective and efficient in terms of paper use and appearance.In addition, based on input from validators and students' responses, the quality and quantity of test instruments adhere to field requirements and students' mastery of technology.Integrating numeracy literacy and metacognitive abilities into digital-based evaluation questions provides a new nuance in learning in the current era of technological sophistication.In the following study, the researchers hope to implement it more broadly with a more comprehensive range of subjects from schools.The results of broader research scope will have an even more significant positive effect on the advancement of education in Indonesia's digital era, which does not only look at results but also pays attention to processes and prioritizes students' primary and psychological abilities.

Table 1 .
Findings of Students' Metacognitive Knowledge Table 1 presents the findings, namely S1, S2 and S3 are dominant in planning indicators by fulfilling all planning sub-indicators.S4, S5, and S6 are prevalent on the monitoring indicator of comprehension monitoring, with S4, S5, and S6 fulfilling six of the seven monitoring sub-indicators on comprehension monitoring.S7, S8, and S9 dominate the evaluation indicators, with S7, S8, and S9 fulfilling five evaluation sub-indicators.A student in planning indicators, namely S1, only fulfills three monitoring sub-indicators, namely understanding indicators and evaluation indicators.In comparison, S1 and S2 achieved six understanding monitoring sub-indicators and met four evaluation sub-indicators.Students with dominance on monitoring indicators of knowledge, namely S4, S5 and S6, only fulfil five planning sub-indicators, while S4 and S6 fulfil three evaluation sub-indicators, and S5 fulfils four evaluation sub-indicators.

Table 4 ,
students who are dominant on evaluation indicators can analyze information from pictures, graphs, tables, and diagrams, cannot use symbols in spatial and form material, and are moderately capable in spatial, form, and measurement concept skills.

Table 5 .
Recapitulation of Validator Scores for the E-test