Preservice Elementary Teachers' Noticing Formative Assessment

Pre-service teachers learn a lot about educational practice, especially regarding formative assessment, through noticing. For this reason, pre-service teachers’ noticing ability determines the learning process regarding formative assessment. Thus, understanding and analyzing pre-service teachers’ noticing skills is very important. This research is a descriptive qualitative research which aims to analyze the observations of pre-service teachers regarding formative assessment using a video. Data collection was carried out using video task analysis regarding formative assessment. This study involved 69 pre-service elementary teachers. The results of the task analysis were coded with the theoretical coding method. The results of the coding were then analyzed descriptively. The focus of noticing for pre-service elementary teachers is the greatest (71%) on the questioning loop aspect, while other aspects are less than 50% of the participants. No one  pre-service elementary teachers observes the five aspects of formative assessment. For the quality of noticing an average of 2.25-2.85 or between the descriptive level and validation.The characteristics of the focus of pre-service elementary teachers’ noticing regarding formative assessment tend to focus on the questioning aspect and are not comprehensive. Most of the noticing quality of pre-service elementary teachers regarding formative assessment is still at the descriptive and validation level.


INTRODUCTION
In teacher education, Pre-service Teachers (PSTs) learn a lot about pedagogy by noticing learning practices, both others and themselves (for example in microteaching and even field experience). Thus, noticing is one of the important components in teacher education that allows PSTs to capture a lot of learning from existing practices. However, noticing has not received important attention. Noticing ability is the ability to attend to and reason about teaching and learning (Mason, 2020;Walkoe et al., 2020). The noticing ability allows the teacher to properly capture empirical evidence about student thinking, the learning environment, student behavior, and even the development of learning trajectories. This evidence is very important in making decisions both in real time in the classroom and in analyzing the implementation of learning. In fact, noticing skills are believed to encourage student-oriented learning in developing mathematical abilities through the use of empirical evidence on student thinking, classroom environment, etc. (Lomicka & Ducate, 2021;Mason, 2020;NCTM, 2014;Xu et al., 2019). Therefore, Teacher education programs must facilitate and support Pre-service Teachers (PSTs) to develop their noticing skills (Fernández et al., 2020;Fisher et al., 2019;Lee, 2019; Van Es, 2011;Wiens et al., 2021). Noticing is an act of observing or recognizing something in the context of learning. Various literatures and researchers define noticing as the ability to attend to important teaching features, to consider what is observed in a meaningful way and to decide how to respond (Baird et al., 2017;Kang & van Es, 2019;Larochelle et al., 2019;Lee, 2019;Mason, 2020;Santagata et al., 2021;Wiens et al., 2021;Xu et al., 2019;. There are two categories in understanding noticing, namely "what" teachers notice and "how" teachers notice ( Van Es, 2011).
In terms of "what" Teachers notice, various studies suggest using an ambitious pedagogy framework ( Van Es et al., 2017). Ambitious pedagogy refers to investigations that focus on student work as a center for learning activities with the aim of students developing procedural fluency, deep and lasting understanding of mathematics, and positive dispositions and identities as mathematics learners ( Van Es et al., 2017). One of the components of ambitious pedagogy is formative assessment. This study only took one component (namely formative assessment) because it wanted to encourage PSTs, as beginners, to focus more on studying each part of ambitious pedagogy. By studying the components, PSTs will make it easier to understand the whole part of ambitious pedagogy. One way to develop and analyze noticing skills is to use video cases (Lee, 2018). Various studies have proven that video is a powerful tool for analyzing noticing ability (Gaudin & Chaliès, 2015;Gotwals et al., 2015;König et al., 2014;Kosko et al., 2020;Lebak, 2018;Roller, 2016;Santagata & Yeh, 2014;Star & Strickland, 2008;Tripp & Rich, 2012;Ulusoy & Çakiroğlu, 2018;van Es et al., 2017;Walkoe, 2014;Warshauer et al., 2021). In formative assessment context, video is a tool that can bridge the complexity of the theory behind formative assessment with what is seen in the classroom (Kang & van Es, 2019;Walkoe et al., 2020;. Videotapes can capture various components of teacher practice and allow researchers to analyze lessons from multiple perspectives. Videotapes allow us to slow down, unload, and critically examine formative assessment practices and routines in the classroom (Gotwals et al., 2015;Lebak, 2018). Because interactions between teachers and students help characterize expertise in formative assessment practice, videos provide ample opportunities for researchers to examine the nuances in these interactions.
Although research on PSTs noticing has been very much done, specifically analyzing the noticing regarding the formative assessment has not been carried out extensively. From existing research, examine teacher noticing of specific interactions related to the formative assessment process during teachers' participation in an on-line video club (Lebak, 2018). As a result, in terms of formative assessment, the focus of teacher noticing is on "questions". While other research analyzing noticing in relation to the ability of PSTs to carry out assessments (Gotwals et al., 2015;Zambak & Magiera, 2018). The various studies mentioned above have not analyzed PSTs noticing regarding formative assessment using a video case. In fact, the noticing ability regarding formative assessment in PSTs is very important to be developed since in teacher education. For this reason, this study aims to analyze PSTs noticing regarding formative assessment using video. In more detail, this study will explore the focus (what) and quality (how) of PSTs noticing regarding formative assessment. The purpose of this study was to analyze the focus and quality of the PSTs noticing video regarding formative assessment.

METHOD
This study used a descriptive qualitative research design. The research was conducted at the Faculty of Education, Pelita Harapan University. The participants in this research are 69 pre-service elementary teachers (10 males and 59 females). Participants were selected purposively because they wanted to analyze noticing skills regarding the formative assessment of Pre-Service Elementary Teachers (PSTs). Data collection was carried out using video task analysis. Video task analysis means that PSTs is given a video recording of the lessons from Inside Mathematics. Inside Mathematics is a platform that provides a resource for educators around the world. This platform was developed by The Charles A. Dana Center, The University of Texas at Austin. The video recording provided is a clip of the formative assessment implementation in mathematics class. This video is a recording of the implementation of formative assessment in grade 3 elementary school on the topic of multiplication and division. There are 7 (seven) video clips with a total of 45 minutes analyzed by PSTs. PSTs already has sufficient English skills to listen to and understand the video. In addition, a transcript of the video is also provided. After watching the video, PSTs will write down the results of their noticing by following the following guiding questions: (a) What did you observe about the Formative Assessment? (b) What kind of formative assessment did you observe from the video? (c) Interpretive questions (What does it mean to you, why did it happen, how did it happen in terms of teaching and learning theory); (d) Based on information from questions (a), (b) and (c), what would you do if you were that teacher or would you do when you were the teacher? The results of the noticing are then submitted by the PSTs in Moodle.
Data analysis by coding the PSTs noticing results. The coding technique was done theoretical coding. Theoretical coding is used because researchers will use a theoretical framework in determining coding (Cohen et al., 2017). In coding focus noticing, the researcher uses a research-based FA practice progression framework (Gotwals et al., 2015;Lebak, 2018). Based on this framework, the focus of noticing on formative assessment is categorized into (a) learning targets, (b) questioning loops; (c) intentional checks; (d) feedback loop; (e) adapting the instruction. Meanwhile, for the coding of noticing quality, the framework of Lebak (2018) used. This framework categorizes the quality of noticing into artificial, descriptive, validation, analytical validation. The categories of noticing quality will be given a 1-4 scale, namely 1-artificial, 2-descriptive, 3-validation and 4-analytical validation. The validation of the coding results was carried out by comparing the results of the coding between the research teams asking for help from other educational researchers, by randomly selecting ten noticing documents and then coding the samples. These results will be compared with the coding results of the research team. The coding framework for PSTs noticing in the formative assessment is shown in Table 1. After the coding process is done, descriptive statistics (frequency, mean, maximum value, minimum value and standard deviation) will be used to provide a quantitative descriptive about the focus and quality of the PSTs noticing for the formative assessment. 4-Analitycal Validation Validation means that reflection contains comments that explicitly tell the implementation of the formative assessment, and also provides positive affirmation (evidence) of the description coupled with analysis and evidence that explains why the practice is good. As well as suggestions for improvements or assessments that should be.
3-Validation Validation means that reflection contains comments that tell explicitly about the implementation of the formative assessment, and also provides positive affirmation (evidence) of the description. 2-Descriptive noticing Descriptive noticing means that reflection contains comments that tell explicitly about the implementation of the formative assessment. 1-Artificial Noticing Comments on reflection only reveal things related to formative assessment but not explicitly and specifically on events regarding formative assessment.

Result
Data analysis of preservice elementary teachers' (PSTs) noticing formative assessment begins with coding on a video task analysis. Table 2 is an example of coding from the results of the PSTs video task analysis. The results of the video task analysis were categorized as "questioning" because the PSTs wrote about questions (questions) that were given by the teacher to students during the formative assessment process. In addition, PSTs also recorded students' responses to these questions (questions). In accordance with the coding framework described above, the results of this video task analysis fall into the "questioning" category. The formative test used by the teacher in the video is really interesting because the teacher only gives one question and students try and find out for themselves, it can be seen which students understand the concept of the material and which ones still need help… .. that means students are not restrained to always have answers according to the wishes of the teacher and according to the stages of the teacher, but students can freely have their own hypotheses and be able to prove the truth. Because in the video, the teacher provides space for a variety of different answers and different ways of solving each student, as well as listening to their opinions

Validation
The results of characteristics of pre-service teachers' focus noticing data can be seen in Table 3. Table 3 above shows only 13 (18.8%) PSTs that observe how the learning targets aspect occurs in the learning process (video recording). In other words, most PSTs do not pay attention to the teacher's actions in delivering learning targets when implementing formative assessments. In the questioning loop aspect, 49 (71%) PSTs observed this aspect. Thus, more than 50% of PSTs were aware of questions, student responses to questions, the quality of questions given by the teacher. There were 31 (44.9%) PSTs that observed the intentional checks aspect. This means that there are 31 PSTs who observe activities such as hand signals, asking students to write on the blackboard to check understanding. Meanwhile, for the feedback loop aspect, 16 (23.2%) PSTs paid attention to this. That is, There are things about the feedback the teacher gives, for example, responses to student income or answers, both verbal and non-verbal. In the aspect of adapting the instruction, there were 27 (39.1%) PSTs who observed this in the results of the video task analysis. This means that the PSTs observe activities such as the teacher giving re-explanation, providing guidance, re-teaching. These results indicate and it means that PSTs tend to focus on the questioning loop aspect when observing formative assessment in the classroom. Important things in the formative assessment as a followup to the questioning loop, (such as feedback loops, adapting the instruction) received little attention from PSTs when doing video task analysis. This means that according to the PSTs, the most important thing in formative assessment is questioning, namely questions, question form, student answers. From the comprehensive level of focus on noticing, no PSTs are aware of and observe the five aspects of the formative assessment. In fact, there were 7 (10,2%) PSTs who did not observe a single aspect regarding the formative assessment. This data means that PSTs are not comprehensive in observing and understanding formative assessment. PSTs only focus on certain aspects in the formative assessment. Data regarding the level of comprehensiveness is shown in Table 4. The results of pre-service teachers' noticing quality data can be seen in table 5. Table 5 shows the descriptive statistics of the quality of the PSTs coding regarding the formative assessment. Noticing quality analysis was carried out on PSTs that observed these aspects (the number of PSTs observing in each aspect is shown in Table 3). From Table 5, it can be seen that the average PSTs noticing quality regarding formative assessment is 2.85. This means that the noticing quality of PSTs on the learning targets aspect has exceeded the descriptive level. The four other aspects (question loop, intentional checks, feedback loop, adapting the instruction) have a sequential average of 2.49, 2.31, 2.25, 1.64. This shows that the noticing quality of PSTs has also exceeded the descriptive level, but has not yet reached the validation level. This means that for the five aspects, the average PSTs still describe the events that appear in the video task regarding formative assessment. On average, PSTs have not validated this description. When viewed in detail (Table 6), the most PSTs frequencies are at the validation level for aspects of learning targets, questioning loops, intentional checks, adapting the instruction. Meanwhile, most of the feedback loop aspects are at the descriptive level. The data regarding the quality of the noticing PSTs above means that most of the PSTs have a noticing quality between descriptive level and validation. This shows that most of the PSTs have been able to describe the conditions that occur in the classroom, whereas most of them are also able to provide evidence from the descriptive.

Discussion
The results of data analysis that have been carried out indicate that PSTs noticing characteristics regarding formative assessment tend to only focus on the questioning loop aspect. This shows that PSTs see formative assessment as more likely in the process of assessing student understanding (giving questions, form, student responses). In addition, the PSTs noticing characteristics regarding formative assessment are not comprehensive. This is indicated by the absence of PSTs that observed all aspects of the formative assessment, even only 5% observed four aspects. The results of this study are consistent with previous studies (Gotwals et al., 2015;Lebak, 2018). The results of this study are in line with the other research, in the context of teachers, it shows that the focus of teacher noticing regarding formative assessment is on the questioning aspect (Lebak, 2018;Warshauer et al., 2021). Learning targets and feedback are the components that are least observed (Lebak, 2018;Walkoe et al., 2020). Thus the results of this study further strengthen the results of previous studies regarding pre-service teachers' focus noticing. Thus, the results of this study further strengthen and expand the context of the results, namely in the context of pre-service teachers (Lebak, 2018). In addition, the results of this study also provide an overview of the level of comprehensiveness of the noticing focus that was not studied by Lebak (2018). The research does not specifically provide an analysis of the level of comprehensiveness of the noticing focus. However, the results of his research which showed that there was one formative assessment component which was only observed by 1.59% of the participants showed that the level of comprehensiveness of the noticing focus of pre-service teachers was also very low. This is in line with the results of this study.
Noticing aspects are not comprehensive and tend to focus on questioning aspects can be influenced by PSTs' belief in formative assessment. Teachers and PSTs often see formative assessment as the same thing as summative assessment (Brown et al., 2015;Gebril & Brown, 2014;Martínez-Sierra et al., 2020;Nortvedt & Buchholtz, 2018). As a result, essential matters and objectives of formative assessment, such as feedback, adapting the instruction are often forgotten by PSTs. This is also supported by other research which reveals the existence of unproductive beliefs regarding assessment, namely the focus on tests rather than on things such as feedback and adapting the instruction (NCTM, 2014). Although various studies have revealed that video is very helpful for PSTs in noticing, there are also other studies which reveal that the focus of noticing is less broad when observing videos than direct observation (Ellis et al., 2015;Kosko et al., 2020;Lebak, 2018;Magiera & Zambak, 2021;Santagata & Yeh, 2014;van Es et al., 2017;van Es & Sherin, 2008;Walkoe, 2014). This means that the use of video can also affect the comprehensiveness of PSTs noticing. Based on the result of data, the quality of the PSTs noticing was mostly at the descriptive and validation levels. This finding is consistent with previous research, which revealed that the noticing quality level of PSTs tends to be at the validation level (Lebak, 2018). This could be due to the lack of comprehensive (detail) aspects observed by PSTs regarding their formative assessment. The results of previous research revealed that when the focus of the noticing was getting more detailed, the quality of the noticing would be better (Stockero et al., 2017;van Es, 2011).
Based on the results and findings above, this research has contributed in several ways. First, this research contributes in terms of context. Unlike the previous research which analyzed teacher noticing, this study analyzed the noticing of PSTs. Thus, this study expands the noticing context of the formative assessment from teachers to PSTs. This is important, because noticing skills must also be developed since in teacher education (PSTs) (Fernández et al., 2020;Fisher et al., 2019;Guner & Akyuz, 2020;Lau & Man, 2018;Lee, 2019;Lomicka & Ducate, 2021;Santagata et al., 2021;Wiens et al., 2021). Second, it is different from previous studies which used video recordings of their friends' lessons (formative assessment implementation) or their own lessons (Gotwals et al., 2015;Lebak, 2018;Schwarts & Karsenty, 2020). This study uses video recordings of other teachers' lessons (formative assessment implementation) that are not recognized by the participants. The implication is that PSTs are more flexible in providing reviews of videos, so that the noticing will be more authentic. This is in accordance with other research, that PSTs tend to prefer to give praise rather than critical analysis, questions when observing the practice of learning (explanation) carried out by their friends (Ellis et al., 2015). Therefore, in this case it can be said that the use of others' learning videos (not their friends) is more likely to appear more authentic noticing (both focus and quality). However, there are limitations to this study. First, the learning context in the video that was observed. The learning context in other countries may influence PSTs' noticing of the implementation of the formative assessment, for example in terms of language, classroom conditions and even the learning media used. Second, the results of this study have not revealed the factors causing the incomprehensive of PSTs' focus noticing. Although theoretically, knowledge and perceptions regarding formative assessment will affect noticing ability, this has not been analyzed empirically.

CONCLUSION
Based on the results, the conclusion of this study is that the focus characteristic of PSTs noticing regarding formative assessment was not comprehensive that it tends to focus on the questioning aspect, and PSTs noticing quality was still at the descriptive and validation level. The results of this study have implications for developing the understanding and practice regarding formative assessment. The incomprehensive focus of noticing shows the importance of developing PSTs' understanding regarding the important aspects of formative assessment. In addition, video noticing can be used to develop and analysis PSTs' understanding of formative assessment.