Measuring Meaningful English Learning Through Video Production: An Instrument Development Report

Students’ low motivation in learning English as a foreign language is an age-old problem for English teachers. To overcome this problem teacher must design and present the learning process in such way which minimize students’ fear and anxiety. One of the more interesting ways to teach English is to use videos in class. The present study aims at developing an instrument to measure students’ perception of the meaningfulness of English learning using video production. This instrument is developed to facilitate the measurement of students’ perception of the meaningfulness of English learning using video. The construct used as the basis of the instrument development is the Meaningful Learning using digital media and video proposed by Paivi Karppinen. The instrument is developed using Borg’s seven steps instrument development process which includes defining the construct to be measured, defining the target population, reviewing related test, developing prototypes (blueprint and test items), evaluating the prototype, revising the instrument


INTRODUCTION
The advances in technology have brought the world even closer with the use of English as the medium.This background has pushed English into an increasingly important position as a language.Nevertheless, despite the fact that the students may very well be in need of English for their future, there are still many students who decidedly avoid English (Evans et al., 2021;Ryan, 1998;Zein, S, 2017).Students avoided English since they view English as a very difficult subject.Such perception has demotivated students from even trying to learn the language.Students' low motivation in learning English as a foreign language is an age-old problem for English teachers (Pertiwi & Sudarsono, 2015;Putra, 2021;Uddiniyah & Silfia, 2019).To overcome this problem teacher must design and present the learning process in such ways which minimize students' fear and anxiety (Jiménez & Rose, 2010;Wachob, 2006).One of the more interesting ways to teach English is to use videos in class.
Previous researches on the use of video in ELT, however, have focused mostly on the use of video viewing English learning.Video viewing in the classroom is more than just a novel approach when compared to the staple textbook-oriented and test-driven approach to everyday teaching (Andini et al., 2018;King, 2002).Video has been proven useful in facilitating language learning and improving motivation (Ariantini et al., 2021;Audina et al., 2022;Dantes et al., 2022;Listiani et al., 2021;Purwanti et al., 2022;Rastari et al., 2023;Ryan, 1998;Suwastini et al., 2023), it creates an interesting and lively English learning compared to conventional learning without video (Mekheimer, 2011;Renes & Strange, 2010;Yusup et al., 2016).Furthermore, video can be used to improve conversation, listening, and pronunciation skills, and it can even be used as realia to stimulate cultural lessons and promotes authentic vocabulary development (Audina et al., 2022(Audina et al., , 2023;;Dantes et al., 2022;Dung, 2021;Rastari et al., 2023;Soeoed et al., 2018;Suwastini et al., 2023;Watkins & Wilkins, 2011).However, despite the fact that video viewing is enjoyable, video viewing is just another form of knowledge presentation.When watching a video, students tend to passively consume a set of knowledge prepared by other people.To have a meaningful learning process, students must be put in a position where they have to create or construct their own knowledge, concepts, and ideas based on the information they have possessed (Ersani et al., 2021;Karppinen, 2005;Sharan, 2015;Suwastini et al., 2021).
Undergraduate level students aspiring to master English and to teach the language must experience a meaningful English learning.The experience of a meaningful English learning will not only prompt them to learn English but also reward them with a sense of achievement (Khodadad & Kaur, 2016;Li & Pan, 2009;Pertiwi & Sudarsono, 2015).Sense of achievement is the feeling of satisfaction that a person experience when she or he manages to do something.In English learning context, when students are successful in performing certain tasks, they will feel satisfied with themselves and their effort which in turn boost their confidence.The sense of achievement is actually part of students' internal motivation system.The satisfaction will push students to learn more, even trying new and more challenging activities or tasks, in order to have the satisfaction (Komarraju et al., 2009;Oz, 2016).
It is this sense of achievement that needs to be capitalized by the teachers by providing activities which will prompt students' satisfaction of their learning process by producing something tangible that they can be proud of.Furthermore, by using video production as a project in English learning, students will learn not only English but also other skills associated with producing videos (Dwifadjrin & Pamungkas, 2020;Hutapea & Suwastini, 2019;Suwastini et al., 2021;Sweeder, 2007).The use of video and video making is a part of the methodological approach which combines pedagogical knowledge, content knowledge, and technological knowledge, which is widely known as TPACK or Technological Pedagogical Content Knowledge (Koehler & Mishra, 2005;Muslimin et al., 2023;Tan et al., 2019).
However, to date, there has never been any instrument developed to measure student's perception of the meaningfulness of the learning through video production project.The increase in the use of video production instead of mere video viewing warrants a good instrument which is able to measure students' English learning.For it is very easy for both the teacher and the students to forget the English learning objectives, amidst the flurry of excitement of video production.Such departure from English learning objectives may prompt students to produce well-made videos with little English learning experience.Therefore, it is important to devise an instrument which can be used to measure students active learning process while producing videos.Thus, this instrument is developed to measure students' perception of the meaningfulness of learning through video production.

METHODS
The development of the instrument is based on the seven major steps of instrument development (Gall et al., 2003).
Step 1 is defining the construct that is going to be measured.
Step 2, after the construct is defined, the target population is specified based on the purpose of the instrument.Step 3 is the process of reviewing other related instruments.
Step 4 the prototype of the instrument is developed in the form of a blueprint of the instrument prototype down to the items of the instrument.Step 5 is the instruments prototype evaluation, conducted by obtaining critical review from experts on the instrument's content validity.Step 6 is revising the instruments based on the result of the evaluation in step 5.The revised version of the instrument is used in the next step.
Step 7 is the empirical steps of instrument development which includes pilot-testing the instrument to get the internal attributes of the test, namely; reliability, construct validity, and also item analysis.The resulting instrument was piloted by EFL pre-service students from three different classes at a local state university in Bali province.These classes were selected since they represent the target population upon which the instrument will be used in the future.Since the instrument measures meaningful learning in classes with video production, the classes selected were classes that have experienced video production in their learning of English.
The target populations of the instrument are male and female students, from any level of education, both formal and informal, who have been learning English using video production.Furthermore, video production also challenges students technical and language skills, prompt students' cooperation and collaboration, and most importantly video production improved students' language competence and boosted students' learning motivation (Nikitina, 2010).Students took part in video production also reported mastery in the subject matter learned through video production as well as feeling positive emotion and motivation during video production (Pirhonen & Rasi, 2017).

Result
The instrument developed in this study is aimed at measuring student's perception of the meaningfulness of learning using video production project.Naturally, the construct used here is meaningful learning.The construct is defined based on meaningful learning with digital and video learning description.A mere presentation of interesting material in the form of video will not automatically lead to in-depth learning.Therefore, students need to study in a meaningful way, in the sense that the students need to a) actively construct their own meaning instead of simply acquiring the knowledge, and b) Learning is a process of facilitation knowledge construction instead of communicating knowledge.The principles are the foundation of meaningful learning with digital and video learning which will be further detailed into variables, indicators, and finally into items.The development of the construct into items is recorded in the blueprint.
Based on the review of related literature and the formulation of construct definition, variables and indicators, the blueprint of the presently developed instrument can be shown in Table 1.Base on Table 1, there are 6 variables that represent the construct of meaningful learning.The indicators of each variable are developed based on the principles of meaningful learning.The items are developed in form of statements to be responded based on the users' agreement using 5 points Likert scale; Strongly Agree, Agree, Undecided, Disagree, Strongly Disagree.Upon the completion of the prototype, it was sent to a panel of 5 judges which are all English teachers holding in minimum, a master degree in English education and a minimum of 10 years teaching experience.The authors shared the prototype to the experts and then they were given the chance to evaluate the material, instruments construction, and language use.The results of the experts' evaluation are as follows.1.)The first thing criticized by the experts is the use of overtly judgmental words in the items.The experts point out that the words such as "forces" and "pushes" in items 1 to 4 is not appropriate because it presents a very strong tone which may invite a negative reaction from the participants.The panel of experts suggested changing the words with a subtler one such as "prompts".2.) The panel of expert pointed out that there are two items, item 19 and item 20, which present the same question, "The teacher taught me how to write my script".This was a technical error since in the first draft of the blueprint, the question was "The teacher taught me how to shoot and edit my video".3.) The experts noted some grammatical mistakes in the items, such as the inconsistencies in the tenses used in the items.Some of the sentences still used simple present tense to describe past events.
Based on the results of the evaluation performed by the panel of experts, revisions were made on the prototype.Since there were only minor revisions for the items, it did not take much time to revise the prototype.The revised version of the Instruments to Measure Perceived Meaningfulness of English Learning Using Video Production is presented in Table 2.The last stage of the instrument development is pilot testing the instrument.The participants selected for the pilot was based on the requirement that they have taken an English course which uses Video Production approach.The participants were taken from three university level EFL classes of a state university in Bali.There were 86 participants consisting of 54 male and 32 female students.Each of the students has been involved in an English class using video production.The data obtained from the tryout were then analyzed using SPSS to obtain the Cronbach's Alpha Coefficient which signifies the internal consistency and the evidence of the instrument reliability.The result of the analysis can be seen in Table 3. Base on Table 3, it can be concluded that the Cronbach's Alpha Value and the Cronbach's Alpha Based on Standardized items value are 0.896 and 0.927 respectively.This means that this instrument achieves a high internal consistency, or in other words all items of this instrument measure the same concept or construct and it is connected to the interrelatedness of the items within the instrument.
Furthermore, since the acceptable value of Cronbach Alpha ranges from .70 -.95 and low Cronbach's Alpha value can be caused by poor correlation between items, items which are not correlated should be removed to improve the Cronbach's Alpha value.Using the Item-Total Statistic performed by SPSS, we can see that if a particular item is to be deleted, the Cronbach's Alpha value will improve.The item-total statistics is displayed in Table 4.As presented in Table 4, if either item 12 or item 22 is deleted, the Cronbach's Alpha value will be higher and achieve 0.903.Since item 22 has lower corrected item-total correlation compared to item 12, it would be better to delete item 22.After deleting item 22, the calculation for the Cronbach's Alpha was run again, and the result of the second calculation is presented in Table 5.

Discussions
Instrument development in this study is based on the principles of meaningful learning (Karppinen, 2005).A meaningful learning is an active and student-centered learning process.Students are engaged by the learning process in a mindful processing of information, where they are responsible for the result (Jiménez & Rose, 2010;Sholeh et al., 2021).It is a constructive and individual process where students incorporate new information into existing knowledge.The whole process involves meaning-making, a process of understanding the world around us.Individuality means that each student is different and should be understood accordingly.Students decide what topics they want to talk about, how to present their topics, and be creative in their ways.A meaningful learning is a collaborative and conversational process.It promotes cooperation and collaboration between students through pair work or group work setting which prompts the students to work together to achieve the group goal (Nair & Yunus, 2021;Taylor et al., 2015).It also promotes collaboration between students and teacher as teacher guides and advises the students during the learning process.
Despite the popularity of the use of video in English classes, previous teaching and research of said usage focus heavily on video viewing instead of video production.Naturally, the instruments developed related to video and English class were mostly to measure students' responses to video viewing.To date, there are only few studies focusing on video production, and each of these studies used different instruments.Previous study measure students' perception of video blogging (Hung & Huang, 2015).The instruments used to measure students' verbal performance using a public speaking scale adapted from Johnson and validated.
Another instrument used in the study was an interview guide which resulted in two themes of the students' reaction.Another study conducted by previous study on Japanese students' video production using their smartphones to improve the number of words spoken by students when performing a monologue (Gromik, 2012).The instruments used were interview guides to obtain students' monologue performance improvement and perception of video production.Based on the above studies, it is safe to conclude that there were limited studies conducted on video production, and none of these researches used an instrument to measure students' perception on video production as a meaningful learning process (Hobbs & Friesem, 2019;Yamashiro & Johnson, 1997).It is hoped that the present instrument can be used in other studies which investigate students' perception of meaningful learning using video production.
However, there are still very few instruments developed to assess or measure students' perception of English learning using video production.This current study has managed to develop an instrument which aimed at measuring students' perception of the meaningfulness of English learning using video production.The instrument constructed through this study considered to have high internal consistency.Despite the high internal consistency achieved by this instrument, future researcher is suggested to adapt, modify, and improve this instrument to better suit their needs as well as to better measure any variables which have not been included in the construct used as the basis of the instrument developed herein.

CONCLUSION
The development of technology has accommodated the widespread use of video and video production in learning, including in English learning.The availability of technological means has prompted the rise of video production in English class.The instrument is developed using Borg's seven steps instrument development process.This instrument achieves a high internal consistency, or in other words all items of this instrument measure the same concept or construct and it is connected to the inter-relatedness of the items within the instrument.

Table 1 .
The Prototype of the Instruments to Measure Perceived Meaningfulness of English Learning Using Video Production

Table 2 .
The Revised Prototype of the Instruments to Measure Perceived Meaningfulness of English Learning Using Video Production

Table 3 .
The Result of Cronbach's Alpha Analysis

Table 4 .
Item-Total Statistics

Table 5 ,
the final instrument has a total of 23 items, with the Cronbach's Alpha Value and the Cronbach's Alpha Based on Standardized items values are 0.903 and 0.932 respectively.This also means that this instrument has high internal consistency.