Assessing Students’ Digital Literacy Skill in Senior High School Yogyakarta

This study aimed at investigating students’ digital literacy skill of students Grade X and XI in Senior High School in Yogyakarta. There were three steps of this research. First, we constructed the item format and aspect of digital literacy skill. Second, the administration of this research to 193 students of science class in Yogyakarta which were selected randomly. The last, analysis data of students’ digital literacy skill based on educational level. For the profile of students’ digital we used descriptive and quantitative statistics (frequency and percent). To determine the significant difference of the students’ level we also used independent sample t-test. Result of this study showed that the students’ digital literacy skill of all students was very low level. In addition, there was significant difference of students’ digital literacy skill based on educational level. The conclusion was students’ digital literacy skill in Yogyakarta was very low and there must an effort to force these skills. Recommendation for the future study is to develop a learning model that can enhance students’ digital literacy skill. For the policy makers, learning based on digital literacy skill must be integrated on the curriculum. ICT learning should be thought to students in Indonesia, especially in Yogyakarta.


Introduction
The growth of digital culture in the twenty-first century has encouraged the use of digital resources and communication tools in school education (Kong, 2014) so students are asked to express their ideas in digital media (Chan, Churchill, & Chiu, 2017). In addition, they must also be proficient at interacting with files include converting and creating graphics from one type to another and using web-based tools to complete sophisticated tasks (Frydenberg, 2015).
The role of digital technology in daily life has increase in over the past few decades (Bekker, Bakker, Douma, van der Poel, & Scheltenaar, 2015). But many educational institutions have not fully embraced digital literacy as basic literacy equivalent to reading, writing and counting (Coffin Murray & Pérez, 2017). In fact, the adoption of digital literacy skills to improve quality is an important problem for the digital learning environment (Techataweewan & Prasertsin, 2018).
For education professionals, the implication of increasing student involvement with digital media is to help students develop knowledge, skills, and dispositions to live, contribute and develop in the 21st century digital world (Redmond, 2015). The success of students, as citizens involved in building the nation in the future has been linked to 'digital literacy' (Pangrazio, 2016).
It is difficult to give a precise definition of "digital literacy" because this term has been used for various meanings in the literature (Güneş & Bahçivan, 2018). Digital literacy often seems to amount to a minimum the skills that will allow users to operate effectively with software tools, or in carrying out basic information search tasks (Buckingham, 2015). Digital literacy has several elements such as critical thinking skills, creativity, building and evaluating information and using digital media effectively, and it can be developed as a result of students' digital writing (Al-Qallaf & Al-Mutairi, 2016).
Digital literacy for learning is more than just knowing how to operate technology, but also having the right information management and critical thinking skills, as well as appropriate online behavior (Tang & Chaw, 2016). In addition, digital literacy not only understands how ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i2.17168 Jurnal Pendidikan Indonesia | 170 tools work but also why these tools are useful in the real world and when to use them (Alexander, B., Adams Becker, S., Cummins, 2016). These skills represent a set of discrete abilities or behaviors expressed by users of digital information systems, often in the process of investigation (Meyers, Erickson, & Small, 2013) in other words, it related to skills and abilities utilizing ICT (Prior, Mazanov, Meacheam, Heaslip, & Hanson, 2016). Digital literacy enables one's participation in social networks to create and share knowledge, and supports a variety of professional computing skills (Josie et al., 2018). The conceptualization of digital literacy as a cultural competency, integrating aspects such as basic technical skills, analyzing the media as an object itself, being critical of content and technology and obtaining learning strategies for finding and utilizing information and learning for learning (Bjørgen & Erstad, 2015). So, the digital environment that develops as an educational tool requires research on learners' digital literacy (Greene, Yu, & Copeland, 2014).
There are many digital literacy components proposed by experts. Alkali & Amichai-Hamburger (2004) state five main skills of digital Literacy: (1) photo-visual skills (reading graphic displays), (2) reproductive skills (utilizing digital devices to make new and meaningful findings from pre-existing material) , (3) branching skills (building knowledge from non-linear, hyper textual, and navigation), (4) information skills (evaluating the quality and validity of information), and (5) social-emotional skills (understanding "rules" that apply in the world virtual and apply this understanding in online cyber communication). Bawden (2008) proposes four core digital literacy competencies are: (1) internet search, (2) hypertext navigation, (3) knowledge assembly, and (4) content evaluation. Karpati (2011) states that for future computer users and ICT professionals, the most important component of digital literacy is: accessing, managing, evaluating, integrating, making and communicating information individually or collaboratively on networks, supported by computers, and web-based environments for study, work, or recreation.
Digital literacy is produced from three dimensions which intersect namely (1) technical (2) cognitive and (3) social-emotional dimensions of digital literacy (Ng, 2012). Hatlevik, O. E., & Christophersen (2013) propose digital literacy competencies include skills in using technology, utilizing technology to process, obtain, evaluate information, produce and communicate information. Bollard, Kerry, Whitney, & Fidock (2014) measure digital literacy skills with aspects of digital device ownership, social use and work of digital devices, and attitudes towards the use of digital technology in training. Kazakoff (2015) states that there are six components of digital literacy: (1) understanding and utilizing digital devices; (2) understand navigation tools in digital devices; (3) Able to think critically and solve digital problems; (4) Active in learning and able to cooperate in creative design through digital devices. Reynolds (2016) offers a new conceptualized modular framework for digital literacy that defines this concept as "task constructivist digital literacy", which consists of 6 practice domains based on social constructivism and constructivism: Create, Manage, Publish, Promote, Research, and Surf. Josie et al., (2018) identified five digital literacy disciplines: (1) information literacy, including the ability to search, retrieve, manipulate, evaluate, synthesize and create digital content (2) computer literacy, including the ability to operate digital hardware and software so as to be able to understand how to use various forms of tools is very important to understand technical (3) media literacy, including the ability to interact with text, sound, images, videos and social media (4) communication literacy, including communication skills in traditional and innovative media and (5) technological literacy, including the ability to adopt various technologies for certain life situations.
There are various researches on digital literacy. Prabhu (2010) focuses on tests of critical thinking certification formulated by Educational Testing Service and Certiport that provide basic guidelines for digital literacy by measuring critical thinking skills in technologysupported environments. Machala & Orešković (2014) measured librarians' information and digital literacy activities at the national lifelong learning portal, and tested the application of API Experience (xAPI) as information and digital literacy assessment instruments. Greene et al. (2014) examined how critical aspects of digital literacy are related to student learning outcomes when using the Internet to investigate everyday health and public ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i2.17168 Jurnal Pendidikan Indonesia | 171 science topics. Lee (2014) focuses on measuring significant differences between before and after digital literacy education through tests and surveys before and after treatment is given. Literat (2014) assessed the psychometric properties of the newly tested self-report assessment tool for media literacy, based on twelve new media literacy skills (NML) developed by (Jenkin, Clinton, Purushotma, Robison, & Weigel, 2006). Ainley, Schulz, & Fraillon (2016) review the definitions of digital literacy and ICT that have been adopted in cross-national studies, investigate approaches to digital literacy and ICT assessment that have been used in the study and articulate criteria that should guide the development of a global measure of literacy skills digital and ICT. Siddiq, Gochyyev, & Wilson, (2017) develop test efforts to measure students' ability to handle digital information, to communicate and collaborate during problem solving. Other research investigates digital literacy among junior high school students with the aim of comparing perceptions of participants' digital literacy competencies and their actual performance in relevant digital tasks (Porat, Blau, & Barak, 2018).
In Indonesia, especially the province of Yogyakarta, research on students' digital literacy skills is still rare. In fact, as one of the major cities in Indonesia, this province must have valid data regarding digital literacy skills. As a student city, Yogyakarta must be a role model for other cities in following the current development of education in the 21st century. In this study, we focused on measuring students' digital literacy skills and proving whether there were differences in student skills based on education levels. The research questions in this question are: (1) What is the level of students' digital literacy skills in Yogyakarta? (2) Are there significant differences in the level of digital literacy skills based on educational levels?

Method
The first stage in measuring digital literacy skill of the students was defining what the definition of digital literacy skill is and selecting the aspect of digital literacy skill that should be targeted in this research. As described in the introduction, one of the difficulties in assessing digital literacy skills is the widespread disagreement among researchers and educators about what aspects can represent students' digital literacy skills. Therefore, we initially carried out a review of available research on standardized digital literacy by considering several things. First, we aim to identify key skills of digital literacy that are commonly developed. These key skills are the focus of our tests and thus we can build the right kind of digital literacy skills. Second, we determine the aspects studied by reflecting the existing digital literacy skills test, so that the selected aspects directly intersect with digital literacy skills. All components reviewed based on (a) are in accordance with clear definition / concept of digital literacy, (b) whether the targeted digital literacy skills are general and (c) whether they represent digital literacy skills as a whole. Based on the results of the review, the components of digital literacy skills examined in this study are as in Table 1. The test was administered to students grade X (N = 96) and grade XI (97) in senior high school in Yogyakarta, was selected randomly. The students did not participate in any of the previous pilot studies and were enrolled in digital literacy aspect. The participant consisted of 197 students between the ages of 13 -17 years. Prior the administration, the participants were given the initial instruction regarding the goal of the test, and general direction about how to respond to the items, and ask to take the test seriously. The test was administered similar with (Tiruneh, De Cock, Weldeslassie, Elen, & Janssen, 2017) where hold in a controlled ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i2.17168 Jurnal Pendidikan Indonesia | 172 classroom setting and great caution was made for all students to hand in the test so that test questions would not circulate. The participants were not given a time limit to answer the questions given. However, we conveyed the possibility of taking about 60 minutes to complete all the questions. About 90% of students can complete within 50 minutes and the rest is completed in 60 minutes. We used SPSS version 16 to analysis data. For the profile of the students' digital literacy skill, descriptive and quantitative statistic were used, including frequency and percent. The independent sample t-test was used to determine the significant difference of the level digital literacy skill of students in grade X and grade XI.

Result and Discussion
Based on the test, students were given 10 question according the aspect of digital literacy skill. The result is reveled in  Among the 10 questions that administrated on the students Grade X, answers of the students in internet searching shows the highest mean of 1.56 (very low). Moreover, the answers in knowledge assembly present the lowest mean of 1.02 (very low). It can be gleaned from the table that students on Grade X have very low level digital literacy skill (mean = 1.275, SD = 0.70). Table 3, shows the frequency and percent of students by each level of digital literacy skill. According Table 3, the students on Grade X have very low digital literacy skills (mean = 1.275, SD = 0.00). Most (89.7% of total 97 students) of the students have very low level of digital literacy skill while few (9.27%) students have low level of digital literacy skill and only 1.03% of the students have average level of digital literacy skill.
Based on the test, students on grade XI also were given 10 question according the aspect of digital literacy skill. The result is reveled in Table 4. The table shows the level of their answer in the test. ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i2.17168 Jurnal Pendidikan Indonesia | 173 Among the 10 questions that administrated on the students Grade XI, answers of the students in internet searching also shows the highest mean of 2.52 (very low). Moreover, the answers in knowledge assembly also present the lowest mean of 0.73 (very low). It can be gleaned from the table that students on Grade XI have very low level digital literacy skill (mean = 1.71, SD = 0.71). Table 5, shows the frequency and percent of students by each level of digital literacy skill. According Table 5, the students on Grade XI have very low digital literacy skills (mean = 1.71, SD = 0.98). Most (80.2% of total 96 students) of the students have very low level of digital literacy skill while few (8.3%) students have low level of digital literacy skill and 11.5% of the students have average level of digital literacy skill.
Data analysis on the students answers form digital literacy test. Using the average score of students on Grade X and XI, the significant difference in their level of students' digital literacy skill was determined using independent sample t-test. The result is shown in Table 6. Based on Table 6, the level of digital literacy skill of students in grade X is significantly different (t = 4.204, P < 0.005) from the students in Grade XI. The students in grade XI mean (1.71) difference of 0.44 over than students in grade X (1.27) indicates that the level of students' digital literacy skill in the grade XI is better than in the grade X. The higher digital literacy skill of the grade XI is attributed to the learning they have during the teaching learning process. But, unfortunately these skills also in the very low level of digital literacy aspect.
Based on the results of the study, we know that all aspects of students' digital literacy skills are still very low for both students of class X and class XI. The order of aspects of students' digital literacy skills based on the highest to lowest average is internet searching, content creation, hyper textual navigation and knowledge assembly.
The aspect of internet searching is the highest value aspect in both classes (mean of grade X = 1.56, mean of grade XI = 2.52) even though it is still at a very low level. This is similar to (Tarimo & Kavishe, 2017), who argue that the majority of internet users in secondary schools do not have the right search skills to access information using the Internet. The reason is the lack of training and skills in internet search strategies (Chirwa, 2018). This is because students usually do not have enough skills to appraise the information on the internet ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i2.17168 appropriately (Chiu, Liang, & Tsai, 2016). This very low level can also be caused by English language skills, where those who are proficient in English are better at searching information using the internet (Shirazi, Heidari, Fard, & Ghodsbin, 2019). The second aspect in digital literacy skill in this study is content creation (mean of grade X = 1.51, mean of grade XI = 2.10). Students' skills in this aspect are also at a very low level, even though this aspect provides opportunities for students to integrate and reflect on media experience in everyday life (Costa, Tyner, Henriques, & Sousa, 2018). This can be caused because they do not make great efforts on the sub-aspects of content creation such as networking with other content creators and their offline presence (Törhönen, Sjöblom, & Hamari, 2018). In addition, they may not be able to use digital devices optimally to generate ideas or develop a new method. In fact, one of the conceptual definition of content creation is to use new ways of doing things (van Laar, van Deursen, van Dijk, & de Haan, 2017).
Hyper textual navigation is one of the difficult aspect of digital literacy skill which very low level in this study (mean of grade X =, mean of grade XI =). One of the causes of students' digital literacy skills that are still very low in this aspect is that students have difficulty using computer equipment and software in selecting and reading information. This is similar to (Cordeiro, de Abreu, & Estadieu, 2017), that based on the type of interactivity, hyper textual navigation skills describe interactive pieces in which users must use computer equipment and software to make reading choices in the database. In addition, students may not often interact with the media to make reading choices in the database using interface technology as a constructor of customized and individualized data sets. Meanwhile, interaction with the media in making choices is also a sub-aspect of hyper textual navigation (Tunalı, 2016).

Conclusion
Based on the results and discussion, students' digital literacy skills are at a very low level. The order of skills of the lowest to highest students is knowledge assembly, hyper textual navigation, content creation, and internet searching. It shows that knowledge assembly is the most difficult aspect of students' digital literacy skill and internet searching is the easy aspect. Nonetheless, students' digital literacy skills in Yogyakarta are still very low.
There is a significant difference between the digital literacy skills of class X students and class XI. In other words, digital literacy skills are also influenced by the level of education. These findings prove that digital literacy-based learning is still very much needed by students. For the teachers, the learning process should always involve digital devices. Because learning with digital devices can improve students' digital literacy skills.
For the policy makers, digital literacy-based learning processes should again be integrated with the current curriculum. In addition, digital literacy learning is organized differently at each level of education. This is because digital literacy skills possessed by students differ based on the level of education. This level of different digital literacy skills requires different subject matter at each level of the curriculum.
Recommendations for further research, researchers should start learning models that are specifically for improving students' digital literacy skills. In addition, further research is needed with other aspects of student digital literacy skills, so that it can add to the information about the level of student digital literacy skills. Other studies can also focus on the main causes why students' digital literacy skills are still very low. This can be a basic guideline for further research to address the problems of students' digital literacy skills.