Analysis of The Alignment Between Chemistry Content on TIMSS And Science-Chemistry Textbooks of Junior High School

This study is conducted to: (1) analyze the level of alignment between chemistry content on TIMSS with science-chemistry textbooks of Junior High School and (2) describe the discrepancies between chemistry content on TIMSS with science-chemistry textbooks of Junior High School in term of cognitive demands as well as subject content covered. This study was descriptive which used content analysis method. The content analysis was conducted on chemistry content of TIMSS and sciencechemistry textbooks of Junior High School. To determine the degree of alignment criteria, this study used the method developed by Porter. Accordingly, the level of alignment is symbolized by Porter's Alignment Index. The results showed that the degree of alignment between TIMSS and and sciencechemistry textbooks of Junior High School was 0.49 (low level of alignment). This imperfect alignment level is characterized by the discrepancies in term of subject content covered and cognitive demands among chemistry content on TIMSS and and science-chemistry textbooks of Junior High School. This can affect toward the achievement of Indonesian students in TIMSS.


Introduction
Trends International Mathematics and Science Study (TIMSS) is an international study of mathematics and science achievements of junior high school students (14 years) and elementary school students (9 years). The study is organized by the International Association for the Evaluation of Educational Achievement (IEA) in Amsterdam, Netherlands. Indonesia has fully participated in this association since 1999. The position of Indonesia as a TIMSS participating country is illustrated in Table 1:  Table 1 demonstrates Indonesia's position during its participation in TIMSS study, which shows unsatisfied results. Indonesia still lies in the lower group and the average score is far below the world average score (benchmark). In addition, this position also confirms no progress and improvement in the achievements over the three periods followed. The static position of Indonesia in TIMSS study certainly raises questions about the improvement efforts to increase the science achievements of Indonesian students.
Further research on the results of international studies followed by Indonesia, including TIMSS, can be a solution to solve this problem. Advanced research can be initiated by identifying factors that contribute to the results of students' achievement in international studies. According to Winkel (2005), there are several external factors that influence student achievement include teaching curriculum and learning effectiveness. Furthermore, Arliani and Hidayati (2013) add that curriculum standards, learning process and assessment should be implemented in balance. If there is an alignment between these three components, an accurate assessment of students' learning outcomes can be achieved. This also happens to TIMSS. The questions tested at TIMSS must be in accordance with the curriculum in Indonesia and they have been studied by students who take the TIMSS study.
A strategy to perceive the alignment among the curriculum, learning and assessment is by conducting alignment analysis. Porter & Smithson (2001) assert that alignment analysis can be conducted not only between the curriculum standards and assessment, but also between curriculum standards and learning and teaching materials. Research related to alignment analysis has been carried out, including: (1) an alignment between standards and textbooks (Kesidou and Roseman, 2002); (2) an alignment between the contents of TIMSS and PISA and assessment standards (Smithson, 2009);(3) an alignment between content standards and tests (Liu & Lu, 2012); (4) an alignment between national exams and content standards (Firman, 2013).
The studies mentioned previously still analyzed the level of alignment between the two components. To date, no research has been conducted about the analysis of three important components of the curriculum, namely: standards, learning and assessment, simultaneously. Edwards (2010) comments that an alignment study is very crucial in the context of the education process improvement. For this reason, there is a need for further research on TIMSS, which aims to analyze the alignment between chemistry content on TIMSS and science-chemistry textbooks of Junior High School in order to precisely identify the relationship of the alignment between the two. Hence, this research provided effective information as a feedback in formulating the policy to improve Indonesia's position in the TIMSS international study. The analytical method applied in this research is the alignment analysis method developed by Porter (2002).

Method
This research was descriptive research. According to Arikunto (2003: 9), descriptive research aims to describe and interpret symptoms or events as they are without providing variable treatment and manipulation. The method of this research was content analysis. The analysis was conducted on documents related to TIMSS and science-chemistry textbooks of Junior High School. The document analysis was carried out to determine the subject content covered and cognitive demands of chemistry content from TIMSS and science-chemistry textbooks of Junior High School. The results of the content analysis were validated by two expert validators and were reduced to a two-dimensional matrix consisting of five categories of subject content covered and six levels of cognitive demands based on Bloom's revised taxonomy by Anderson. The next step was converting the table by changing all cell values in the form of a proportion of the total of each matrix. Finally, the alignment index was calculated by using the Porter equation. Thus, the Porter 's Alignment Index value ranges from 1.0 (very aligned) to 0 (not aligned) (Porter, 2002). For the alignment index criteria, according to Fonthal, G (2004), if the index values are <0.5, they are classified into low category while if 0.5 <alignment index <0.66, it is classified into medium category, and if the index is > 0.66, it is classified into high category. ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i1.16101 Jurnal Pendidikan Indonesia | 125

Results and Discussion
Porter's Alignment Index As stated earlier, the analytical method developed by Porter was utilized to analyze the chemistry content from TIMSS and science-chemistry textbooks of Junior High School. The results of the documents analysis of TIMSS and science-chemistry textbooks of Junior High School, which have been reduced in the form of a matrix of cognitive demands and subject content covered, were used to calculate the alignment index by using the Porter formula. Based on the results of calculation, the alignment index between chemistry content from TIMSS and science-chemistry textbooks of Junior High School was represented by the Porter index. The value of the Porter index alignment between science-chemistry textbooks of Junior High School and chemistry content from TIMSS is 0.49. Based on the criteria, the alignment is at a low level.

The Alignment and Difference between Chemistry Contents from TIMSS and Sciencechemistry Textbooks of Junior High School
The difference of cognitive demands and subject content covered could also be calculated based on the subject content covered matrix and cognitive demands from TIMSS and science-chemistry textbooks of Junior High School. The following tables and figures show differences in subject content covered and cognitive demands between TIMSS and science-chemistry textbooks of Junior High School.  The negative sign in Table 2 shows that the materials in the subject content of the Science-chemistry Textbooks of Junior High School is less than TIMSS while the positive sign shows that the coverage of the subject content of the Science-chemistry Textbooks of Junior High School is greater than TIMSS. The most significant difference is on the subject of Mixed Separation (physics and chemical), which is equal to 0.024, while the smallest difference is the subject of Atom, ion and molecule, that is equal to 0.001.  The most significant difference is the cognitive demands at the level of applying (C3) and analyzing (C4) between TIMSS and science-chemistry textbooks of Junior High School, respectively +0,037 and -0,037. Meanwhile, the smallest difference is cognitive demands between TIMSS and Science-chemistry Textbooks of Junior High School at the level of creating (C6), which is equal to 0.009. The level of evaluating (C5), both in Sciencechemistry Textbooks of Junior High School and the chemistry content on TIMSS were not found.
Similar to the tables and graphs of differences in subject content covered, the negative sign in the tables and graphs of differences in cognitive demands shows that in the Sciencechemistry Textbooks of Junior High School the cognitive demands for these levels are less than TIMSS. Whereas the positive sign shows that in Science-chemistry Textbooks of Junior High School the cognitive demands for these levels are more, compared to TIMSS. The difference in the proportion of the subject content covered and cognitive demands between Science-chemistry Textbooks of Junior High School and TIMSS can also be seen from the distribution of subject content covered diagrams and the cognitive demands in Figure 3.   Figure 3, it can be seen that the topic of material properties is more dominantly tested in TIMSS, compared to other topics. Meanwhile, the distribution of the topic of elements, compounds and mixtures; chemical changes; and atoms, ions, and molecules have the proportions which are almost similar. Then, only about 2% of the questions about mixed separation were tested. For the cognitive demands in TIMSS, it was found that the distribution for cognitive ability in the level of understanding (C2) is more dominant while the ratio for the level of remembering (C1), applying (C3) and analyzing (C4) is almost similar, which is around 20%; then, the distribution for the level of creating (C6 ) is around 2%. Finally, the level of evaluating (C5) was not tested in chemistry content TIMSS.
On the contrary, the analysis results of the science-chemistry textbooks of Junior High School revealed that the distribution of the subject content covered in science-chemistry textbooks of Junior High School presented the same proportion for the topics of elements, mixed compounds; the material properties and chemical changes, which is as much as 23%, and the distribution for the topic of atom, ion, and molecular is 16%. The most obvious difference between TIMSS and science-chemistry textbooks of Junior High School lies in the topic of mixed separation, in which the proportion in TIMSS is only 2% while in sciencechemistry textbooks of Junior High School this topic has a proportion as much as 16%.
The interesting thing about the cognitive demands is that the level of evaluating (C5) was not tested neither in the TIMSS nor in the science-chemistry textbooks of Junior High School. Whereas, the cognitive demands in the Chemistry Science-Middle School textbook were dominated by the level of understanding (C2) for 47%, followed by the level of applying (C3) for 40%, remembering (C1) for 7%, and creating (C6) and analyzing (C4) for 3% each.

Discussion
One of the international studies attended by Indonesia to measure students' achievement is TIMSS. TIMSS is a study that measures the mathematics and science achievements of junior high school students which has been held every 4 years since 1999. However, after participating for three periods, the position of Indonesia is still below the international benchmark. This indicates that Indonesian students have not mastered the concepts tested in TIMSS. This unpreparedness finally has an impact on the low achievement of Indonesian students in the TIMSS study.
Thus, some efforts are needed to improve the performance of Indonesian students in the TIMSS study. This is in accordance with one of the objectives of TIMSS, namely as input in the formulation of policies for improving the education quality. As stated in the TIMSS framework, TIMSS measures students' scientific and mathematical achievements not only from the content domain but also the cognitive domain.
The content domain consists of chemistry, physics, biology and geology, while the cognitive domain consists of knowledge, application and reasoning. The test items are designed based on the results of the curriculum reviews of all participating countries. As a result, TIMSS has a wide range of scientific materials. The alignment analysis between the questions tested in TIMSS and the curriculum and the learning process in our country is one of the efforts to design and decide the formulation of policies related to improving the education quality.
Based on the findings, the level of alignment between TIMSS and the sciencechemistry textbooks of Junior High School is at a low level, which is as much as 0.49. This result led to the low achievement gained by Indonesian students in the TIMSS study. Edwards (2010) states that one of the causes of the students' low achievement in a test is caused by a discrepancy between the standards of the curriculum, learning process, and teaching materials and the assessment characterized by a low value of the alignment index.
The imperfection of the alignment index value is reflected in the difference in subject content covered and the cognitive demands between TIMSS and science-chemistry textbooks of Junior High School. This is also in line with the result of research by Firman (2013). The analysis results of the subject content covered and cognitive abilities tested in TIMSS illustrates that the test items in TIMSS do not only test the understanding of chemical concepts, but also measure the extent to which students can apply the concepts they understand to solve simple problems and even problems requiring systematic reasoning. This is in accordance with the assessment framework which states that the cognitive domain expected from students is knowledge of facts and procedures, the use of concepts to solve concepts and use reasoning abilities (Mullis et. Al., 2011).
Although some of the TIMSS chemistry questions compiled were in the form of multiple choices, students should understand the concepts and the reasoning to solve these questions.
This research also found some differences in the subject content covered between TIMSS and science-chemistry textbooks of Junior High School as one of the standards and learning resources during the learning process. The scope of the material in the sciencechemistry textbooks of Junior High School was indeed broader than the scope of the material available at the TIMSS for the topics such as elements, compounds and mixtures; mixed separation; atoms, ions and molecules. Students' knowledge was tested on the topic of mixed separation. ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i1.16101 Jurnal Pendidikan Indonesia | 129  Figure 4 represents one of the chemistry contents in TIMSS in 2007 that requires students to determine the appropriate mixture separation method. Based on this problem, it can be seen that the problem related to the mixed separation given was an applicable problem and closest to everyday life. Meanwhile, in the science-chemistry textbook of Junior High School, students were required to understand the method of mixed separation such as centrifugation, chromatography, and distillation that cannot be directly practiced by students.
During the learning process, students listened to the teacher's explanation almost all the time. This means that students tended to memorize because the demands of the dominant cognitive ability of memorizing and understanding. The students had lack experience in applying the concepts they had to solve problems, for example through a trial process. Hence, the students would find difficulties when they faced the problems that were applicable and required reasoning. The implementation of learning the cognitive level of creating (C6) could not be applied although the students were expected to master this level on certain topics.
The difference between what students learnt and what was tested in TIMSS certainly affected students' achievement. It is unfair to ask the students to solve problems while their concepts and cognitive abilities have not reached that level. As revealed by Gunilla & Henrikson (2008), the inconsistency between learning and curriculum and assessment, in this case TIMSS will provide inappropriate information related to students' knowledge and abilities. This will certainly harm the students.
In addition, there were several other materials which were not tested in TIMSS but were studied and included in the science-chemistry textbooks of Junior High School. The materials were environmental pollution, global warming, additives and addictive substances. These materials were adjusted to the demands of content standards in the 2013 curriculum for Science-Chemistry subjects at junior high school. In addition, the topics about atoms, ions and molecules were discussed more deeply in the textbook than the questions tested in TIMSS. It was because students have been introduced to the topic of electron configuration. This difference certainly becomes a cause of the low level of alignment between chemistry content in TIMSS and science-chemistry textbooks of Junior High School. ISSN: 2541-7207 DOI: 10.23887/jpi-undiksha.v8i1.16101 Jurnal Pendidikan Indonesia | 130 The imperfection of the alignment level between the TIMSS and the science-chemistry textbooks of Junior High School is a problem that must be solved immediately in order to prepare Indonesian students to deal with the next TIMSS. This is because the results of the TIMSS study does not only show the extent of Indonesian students' understanding, but also illustrates the quality of our education if it is compared to other participating countries. The improvement efforts being made must certainly start from the improvement of the learning quality and learning resources.
There are several things that can be attempted in order to improve Indonesia's performance in the TIMSS study, namely: a. Making the results of international studies such as TIMSS and PISA as references for formulating curriculum policies in order to keep up with other participating countries. In the curriculum development, both content and cognitive aspects must be considered and more balanced. The content chosen should be a contextual and applicable content. The allocation of learning time should be arranged based on the essence of the materials being studied. It is necessary to consider the time allocation used for the topic of chemistry in life and environmental pollution because these topics are actually not the basic chemical materials which can actually be delivered through extracurricular activities and counseling, not through learning process. It would be better if the time allocation used to discuss these materials is allocated to other basic science materials that are more essential and strategic. For the cognitive aspect, it is time to move from the learning tradition to memorize, which makes students unable to develop their cognitive abilities. Our students must be trained to get used to using their reasoning abilities in learning. b. Improving and increasing the quality of standards and assessment by paying attention to the proportion of material and cognitive abilities. For example, although the questions in TIMSS are multiple choice, they can measure students' abilities in reasoning and problem solving. c. Sustainably improving the teaching and learning process. During the learning process, the students are trained to apply their chemical concepts and reasoning abilities when solving problems that exist in everyday life. Familiarizing the students to find their own concepts and use them to solve problems. Practice tasks, assignments or tests that are used in class, should be arranged by considering the proportion of cognitive abilities that are tested in a balanced manner. The variation of questions referring to the TIMSS will make the students get used to the TIMSS model so that the students will not feel awkward when they are asked to answer the questions like the ones in TIMSS,. d. Providing students' learning facilities and resources such as textbooks, class facilities, and various learning models that are not monotonous. Learning resources are used by teachers and students must also be in accordance with the competencies in the curriculum. e. Teachers' professionalism and competence need to be improved, especially in the expertise specification. Recently, the science subject at junior high school including biology, physics and chemistry are only taught by one teacher at one time. Most of the teachers who teach junior high school chemistry materials are graduated from the departments of biology and physics. As a result, the teaching loads increases and it is concerned that the learning process will not run optimally. f. Improving the teachers' competence and creativity in planning and implementing learning by using various methods and media as well as the assessment used so that it can improve the learning quality. Based on all the findings in this study, it concludes the finding into an improvement in the level of alignment between TIMSS and the science-chemistry textbooks of Junior High School as a learning resource. Therefore, the efforts mentioned previously are expected to be an input for policy makers to improve the education quality and Indonesian students' achievements in TIMSS.

Conclusion and Suggestions
Based on the results of this research, it can be concluded that the alignment level between chemistry content in TIMSS and the science-chemistry textbooks for junior high schools is still at a low level with an alignment index of 0.49. The imperfection of the alignment level between chemistry content in TIMSS and the science-chemistry textbooks for junior high schools is indicated by the difference in the proportion of subject content covered which is quite dominant in the topic of mixed separation. Meanwhile, the proportion of cognitive ability of applying (C3) and analyzing (C4) on chemistry content in TIMSS and the science-chemistry textbooks for junior high schools also had the most dominant differences. This certainly affects the Indonesian students' achievement in TIMSS, especially in chemistry.
Based on the conclusions obtained above, several suggestions are provided including: a. Practical Interest: The proportion of each coverage of chemical materials in the Science-Chemistry textbook of Junior High School needs to be reviewed by considering the balanced content and cognitive aspects as well as the essence of materials to be studied by junior high school students.
The learning process that only emphasizes on the ability to remember needs to be improved by designing and implementing learning that can help students to involve other cognitive abilities such as applying and analyzing so that students are accustomed to using concepts they know and their reasoning abilities to solve chemical problems. To do this, it certainly needs to be supported by the existence of adequate and effective learning resources, in this case the textbook. b. For Science Development: Other researchers who want to develop this research into a broader scope or to use other subjects, this research can be used as a reference. Other researchers in the fields of Biology, Physics and Mathematics can continue this alignment analysis study for TIMSS in the fields of Biology, Physics and Mathematics so that it becomes a single piece of information that will be beneficial for decision making and policy formulation.