Online Project-Based Learning (O-PjBL): Effectiveness in Teachers Training and Coaching in Vocational Education

learning


INTRODUCTION
Vocational education globally has the primary mission of equipping job skills according to students' interests (Billett, 2011;Clark & Winch, 2007;Sudira, 2012). In addition, the aspect of sustainable development is an important aspect that must be carried out and maintained continuously (Pavlova, 2009). Various countries worldwide have developed their respective vocational education (Eichhorst et al., 2021) to support the sustainability of these aspects (Purvis et al., 2019). The ongoing revitalization of vocational education is one way to develop it (Ahmad, 2020;Kayode & Adeyemi, 2016;Made Sudana et al., 2019). In addition, the change in the vocational learning paradigm is an issue that periodically occurs as a result of the development of science and technology (Gunadi et al., 2020;Kanwar et al., 2019;Savickas, 2011). The pace of science and technology development in the 21st-century has brought about significant changes in various aspects, including the paradigm in vocational learning (Shahroom & Hussin, 2018;Sharma, 2019;Trilling & Fadel, 2012). Vocational education, which is identical to the characteristics of work competency-based learning (Mutohhari, Sutiman, et al., 2021;Nurtanto et al., 2020), has now been integrated with project-based learning (Guo et al., 2020;Hussin et al., 2019). Improving 21st-century skills in students, such as problem-solving, critical thinking, creativity, collaboration, and communication, is the primary goal of implementing project-based learning (Kizkapan & Bektas, 2017;Ummah et al., 2019). In addition, full student involvement in project-based learning activities is an advantage of this model (Bagheri et al., 2020;Haryanto et al., 2021;Pawar et al., 2020). Various majors in vocational education have implemented many project-based learning models (Nilsook et al., 2021;Pawar et al., 2020;Purvis et al., 2019;Vargas et al., 2018). Collaboration between disciplines is the main concept in its application in vocational education (Dogara et al., 2019;Ngereja et al., 2020).
In the field of vocational education, PjBL has been proposed as a consistent learning model (Beneroso & Robinson, 2022;Dym et al., 2005) for increasing knowledge, student satisfaction, and improving 4Cs (Muhammad et al., 2020;Mutohhari, Sutiman, et al., 2021). Various problems of vocational teachers have been discussed, including the understanding, implementation strategies, and evaluation of PjBL (Sudjimat et al., 2019). This situation is exacerbated by the COVID-19 pandemic, which has led to online learning policies being implemented to anticipate widespread adverse impacts. Empirical studies reveal that the fundamental problem in implementing project-based learning is that its management is quite complex compared to other learning models (Sadrina et al., 2018). As a result, vocational teachers find it difficult to appropriately determine the concept of implementing PjBL (Ichikawa et al., 2019). In addition, student learning outcomes are at stake, and student motivation and interest have decreased (Prasetya et al., 2019). Other researchers also revealed the problem of PjBL implementation, namely the difficulty of teachers (Mutohhari, Sutiman, et al., 2021;Shpeizer, 2019;Strevy, 2014). The problems described can be solved through the training and coaching of vocational teachers (Mikkonen et al., 2017). Furthermore, vocational teachers understand the concept of implementation, preparation of PjBL tools, implementation, and evaluation of PjBL. The training program significantly improves the quality of learning, especially interactive classroom management, and motivated students, increased learning interest and self-satisfaction (Almahasees et al., 2021;Darling-Hammond et al., 2019). This problem is further exacerbated by the COVID-19 pandemic, which has transformed offline learning into online. The difficulty of vocational teachers in managing project-based learning is increasing, and learning outcomes are also decreasing (Noviyanti et al., 2021;Pan et al., 2021). Based on the teacher's statements during the interview, their readiness to implement the project-based learning model is minimal. Teacher literacy in developing online learning concepts is still minimal (Astuti et al., 2021;Kholifah et al., 2020;Mutohhari, Sofyan, et al., 2021). Moreover, you have to apply a project-based practical learning model held online. Furthermore, it must use a useful projectbased learning model born online. In addition, students' awareness and ability to be active in participating in O-PjBL is also a crucial problem (Aguskin & Maryani, 2020;Almulla, 2020). The difficulties of vocational education teachers in implementing project-based learning must be resolved immediately so that the 21st-century skills of students currently needed can be adequately achieved (Malik, 2018;Nurtanto et al., 2021). Various efforts have been made to overcome these difficulties, starting from developing models, media, and learning resources that can support the implementation of efficient online learning (Mutohhari, Sudira, et al., 2021;Rabiman et al., 2021). However, the developments that have been carried out have not been significantly influential in overcoming the problem of managing O-PjBL in vocational education (Onyema et al., 2020). The various developments that have been carried out will be more effective if accompanied by socialization and intensive training for teachers. Training and assistance related to the management of O-PjBL models are crucial for understanding and managing these models, which are carried out online (Melki & Bouzid, 2021). However, training and coaching, starting from planning, implementation, and evaluation, must be carried out correctly in integrated project-based learning on digital technology using networks. Thus, this study aims to analyze and formulate the concept of training and coaching of O-PjBL models and evaluate them.

METHOD
This research is a quantitative study to test the perception and a quasi-experimental to test the effectiveness by adopting the Nonequivalent Control Group Design, which refers to the opinion (Thyer, 2012). The study involved vocational education teachers in Yogyakarta Province. The study used a quantitative data approach with orientation to the collected data and then analyzed using a t-test with four different methods to measure the effectiveness of training and coaching O-PjBL models for vocational education teachers. Data were collected through questionnaires, learning outcomes tests, and observations. The research was conducted in conjunction with community service carried out in vocational education in Yogyakarta Province. A total of 70 vocational education teachers in the province of Yogyakarta were involved as subjects of this study, which included 35 subjects as perception test samples and 70 samples as experimental samples, with the division of each class (experimental & control) as many as 35 subjects. The details of the research sample are grouped based on certain aspects and are presented in Table 1. Data was collected using a questionnaire method to measure teacher perceptions of training and coaching, learning outcomes test methods, and observation methods to measure the effectiveness of training and coaching in improving teacher skills in managing O-PjBL models. The first research instrument used a question with a four scale, namely Strongly Agree (SA), Agree (A), Disagree (D), and Strongly Disagree (SD). Meanwhile, the second instrument is a learning outcome test which consists of theoretical questions that have been presented through training. And the last instrument is an observation sheet with four assessment criteria, namely Very Good (VG), Good (G), Not Good (NG), and Very Not Good (VNG), to assess the draft of the learning plan. Table 2 presents the research instrument grid. Test (Guo et al., 2020;Mustapha et al., 2020) Observation (Guo et al., 2020;Pawar et al., 2020) The relevance of the material/assistance delivered The suitability of the content of the material/training with the needs of the participants Quality of presentation & delivery of materials/training Mastery of materials/training Appropriate time allocation Definition of online project-based learning Characteristics of online project-based learning Syntax of online project-based learning Examples of implementation of online project-based learning Product conformity with competency goals & achievements Consistency and clarity of the material and teaching media Clarity of learning scenario syntax Completeness of scientific aspects of learning Time allocation accuracy Appraisal technique accuracy 3 2 2 2 1 1 3 3 3 2 2 2 1 1 2 Descriptive analysis, descriptive and inferential statistical analysis was used to analyze the collected data. Descriptive research is used to describe implementing training and coaching programs. Descriptive statistics were used to analyze the average and percentage of perception test results concerning the criteria presented in Table 3. Meanwhile, inferential statistical analysis used Dunnett, Tukey's t-test, paired samples, and independent samples to analyze the difference in mean and the effect between aspects or between classes. Less Good Not Good (Mardapi, 2012)

Result PDCE in the Implementation of Training and Coaching for Vocational Teachers
Training and coaching for vocational teachers to prepare integrated online learning Project Based Learning (O-PjBL) is carried out in several phases, including (1) phase 1pre-test of teacher understanding of PjBL; (2) phase 2project analysis based on subjects and preparation of learning tools; (3) phase 3presentation of the concept of learning tools with the O-PjBL approach; (4) phase 4-post-test and product assessment (Presented in Figure. 1). Training and coaching activities are carried out simultaneously. The concept of PjBL learning focuses on the final result, namely creative products that are packaged during learning. The product in question is a work including SOPs, drawings, crafts, stages, or procedures up to appropriate technology. This confirms that PjBL can be applied flexibly in all subjects, including the social field. The "Plan" stage is the teacher analyzes the syllabus and lesson plans and sorts out the competencies that can be applied to project-based learning. Because for the first time implementing PjBL in online learning and there are still limited teachers actively implementing PjBL, the approach chosen is the silo. The silo approach is that PjBL is applied by subject teachers independently without involving other teachers following the project field. Vocational teachers determine the PjBL concept with several considerations, namely routine problems in daily life, easy processing of raw materials, and simple for students to work. The project proposed by the teacher is based on the following subjects (see Table 4.) Creating a mocopat song Javanese language 15 Creating gurgles Javanese language 16 Acid-base and solution pH Chemical 17 hand sanitizer Chemical 18 Hosting control panel Network system administration 19 CLI operating system Basic Computers and Networks 20 Cutout image Engineering drawings Next, the "Do" stage is strengthens understanding of PjBL during the four JP (study hours) training. The first knowledge about the definition of PjBL and its implementation in vocational learning. The second knowledge is about the preparation of PjBL-based learning tools. After the exposure of the material and understanding of vocational teachers to PjBL increased, the applicative process continued, namely the preparation of a learning implementation plan. Concerning training and coaching aimed at assisting in making lesson plans and limited implementation practices carried out in groups consisting of three types of teachers, namely normative, adaptive and productive. Combining the three types of teachers is to produce concepts and designs of integrated learning plans between disciplines. In addition, the purpose of the merger is also to improve time efficiency and provide students with an understanding of multidisciplinary science, an example of an interdisciplinary approach that has been widely used in STEM integrated project-based learning.The "Check" stage is the implementation of observation of the preparation of learning tools and the development of vocational teachers. In this stage, the teacher is divided into three rooms, with each room consisting of 12 people. In coaching and monitoring, participants are targeted to draft a lesson plan product within 4 hours. The learning implementation plan contains the technical implementation of the project-based learning model carried out online. Then after the draft is finished, the lesson plans are practiced in a limited way in the room.
The "Evaluation" stage is the evaluation stage, containing a final evaluation and a perception test of the training and coaching materials. In addition, the evaluation stage also includes preparing the primary draft of the lesson plan, which is carried out individually. Participants are given three days to complete the draft. After the draft is collected, the finished lesson plan is assessed with a minimum average criterion of "good" based on fourteen indicators covering four aspects. Namely, (1) aspects of the suitability of the learning context with learning outcomes, (2) aspects of fundamental competency analysis that are appropriate for implementing the O-PjBL model, (3) aspects of clarity syntax, and (4) aspects of the accuracy of learning evaluation techniques. Implementing the concept of training and coaching of O-PjBL models for vocational education teachers consists of four steps, namely Plan, Do, Check, and Evaluate (PDCE). Table 5 presents the stages' details and the implementation methods and activities. Discussions, questions and answers, project-based learning, presentation.
Initial and final understanding test, evaluation of resources, training and coaching.

Vocational Teachers' Perception of O-PjBL Training and Coaching
The concept of training and coaching implemented then analyzed the vocational teachers' perceptions who follow it. The perception test aims to find out the teachers' statements regarding the quality and suitability of the training and coaching content. The perception test involved 35 participants who were vocational education teachers. Aspects tested for perceptions were aspects of delivering material by the presenters and aspects of coaching and monitoring the making of O-PjBL learning plans. Meanwhile, the indicators of vocational teachers' perceptions that were explored based on statements consisted of the relevance of training materials/assistance with instructional objectives, the suitability of the content of the material/training with the needs of the participants, Quality of presentation and delivery of material/training, Mastery of materials/training, Appropriateness of time allocation. This perception test involved 35 participants divided into three characteristics, namely gender, age range, and employment status. The perception test results using descriptive statistics are presented in Figure 2.

Figure 2. Vocational Teachers' Perception
The total score of participants' perceptions of the material, training, and coaching is 1.505. The average is 43 with 86.55% and is included in the very good category. Based on these results, it can be interpreted that the participants are very satisfied with the content of the O-PjBL training and coaching carried out. Vocational education teachers stated that content relevance, suitability, presentation quality, mastery of content, and time allocation for training and coaching were on the "very good" criteria. Then, the following analysis tests the difference in the average perceptions of participants using comparative analysis using three different types of analysis to determine the perception of vocational teachers' perception of online project-based training and coaching. The aspect category is divided into three categories, namely based on gender (male; female), employment status (honorary teacher, civil servant, certified teacher), and age range (≤ 25 years; 26-30 years; 31-35; > 35 years old). Participants' perceptions of training and coaching based on gender categories were analyzed using an independent sample t-test analysis. Table 6 shows the results of the independent t-test analysis of the sample of participants by gender. Based on the independent sample t-test results on the gender aspect, it is known that there is no significant difference between the perceptions of male and female participants on O-PjBL training and coaching. This means that the perception of equality occurs in the gender aspect of the training and assistance carried out. Then table 7 shows the perception test results using the Dunnett C test on employment status on O-PjBL training and coaching. Based on the Dunnett C test results on the aspect of employment status, the results obtained in the form of a calculated F value of 0.805 with a significance value of 0.456. The results concluded no difference in the average perception between honorary teachers, civil servants, and certified teachers on O-PjBL training and coaching. Then Table 8 shows the perception test results using the Tukey test on the aspect of the age range of the participants for O-PjBL training and coaching.

Effectiveness of O-PjBL Training and Coaching
Effectiveness, in this case, refers to the extent to which teacher skills improve in managing O-PjBL in their respective fields. Testing the effectiveness of training and coaching of O-PjBL models was carried out on two aspects: participants' understanding of the training and coaching materials and the results of the integrated learning plan product of the O-PjBL model. The pre-test -post-test data and the value of the draft lesson plan were analyzed using paired sample t-test and independent sample t-test, which was preceded by analysis prerequisite tests (normality and homogeneity). Based on the normality test results, it is known that all classes have a significance value of more than 0.05, or it can be called > 0.05. Thus, it can be concluded that the data is normally distributed to be used for further analysis, namely the paired sample t-test. Likewise, the homogeneity test obtained a significant value in all classes greater than 0.05 or > 0.05. Thus, it can be concluded that the data variance is homogeneous to be used for further analysis, namely the independent sample t-test. Then, after the analysis prerequisite test is fulfilled, the first is to conduct a paired sample t-test with alternative hypotheses. (Ha 1) "there is a significant positive effect of training and coaching on understanding the O-PjBL model for vocational education teachers". (Ha 2) "there is a significant positive effect of training and coaching on the quality of the integrated learning plan of the O-PjBL model for vocational education teachers". The results of hypothesis testing 1 and 2 are shown in Table 9.

Pair
Mean Diff. t value Sig Pretest experimentposttest experiment Pretest controlposttest control Initial lesson plans experimentfinal lesson plans experiment Initial lesson plans controlfinal lesson plans control -25,000 -10,000 12,340 5,450 -10,481 -2,488 -8,147 -2,081 0,000 0,042 0,000 0,048 The significance value of the experimental class pre-test -post-test on the aspect of material understanding is 0.000 at a significance level of 5%. The significance value is smaller than 0.050, so it can be concluded that there is an increase in the understanding of the material mastered by the participants after receiving training and coaching. Thus, the first alternative hypothesis is accepted, or it can be concluded that training and coaching have been proven to improve the understanding of the management material for the O-PjBL model for vocational education teachers. Likewise, the significance value of the pre-test -post-test pairs of the experimental class on the aspect of the learning plan product outcome is 0.000 at a significance level of 5%. The significance value is less than 0.050, so it can be concluded that there is an increase in the quality of the participants' learning plans after receiving training and coaching. Thus, the second alternative hypothesis is accepted, or it can be concluded that training and coaching are proven to improve the quality of learning plans integrated with the O-PjBL model for vocational education teachers.
Then, the subsequent data analysis was to test the average difference between the experimental class and the control class using the independent sample t-test. The alternative hypotheses in this analysis are as follows: (Ha 3) "there is a difference in the average understanding of the training and coaching participants between the experimental class and the control class"; (Ha 4) "there is an average difference in the quality of the learning plan integrated with the O-PjBL model between the experimental class and the control class. Table 10 presents the results of the independent sample t-test to test the 3rd and 4th hypotheses. The significance value of the independent sample t-test on the aspect of material understanding is 0.004 at a significance level of 5%. The significance value is smaller than 0.050, so it can be concluded that there is a difference in the average level of understanding of the training and coaching materials between the post-test experimental class and the control class. Thus, the third alternative hypothesis is accepted, or it can be concluded that the class that received the training and coaching proved to have a significantly higher level of understanding of the management material of the O-PjBL model. Likewise, the significance value of the independent sample ttest on the quality aspect of the lesson plan is 0.012 at a significance level of 5%. The significance value is smaller than 0.050, so it can be concluded that there is a difference in the average quality of the final lesson plan between the experimental class and the control class. Thus, the fourth alternative hypothesis is accepted, or it can be concluded that the class that received training and coaching was proven to develop a learning plan that was integrated with the O-PjBL model significantly better.

Discussion
The high positive response to the training and coaching of O-PjBL models indicates satisfaction and excellent quality in its implementation. All participants from the three aspects of the characteristics simultaneously, there is no difference in their perception of the implementation of O-PjBL training and coaching. Very good perception of vocational education teachers on the training and coaching can not be separated from various essential factors. Learning management with new systems and media is a fundamental factor in online learning management training and assistance (Simon & Maskit, 2016). The suitability of the material with the context and purpose is also an equally important factor in influencing the positive perceptions of participants (Nemec, 2018). In addition, a comprehensive explanation regarding the ease of implementation that can be collaborated in various disciplines online made participants enthusiastic and gave an excellent response (Kılınç, 2019). Improving teacher competence is an innovation in improving the process of implementing online learning that cannot be separated from a mature concept analysis (Holmes et al., 2019). The concept of project-based learning that can be implemented well online can train students to stay productive during online learning (Chanpet et al., 2020). The effectiveness of the training and coaching cannot be separated from good organization in its implementation (Nemec, 2018). A mature and well-structured concept, from implementation to evaluation, is the most crucial factor in determining the O-PjBL model training and coaching (Abuhmaid, 2020). In addition, the right target of training and coaching for subjects that require O-PjBL model management competencies makes it well received so that the effectiveness of the training and coaching is obtained (Silwal & Bhatta, 2017).
The difference in effectiveness between the experimental class and the control class indicates that the presence of treatment in the form of training and coaching for the O-PjBL model is very helpful in increasing teacher competence in managing effective and efficient online learning (Abuhmaid, 2020). This also indicates that the O-PjBL model is needed to help achieve 21st-century skills and provide good online learning literacy (Wahyudi & Winanto, 2018;Yunus et al., 2021). The effectiveness of the training and coaching of the O-PjBL model is proven in improving the understanding and quality of the resulting lesson plans (Salma et al., 2021). Thus, the estuary of understanding the O-PjBL model and being able to develop an integrated learning plan with the model will provide a new picture and create new opportunities in its management. Previous research revealed that teachers need reinforcements in implementing project-based learning. The study revealed teachers' difficulties managing project-based learning in the online learning period (Hernáiz-Pérez et al., 2021). Then other relevant research shows the effectiveness of project-based training and coaching for teachers during the COVID-19 pandemic. This training and coaching have been proven to improve understanding and practice of implementing O-PjBL models (Salido López, 2020). Other studies also reveal positive perceptions given by teachers to model implementation training during the COVID-19 period, which states that teachers have very high satisfaction with the implementation of the training and coaching. In addition, teachers in making lesson plans are even better, especially for online learning as it is today (Sartor-Harada et al., 2020). This study provides a view regarding the importance of training and coaching for teachers to improve their ability to manage project-based learning during the online learning period. In addition, this study offers a concept that can be used by other training providers in boosting project-based online learning management skills for vocational education teachers. However, this research has limitations on the coverage of vocational education, which is less comprehensive. The most significant rule of the researchers and the team was the COVID-19 pandemic which hampered the training process, so it did not match the previous schedule. This research is limited to improving teachers in designing and compiling lesson plans and limited implementation. Future research is expected to provide a more comprehensive overview and research results related to implementing teacher training and coaching in managing O-PjBL learning.

CONCLUSION
The implementation of training and coaching of the O-PjBL model conceptually consists of four steps, namely Plan, Do, Check, and Evaluate (PDCE). The application of the concept was responded to and received well, as evidenced by the perception test, which showed very good results. The trial application of the concept of training and coaching is also effective in increasing understanding of O-PjBL materials and improving the quality of integrated learning plans for O-PjBL. The mature concept that is designed and followed by a wellplanned implementation is an important factor in the effectiveness of the O-PjBL model training and coaching. In addition, the precise target of training and coaching in line with the competency needs of online learning management for vocational education teachers is also a fundamental factor in implementing O-PjBL training and coaching to be effective and well perceived. This concept is expected to provide a reference for the management of adequate training and coaching, especially in increasing interactive and productive vocational online learning.