Kirkpatrick Four-level Model Evaluation: An Evaluation Scale on the Preservice Teacher’s Internship Program

Muhammad Bukhori Dalimunthe

doi:10.23887/jere.v6i2.43535

Authors

Muhammad Bukhori Dalimunthe Universitas Negeri Medan http://orcid.org/0000-0002-2422-5568

DOI:

https://doi.org/10.23887/jere.v6i2.43535

Keywords:

Evaluation, Apprenticeship, Kirkpatrick's Model

Abstract

Internship programs are organized depending on the program provider (school), so many students have a big difference in the internship experience. On the other hand, the campus has limitations in running the program, one of which is evaluating the achievement of the internship program. The purpose of this study is to build validity on the evaluation scale of the apprenticeship program for pre-service teachers using Kirkpatrick's four-level evaluation model. This type of research is quantitative with a confirmatory quantitative approach. The sample is 212 pre-service teachers. The methods used in collecting data are observation, tests, and questionnaires. Instruments used for test sheets and questionnaires. Data analysis used a confirmatory factor analysis (CFA) approach with the SmartPLS v3.0 application. The analysis results are that the four-level evaluation model meets construct validity and proves accurate validation on the internship program evaluation scale. In particular, the internship program evaluation scale effectively tracks program achievements at each level. The implication is that academic stakeholders on campus can use this evaluation scale as a measuring tool to experiment with the success of the internship program. In addition, they can review the achievements of pre-service teachers in implementing apprenticeship programs based on each level of the Kirkpatrick model.

Author Biography

Muhammad Bukhori Dalimunthe, Universitas Negeri Medan

Department of Economics Education

Faculty of Economics

Universitas Negeri Medan

References

Adams, T., Koster, B., & Brok, P. Den. (2020). Student teachers’ classroom management during the school internship. European Journal of Teacher Education, 1–19. https://doi.org/10.1080/02619768.2020.1860011.

Ahanchian, M., Sharafi, S., Vafaee, M., & Hajiabadi, F. (2017). Evaluate the Effectiveness of internship program in nursing student using Kirkpatrick’s model TT. gums-rme, 9(1), 17–19. https://doi.org/10.18869/acadpub.rme.9.1.17.

Amir, T., & Ismail, O. (2014). An evaluation of the internship program at the college of education, Sultan Qaboos University, Muscat. International Journal of Humanities, Social Sciences and Education, 1(10), 5–10. https://doi.org/10.13140/RG.2.2.10633.47200.

Aykac, N., Yildirim, K., Altinkurt, Y., & Marsh, M. M. (2017). Understanding the underlying factors affecting the perception of pre-service teachers’ teacher identity: A new instrument to support teacher education. Üniversitepark Bülten, 6(1), 67. https://doi.org/10.22521/unibulletin.2017.61.6.

Badu, S. Q. (2013). Implementasi Evaluasi Model Kirkpatrick Pada Perkuliahan Masalah Nilai Awal Dan Syarat Batas. Jurnal Penelitian dan Evaluasi Pendidikan, 16, 102–129. https://doi.org/10.21831/pep.v16i0.1108.

Baharuddin, M. R. (2021). Adaptasi Kurikulum Merdeka Belajar Kampus Merdeka (Fokus: Model MBKM Program Studi). Jurnal Studi Guru dan Pembelajaran, 4(1). https://doi.org/10.30605/jsgp.4.1.2021.591.

Bush, A., & Grotjohann, N. (2020). The impact of a long-term internship on pre-service teacher collaboration. Journal of Innovation in Psychology, Education and Didactics, 24(1).

Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7(3), 309–319. https://doi.org/10.1037/1040-3590.7.3.309.

Creswell, J. W. (2014). Research design: Quantitative, qualitative and mixed methods (4th ed). Thousand Oaks, CA: Sage.

Dalimunthe, M. B., Nugrahadi, E. W., Fitrawaty, & Dalimunthe, R. Z. (2019). Evaluation of Teacher Professional Training Program with Contextual, Input, Process, and Product (CIPP) Models. RedWhite Press PP - Jakarta. https://doi.org/10.32698/hum0213.

Dewanto, A. C. (2018). Efektivitas PPL untuk meningkatkan efikasi diri mengajar pada mahasiswa pendidikan matematika. Delta: Jurnal Ilmiah Pendidikan Matematika, 6(1), 1–9. https://doi.org/10.31941/delta.v6i1.662.

Dewi, L. R., & Kartowagiran, B. (2018). An evaluation of internship program by using Kirkpatrick evaluation model. REID (Research and Evaluation in Education), 4(2), 155–163. https://doi.org/10.21831/reid.v4i2.22495.

Direktorat Pembelajaran. (2017). Panduan Program Pengenalan Lapangan Persekolahan. In Direktorat Jenderal Pembelajaran dan Kemahasiswaan. Kementerian Riset, Teknologi, dan Pendidikan Tinggi.

Effendi, Zainuddin, Z., & Ahmad, M. S. (2022). Implementasi evaluasi model kirkpatrick terhadap kualitas sistem pembelajaran. Jurnal Educatio, 8(1). https://doi.org/10.29210/1202221160.

Goldhaber, D. (2018). Evidence-Based Teacher Preparation: Policy Context and What We Know. Journal of Teacher Education, 70(2), 90–101. https://doi.org/10.1177/0022487118800712.

Gracia, E. P., Rodríguez, R. S., Pedrajas, A. P., & Carpio, A. J. (2021). Teachers’ professional identity: validation of an assessment instrument for preservice teachers. Heliyon, 7(9), e08049. https://doi.org/10.1016/j.heliyon.2021.e08049.

Gravett, S., Petersen, N., & Ramsaroop, S. (2019). A University and School Working in Partnership to Develop Professional Practice Knowledge for Teaching . In Frontiers in Education (Vol 3, bl 118).

Hair, J., Hollingsworth, C. L., Randolph, A. B., & Chong, A. Y. L. (2017). An updated and expanded assessment of PLS-SEM in information systems research. Industrial Management & Data Systems, 117(3), 442–458. https://doi.org/10.1108/IMDS-04-2016-0130.

Hair Jr, J. F., Hult, G. T. M., Ringle, C., & Sarstedt, M. (2016). A primer on partial least squares structural equation modeling (PLS-SEM). Sage publications.

Henseler, J., Hubona, G., & Ray, P. A. (2016). Using PLS path modeling in new technology research: updated guidelines. Industrial Management & Data Systems, 116(1), 2–20. https://doi.org/10.1108/IMDS-09-2015-0382.

Henseler, J., Ringle, C. M., & Sarstedt, M. (2015). A new criterion for assessing discriminant validity in variance-based structural equation modeling. Journal of the Academy of Marketing Science, 43(1), 115–135. https://doi.org/10.1007/s11747-014-0403-8.

Ismail, I., Hasan, H., & Musdalifah, M. (2018). Pengembangan Kompetensi Mahasiswa Melalui Efektivitas Program Magang Kependidikan. Edumaspul: Jurnal Pendidikan, 2(1), 124–132. https://doi.org/10.33487/edumaspul.v2i1.48.

Jamil, N. A., Shariff, S. M., & Abu, Z. (2013). Students’ Practicum Performance of Industrial Internship Program. Procedia - Social and Behavioral Sciences, 90, 513–521. https://doi.org/10.1016/j.sbspro.2013.07.121.

Jöreskog, K. G., Olsson, U. H., & Wallentin, F. Y. (2016). Multivariate analysis with LISREL. Springer. https://doi.org/10.1007/978-3-319-33153-9.

Kazaz, I., & Alagözlü, N. (2020). Evaluation of teaching practicum for pre-service English language teachers: A scale development study. Dil ve Dilbilimi Çalışmaları Dergisi, 16(3), 1562–1593. https://doi.org/10.17263/jlls.803902.

Kirkpartrick, D. (1959). Techniques for evaluation training programs. Journal of the American Society of Training Directors, 13, 21–26.

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2007). Implementing the Four Levels: A Practical Guide for Effective Evaluation of Training Programs. Berrett-Koehler Publishers, Inc.

Kirkpatrick, M., Akers, J., & Rivera, G. (2019). Use of Behavioral Skills Training with Teachers: A Systematic Review. Journal of Behavioral Education, 28(3), 344–361. https://doi.org/10.1007/s10864-019-09322-z.

Liu, S.-H. (2016). Teacher education programs, field-based practicums, and psychological factors of the implementation of technology by pre-service teachers. Australasian Journal of Educational Technology, 32(3 SE-Articles). https://doi.org/10.14742/ajet.2139.

Mahmoodi, M., Rashtchi, M., & Abbasian, G. R. (2019). Evaluation of in-service teacher training program in iran: Focus on the Kirkpatrick model. Education and Self Development, 14(4), 20–38. https://doi.org/10.26907/esd14.4.03.

Makarim, N. A. (2021). Indikator Kinerja Utama Perguruan Tinggi Negeri dan Lembaga Layanan Pendidikan Tinggi. Kementerian Pendidikan dan Kebudayaan.

Marufi, M., Ilyas, M., Anas, A., & Islamiah, R. Y. (2018). Program Pengembangan Kewirausahaaan Fakultas Keguruan dan Ilmu Pendidikan. Caradde: Jurnal Pengabdian Kepada Masyarakat, 1(1). https://doi.org/10.31960/caradde.v1i1.13.

Narayanan, V. K., Olk, P. M., & Fukami, C. V. (2010). Determinants of internship effectiveness: An exploratory model. Academy of Management Learning & Education, 9(1), 61–80. https://doi.org/10.5465/amle.9.1.zqr61.

Nghia, T. L. H. (2018). External stakeholders’ roles and factors influencing their participation in developing generic skills for students in Vietnamese universities. Journal of Education and Work, 31(1), 72–86. https://doi.org/10.1080/13639080.2017.1386774.

Nghia, T. L. H., & Duyen, N. T. M. (2019). Developing and validating a scale for evaluating internship-related learning outcomes. Higher Education, 77(1), 1–18. https://doi.org/10.1007/s10734-018-0251-4.

Nghia, T. L. H., & My Duyen, N. T. (2018). Internship-related learning outcomes and their influential factors. Education + Training, 60(1), 69–81. https://doi.org/10.1108/ET-02-2017-0030.

Piryani, R. M., Dhungana, G. P., Piryani, S., & Sharma Neupane, M. (2018). Evaluation of teachers training workshop at Kirkpatrick level 1 using retro-pre questionnaire. Advances in Medical Education and Practice, 9, 453–457. https://doi.org/10.2147/AMEP.S154166.

Praslova, L. (2010). Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educational Assessment, Evaluation and Accountability, 22(3), 215–225. https://doi.org/10.1007/s11092-010-9098-7.

Putman, S. M., & Polly, D. (2021). Examining the Development and Implementation of an Embedded, Multi-Semester Internship: Preliminary Perceptions of Teacher Education Candidates, Clinical Educators, and University Faculty. Peabody Journal of Education, 96(1), 99–111. https://doi.org/10.1080/0161956X.2020.1864250.

Ramadhani, M. A., & Rahayu, A. (2020). Competency Improvement through Internship: An Evaluation of Corporate Social Responsibility Program in Vocational School. International Journal of Evaluation and Research in Education, 9(3), 625–634. https://doi.org/10.11591/ijere.v9i3.20571.

Streiner, D. L. (2003). Starting at the Beginning: An Introduction to Coefficient Alpha and Internal Consistency. Journal of Personality Assessment, 80(1), 99–103. https://doi.org/10.1207/S15327752JPA8001_18.

Sunardi, S., & Sudjimat, D. A. (2016). Magang Industri Untuk Meningkatkan Relevansi Kompetensi Profesional Guru Produktif SMK. Teknologi dan Kejuruan, 39(2). https://doi.org/10.17977/tk.v39i2.7791.

Tindowen, D. J., Bangi, J., & Parallag Jr, C. (2019). Pre-Service Teachers’ Evaluation on their Student Internship Program. International Journal of Learning, Teaching and Educational Research, 18(10), 279–291. https://doi.org/10.26803/ijlter.18.10.18.

Tovey, J. (2001). Building Connections between Industry and University: Implementing an Internship Program at a Regional University. Technical Communication Quarterly, 10(2), 225–239. https://doi.org/10.1207/s15427625tcq1002_7.

Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). A journey around alpha and omega to estimate internal consistency reliability. Annals of Psychology, 33(3), 755–782. https://doi.org/https://doi.org/10.6018/analesps.33.3.268401.

Widhiarso, W. (2017). Semantik Diferensial. Fakultas Psikologi UGM. Yogyakarta, 10.