Development of a Creative Problem-Solving test-based Higher Order Thinking Skills: A Rasch Model Measurement

Farida Farida(1), Yosep Aspat Alamsyah(2), Suherman Suherman(3*)

(1) Universitas Islam Negeri Raden Intan Lampung
(2) Universitas Islam Negeri Raden Intan Lampung
(3) University of Szeged, Hungary and Universitas Islam Negeri Raden Intan Lampung
(*) Corresponding Author

Abstract


Creative Problem Solving (CPS) is increasingly valued in independent curriculum education, yet the comprehensive psychometric validation of CPS assessment tools remains limited. This study aimed to develop and validate an instrument to measure CPS skills among students using the Rasch model. A total of 137 higher education students participated. Analysis showed the assessment items fit the Rasch Model well, with strong item-person alignment, effective discrimination among difficulty levels, and good reliability. The Wright map revealed clear patterns between item difficulty and student ability. One item showed Differential Item Functioning (DIF) favouring male students. Furthermore, fourth-semester female students outperformed those in later semesters. Significant differences in CPS ability were found between gender and between urban and rural students, emphasising the need for targeted differentiation strategies in education. These findings underscore the importance of using validated and inclusive assessments to inform equitable teaching practices and curriculum design that support diverse learners in developing CPS skills.

Keywords


Creative problem solving; Rasch evaluation; higher education student; higher oreder thinking

Full Text:

PDF (Indonesian)

References


Abdulla Alabbasi, A. M., Hafsyan, A. S., Runco, M. A., & AlSaleh, A. (2021). Problem finding, divergent thinking, and evaluative thinking among gifted and nongifted students. Journal for the Education of the Gifted, 44(4), 398–413. https://doi.org/10.1515/ctra-2018-0019

Abosalem, Y. (2016). Assessment techniques and students’ higher-order thinking skills. International Journal of Secondary Education, 4(1), 1–11. https://doi.org/10.11648/j.ijsedu.20160401.11

Andrich, D. (2017). Advances in social measurement: A Rasch measurement theory. In Perceived health and adaptation in chronic disease (pp. 66–91). Routledge.

Bond, T., & Fox, C. M. (2015). Applying the Rasch Model: Fundamental Measurement in the Human Sciences, Third Edition (3rd ed.). Routledge. https://doi.org/10.4324/9781315814698

Boone, W. J., Staver, J. R., & Yale, M. S. (2013). Rasch analysis in the human sciences. Springer.

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4

Burns, D. P., & Norris, S. P. (2009). Open-minded environmental education in the science classroom. Paideusis, 18(1), 36–43. https://doi.org/10.7202/1072337ar

Cappelleri, J. C., Lundy, J. J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical Therapeutics, 36(5), 648–662. https://doi.org/10.1016/j.clinthera.2014.04.006

Care, E., & Kim, H. (2018). Assessment of twenty-first century skills: The issue of authenticity. Assessment and Teaching of 21st Century Skills: Research and Applications, 21–39. https://doi.org/10.1007/978-3-319-65368-6_2

Carnevale, A. P., & Smith, N. (2013). Workplace basics: The skills employees need and employers want. In Human Resource Development International (Vol. 16, Issue 5, pp. 491–501). Taylor & Francis. https://doi.org/10.1080/13678868.2013.821267

Caty, G. D., Arnould, C., Stoquart, G. G., Thonnard, J.-L., & Lejeune, T. M. (2008). ABILOCO: a Rasch-built 13-item questionnaire to assess locomotion ability in stroke patients. Archives of Physical Medicine and Rehabilitation, 89(2), 284–290. https://doi.org/10.1016/j.apmr.2007.08.155

Chan, S. W., Looi, C. K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: A rasch model measurement analysis. Journal of Computers in Education, 8(2), 213–236. https://doi.org/10.1007/s40692-020-00177-2

Cho, S., & Lin, C.-Y. (2010). Influence of family processes, motivation, and beliefs about intelligence on creative problem solving of scientifically talented individuals. Roeper Review, 33(1), 46–58. https://doi.org/10.1080/02783193.2011.530206

Eberle, B., & Stanish, B. (2021). CPS for kids: A resource book for teaching creative problem-solving to children. Routledge.

Farida, F., Alamsyah, Y. A., Anggoro, B. S., Andari, T., & Lusiana, R. (2024). Rasch Measurement Validation of an Assessment Tool for Measuring Students’ Creative Problem-Solving through the Use of ICT. Pixel-Bit. Revista de Medios y Educación, 71, 83–106. https://doi.org/10.12795/pixelbit.107973

Farida, F., Supriadi, N., Andriani, S., Pratiwi, D. D., Suherman, S., & Muhammad, R. R. (2022). STEM approach and computer science impact the metaphorical thinking of Indonesian students’. Revista de Educación a Distancia (RED), 22(69). https://doi.org/10.6018/red.493721

Florida, R. (2014). The creative class and economic development. Economic Development Quarterly, 28(3), 196–205. https://doi.org/10.1177/0891242414541693

Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., & Csapó, B. (2013). Complex problem solving in educational contexts—Something beyond g: Concept, assessment, measurement invariance, and construct validity. Journal of Educational Psychology, 105(2), 364. https://doi.org/10.1037/a0031856

Hao, J., Liu, L., von Davier, A. A., & Kyllonen, P. C. (2017). Initial steps towards a standardized assessment for collaborative problem solving (CPS): Practical challenges and strategies. Innovative Assessment of Collaboration, 135–156. https://doi.org/10.1007/978-3-319-33261-1_9

Harding, S.-M. E., Griffin, P. E., Awwal, N., Alom, B. M., & Scoular, C. (2017). Measuring collaborative problem solving using mathematics-based tasks. AERA Open, 3(3), 2332858417728046. https://doi.org/10.1177/2332858417728046

Komarudin, K., & Suherman, S. (2024). An Assessment of Technological Pedagogical Content Knowledge (TPACK) among Pre-service Teachers: A Rasch Model Measurement [Evaluación del conocimiento tecnológico pedagógico del contenido (TPACK) entre los profesores en formación: modelo de medición Rasch]. Pixel-Bit. Revista de Medios y Educación, 71, 59–82. https://doi.org/10.12795/pixelbit.107599

Kyngdon, A. (2008). The Rasch model from the perspective of the representational theory of measurement. Theory & Psychology, 18(1), 89–109. https://doi.org/10.1177/095935430708692

Lestari, W., Sari, M. M., Istyadji, M., & Fahmi, F. (2023). Analysis of Implementation of the Independent Curriculum in Science Learning at SMP Negeri 1 Tanah Grogot Kalimantan Timur, Indonesia. Repository Universitas Lambung Mangkurat. https://doi.org/10.36348/jaep.2023.v07i06.001

Linacre, J. M. (2012). Winsteps® Rasch measurement computer program user’s guide. Beaverton, Oregon: Winsteps. Com. https://www.winsteps.com

Lorusso, L., Lee, J. H., & Worden, E. A. (2021). Design thinking for healthcare: Transliterating the creative problem-solving method into architectural practice. HERD: Health Environments Research & Design Journal, 14(2), 16–29. https://doi.org/10.1177/193758672199422

Mitchell, I. K., & Walinga, J. (2017). The creative imperative: The role of creativity, creative problem solving and insight as key drivers for sustainability. Journal of Cleaner Production, 140, 1872–1884. https://doi.org/10.1016/j.jclepro.2016.09.162

Montgomery, K. (2002). Authentic tasks and rubrics: Going beyond traditional assessments in college teaching. College Teaching, 50(1), 34–40. https://doi.org/10.1080/87567550209595870

Panayides, P., Robinson, C., & Tymms, P. (2010). The assessment revolution that has passed England by: Rasch measurement. British Educational Research Journal, 36(4), 611–626. https://doi.org/10.1080/01411920903018182

Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2), 020111. https://doi.org/10.1103/PhysRevPhysEducRes.15.020111

Rusch, T., Lowry, P. B., Mair, P., & Treiblmaier, H. (2017). Breaking free from the limitations of classical test theory: Developing and measuring information systems scales using item response theory. Information & Management, 54(2), 189–203. https://doi.org/10.1016/j.im.2016.06.005

Samson, P. L. (2015). Fostering student engagement: Creative problem-solving in small group facilitations. Collected Essays on Learning and Teaching, 8, 153–164. https://doi.org/10.22329/celt.v8i0.4227

Sousa, F. C., Monteiro, I. P., Walton, A. P., & Pissarra, J. (2014). Adapting creative problem solving to an organizational context: A study of its effectiveness with a student population. Creativity and Innovation Management, 23(2), 111–120. https://doi.org/10.1111/caim.12070

Stankovic, J. A., Sturges, J. W., & Eisenberg, J. (2017). A 21st century cyber-physical systems education. Computer, 50(12), 82–85. https://doi.org/10.1109/MC.2017.4451222

Steiner, G. (2009). The Concept of Open Creativity: Collaborative Creative Problem Solving for Innovation Generation-a Systems Approach. Journal of Business & Management, 15(1).

Suastra, I. W., Ristiati, N. P., Adnyana, P. P. B., & Kanca, N. (2019). The effectiveness of Problem Based Learning-Physics module with authentic assessment for enhancing senior high school students’ physics problem solving ability and critical thinking ability. Journal of Physics: Conference Series, 1171(1), 012027. https://doi.org/10.1088/1742-6596/1171/1/012027

Suherman, S., & Vidákovich, T. (2022). Assessment of Mathematical Creative Thinking: A Systematic Review. Thinking Skills and Creativity, 101019. https://doi.org/10.1016/j.tsc.2022.101019

Suherman, S., & Vidákovich, T. (2025). Ethnomathematical test for mathematical creative thinking. Journal of Creativity, 35(2), 100099. https://doi.org/10.1016/j.yjoc.2025.100099

Suryanto, H., Degeng, I. N. S., Djatmika, E. T., & Kuswandi, D. (2021). The effect of creative problem solving with the intervention social skills on the performance of creative tasks. Creativity Studies, 14(2), 323–335. https://doi.org/10.3846/cs.2021.12364

Tang, T., Vezzani, V., & Eriksson, V. (2020). Developing critical thinking, collective creativity skills and problem solving through playful design jams. Thinking Skills and Creativity, 37, 100696. https://doi.org/10.1016/j.tsc.2020.100696

Utami, Y. P., & Suswanto, B. (2022). The Educational Curriculum Reform in Indonesia: Supporting “Independent Learning Independent Campus (MBKM)”. SHS Web of Conferences, 149. https://doi.org/10.1051/shsconf/202214901041

Val, E., Gonzalez, I., Lauroba, N., & Beitia, A. (2019). How can Design Thinking promote entrepreneurship in young people? The Design Journal, 22(sup1), 111–121. https://doi.org/10.1080/14606925.2019.1595853

van Hooijdonk, M., Mainhard, T., Kroesbergen, E. H., & van Tartwijk, J. (2020). Creative problem solving in primary education: Exploring the role of fact finding, problem finding, and solution finding across tasks. Thinking Skills and Creativity, 37, 100665. https://doi.org/10.1016/j.tsc.2020.100665

Wang, M., Yu, R., & Hu, J. (2023). The relationship between social media-related factors and student collaborative problem-solving achievement: An HLM analysis of 37 countries. Education and Information Technologies, 1–19. https://doi.org/10.1007/s10639-023-11763-z

Wolcott, M. D., McLaughlin, J. E., Hubbard, D. K., Rider, T. R., & Umstead, K. (2021). Twelve tips to stimulate creative problem-solving with design thinking. Medical Teacher, 43(5), 501–508. https://doi.org/10.1080/0142159X.2020.1807483




DOI: http://dx.doi.org/10.30998/jkpm.v10i2.28956

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Farida Farida, Yosep Aspat Alamsyah, Suherman Suherman

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Alamat: Jln. Nangka No. 58C (TB. Simatupang), Kel. Tanjung Barat, Kec. Jagakarsa, Jakarta Selatan 12530 Flag Counter