A Validated Instrument to Assess Representational Interpretation Skills in Basic Chemistry Laws: A Development and Validation Study
)
| 
Country:
(1) Department of Chemical Education, Universitas Lampung, Indonesia.
(2) Department of Chemical Education, Universitas Lampung, Indonesia.
Corresponding Author
10.61436/bsscd/v4i1.pp01-13
| Metrics→ |
| Indexing Site → |
|
|
|
|
Copyright (c) 2025 Lisa Tania, Andrian Saputra
The abstract and symbolic nature of chemistry presents significant learning challenges for students, particularly in the understanding of foundational concepts through a variety of representations. It's tougher to teach well, especially in Indonesia, because there are not many trustworthy instruments to measure students' interpretative skills, which are very important for representational competence. The goal of this study was to fill this gap by creating and testing a reliable assessment tool called the Assessment of Representational Competence in Fundamental Chemical Laws (ARC-FCL), which was meant to measure the interpretative skills of Indonesian senior high school students. The Define, Design, and Develop stages of the 4D model were employed in the Research and Development process of this study design. The Kozma-Russell framework and the Three-Dimensional Learning Assessment Protocol (3D-LAP) helped shape the instrument's development. The ARC-FCL test has two levels of multiple-choice and essay questions about the laws of Lavoisier, Dalton, and Gay-Lussac. Two chemistry education professors and one experienced instructor reviewed the content to ensure its validity. Researchers got real-world data for psychometric analysis from 30 11th-grade students at a senior high school in Lampung, Indonesia. The study looked at item validity using Pearson correlation, item discrimination, and internal consistency reliability using Cronbach alpha. All three designed items had great validity, with correlation values over the key threshold of 0.361 and good discrimination indices between 0.50 and 0.75. The instrument was somewhat reliable, with a Cronbach alpha coefficient of 0.572. This is fine for a formative diagnostic tool at this early stage of development. Item analysis also showed that students had the most trouble with the question about the Law of Multiple Proportions. In conclusion, this study developed a valuable and valid instrument for identifying the specific challenges faced by students when attempting to comprehend sub-microscopic representations of fundamental chemical principles. As a formative assessment instrument that fits with the Kurikulum Merdeka, the ARC-FCL is quite useful for Indonesian teachers. It encourages a move from memorizing things by heart to comprehending them better and learning how to do science. Future work should be focused on adding more items to the bank to make it more reliable and cover more ground.
Keywords: representational competence, assessment instrument, interpretive skills, basic chemical laws, 4D model, instrument validation.
Adadan, E. (2013). A rubric for assessing high school students’ conceptual understanding of chemical bonding. Educational Assessment, 18(1), 1-27.
Adlim, A., & Saminan, S. (2021). The need for assessment instruments for chemical literacy in the context of socio-scientific issues of the COVID-19 pandemic. Jurnal Pendidikan IPA Indonesia, 10(1), 12-23. https://doi.org/10.15294/jpii.v10i1.26121
Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183-198.
Briggs, D. C., & Alonzo, A. C. (2012). The psychometric modeling of learning progressions. Journal of Educational Measurement, 49(4), 427-447.
Chandrasegaran, A. L., Treagust, D. F., & Mocerino, M. (2007). The development of a two-tier multiple-choice diagnostic instrument for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple levels of representation. Chemistry Education Research and Practice, 8(3), 293–307. https://doi.org/10.1039/B7RP90006F
Cooper, M. M., Stieff, M., & Dori, Y. J. (2016). The language of chemistry. In The Wiley handbook of science teaching and learning (pp. 371-396). John Wiley & Sons.
Devetak, I., & Glažar, S. A. (2014). The influence of animations on students' understanding of chemical concepts at the submicroscopic level. Chemistry Education Research and Practice, 15(4), 603-614.
diSessa, A. A. (2004). Metarepresentation: Native competence and targets for instruction. Cognition and Instruction, 22(3), 293–331. https://doi.org/10.1207/ s1532690xci2203_2
Effendy. (2017). The paradoxical situation of chemistry education in Indonesia and the need for a new direction. Acta Chimica Asiana, 1(1), 27-36. https://doi.org/10.24198/aca.v1i1.13745
Gilbert, J. K., & Treagust, D. F. (Eds.). (2009). Multiple representations in chemical education. Springer. https://doi.org/10.1007/978-1-4020-8872-8
Gkitzia, V., Salta, K., & Tzougraki, C. (2019). Development and application of a diagnostic instrument to assess Greek students’ understanding of the basic concepts of the particulate nature of matter. Chemistry Education Research and Practice, 20(2), 433-447. https://doi.org/10.1039/C8RP00267A
Gkitzia, V., Salta, K., & Tzougraki, C. (2019). Student understanding of the structure of matter in the three states: A study of the effect of an intervention based on a historical and a simulation approach. Chemistry Education Research and Practice, 20(4), 804-820.
Hadenfeldt, J. C., Bernholt, S., Liu, X., & Parchmann, I. (2013). Using a Rasch-based learning progression for matter to facilitate the interpretation of a large-scale assessment in chemistry. Journal of Chemical Education, 90(6), 700-708.
Johnstone, A. H. (1991). Why is science difficult to learn? Things are seldom what they seem. Journal of Computer Assisted Learning, 7(2), 75–83. https://doi.org/10.1111/j.1365-2729.1991.tb00230.x
Keig, P. F., & Rubba, P. A. (1993). Translation of representations of the structure of matter and its relationship to reasoning, gender, and prior knowledge. Journal of Research in Science Teaching, 30(8), 883-901.
Kozma, R. B. (2003). The material features of multiple representations and their cognitive and social affordances for science understanding. Learning and Instruction, 13(2), 205–226. https://doi.org/10.1016/S0959-4752(02)00021-X
Kozma, R., & Russell, J. (2002). Students becoming chemists: Developing representational competence. In J. K. Gilbert, M. de Jong, S. Justi, D. F. Treagust, & J. H. Van Driel (Eds.), Chemical education: Towards research-based practice (pp. 121-145). Kluwer Academic Publishers.
Kozma, R. B., & Russell, J. (2005). Students becoming chemists: Developing representational competence. In J. K. Gilbert (Ed.), Visualization in science education (pp. 121–145). Springer.
Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., Fata-Hartley, C. L., Ebert-May, D., & Cooper, M. M. (2016). Characterizing college science assessments: The Three-Dimensional Learning Assessment Protocol. PLoS ONE, 11(9), e0162333. https://doi.org/10.1371/journal. pone.0162333
Laverty, J. T., Underwood, S. M., Matz, R. L., Posey, L. A., Carmel, J. H., Caballero, M. D., Fata-Hartley, C. L., Ebert-May, D., & Couch, B. A. (2016). A computer-based assessment for measuring students’ foundational knowledge of the chemistry of biology. CBE—Life Sciences Education, 15(2), ar17.
Loh, A. S. L., Subramaniam, R., & Tan, K. C. D. (2014). Exploring students’ understanding of the states of matter and solution chemistry using a three-tier diagnostic instrument. International Journal of Science Education, 36(15), 2475-2499. https://doi.org/10.1080/09500693.2014.920551
Loh, Y. L., Subramaniam, R., & Tan, S. K. (2014). Using a three-tier diagnostic instrument to assess secondary school students' understanding of some chemical concepts. International Journal of Science Education, 36(5), 783-806.
Nakhleh, M. B. (1992). Why some students don’t learn chemistry: Chemical misconceptions. Journal of Chemical Education, 69(3), 191. https://doi.org/10.1021/ed069p191
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. National Academy Press.
Rahmawati, Y., & Koul, R. (2019). The challenges of implementing a new science curriculum in Indonesian secondary schools. In D. F. Treagust, R. Koul, & W. F. Lin (Eds.), Science Teacher Education in the Asia-Pacific (pp. 81-94). Routledge.
Schnotz, W. (2002). Towards an integrated view of learning from text and pictures. In J. Otero, J. A. León, & A. C. Graesser (Eds.), The psychology of science text comprehension (pp. 385–416). Lawrence Erlbaum Associates Publishers.
Stieff, M., Hegarty, M., & Deslongchamps, G. (2011). Identifying representational competence with multi-representational displays. Cognition and Instruction, 29(1), 123–145. https://doi.org/10.1080/07370008.2010.507318
Sun, D., & Furtak, E. M. (2016). A systematic review of the literature on the formative assessment of scientific argumentation. Educational Research Review, 18, 50-67.
Taber, K. S. (2013). Three levels of chemistry educational research. Chemistry Education Research and Practice, 14(2), 151-155.
Tang, K. S. (2016). The interplay of representations and patterns of interaction in a science classroom. Science Education, 100(4), 611-641.
Thiagarajan, S., Semmel, D. S., & Semmel, M. I. (1974). Instructional development for training teachers of exceptional children: A sourcebook. Indiana University.
Treagust, D. F. (1986). Evaluating students' misconceptions by means of diagnostic multiple choice questions. Research in Science Education, 16(1), 199-207. https://doi.org/10.1007/BF02356835
Treagust, D. F., Chittleborough, G. D., & Mamiala, T. L. (2003). The role of submicroscopic and symbolic representations in chemical explanations. International Journal of Science Education, 25(11), 1353–1368. https://doi.org/10.1080/0950069032000075426
Tytler, R., Prain, V., Hubber, P., & Waldrip, B. (Eds.). (2013). Constructing representations to learn in science. Sense Publishers.
Wu, H. K., & Shah, P. (2004). Exploring visuospatial thinking in chemistry learning. Science Education, 88(3), 465-492.
Refbacks
- There are currently no refbacks.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The copyright is reserved to Bulletin of Social Studies and Community Development that is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Address:
STKIP Al Islam Tunas Bangsa: Jalan Z.A. Pagar Alam No. 41 Gedong Meneng Bandar Lampung
Institute of Multidisciplinary Research and Community Service: Jalan Airan Raya, Way Huwi, Lampung Selatan