Constructing Large-Scale Assessment Test Items to Measure Mathematical Literacy: Towards a Theory of Assessment Literacy in Mathematics

Final Defense
Constructing Large-Scale Assessment Test Items to Measure Mathematical Literacy: Towards a Theory of Assessment Literacy in Mathematics

by Mark Lester B. Garcia
PhD Mathematics Education Candidate

Date: Monday, 02 December 2024
Time: 8:30 am
Venue: SECA 321 (MJR Room)

Advisers:
Catherine P. Vistro-Yu, EdD
Ateneo de Manila University

Panelists:
Maria Alva Q. Aberin, PhD (Critic Reader)
Ateneo de Manila University
Kevin Carl P. Santos, PhD (Critic Reader)
University of the Philippines Diliman
Angela Fatima H. Guzon, PhD
Ateneo de Manila University
Reginaldo M. Marcelo, PhD
Ateneo de Manila University
Prof. Emeritus Kaye Stacey, PhD
University of Melbourne

This study proposes a cognitive-behavioral framework for assessment literacy in mathematics. It is grounded in the following theoretical frameworks, namely: Xu & Brown’s (2017) Teacher Assess- ment Literacy in Practice (TALiP), Hayes’ (2012) Cognitive Model of Writing, and Webb’s (1993) proposed theory for assessment literacy in mathematics. In this comprehensive mixed-methods study, the researcher examined teacher-related variables such as teacher background and item properties, and introduced an intervention that came in the form of an intensive item-writing pro- fessional development program for teachers. As such, the researcher sought to determine: (1) how teachers construct PISA-like mathematics items; (2) the relationships among variables related to mathematics teachers’ backgrounds and the variables related to the quality of the items they con- struct, and; (3) the knowledge and skills needed in constructing mathematics items presented in real-world contexts.

The findings of this study suggest that: (1) item-writing is a cognitive, problem-solving, and text production task characterized by sequential phases, decision-making around content and con- text, and iterative writing requiring both skill and instinct; (2) teacher background variables show mostly strong positive relationships (with statistically significant differences in terms of assess- ment literacy between veteran and novice teacher-participants), item quality variables have weak positive relationships, and teacher background variables and item quality variables have mostly negative relationships; and (3) writing context-based test items employs a blend of domain-specific item construction skills and general assessment literacy skills. The results of this study address the gap in item-writing research and help inform practice in the context of writing PISA-like mathe- matics items. Such insights are both valuable and timely in developing theoretically grounded and targeted individual support for mathematics teachers or institutional interventions for schools, es- pecially that national preparations for PISA 2025 have commenced.