Self-reported learning strategies in the large-scale assessment study PISA: The comparison of two response formats

Ana Tupac-Baier, Research Associate TUM

Doctoral Supervisor: Prof Dr. Kristina Reiss➚
Mentor: Dr. Mirjam Weis
Supervisors: Prof. Dr. Anja Schiepe-Tiska, Dr. Jörg-Henrik Heine

Theoretical background

Large-scale assessments such as the PISA survey collect data on students’ motivational orientations, attitudes, and behaviours through self-report questionnaires. These include learning strategies which schools and teaching methods aim to promote and which are associated with academic achievement and lifelong learning. The PISA survey typically measures learning strategies using a traditional rating scale format. However, research findings indicate that the rating scale format is prone to response biases, which challenges the validity of the questionnaire and poses difficulties for international comparisons.

A central theoretical framework for embedding learning strategies is Boekaert's (1999) three-layer model of self-regulated learning, which distinguishes three types of learning strategies: memorization strategies (learning by heart), elaboration strategies (deep understanding through connecting with prior knowledge and everyday contexts), and metacognitive control strategies (monitoring the learning process).

The aim of the doctoral project is to compare two response formats – the forced-choice scale and the rating scale – for assessing self-reported mathematical learning strategies, with the aim of examining their construct and criterion validity and investigating their relationship with socially desirable responding as an indicator of response bias.

Research questions

This doctoral project addresses four research questions. The first concerns the reliability of the two response formats: Do the two response formats measure learning strategies with the same accuracy of measurement? The second question focuses on construct validity and examines whether the use of different response formats captures the same latent variable. The third question investigates how socially desirable responding is related to each response format. Finally, the fourth question concerns criterion validity and explores whether the relationship between mathematical literacy and learning strategies differs depending on response format.

Methods

The sample consisted of n = 5739 German ninth graders who participated in the main PISA study in 2012, with mathematics constituting the major domain. Mathematical learning strategies included control, elaboration, and memorization strategies, which were measured using both a forced-choice scale, scaled according to the Thurstonian IRT model, and a rating scale, scaled according to the cumulative partial credit model. Socially desirable responding was assessed with a rating scale and comprised self-deception and impression management. As a criterion variable, mathematical literacy was operationalized through the five plausible values provided in the PISA assessment. The analyses included reliability estimates of the scales which were applied, Pearson correlations within a multitrait-multimethod matrix, and linear regression analyses.

For further information and questions, please contact:

• 

  © Andreas Heddergott

  Ana Tupac-Baier

  Research Associate | TUM

  Show profile