ReCo – Automatically Processing Text Responses
from Large-Scale Assessments
Lead: DIPF
Joint research with: TUM
Current project members: Dr. Fabian Zehner➚, Nico Andersen➚, Prof. Dr. Frank Goldhammer➚)
Associate project members: TUM (Julia Mang as contact person for PISA data and structures)
The project Automatic Response Coding (ReCo) centers around text responses in tests. The computer programme ReCo automatically assesses whether a text response is correct, for example “The author aims at saving the trees.” as an answer in the PISA test. Moreover, ReCo extracts further features, for example whether a student adds knowledge to their response beyond the explicit information in the text.
The computer programme ReCo was originally developed at the Technical University of Munich and the Centre for International Student Assessment (ZIB). In cooperation with both institutions, the Centre for Technology Based Assessment (TBA Centre) at DIPF is now developing the software package further. The project comprises the general software development as well as scientific studies employing ReCo. For this, TBA is responsible for the conceptual and technical development on the one hand and acts as the project manager for projects such as ReCo-Multi on the other hand. Also, TBA supports external research groups using the program.
Further information about this project can be found here
Selected publications from this project
Zehner, F., Kroehne, U., Hahnel, C. & Goldhammer, F. (2020). PISA reading: Mode effects unveiled in short text responses. Psychological Test and Assessment Modeling, 62(1), 85–105.
Zehner, F., Goldhammer, F., Lubaway, E. & Sälzer, C. (2019). Unattended consequences: How text responses alter alongside PISA's mode change from 2012 to 2015. Education Inquiry, 10(1), 34–55. doi: 10.1080/20004508.2018.1518080
Zehner, F., Goldhammer, F. & Sälzer, C. (2018). Automatically analyzing text responses for exploring gender-specific cognitions in PISA reading. Large-scale Assessments in Education, 6:7. doi: 10.1186/s40536-018-0060-3
Zehner, F. (2016). Automatic processing of text responses in large-scale assessments (Dissertation).Technische Universität München, München. doi:10.13140/RG.2.2.26846.84800
Zehner, F., Sälzer, C. & Goldhammer, F. (2016). Automatic coding of short text responses via clustering in educational assessment. Educational and Psychological Measurement, 76(2), 280–303. doi:10.1177/0013164415590022
Zehner, F., Goldhammer, F. & Sälzer, C. (2015). Using and improving coding guides for and by automatic coding of PISA short text responses. In Proceedings of the IEEE ICDM Workshop on Data Mining for Educational Assessment and Feedback (ASSESS 2015), Atlantic City. doi: 10.1109/icdmw.2015.189