Cognitive Interviews as a Technique for Instrument Adaptation in Educational Contexts

Abstract

The objective of the study presented in this article was to qualitatively evaluate the cognitive receptivity and general understanding of new questions for the Quality and Context of Education Questionnaires, using the cognitive interviewing technique. The use of this technique to evaluate the receptivity and comprehension to questions on non-academic aspects (school climate, self-esteem and motivation, among others) by different actors is described. A non-probabilistic sample was used, according to predefined quotas, in accordance with heterogeneity variables, which allowed the selection of 20 schools in the Metropolitan, Antofagasta and Biobío regions of Chile. The participants were students of these schools in 4th and 8th grade (78 and 48, respectively), teachers (19) and parents (28) of the same grades, selected by convenience. The 1,059 units of analysis were distributed in 20 cognitive interview guidelines, allowing a limited number of items per application, safeguarding the quality of the guideline and avoiding over-demanding of the interviewees. The analysis of information was organized in three stages: (a) data definition, (b) data processing and (c) identification of findings. The application of cognitive interviews made it possible to identify comprehension difficulties per question and its components (items and statements) in the various actors, and problems of relevance derived from the assumptions that several items or statements contained, providing suggestions for adapting these questions. The process confirms the relevance of the technique for piloting and adapting questionnaire questions in the educational context.