A critical-cultural approach to standardized evaluation instruments. A mixed methods study of the CLASS Toddler application in Italy

The increasing globalization of measures to assess ECEC quality may lead to pitfalls if the complexities of cross-cultural use of these instruments, their cultural consistency, and ecological validity, are not taken into account (Dahlberg, Moss & Pence,2007; Tobin et al.,2009; Vandenbroeck & Peeters,2014). Even tools with a solid theoretical and empirical background and wide international diffusion – such as the CLASS – cannot be considered culture-free since they still unavoidably reflect cultural values and methodological assumptions typical of their ‘cultural cradle’ (Pastori & Pagani, 2017; Pagani, submitted). Despite its relevance, this issue has received only marginal attention in the literature (Douglas, 2004; Fenech,2011; Ishimine & Taylor, 2014; Mathers et al.,2007; Sheridan,2007). The present study is an attempt to narrow this gap, focussing on the CLASS Toddler (Classroom Assessment Scoring System – La Paro, Hamre & Pianta, 2012). A mixed methods convergent parallel design (Creswell & Plano Clark, 2011) was adopted to analyse the implications of the CLASS application to the Italian ECEC context. Through a qualitative approach, Italian infant-toddler centres teachers were involved in sustained and focused observations and dialogues (focus groups and video observations) to observe and compare the perspective of the tool to the local-cultural and pedagogical perspective. The qualitative exploration was complemented using the quantitative analysis of the tool. Quantitative data (i.e., classroom observations coded using the CLASS) was used to describe classroom quality as postulated by the tool in Italian infant-toddler centers, and to examine the applicability and generalizability of the CLASS framework to the Italian context at the statistical level. The two strands of data were then compared to offer a more thorough understanding of the issue at hand. Results highlight the value of adopting a criticalcultural approach to evaluation tools (Pastori & Pagani, 2017; Pagani,submitted), and offer interesting insights into a methodological reflection on the potential offered by integrating a reflective discussion with the use of standard-based instruments.