Making Learning Visible as a way of Teacher Professionalization. Exploratory Study Based on the Participatory Assessment of a Training Program

This contribution presents a mixed-methods sequential explanatory design (Creswell & Clark, 2017) focused on training conditions to support teacher capacity for assessing, in both a descriptive and formative approach, student's learning outcomes as a way to enhance professional development.In recent decades, researchers have shown an increased interest in professional development (Kennedy, 2016); most recently, several reviews have been written and several meta-analyses (Kennedy, 2016; Sims et al. 2021;) have been conducted in an effort to identify the ideal conditions for effective professional development (PD) (Kennedy, 2016a; Timperley et al., 2007; Walter & Briggs, 2012) and to clearly define the term "professional development", which is often used as an umbrella term to refer to any opportunities that serve to enhance teachers' knowledge. In this contribution, PD specifically refers to a “structured, facilitated activity for teachers intended to increase their teaching ability” (Sims, 2021). This definition allows us to include a wide range of skills and abilities, but at the same time is useful in its exclusion of PD activities and programmes consisting of mere updates for teachers on general topics. Results of the meta-analyses reveal that several design features (Kennedy, 2016) and factors (Sims et al., 2021) characterize effective PD and that these can be divided into two main categories: external factors, namely features established before the intervention (content knowledge; program intensity and length; role of the instructor; mandatory/voluntary participation; online/presential); b) internal factors, namely features developed during the intervention (feedback; goal setting; self-monitoring). This contribution seeks to place the teacher's ability to collect educational evidence (firstly, to make student’s learning visible; secondly, to analyze and assess it) at the center of PD, starting from the consideration of the aforementioned factors in designing a formative intervention. The hypothesis, to be explored in this contribution through an empirical study based on the evaluation of the formative impact of a training program, is consistent with Guskey’s model of teacher change (2002) and the idea that professional change primarily occurs as the result of gaining evidence of students’ learning rather than as the result of a change in teachers’ attitudes and beliefs. The feedback, in the form of an analysis of the students' learning process, that teachers receive on their educational practices is intended here to be an effective motivator toward PD. In continuity with Guskey's theory, the purpose of this contribution is to provide a qualitative exploration of the formative impact of a training program focused on supporting the teacher’s capacity for making students’ learning visible. In particular, the study aims to identify a) the set of professional skills enhanced through the training program as perceived by teachers; b) the training program methodologies and conditions that best foster the development of these skills. The skills we are considering belong to the area of teacher professionalization, identified as “pedagogical-didactic,” and relate to the ability to design activities consistent, as well as the gathering and analysis of evidence of learning outcomes, the formative feedback provided to students. Most of these skills can be associated with the professional profile of the so-called “teacher-as-researcher”. This connection between PD and the definition of teacher as researcher relies on the idea that teacher professionalization is dependent on the teachers ability to a) adopt an inquiry-based approach to his or her own teaching and, b) use this approach to observe and to collect data on the educational process and then to reflect on it. This mixed-methods sequential explanatory design (Creswell & Clark, 2017) is situated within the ecological paradigm (Mortari, 2010), and it is structured in the evaluation process of a teachers’ training program. The evaluation process follows a participatory evaluation (Bezzi, 2010) and 'fourth generation' approach (Guba & Lincoln, 1989) and is oriented towards the gathering of the perspectives and the perceptions of the teachers involved in the training. The main topic of the training program, carried out by teachers during the second half of the 2021-2022 school year, was the descriptive and formative tools and strategies of assessment at the Primary School level. The training was structured in three online meetings totalling 16 hours. The activities were focused on the documentation and analysis of the student's learning process as it was evidenced in given assignments. Participants were asked to conduct the activities and upload the assignments and students' work to an online platform before each of the online meetings. The training program was designed taking into account 3 main conditions already shown to be effective in PD (Kennedy, 2016; Sims, 2021): a) collective participation was promoted through group projects during the online meetings and with the assignment in order to facilitate teacher collaboration and the development of a learning community (Yoon et al., 2007); b) training facilitators were experienced teacher professional developers; c) attendance was voluntary (teachers participants were from Primary Schools in the Milan-region). Participants included 200 primary school teachers from in and around Milan. The evaluation process focused on the three dimensions of the training program: (i) the change in teachers' perceptions of their ability to assess students' learning; (ii) the effectiveness of training methodologies as perceived by participants; (iii) the change in students' learning assessment practices. The first and second dimensions were explored through the use of an ex-ante and ex-post questionnaire distributed to participants and the conducting of focus group interviews at the end of the training program. The third dimension was studied by way of an analysis of the participants’ assignments as uploaded before each online meeting. A factorial and exploratory regression analysis was carried out for the ex-ante and ex-post questionnaires (N=99) based on likert-scale items. Thematic analysis (Braun & Clarke, 2006) following an inductive and deductive labeling process, was applied to the textual data of the focus groups. Content analysis was applied to the data taken from the participants’ assignments. Results of the exploratory factorial analysis show that 3 main areas of professional development were promoted during the intervention: a) collaboration with colleagues; b) role of students in the assessment; c) analysIs and assessment of learning outcomes. Among them, collaboration with colleagues, both in terms of cooperation in the assessment process and in designing activities and tests, is the area that saw the most drastic change, while the other 2 areas present a limited and not significant evolution. As regards the effectiveness of the training conditions and methods, results show that the analysis of assessment tools and activities provided by facilitators was perceived as the most useful aspect of the training program along with the opportunity to converse with colleagues in group activities. Feedback received by participants on their assignments has a lower, but considerable impact. Findings from thematic analysis of the focus group illustrate the growing ability of teachers to analyze the learning process of students in a more systematic way, both using documentational tools to make learning visible (recording of group discussion; student self-assessments; recording or transcription of students’ metacognitive reflection) rather than more conventional tools (students’ tests; students texts and products) and adopting a more detailed method for analyzing this data. These findings are largely based on the results of the focus group interviews and on the analysis of the participants' assignments.