The research-intervention was conducted by the Communication and Social Research Department of the Sapienza University in collaboration with the Institute for Environmental Protection and Research (ISPRA) starting from february 2010. Scope of the study-campaign was to evaluate the efficiency of an information campaign about radioactivity and connected risks, performed by ISPRA and addressed to students aged 14-19 years in 4 towns of the Lazio region (teenagers are the best target in case of interventions aimed at generating knowledge and attitudes to face future challanges and they also represent the potential source of information diffusion within the families). The research was carried out according to a quasi-experimental design with pre-test, post-test and a not equivalent control group, the characteristics of which were aimed at optimizing the risk of non validity in the specific survey situation The towns have been selected based on the presence of radiation sources in their territory: • Rome and Latina have some man-induced radiactivity (nuclear research center of Enea-Casaccia and a shut-down nuclear power plant in Borgo Sabotino); • Viterbo is concerned by natural radioactivity (radon released from the volcanic stones used in the construction of buildings); • Frosinone, not having any problem related to radioactivity, is the “control town”. The sampling performed was conceived in clusters: there were identified different institutes from which to choose 5 classes in order to have the hugest possible variety of educational fields and teachers. The schools, 24 in total, have been selected in base of their proximity to the radiation sources and each of them have been casually and entirely assigned to the experimental or the control group. The survey have involved 2635 teenagers from all five school years and with a fair distribution between high schools, technical and professional institutes. The students had to fill in 2 tests (one before and one after the training session carried out by the ISPRA experts and addressed to students of the experimental group), prepared in the form of standardized, semi-structures and autonomously filled-in questionnaires, in large part identical in the two deliveries. The training session carried out by the ISPRA experts consisted, in the schools of the experimental group, of a 2-hours lesson focused on issues related to radioactivity and risks deriving from the exposure to natural or man-induced sources of radioactivity. During the lessons some specially trained personnel took care of: • detecting the students’ attendance; • assessing the structural conditions of the classrooms; • evaluating the study environment (treshold of interest, overall attention and participation, level of knowledge of the students and understanding by them of the technical terms, legibility of the tutorial texts used and duration of the presentations). The main part of the questionnaire (49 questions) is aimed at testing the students’ competency on issues related to radioactivity. Other questions are about risk perception, knowledge of environmental issues, sustainable living behaviour moods adopted and their frequency, socio-political participation, source of information used to get the latest news and frequency of fruition, personal data. The analysis performed implies the control of the causal effect of the experimental variable through the introduction of theoretically relevant third variables (contextal and of mechanism). In particular it means to apply the concepts and the models of the tri-variable analysis – namely of the specification (contextual variables) and of the interpretation with an intervening antecedent variable (variables of mechanism) – to the study of the effect induced by the experiment. Specific hypothesis, correctly converted in reliable operative procedures, allows a better control of the model, especially in view of what has been defined the “theoretically expanded concept of the experiment”. In this perspective it is essential to take into account not only the manipulated variable (considered the cause), but also a series of contextual and dispositional characteristics (of people or referring to people) which can affect the relation between the cause and the effects. The results of the study-campaign will be evaluated according to the above model, with particular attention on emphasising the below explanatory aspects which can interact with the “experimental processing” X (the training session) so to influence the efficiency of the same “experimental processing” in quality of intervening variables: - partially independent from X but able to feed back to X (contextual variables); - partially triggered by X (variables of mechanism: causal intermediators). The contextual variables are the specific aspects of the environment serving as background for the experiment and certain features of the involved individuals. These variables: • represent the circumstances of the experiment; • are to be considered theoretically or logically independent from the experimental processing; • have an ex ante influence on the planning of the experimental processing; • influence in itinere the realisation of the training session; • can have a consequence on the outcome of the experimental processing, amplifing or reducing its effect. For the above reasons they can constitute the context of the relation between X (training session) and Y (outcome of the experiment), even if it is mediated by a mechanism (the contextual variables can affect on the relation as a whole including the intervening mechanism between X and Y). The further analysed contextual variables are: 1) the town where the school is situated; 2) educational profile of the school (hereinafter referred as: type of school); 3) the structural characteristics of the classroom and the study environment/atmosphere which characterized the training session (obviously only for the experimental groups); 4) the initial competences (knowledge) about radioactivity of the students; 5) the perception of the risk related to exposure to ionizing radiation sources as conceived at the time of the first test. The variables of mechanism are conceptualized as follows: 1) individual characteristics; 2) variables of dispositional nature; 3) variables possessed by the individuals indipendently from the experimental situation (so antecedent to the experiment); 4) variables arising at certain conditions and in response to precise stimula. Consequently even if the mechanism is independent from X, it’s possible that it occurs due to X. In this case the only analysed variable of mechanism will be the individual deepening of the issue related to the ionizing radiations as performed between the two testings. Thus the pre-test and the training session in the experimental group and the only pre-test in the control group are conceived as condition/stimulus/force sufficient to activate a previous inclination of the individuals to deepen an issue. The kind of relation arising has the characteristic [X (only pre-test; pre-test and experimental variable) → Z (inclination to deepen knowledge) → Y (cognitive improvement)] that Z, even if activated by X, is antecedent to the same X. Z is not considered a variable depending from X, even if it is activated in the time elapsing between X and Y, and even if its activation is induced by X. The term mechanism used here refers to an element which is antecedent to X and is causally relevant, and not a simple causal intermediator which occurs as an effect of X and cause of Y in the relation between X and Y. It was asked to the interviewed students if they have deepened by themselves their knowledge about the ionizing radiations during the time between the two tests (for the control group) or between the training session and the post-test (for the experimental group). This procedure can be called retrospective posttest. The overall picture of the results reveals a substantial success of the experiment, which tends to reproduce itself independently from the contextual variables, except for: 1. structural and relational dimensions characterizing the “internal story of the experiment”; 2. place; 3. initial level of knowledge in the field of radioactivity. Concerning the place some interesting findings appeared: starting from the falling of the hypothesis of major awareness on the issue in the neighborhood of radiation sources, till to the far too good performance of the students of Rome and the quite disapponting performance of the students of Viterbo. Different authors agree that the man-made sources of radiation are generally perceived more dangerous compared to the natural sources. The type of institute, able in theory to account the education, the background and the interests of the students, proved to be non-discriminatory for the purpose of receiving the training session. Conversely and in line with the expectations there was a way of bringing to light important elements in establishing a cause-effect relationship with a certain grade of specification. It can be seen that favorable conditions both at structural level and at the level of study environment/atmosphere of the training session have positive influence on its outcome. The initial level of competence of the students turned out to be able to influence in a differentiated way also the worsening/improvement; so, with reference to the experimental group it could be observed a marked improvement within the students with a low initial competency level and a braking in fall of the scores of students with a high level of previous knowledge. The perception of risk related to ionizing radiation was not found determinant as a contextual variable. Concerning a possible configuration of risk perception as a variable of impact it has to be mentioned that it would be impermeable to the cognitive improvements achieved with the experiment. In line with the expectations there was a way of bringing to light important elements in the establishing of a correct cause-effect relationship, like the individual deepening of the topic radioactivity, which not only has a direct effect on the acquisition of competencies by the students in the given field, but it also interacts with the training session performed by the ISPRA experts enhancing its effects. The work presented was aimed at showing how a theoretically enlarged concept of the experiment can conduct to a better interpretation of the experimental result and therefore a deeper control of the model. In opposition to an old-behaviorist concept of the experiment, which would allow to proceed based on the simple scheme stimulus-response, it was considered not only the experimental variable, taken as the main stimulus, but also contextual and dispositional characteristics, in theory able to intervene in the relationship cause-effect, but not strictly “manipulable” by the researcher. Each of the above factors contributed to enable a better understanding of the relationship between the experimental training and the worsening/improvement of the students, indicating the cases where this relation results enhanced or weakened, and creating the conditions to avoid the simplicity inherent of a behaviorist approach to the experiment.

Decataldo, A., Faggiano, M., Fasanella, A., Maggi, M. (2012). Effects, contexts, mechanisms operating in a quasi-experimental design. SOCIOLOGIA E RICERCA SOCIALE, 97, 85-126 [10.3280/SR2012-097008].

Effects, contexts, mechanisms operating in a quasi-experimental design

DECATALDO, ALESSANDRA;
2012

Abstract

The research-intervention was conducted by the Communication and Social Research Department of the Sapienza University in collaboration with the Institute for Environmental Protection and Research (ISPRA) starting from february 2010. Scope of the study-campaign was to evaluate the efficiency of an information campaign about radioactivity and connected risks, performed by ISPRA and addressed to students aged 14-19 years in 4 towns of the Lazio region (teenagers are the best target in case of interventions aimed at generating knowledge and attitudes to face future challanges and they also represent the potential source of information diffusion within the families). The research was carried out according to a quasi-experimental design with pre-test, post-test and a not equivalent control group, the characteristics of which were aimed at optimizing the risk of non validity in the specific survey situation The towns have been selected based on the presence of radiation sources in their territory: • Rome and Latina have some man-induced radiactivity (nuclear research center of Enea-Casaccia and a shut-down nuclear power plant in Borgo Sabotino); • Viterbo is concerned by natural radioactivity (radon released from the volcanic stones used in the construction of buildings); • Frosinone, not having any problem related to radioactivity, is the “control town”. The sampling performed was conceived in clusters: there were identified different institutes from which to choose 5 classes in order to have the hugest possible variety of educational fields and teachers. The schools, 24 in total, have been selected in base of their proximity to the radiation sources and each of them have been casually and entirely assigned to the experimental or the control group. The survey have involved 2635 teenagers from all five school years and with a fair distribution between high schools, technical and professional institutes. The students had to fill in 2 tests (one before and one after the training session carried out by the ISPRA experts and addressed to students of the experimental group), prepared in the form of standardized, semi-structures and autonomously filled-in questionnaires, in large part identical in the two deliveries. The training session carried out by the ISPRA experts consisted, in the schools of the experimental group, of a 2-hours lesson focused on issues related to radioactivity and risks deriving from the exposure to natural or man-induced sources of radioactivity. During the lessons some specially trained personnel took care of: • detecting the students’ attendance; • assessing the structural conditions of the classrooms; • evaluating the study environment (treshold of interest, overall attention and participation, level of knowledge of the students and understanding by them of the technical terms, legibility of the tutorial texts used and duration of the presentations). The main part of the questionnaire (49 questions) is aimed at testing the students’ competency on issues related to radioactivity. Other questions are about risk perception, knowledge of environmental issues, sustainable living behaviour moods adopted and their frequency, socio-political participation, source of information used to get the latest news and frequency of fruition, personal data. The analysis performed implies the control of the causal effect of the experimental variable through the introduction of theoretically relevant third variables (contextal and of mechanism). In particular it means to apply the concepts and the models of the tri-variable analysis – namely of the specification (contextual variables) and of the interpretation with an intervening antecedent variable (variables of mechanism) – to the study of the effect induced by the experiment. Specific hypothesis, correctly converted in reliable operative procedures, allows a better control of the model, especially in view of what has been defined the “theoretically expanded concept of the experiment”. In this perspective it is essential to take into account not only the manipulated variable (considered the cause), but also a series of contextual and dispositional characteristics (of people or referring to people) which can affect the relation between the cause and the effects. The results of the study-campaign will be evaluated according to the above model, with particular attention on emphasising the below explanatory aspects which can interact with the “experimental processing” X (the training session) so to influence the efficiency of the same “experimental processing” in quality of intervening variables: - partially independent from X but able to feed back to X (contextual variables); - partially triggered by X (variables of mechanism: causal intermediators). The contextual variables are the specific aspects of the environment serving as background for the experiment and certain features of the involved individuals. These variables: • represent the circumstances of the experiment; • are to be considered theoretically or logically independent from the experimental processing; • have an ex ante influence on the planning of the experimental processing; • influence in itinere the realisation of the training session; • can have a consequence on the outcome of the experimental processing, amplifing or reducing its effect. For the above reasons they can constitute the context of the relation between X (training session) and Y (outcome of the experiment), even if it is mediated by a mechanism (the contextual variables can affect on the relation as a whole including the intervening mechanism between X and Y). The further analysed contextual variables are: 1) the town where the school is situated; 2) educational profile of the school (hereinafter referred as: type of school); 3) the structural characteristics of the classroom and the study environment/atmosphere which characterized the training session (obviously only for the experimental groups); 4) the initial competences (knowledge) about radioactivity of the students; 5) the perception of the risk related to exposure to ionizing radiation sources as conceived at the time of the first test. The variables of mechanism are conceptualized as follows: 1) individual characteristics; 2) variables of dispositional nature; 3) variables possessed by the individuals indipendently from the experimental situation (so antecedent to the experiment); 4) variables arising at certain conditions and in response to precise stimula. Consequently even if the mechanism is independent from X, it’s possible that it occurs due to X. In this case the only analysed variable of mechanism will be the individual deepening of the issue related to the ionizing radiations as performed between the two testings. Thus the pre-test and the training session in the experimental group and the only pre-test in the control group are conceived as condition/stimulus/force sufficient to activate a previous inclination of the individuals to deepen an issue. The kind of relation arising has the characteristic [X (only pre-test; pre-test and experimental variable) → Z (inclination to deepen knowledge) → Y (cognitive improvement)] that Z, even if activated by X, is antecedent to the same X. Z is not considered a variable depending from X, even if it is activated in the time elapsing between X and Y, and even if its activation is induced by X. The term mechanism used here refers to an element which is antecedent to X and is causally relevant, and not a simple causal intermediator which occurs as an effect of X and cause of Y in the relation between X and Y. It was asked to the interviewed students if they have deepened by themselves their knowledge about the ionizing radiations during the time between the two tests (for the control group) or between the training session and the post-test (for the experimental group). This procedure can be called retrospective posttest. The overall picture of the results reveals a substantial success of the experiment, which tends to reproduce itself independently from the contextual variables, except for: 1. structural and relational dimensions characterizing the “internal story of the experiment”; 2. place; 3. initial level of knowledge in the field of radioactivity. Concerning the place some interesting findings appeared: starting from the falling of the hypothesis of major awareness on the issue in the neighborhood of radiation sources, till to the far too good performance of the students of Rome and the quite disapponting performance of the students of Viterbo. Different authors agree that the man-made sources of radiation are generally perceived more dangerous compared to the natural sources. The type of institute, able in theory to account the education, the background and the interests of the students, proved to be non-discriminatory for the purpose of receiving the training session. Conversely and in line with the expectations there was a way of bringing to light important elements in establishing a cause-effect relationship with a certain grade of specification. It can be seen that favorable conditions both at structural level and at the level of study environment/atmosphere of the training session have positive influence on its outcome. The initial level of competence of the students turned out to be able to influence in a differentiated way also the worsening/improvement; so, with reference to the experimental group it could be observed a marked improvement within the students with a low initial competency level and a braking in fall of the scores of students with a high level of previous knowledge. The perception of risk related to ionizing radiation was not found determinant as a contextual variable. Concerning a possible configuration of risk perception as a variable of impact it has to be mentioned that it would be impermeable to the cognitive improvements achieved with the experiment. In line with the expectations there was a way of bringing to light important elements in the establishing of a correct cause-effect relationship, like the individual deepening of the topic radioactivity, which not only has a direct effect on the acquisition of competencies by the students in the given field, but it also interacts with the training session performed by the ISPRA experts enhancing its effects. The work presented was aimed at showing how a theoretically enlarged concept of the experiment can conduct to a better interpretation of the experimental result and therefore a deeper control of the model. In opposition to an old-behaviorist concept of the experiment, which would allow to proceed based on the simple scheme stimulus-response, it was considered not only the experimental variable, taken as the main stimulus, but also contextual and dispositional characteristics, in theory able to intervene in the relationship cause-effect, but not strictly “manipulable” by the researcher. Each of the above factors contributed to enable a better understanding of the relationship between the experimental training and the worsening/improvement of the students, indicating the cases where this relation results enhanced or weakened, and creating the conditions to avoid the simplicity inherent of a behaviorist approach to the experiment.
Articolo in rivista - Articolo scientifico
Traninig programm, evaluation, quasi-experimental research design, mechanisms, contexts
English
2012
97
85
126
none
Decataldo, A., Faggiano, M., Fasanella, A., Maggi, M. (2012). Effects, contexts, mechanisms operating in a quasi-experimental design. SOCIOLOGIA E RICERCA SOCIALE, 97, 85-126 [10.3280/SR2012-097008].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/34782
Citazioni
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
Social impact