Quality over quantity: Focusing on high-conflict trials to improve the reliability and validity of attentional control measures

In conflict tasks, congruency effects are thought to reflect attentional control mechanisms needed to counteract response conflict elicited by incongruent stimuli. Although congruency effects are well-replicable experimentally, recent studies have evidenced low correlations between congruency effects measured across different paradigms, leading to a heated debate over whether these low correlations indicate a lack of construct validity or are rather attributable to high measurement error, as indicated by the poor reliability typically displayed by congruency effects. In the present study, we investigated whether the poor reliabilities of congruency effects are due to their poor theoretical specification. Specifically, we tested whether the psychometric properties of congruency effects can be improved by focusing exclusively on those trials in which response conflict is theoretically expected to be highest. We considered two factors modulating the degree of response conflict: previous trial congruency, with higher conflict following congruent trials, and the time elapsed since stimulus onset, with higher conflict in fast responses. Data from 195 participants completing a Simon and a spatial Stroop paradigm showed that generally poor split-half reliabilities for the full set of trials improved greatly when excluding postincongruent and slow trials. Importantly, between-task correlations also increased substantially when controlling for these factors, suggesting that, with increased reliability, these tasks capture common attentional control ability. Our results suggest that individual differences in conflict tasks can provide valid and reliable measures of inhibition as a major component of attentional control when focusing on the trials with the theoretically highest response conflict.