In Search of the Lost Interaction: A Theoretical and Methodological Framework for Researching Interactions

Abstract

We suggest that psychological research into interaction effects might benefit from analyzing potential interactions from the perspective of the Positive Predictive Value (PPV). The PPV denotes the post-study probability that a claimed effect is true, based on the pre-study probability that said effect exists, the power of the respective test and the significance level used for testing. We use the PPV in order to propose a framework structuring potential interaction effects based on their (theoretical) plausibility and their shape. Specifically, the position of a hypothesized interaction in the proposed framework may inform sample-size planning and the choice of alpha levels prior to a study; and it may inform confidence into results after a study. Finally, we present a heuristic approach for planning research on interactions based on R (the pre-study probability that an effect exists), the PPV (the post-study probability that a claimed effect is true) and α (the significance level used for significance testing). In doing so, we aim to provide a nuanced view on the feasibility of investigating into interactional hypotheses, a view that is critical where needed but that at the same time does not discourage research on interactions.