Evaluating statistical and clinical significance of intervention effects in single-case experimental designs: an SPSS method to analyse univariate data - summary of an article by : Marija, de Haan, Hogendoorn, Wolters and Huizenga

Evaluating statistical and clinical significance of intervention effects in single-case experimental designs: an SPSS method to analyse univariate data
Marija, de Haan, Hogendoorn, Wolters and Huizenga (2015)

Abstract

Single-case experimental designs are useful to investigate individual client progress. This can help the clinician to investigate whether an intervention works as compared with a baseline period or another intervention, and whether symptom improvement is clinically significant.

Introduction

In single-case experimental designs (SCEDs), a single participant is repeatedly assessed on one or multiple indices (symptoms) during various phases. Advantages of this are: 1) It can be used to test novel interventions. 2) In heterogenous populations, it can be the only way to investigate treatment outcomes. 3) It offers the possibility to systematically document knowledge of researchers and clinicians.

Method

The AB design consist of two phases; baseline and a treatment.

The AB method can be conceptualized as an interrupted time series. To analysis the differences between baseline and treatment two requirements must be fulfilled. These are:  1) the overall pattern in the time series has to be modelled adequately, an adequate model consists of two linear functions, one for baseline and one for the treatment. Each of these functions is described by an intercept and a slope. 2) adequate modelling of potential correlations between residuals is needed, this is adequate modelling for that which remains after the overall pattern has been accounted for. Autocorrelation is the correlation between residuals of the observations. The residuals are not independent. If residuals are correlated, the correlations are likely to decrease with increasing separation between time points.

Analyses investigating treatment efficacy

 Y(i) = b0 +b1 * phase(i) + b2*time_in_phase (i) + b3 * phase(i) * time_in_phase(i) + error(i)

Y(i) is the outcome variable at time point i.
Phase(i) denotes the phase in which time point I is contained (0 for baseline and 1 for treatment).
Time_in_phase (i)is the time points in each phase.
Error (i) is the residual at point i.
b0 is the baseline intercept
b1 is the treatment-baseline difference in intercepts
b2 is the baseline slope
b3 is the treatment-baseline difference in slopes

Intercept differences between phases can be assessed by testing whether b1 differs from 0. Slope differences can be assessed by testing b3.

B0 and b1 refer to symptom scores when time_in_phase is zero. This depends on the coding of time_in_phase.

It is best to both describe the general trend adequately and to account for remaining correlations. The correlations can be quantified by the AR(1) parameter rho incorporated in the autoregressive model. This parameter might be underestimated for short time series. Solutions for this are: 1) apply a small sample correlation to the estimated AR(1) parameter 2) Test parameters by implementing a permutation-based procedure 3) test may be carried out at some stringent levels.

Analysis investigating reliable change

B0 gives an estimator of the end point of the baseline phase. B0 + b1 gives an estimator of the end of the ‘treatment’ phase. These can be used to estimate reliable change.

If the linear change model is correct, these estimators are likely to be more precise estimators of final outcomes than observed final outcomes, as they are less contaminated by error.

The reliably change index (RCI):

RCI = X2 – X1/Sdiff

X1 and X2 represent a participant’s pre- and posttreatment scores.
Sdiff denotes the standard deviation of the difference between two test scores.

RCI = [(b0+b1)-b0]/Sdiff = b1/Sdiff

Discussion and conclusion

The method can be generalized in various ways: 1) It can be generalized to more than two phases. Additional phase variables and their interaction with time_in_phase should be added 2) the method can be used if symptoms are probed at irregular time intervals 3) longer-term follow-up data can be seen as an additional phase 4) results across participants can be combined by weighting intercept or slope differences by their respective standard errors.