Evidence-based Clinical Practice – Lecture 7 (UNIVERSITY OF AMSTERDAM)

The advantages of analysing multiple individuals are that it allows for generalization of findings to others. However, it does not tell us anything about the individual. The analysis of an individual does not allow for generalization to others but does tell us something about the individual.

In an N=1 analysis, the baseline is compared with a treatment for a single individual. Normative comparisons make use of comparing a single individual to a distribution of the norm (e.g. normal distribution of test scores of a normative sample).

The question of whether there is an improvement requires a comparison while the question of whether there is a significant improvement requires multiple observations (e.g. baseline). There are several ways in which N=1 designs bridge the gap between science and practice:

1. Multiple N=1 studies in practice indicating improvement implies the need for a randomized controlled trial as this reduces costs (i.e. intervention idea starts in clinical practice).
2. RCTs that indicate improvement implies the need for N=1 studies to check the improvement in practice.
3. Conducting N=1 studies encourages practitioners to document their experiences which could be used in a meta-analysis.
4. Conducting N=1 studies may be the only way to study very heterogeneous groups.

There are several methodological criteria for an N=1 study:

1. Design
   The design should consist of at least two phases (e.g. baseline; intervention). A baseline is very important.
2. Dependent variable
   The dependent variable needs to be measured in a reliable way (1), there needs to be good interrater reliability (2) and the raters need to be blind to the phase (e.g. of treatment) the participant is in (3).
3. Analysis
   Researchers should arrive at the same conclusion using the data and the results should be trustworthy.
4. Importance
   The generalization to other constructs (1), other clients (2) and other therapists (3) needs to be assessed.

In an N=1 design, the end of phases is compared and the change of phases is compared. This implies that there should be a higher or lower score for the dependent variable after the treatment phase than after the end of the baseline phase. Furthermore, the change in the treatment phase should be higher than the change in the baseline phase.

An N=1 analysis is a form of regression analysis.

The interpretation of the bs depends on the coding of the independent variables.

A problem of the N=1 analysis using the regression analysis is that the assumption of uncorrelated errors is violated. This leads to results that are too liberal. This can be solved by modelling the dependence in the data. This makes use of autoregressive order 1. It makes sure that the correlation between time points is only dependent on their distance. The gist of this is that timepoints that are close to each other become more alike while timepoints that are further away become less alike.

The parameter (i.e. rho) used in the first-order autoregressive model may be underestimated in short time series. This can be solved by applying a small sample correction (1), testing parameters by implementing a permutation-based procedure (2) and carrying out more stringent tests (3).

The first-order autoregressive model (i.e. AB design) assumes that the overall pattern in the data is modelled correctly (1), the error correlation structure is modelled adequately (2), missing data is random (3) and the number of timepoints is sufficient to estimate the AR(1) parameter (4). The autoregressive model is reliable if the number of time points per phase is 10 or more. The AB design method using the autoregressive model can be generalized to more than two phases (1), can be used if session data is missing at random (2), can be used if symptoms are probed at irregular time points (3). Results across participants can be combined using meta-analytic techniques and follow-up data can be included as a separate phase.

There are four requirements for normative comparisons:

1. The normative samples should be up to date (1), large (2), cover the entire age range (3) and originate from the typical population (4).
2. The normative samples should be matched on age (1), sex (2) and premorbid IQ (3).
3. The normative samples should correct for multiple testing.
4. The normative samples should offer the possibility to test deviating profiles.

In regular normative samples, the sample is not always up to date because of the Flynn effect (1), it is often too small (2), the age of over than 80 is often missing (3) and is not always of a typical population (4). Furthermore, the typical samples may not be completely matched on demographic characteristics (i.e. test scores not matched on demographics). This can be solved by using stratified norm tables (e.g. age 20-40; 40-60). However, the sudden jumps and stratified norms yield small samples. Another solution to this may be using regression-based norms.

An additional problem of regular normative samples is that they are typically collected for one test at a time and combining them is not always possible.

A familywise error refers to the probability of at least one atypical result. This probability increases with the number of tests. Therefore, the normative samples should correct for multiple testing. The normative samples should allow for testing deviating profiles (e.g. atypical score on IQ and memory). In a multivariate normative comparison (i.e. atypical score on two tests), the atypicality also depends on the covariance between the tests.

These problems are solved when using the ANDI norms database. In the ANDI database, datasets of several research groups are combined into a single database. However, limitations of the ANDI database are that the data is not necessarily based on a random sample. Furthermore, some studies had lenient inclusion criteria and some participants might thus not have had psychopathology.