Evidence-based Clinical Practice – Lecture 5 (UNIVERSITY OF AMSTERDAM)

Nested data (i.e. multilevel data) refers to data gathered in multiple groups (e.g. intervention study in multiple classrooms). Nested data that is analysed in the usual way can have strong consequences for the significance of the results (i.e. results being significant while they are actually not). Nested data being analysed in the wrong way leads to drawing false conclusions that something is effective while it is not.

The unit of analysis problem refers to the situation where the moderator variable exists at a different level than the independent and the dependent variable. Using the regular statistical analyses with nested data violates the assumption of independence of observations because it ignores the fact that individuals share a common environment. This may lead to an increase in type-I errors due to the small standard errors.

The multilevel analysis makes use of a general linear model. In the multilevel analysis, data are measured at different levels. For example, multiple children (i.e. level 1) are within multiple classrooms (i.e. level 2). Classrooms are level two because multiple children can be in a classroom but a child cannot be in multiple classrooms. In the multilevel analysis, the dependent variable is continuous and the independent variable is nominal or continuous (i.e. the same as the GLM).

The gamma00 (i.e. ) refers to a general change. There is a general change in the population but the change differs per level 2 variable (e.g. therapist). However, there is also variation within level 1 variables (e.g. individuals). This can be seen as the error.

In nested data analysed in the same way as non-nested data leads to the assumption of uncorrelated errors being violated. This means that the tests are too liberal and the results cannot be trusted. The assumption is violated because each level 1 variable (e.g. individuals) in a certain level 2 variable (e.g. classroom) has the same error, meaning that the errors become correlated (e.g. each individual in a classroom has the same error). A multilevel analysis accounts for these correlated errors.

The formula for a multilevel analysis without independent variables is the following:

The level 2 independent variables are often grand mean centred such that 0 reflects the mean of all groups. In this case, a score of 0 on a predictor would mean that  can be interpreted as the mean level of the variable, adjusted for individuals in that group. Y refers to the level two slope and can be interpreted as the change in that group’s mean for the dependent variable per 1-unit increase in the independent variable.

The intraclass correlation refers to the percentage of the total variance that is between group. This reflects the degree to which scores on the dependent variable are due to a nested factor.

The one-way ANOVA with random effects tests whether the groups differ on a level one variable. The groups are treated as random rather than fixed and it tests whether there is sufficient variability in the group intercepts to predict with a level-two independent variable The means-as-outcomes model assesses whether group-level variables can be used to predict group means. Compared to the one-way ANOVA with random effects, it adds a group-level predictor which is multiplied by the slope.

Intercepts- and slopes-as-outcomes model assesses whether the varying slopes between level one independent variables and dependent variables can be predicted by one or more level two independent variable. In this model, the varying intercepts and slopes are predicted whereas, in the means-as-outcomes model, only the varying intercepts were predicted. Random coefficients regression assesses whether one or more level one variables predict another level one variable. It is a simple regression analysis with the possibility for randomly varying intercepts and slopes across level two. It can demonstrate whether slopes and intercepts significantly across groups.

In multilevel analysis, the level 1 formula (e.g. for the individual) is substituted into the level 2 formula (e.g. for the classroom). A zero refers to an intercept (i.e. an effect which is not due to an independent variable). In the interpretation of the MLA, the example of multiple clients (i.e. level 1 variable) being treated by multiple therapists (i.e. level 2 variable) is used.

The fixed effects refer to what is fixed over groups (i.e. ). The random-effects refer to what is random over groups. In the therapist example, this is  (i.e. between therapist variance in change; between level 2 variable variance in change) and  (i.e. between client variance in change; between level 1 variable variance in change). In a multilevel analysis, it is possible to combine the random- and fixed effects.s

The formula for a multilevel analysis with a level 1 independent variable is the following:

The interpretation of this formula on SPSS is similar to the interpretation of MLA without independent variables.

Multilevel analysis accounts for dependency in the data due to grouping. It makes sure that the tests can be trusted. It can incorporate independent variables at both level 1 (e.g. individual) and level 2 (e.g. group) or both levels simultaneously. The multilevel analysis is suited for individual patient meta-regression. 

The formula for a multilevel analysis with a level 2 independent variable is the following:

The effects of level 2 variables cannot be random.

Using multilevel analysis can allow researchers to disentangle therapist and client factors on outcome (1), examine processes and outcomes of group psychotherapy (2) and allows participants with missing data to still be added into the analysis (3).