Factorial designs - summary of chapter 14 of statistics by A. Field (5th edition)

Statistics
Chapter 14
Factorial designs

Factorial designs

Factorial design: when an experiment has two or more independent variables.
There are several types of factorial designs:

• Independent factorial design: there are several independent variables or predictors and each has been measured using different entities (between groups).
• Repeated-measures (related) factorial design: several independent variables or predictors have been measured, but the same entities have been used in all conditions.
• Mixed design: several independent variables or predictors have been measured: some have been measured with different entities, whereas others used the same entities.

We can still fit a linear model to the design.
Factorial ANOVA: the linear model with two or more categorical predictors that represent experimental independent variables.

Independent factorial designs and the linear model

The general linear model takes the following general form:

Y_i =b₀ + b₁X_1i+b₂X_2i+... +b_nX_ni+Ɛ_i

We can code participant’s category membership on variables with zeros and ones.

For example:

Attractiveness_i = b₀+b₁A_i+b₂B_i+b₃AB_i+Ɛ_i

b₃AB is the interaction variable. It is A dummy multiplied by B dummy variable.

Behind the scenes of factorial designs

Calculating the F-statistic with two categorical predictors is very similar to when we had only one.

• We still find the total sum of squared errors (SS_T) and break this variance down into variance that can be explained by the model/experiment (SS_M) and variance that cannot be explained (SS_R)
• The main difference is that with factorial designs, the variance explained by the model/experiment is made up of not one predictor, but two.

Therefore, the sum of squares gets further subdivided into

• variance explained by the first predictor/independent variable (SS_A)
• variance explained by the second predictor/independent variable (SS_B)
• variance explained by the interaction of these two predictors (SS_AxB)

Total sum of squares (SS_T)

We start of with calculating how much variability there is between scores when the ignore the experimental condition from which they came.

The grand variance: the variance of all scores when we ignore the group to which they belong.
We treat the data as one big group.
The degrees of freedom are: N-1

SS_T = s²_Grand(N-1)

The model sum of squares (SS_M)

The model sum of squares is broken down into the variance attributable to the first independent variable, the variance attributable to the second independent variable, and the variance attributable to the interaction of those two.

The model sum of squares: the difference between what the model predicts and the overall mean of the outcome variable.
What the model predicts is the group mean.
We work out the model sum of squares by looking at the difference between each group mean and the overall mean.

SS_M = Σ^k_g=1n_g(ẍ_g-ẍ_grand)²

n = the number of scores in each group

Grand mean = the mean of all scores

The degrees of freedom are k-1

The main effect of A or B

To work out the variance accounted for by the first predictor/independent variable (for example, type of cat) we group Y ratings according to which type of cat was being rated.
So, we ignore the other independent variable, and place all the ratings of (for example) cat type 1 in one group, and all the ratings of (for example) cat type 2 into another.

We apply the same equation for the model sum of squares as above.

The interaction effect, SS_AxB

The final stage is to calculate how much variance is explained by the interaction of the two variables.

The SS_M is made up of three components (SS_A, SS_B, and SS_AxB)
Given that we know SS_A and SS_B, we can calculate the interaction term by subtraction

SS_AxB = SS_M – SS_A – SS_B

The degrees of freedom can be calculated in the same way, but are also the product of the degrees of freedom for the main effects. Two methods:

• df_AxB = df_M – df_A-df_B
• df_AxB = df_A x df_B

The residual sum of squares, SS_R

The residual sum of squares represents errors in prediction from the model. In experimental designs, it also reflects individual differences in performance or variance that can’t be explained by factors that were systematically manipulated.

The value is calculated by taking the squared error between each data point and its corresponding group mean.

SS_R = Σ^k_g=2s²_g(n_g-1)

s²_group1 (n₁-1)+s²_group2 (n2-1)+….+ s²_groupn (n_n-1)+

Degrees of freedom for each group will be one less than the number of scores per group.
We add the degrees of freedom for each group to get a total.

The F-statistics

Each effect in a factorial design has its own F-statistc.
In a two-way design this means we compute F for the two main effects and the interaction.
To calculate these, we first calculate the mean squares of each effect by taking the sum of squares and dividing by the respective degrees of freedom.

MS_A = SS_A/df_A

MS_B = SS_B/df_B

MS_AxB = SS_AxB/df_AxB

MS_R = SS_R/df_R

The F-statistic for each effect is calculated by dividing its mean squares by the residual mean squares.

F_A = MS_A/MS_R

F_B = MS_B/MS_R

F_AxB = MS_AxB/MS_R

Model assumptions in factorial design

When using the linear model to analyse a factorial design the sources of potential bias discussed in chapter 6 apply.

• and homogeneity of variance

Output from factorial design

The main effect of A

“Tests of between-subjects effects” tells us whether any of the independent variables had a significant effect on the outcome.

The interaction effect

“Tests of between-subjects effects” also tells us about the interaction effect.

You should not interpret a main effect in the presence of a significant interaction involving that main effect.

Contrasts

Look at ‘contrast results (K Matrix)
The top of the table shows the contrast for Level 1 vs Later. The control vs experimental.
And then Level 2 vs Level 3.

Simple effects analysis

A particular effective way to break down interactions is simple effects analysis.
Looks at the effect of one independent variable at individual levels of the other independent variable .

Post hoc analysis

The Bonferroni post hoc test break down the main effect A and can be interpreted as if A was the only predictor in the model.

Factorial ANOVA

• Two-way independent designs compare several means when there are two independent variables and different entities have been used in all experimental conditions.
• In the table called ‘tests of between-subject effects’, look at the column called sig for all main effects and interactions.
• To interpret a significant interaction, plot an interaction graph and conduct simple effects analysis.
• You don’t need to interpret main effects if an interaction effect involving that variable is significant.
• If significant main effects are not qualified by an interaction then consult post hoc tests to see which groups differ. Significance is shown by values smaller than 0.05 in the columns labelled sig. And bootstrap confidence intervals that do not contain zero.
• Test the same assumptions as for any linear model.

Interpreting interaction graphs

• Non-parallel lines on an interaction graph indicate some degree of interaction, but how strong an whether the interaction is significant depends on how non-parallel the lines are
• Lines on an interaction graph that cross are very non-parallel, which hints at a possible significant interaction. But, crossing lines don’t always reflect a significant interaction.

Calculating effect sizes

r = square root ((F(1, df_R))/(F(1,df_R) +df_R))

Reporting results of factorial design

We report the details of the F-statistic and the degrees of freedom for each effect. Also p and effect size.

Like: F(1, 8) = 6,06, p = 0.04, r = 0.4.