Statistics
Chapter 8
Correlation

Modeling relationships

The data we observe can be predicted from the model we choose to fit the data plus some error in prediction.

Outcome_i = (model) + error_i
Thus
outcome_i = (b₁X_i)+error_i

z(outcome)_i = b₁z(X_i)+error_i

z-scores are standardized scores.

A detour into the murky world of covariance

The simplest way to look at whether two variables are associated is to look whether they covary.
If two variables are related, then changes in one variable should be met with similar changes in the other variable.

Covariance (x,y) = Σⁿ_i=1 ((x_i-ẍ)(y_i-ÿ))/N-1

The equation for covariance is the same as the equation for variance, except that instead of squaring the deviances, we multiply them by the corresponding deviance of the second variable.

A positive covariance indicates that as on variable deviates from the mean, the other variable deviates in the same direction.
A negative covariance indicates that as one variable deviates from the mean, the other deviates from the mean in the opposite direction.

The covariance depends upon the scales of measurement used: it is not a standardized measure.

Standardization of the correlation coefficient

To overcome the problem of dependence on the measurement scale, we need to convert the covariance into standard set of units → standardization.
Standard deviation: a measure of the average deviation from the mean.
If we divide any distance from the mean by the standard deviation, it gives us that distance in standard deviation units.
We can express the covariance in a standard units of measurement if we divide it by the standard deviation. But, there are two variables and hence two standard deviations.

Correlation coefficient: the standardized covariance

r = cov_xy/(s_xs_y)

s_x is the standard deviation for the first variable
s_y is the standard deviation for the second variable.

By standardizing the covariance we end up with a value that has to lie between -1 and +1.
A coefficient of +1 indicates that the two variables are perfectly positively correlated.
A coefficient of -1 indicates a perfect negative relationship.
A coefficient of 0 indicates no linear relationship at all.

The significance of the correlation coefficient

We can test the hypothesis that the correlation is different from zero.
There are two ways of testing this hypothesis.

We can adjust r so that its sampling distribution is normal:

z_r = ½ log_e((1+r)/(1-r))

The resulting z_r has a standard error given by:

Se_zr = 1/(square root(N-3))

We can adjust r into a z-score

z = z_r/Se_zr

The t-statistic for r is:

t_r = (r * square root(N-2))/ (square root(1-r²))

The correlation coefficient is a commonly used measure of the size of an effect.

• values of +/- 0.1 represent a small effect
• values of +/- 0.5 represent a large effect
• values of +/- 0.3 represent medium effect

Confidence intervals for r

Confidence intervals tell us about the likely value in the population.
Lower boundary of the confidence interval = z_r – (1,96 X SE_Zr)
Upper boundary of the confidence interval = z_r + (1,96 X SE_Zr)

Bivariate correlation

The data must be linear and normally distributed.
The outcome variable needs to be measured at the interval ratio level, as does the predictor variable.

It would be advisable to use a bootstrap to get robust confidence intervals.

Spearman’s correlation coefficient

A non-parametric statistic that is useful to minimize the effects of extreme scores or the effects of violations of the assumptions discussed in chapter 6.
Spearman’s test works by first ranking the data, and then applying Pearson’s equation to those ranks.

Kendall’s tau (non-parametric)

A non-parametric correlation.
Should be used when you have a small data set with a large number of tied ranks, if you rank the scores and many scores have the same rank.
A better estimate of the correlation in the population.

Biserial and point-biserial correlations

Often it is necessary to investigate relationships between two variables when one of the variables is dichotomous (categorical with only two categories).
The biserial and point-biserial correlation should be used in these situations.

The difference between the use of biserial and point-biserial correlations depends on whether the dichotomous variable is discrete of continuous.
A discrete,or true, dichotomy: one for which there is no underlying continuum between the categories.
A continuous dichotomy: a dichotomy for which a continuum exists.

The point-biserial correlation (r_pb) is used when one variable is a discrete dichotomy
Biserial correlation (r_b) is used when one variable is a continuous dichotomy.

Summary

• Spearman’s correlation coefficient, r_s, is a non-parametric statistic and requires only ordinal data for both samples.
• The point-biserial correlation coefficient, r_pb, quantifies the relationship between a continuous variable and a variable that is a discrete dichotomy (there is no continuum underlying the two categories. Like death or alive)
• the biserial correlation coefficient, r_b, quantifies the relationship between a continuous variable and a variable that is a continuous dichotomy (there is a continuum underlying the two categories. Like passing an exam).
• Kendall’s correlation coefficient, τ, is like Spearman’s but probably better for small samples.

Partial and semi-partial correlation

Semi-partial (or part) correlation

There is a type of correlation that can be done that allows you to look at the relationship between two variables, accounting for the effect of a third variable.

You can transform the correlation coefficient into proportion of variance by squaring them.
If we multiply the resulting proportions by 100 we turn them into percentages.

The semi-partial correlation expresses the unique relationship between two variables as a function of their total variance.

Imagine we want to look at the relationship between two variables X and Y, adjusting for the effect of Z.
The semi-partial correlation squared is the uniquely shared variance between X and Y, expressed as a proportion of the total variance in Y.

Partial correlation

Express the variance in terms of variance in Y left over when other variables have been considered.

Summary

• a partial correlation quantifies the relationship between two variables while accounting for the effects of a third variable on both variables in the original correlation
• A semi-partial correlation quantifies the relationship between two variables while accounting for the effects of a third variables on only one of the variables in the original correlation.

Comparing correlations

Comparing independent rs

To compare correlations we can convert them to z_r.
We can calculate the z-score of the difference between these correlations using:;

z_dfiiference= (z_r1-z_r2)/square root((1/(N₁-3))+(1/(N₂-3)))

We can look up this z-score in the appendix.

Comparing dependent rs

you can use a t-statistic to test whether a difference between two dependent correlations are significant.

T_difference = (r_xy -r_zy) square root(((n-3)(1+r_xz)) / (2(1-r²_xy -r²_xz -r²_zy +2r_xyr_xzr_zy)))

This value can be checked agains the appropriate critical value of t with N-3 degrees of freedom.

Calculating the effect size

Correlation coefficients are effect sizes.

How to report correlation coefficents

You report how big they are, their confidence intervals and significance value.