Statistics, the art and science of learning from data by A. Agresti (fourth edition) – Chapter 11 summary

INDEPENDENCE AND DEPENDENCE (ASSOCIATION)
Conditional percentages refer to a sample data distribution, conditional on a category. They form the conditional distribution. If the probabilities for two different categorical variables are the same in the same category, then these variables are independent. If the probabilities for two different categorical variables differ, then these variables are dependent. Dependence refers to the population, so if there is barely any difference between two categorical variables in a sample, it could be independent, even though they differ.

TESTING CATEGORICAL VARIABLES FOR INDEPENDENCE
The expected cell count is the mean of the distribution for the count in any particular cell. The formula for the expected cell count is the following:

The chi-squared statistic summarizes how far the observed cell counts in a contingency table fall from the expected cell counts for a null hypothesis. It is the test statistic for the test of independence. The formula for the chi-squared statistic is:

The sampling distribution using the chi-squared statistic is called the chi-squared probability distribution. The chi-squared probability distribution has several properties:

1. Always positive
2. Shape depends on degrees of freedom
3. Mean equals degrees of freedom
4. As degrees of freedom increases the distribution becomes more bell shaped
5. Large chi-square is evidence against independence

The degrees of freedom in a table with r rows and c columns can be calculated as following:

If a response variable is identified and the population conditional distributions are identical, they are said to be homogeneous. The chi-squared test is then referred to as a test of homogeneity. The degrees of freedom value in a chi-squared test indicates how many parameters are needed to determine all the comparisons for describing the contingency table. The chi-squared test can test for independence, but it cannot provide information about the strength and the direction of the associations and provide information about the practical significance, only about the statistical significance. When testing particular proportion values for a categorical variable, the chi-squared statistic is referred to as a goodness-of-fit statistic. The statistic summarizes how well the hypothesized values predict what happens with the observed data.

DETERMINING THE STRENGTH OF THE ASSOCIATION
A measure of association is a statistic or a parameter that summarizes the strength of the dependence between two variables. The association can be measured by looking at the difference of two associations. The formula for the difference of the two proportions is the following:

The ratio of two proportions is also a measure of association. This is also called the relative risk. The relative risk uses the following formula:

The relative risk has several properties:

1. The relative risk can equal any non-negative number
2. When p1=p2, the variables are independent and the relative risk = 1,0.
3. Values farther from 1,0 represent stronger associations.

For fixed conditional distributions, the value of X2 is directly proportional to the sample size.

USING RESIDUALS TO REVEAL THE PATTERN OF ASSOCIATION
The difference between an observed and expected count is called a residual. A standardized residual reports the number of standard errors that an observed count falls from its expected count and uses the following formula:

SMALL SAMPLE SIZES: FISHER’S EXACT TEST
Fisher’s exact test is a small sample test of independence. Ordinal variables are categorical variables that have a natural ordering of the categories. There is a low end and a high end of the categorical scale.