What is the formula of a simple regression?

What is a simple regression?

• Regression analysis is a statistical method used to model the relationship between a dependent variable and one or more independent variables.
• It helps us understand how changes in one variable affect another, allowing for predictions and analysis of trends.  

The general formula for a simple regression

The general formula for a simple regression is:

\[Y = a + bX + e\]

Where:

• Y : dependent variable
• X : independent variable
• a : the intercept (the value of y when x = 0)
• b : the slope of the regression line
• e : error, or the difference between the estimated and observed value of Y

Assumptions for a simple regression

The following assumptions have to be met:

• There has to be homogeneity of variances. That means that the variance of Y is the same for each value of X in the population.
• The values of Y that are in accordance with the X-values have to be normally distributed.

Step-by-step example of a simple regression formula

For example, you have to pay 5 euros per hour next to the 30 euros entrance fee for a tennis club. You stay for 3 hours. In this case, the regression formula is:

• Y : dependent variable: how much you should pay
• X: independent variable: number of hours
• a : the intercept: entrance fee
• b : slope of the regression line: euros per hour
• e : error: additional tips that you would like to give

\[Y = a + bX + e =\] \[30 + 5 \times 3 + tips =\] \[45 + tips\]

Interpreting the results of a simple regression

Y

• is the dependent variable, also known as the outcome variable.
• It's what you're trying to predict or explain.

X

• is the independent variable, also known as the predictor or explanatory variable.
• It's the variable you're using to predict the value of Y.

Intercept (a):

• Represents the predicted value of Y when X is 0.
• If X can't reasonably be 0 in your context, the intercept might not have a meaningful interpretation.
• High intercept: The predicted value is high when the independent variable is zero.
• Low intercept: The predicted value is low when the independent variable is zero.
• Positive intercept: The regression line crosses the y-axis above zero.
• Negative intercept: The regression line crosses the y-axis below zero.

Slope (b):

• Represents the change in Y for every one-unit increase in X.
• If b is positive, Y increases as X increases.
• If b is negative, Y decreases as X increases.
• The magnitude of b indicates the strength of the relationship between X and Y.

Error:

• Represents the unexplained variation in Y that's not accounted for by the regression model.
• A smaller error term indicates a better fit of the model to the data.

Reporting the results of a simple regression

The results of a simple regression are usually reported by describing:

• whether the regression model was significant,
• the regression coefficient (b),
• the p-value,
• and the amount of explained variance (R²).

For example:

A simple linear regression showed that study time significantly predicted exam scores, b = 3.00, p < .01, R² = .45.

This means that:

• study time was a significant predictor of exam scores,
• exam scores increased by 3 points for each additional hour of study,
• and 45% of the variance in exam scores was explained by study time.