Practice Questions for Reliability and Validity

Questions

1. What is the difference between reliability and validity, two central terms within statistics?

2. Of which two parts consists the total variance in a data set of scores?

3. Between which two numbers does reliability range?

4. Which three kinds of reliability can be distinguished?

5. How can the split-half reliability be computed?

6. What is the difference between internal and external validity?

7. A study-counselor tries to predict study success. He administers a questionnaire about motivation to first year students. At the end of the year, he determines whether the students finished their year successfully. Next, he determines the correlation with the score on the questionnaire. What kind of validity is involved here?

8. A researcher has established that higher levels of testosterone in young men coincides with increased risk behavior when driving. In a follow-up study, he finds the same association for young women. What kind of validity is involved here?

Answers

1. What is the difference between reliability and validity, two central terms within statistics?
The reliability refers to the extent to which a measurement instrument provides consistent results. A reliable instrument will provide similarly results when doing a measurement twice. Validity describes whether the measured construct is indeed measured by the instrument.

2. Of which two parts consists the total variance in a data set of scores?
The total variance consists of the variance from the true scores and the variance from measurement errors (error variance and systematic variance).

3. Between which two numbers does reliability range?
Between 0 and 1.

4. Which three kinds of reliability can be distinguished?

• Split-half reliability

• Inter-item reliability

• Inter-rater reliability

5. How can the split-half reliability be computed?

For the split-half reliability, the items are divided between two sets. Next, a total score is calculated for each set. Then, the correlation between both sets is computed. If the items in both sets measure the same construct, the correlation between the sets should be high.

6. What is the difference between internal and external validity?
Internal validity implies that the researcher draws conclusions about the effects of the independent variable. External validity refers to the extent to which the results can be generalized to other conditions or samples than in the study.

7. A study-counselor tries to predict study success. He administers a questionnaire about motivation to first year students. At the end of the year, he determines whether the students finished their year successfully. Next, he determines the correlation with the score on the questionnaire. What kind of validity is involved here?
Predictive criterion validity.
We speak of predictive criterion validity, when a measurement instrument is able to distinguish between people on a behavior criterion in the future, thus, whether the measurement can predict well. This is especially important in the educational setting.

8. A researcher has established that higher levels of testosterone in young men coincides with increased risk behavior when driving. In a follow-up study, he finds the same association for young women. What kind of validity is involved here?
External validity. External validity refers to the extent to which found study results can be generalized to other samples.