"Furr & Bacharach (2014). Estimating and evaluating convergent and discriminant validity evidence.” - Article summary

There are four procedures to present the implications of a correlation in terms of our ability to use the correlations to make successful predictions:

1. Binomial effect size display (dichotomous)
   This illustrates the practical consequences of using correlations to make decisions. It can show how many successful and unsuccessful predictions can be made on the basis of a correlation. It uses the following formula:
   
2. Binomial effect size display can be used to translate a validity correlation into an intuitive framework. However, it frames the situation in terms of an ‘equal proportions’ situation.
3. Taylor-Russell tables (dichotomous)
   These tables inform selection decisions and provide a probability that a prediction will result in a successful performance on a criterion. The size of the validity coefficient (1), selection proportion (2) and the base rate (3) are required for the tables.
4. Utility analysis
   This frames validity in terms of a cost-benefit analysis of test use.
5. Analysis of test sensitivity and test specificity
   A test is evaluated in terms of its ability to produce correct identifications of a categorical difference. This is useful for tests that are designed to detect a categorical difference.

Validity correlations can be evaluated in the context of a particular area of research or application.

A nomological network refers to the interconnections between a construct and other related construct. There are several methods to evaluate the degree to which measures show convergent and discriminate associations:

1. Focusses associations
   This method focusses on a few highly relevant criterion variables. This can make use of validity generalization.
2. Sets of correlations
   This method focusses on a broad range of criterion variables and computes the correlations between the test and many criterion variables. The degree to which the pattern of correlations ‘makes sense’ given the conceptual meaning of the construct is evaluated.
3. Multitrait-multimethod matrices
   This method obtains measures of several traits, each measured through several methods. The purpose is to set clear guidelines for evaluating convergent and discriminant validity evidence. This is done by evaluating trait variance and method variance. Evidence of convergent validity is represented by monotrait-heteromethod correlations.

The correlations between measures are called validity coefficients. Validity generalization is a process of evaluating a test’s validity coefficients across a large set of studies. Validity generalization studies are intended to evaluate the predictive utility of test’s scores across a range of settings, times and situations. These studies can reveal the general level of predictive validity (1), reveal the degree of variability among the smaller individual studies (2) and it can reveal the source of the variability among studies (3).

A requirement for convergent and discriminant validity in the multitrait-multimethod matrices could be that the correlations of a construct should be more highly correlated with other measures of this construct than with other constructs using the same method.