What types of response bias are there? - Chapter 10

Chapters 10 and 11 deal with threats to psychometric quality. Two types of threats are central here: reaction bias and test bias. Chapter 10 is about response bias. Test bias is discussed in chapter 11.

What is response bias?

Consciously or unconsciously, cooperative or not, zelfverbet honoring or rather self-effacing, response bias plays a constant role in psychological measurement. Reaction bias means that respondents' reactions (negatively) influence the quality of the psychological measurement. Bias means the bias or bias of responses / outcomes, which are often incorrect.  

What types of response bias are there?

There are different types of the reaction bias, each type being influenced by other factors:

• Influenced by the content or design of a test
• Influenced by factors of the test context
• Influenced by conscious possibilities to react in an invalid way
• Influenced by unconscious factors

These factors led to six types of response bias : (1) acquiescence bias (saying yes and saying no); (2) extreme (vs. average) responses; (3) social desirability (" faking good" ), (4) malingering (" faking bad"), (5) random or carefree reaction, (6) gambling.

1. Acquiescence bias (saying yes and saying no)

The acquienscence bias arises when an individual goes along / agrees with statements without looking back / paying attention to the meaning of the statements. This is common with psychological character tests, questionnaires about your own point of view / opinion, interest questionnaires and clinical investigations.

When someone answers all questions in '' one direction '' (that is, either only answers positively or only answers negatively), then these responses can display a valid set of responses, or they may just display a response bias.

The correlation between the same type of tests (from this respondent) is strong. Because if the respondent gives an acquiescent response, there is a good chance that this respondent will also give an acquiescent response in other tests.

The causes of this reaction bias are:

• The items are complex (too difficult) or ambiguous (look alike).
• The test situation creates distractions.
• The respondent simply does not understand the material, so fill in something.

Saying no: bias creates low test scores in the same (negative) direction.
Saying yes: bias creates high test scores in the same (positive) direction.
Consequence: higher / stronger correlations are created than there actually are.

2. Extreme (vs. average) responses

Even though two respondents have the same level of the relevant characteristic / proposition that is stated in a test, one respondent is more inclined to give '' extreme '' answers, while the other respondent prefers to give ''average'' answers .

Example: the statement is 'I am spiritual' and the response options are: not at all, not really, neither of them / neutral, a bit, completely. An 'extreme' respondent gives one of the most extreme answers; or "not at all" or "not at all". An 'average' respondent gives a less extreme answer; or "not real" or "a little."

These extreme or average responses are not in themselves a bias. It reflects the character level of the individual. People with more extreme characteristics should give more extreme reactions and people with more average characteristics should give more average reactions.

Problems do arise such as:

• People with identical character traits use different levels, for example one respondent uses extreme responses and the other respondent uses average responses.
• People with different character traits use the same level, for example, both respond extremely or both respond on average.

3. Social desirability ("faking good")

The social desirability of response bias is when the respondent's intention is to respond in a way that is socially acceptable, apart from his / her actual character traits.

This is influenced by:

• The test content: when the subject of the test links with social desirability.
• The test context: when the consequences of the given reactions are important.
• The personality of the respondents: some people are more inclined to respond socially desirable.

Correlations here are also higher between variables than they actually are.

Del Paulhus did a lot of research into socially desirable responses as an aspect of personality. According to him there are two processes:

1. Impression management: intention to appear socially desirable (sometimes called ' faking good ').
2. Self-deception: unrealistic positive image of yourself. For example, overestimating psychological characteristics.

Another distinction can also be made:

• State-like: impression management (consciously responding in a way that is appropriate in a certain situation).
• Trait-like: self-deception (one has more aptitude for self-deception than the other).

4. Malingering ("faking bad")

Although many researchers are concerned about the problem of social desirability, the opposite can also occur. In some situations, respondents may be more inclined to exaggerate the nature and severity of their psychological problems. Or even pretending that something is wrong, while nothing is really wrong. This is the opposite of the social desirability bias. According to some studies, this form of reaction bias occurs in some test contexts in 7.3% to 27% of cases in psychological evaluations, and even up to 35-45% in forensic evaluations.  

5. Respond randomly or carefree 

Sometimes respondents give answers that are completely or partially random. This can be caused by various factors: a lack of motivation, fatigue and so on. The random or carefree reaction leads to meaningless scores. For example, a respondent may choose to enter the same answer for each item (for example, "neutral" or "agree"). By answering each item randomly, regardless of the item content, the test scores are meaningless. It is estimated that this type of reaction bias occurs in 1-10% of the respondents.

6. Gambling

The last type of reaction bias is gambling. Gambling occurs with questions that only have one correct answer. The result is inconsistency between observed differences and actual differences between respondents, because one is lucky with gambling and the other not. 

What methods are there for dealing with response bias? 

There are three broad strategies for dealing with reaction bias: 

1. Managing the test context.
2. Managing the test content and / or scores.
3. Use specially designed 'bias' tests.

In addition, we can distinguish three goals when dealing with reaction bias:

1. Minimizing the occurrence of reaction bias
2. Minimizing the effects of reaction bias.
3. Discovering reaction bias, possibly intervene.

These strategies and goals can be combined to summarize different methods for reaction bias. We will first list these and then discuss them in more detail.  

1. Strategy 1 + goal 1 = anonymize, minimize frustration, warnings
2. Strategy 2 + goal 1 = simple items, neutral items, forced choices, minimal choice
3. Strategy 2 + goal 2 = balanced scales, opportunity corrections
4. Strategy 2 + goal 3 = embedded validity scales
5. Strategy 3 + goal 3 = social desirability tests, extremity tests , acquiescence tests

1. Minimize the occurrence of the response bias + manage test context 

The occurrence of reaction bias can be minimized by managing the way in which the test is presented to the respondent and by managing the conditions that are set for the respondent within the test situation.

• Reducing situational factors that can cause socially desirable responses.
• Tell the respondent that it is being processed anonymously, so the respondents are less inclined to respond socially desirable. The personal responses do not have any consequences for the respondent, so they are more likely to respond honestly.

A disadvantage of anonymity is that they want to make less effort or have a low motivation. With quick and random response as a result.

Solution: Tell respondents that the validity of their responses to the items is being measured. In other words, false / invalid responses are detected and deleted. This is especially a good solution for simulation (exaggerating psychological problems).

2. Minimizing response bias + manage test content 

Choosing certain forms of test content to reduce the occurrence of reaction bias.

• Formulate the items so that they are easy to understand.
• Use neutral terms in the items. So that no link can be made with socially desirable / acceptable answers by respondents.
• Forced choice ( forced-choice ): there are only two answers (which clearly differ from each other) from which you can choose. You must indicate which term best fits your personality. For example, timid or argumentative.

This is a good solution for extremity choices . For example, yes or no answers.

3. Minimize effects of response bias + manage test content / scores 

Using specialized score procedures to reduce the effect of reaction bias.

One of those specialized score procedures is the so-called 'balanced scales'. This is used as a solution for the acquisition bias (saying yes and saying no). A problem is that no distinction can be made between those who have really high or low scores and those who just choose a positive or negative direction for each item. Balanced scales are tests or questionnaires that deliberately contain certain positively oriented items and negatively oriented items and not just contain positively-oriented or only negatively-oriented items.

Consequence: in this way, the respondents must pay attention to the type of question, negative or positive. Moreover, the people who don't do that are picked out like this. In this way, a distinction is made between the respondents who are truly honest and observant and the respondents who respond in an one-way street (invalid way).

Another specialized score procedure is to have incorrectly answered items weighed differently than unanswered items. This is mainly used as a gambling solution. For example, a correctly answered item receives one point, an incorrect item results in a ¼ point deduction, and an unanswered item receives zero points. This minimizes the effect of gambling.

4. Managing test content to discover reaction bias + intervention

Identifying respondents who possess a form of reaction bias. Valid scales are used that examine the patterns of responses during the test and evaluate the extent to which that pattern reflects different things such as random responses, acquiescence responses, artificial 'good' responses and artificial 'incorrect' responses.

• L-scale: measuring instrument to detect social desirability of bias.
• F-scale: measuring instrument for, for example, detecting simulation.
• K-scale: measuring instrument to detect ' faking good '.
• VRIN scale: measuring instrument to detect random reactions.
• TRIN scale: measuring instrument to detect acquiescence bias.

We can detect the acquiescence bias by means of the reverse scale and the balanced scale .

Intervention: after discovering reaction bias, action is taken where it is possible.

• Do not include test scores of an individual in further analysis.
• Keeping (suspicious) scores, but handling the scores with care.
• Retain (invalid) scores, and use statistical control procedures for potential invalid scores.

5. Use special 'bias' test to discover reaction bias + intervention

Use several scales to measure reaction bias.

These scales allow test users to identify and remove potentially invalid responses. And they allow test users to statistically check the effects of reaction bias.

Use of these scales in two ways:

1. To better understand the reaction bias by studying causes, implications, correlations with other variables, etc.
2. Use scales to measure the extent of (possible) reaction bias in test scores, which may have influenced the test scores .

There are also scales that measure individual differences in the tendency to respond socially desirable. For example ' Marlow-Crowne Social Desirable Scale '. He does this by asking questions that contain true or false answers.

There are also tests that can reveal simulation. One such test is the Dot Counting Test (DCT). In this test, people must count the points on twelve different cards as quickly as possible. Simulation occurs when respondents need just as much time to count both cards with dots that are randomly distributed and cards with dots that are divided into groups.

Terminology

Finally, two terms are briefly discussed for clarification: response sets and response styles. Reaction sets are temporary aspects of test situations or of the test itself. (= situation related). Response styles are stable characteristics of individuals (= person related). 

Chapters 10 and 11 deal with threats to psychometric quality. Two types of threats are central here: reaction bias and test bias. Chapter 10 is about response bias. Test bias is discussed in chapter 11.