Samenvattingen bij de voorgeschreven artikelen van Wetenschapstheorie (RUG) 21/22

Psychology is defined as the science of human behaviour, and social psychology is defined as the branch that deals with human interaction. A goal of science is to establish general laws through systematic observations. Social psychologists thus want to discover causal relationships to establish basic principles that explain the phenomena studied in social psychology. The idea behind this goal was that if general principles of human behaviour could be established, it might be possible to reduce social conflict, mental illness, and to create optimal social conditions that would benefit society. Some even hoped to translate these principles into mathematical forms. However, in contrast to events in the natural sciences, concepts in the social sciences are not stable. Instead, social concepts are nonrepeatable and fluctuate a lot over time. This means that knowledge cannot accumulate. In this paper, the author discusses how social psychology is a historical inquiry.

What was the impact of science on social interaction?

Social science can be viewed as a communications system. The scientist receives messages, transmitted by humans. These messages are ‘noise’, which need to be decoded with the use of scientific theories. This were Back’s (1963) ideas. However, his ideas had to be extended by including another task of the scientist: he or she needs to be a communicator. When a scientists discovers ideas, he or she needs to communicate these, so to help others. Thus, science and society are in a feedback loop.

What is the prescriptive bias of psychological theory?

When social scientists generate knowledge about social interactions, they also communicate personal values. This means that he or she sends a dual message: messages that describe what happens, and messages that prescribe what is desirable. Consider research on personal dispositions. Many people would not like to be characterized as compulsive, or close-minded. This is because of cultural processes. However, these reactions are also created by psychologists who describe certain concepts. For example, when psychologists state: “The authoritarian personality tries to combine societal beliefs with irrational beliefs”. This communicates that an authoritarian personality is undesirable. Findings may also drive people’s behavior, for example when females read that they are more persuasible than males, they might try to overcompensate. In sum, the concepts in the field of psychology are not value-free. However, value-free terms are not as interesting to the reader. In addition, psychologists themselves are also humans, who try to express themselves in their writings. Therefore, it is hard to change this labelling of concepts in terms of values. The author suggests to maintain as sensitive as possible to biases and to communicate them as openly as possible. Even if it is unavoidable to express values, one can avoid labelling them as objective reflections of truths.

What is the relationship between knowledge and behavioral liberation?

Based on Rosenthal’s (1966) research that found that even subtle cues about experimenter expectations can alter the behaviour of the subject, it is now common practice to avoid telling the participant about the theory behind an experiment. There are thus ‘naïve subjects’ required. This is also a difference between the natural and social sciences, in which communication in the latter can have a vital impact on behavior. However, when people learn about psychological processes, they might become more knowledgeable in terms of thinking about alternatives. Thus, being conscious of certain behaviors may modify behaviours and thus lead to ‘behavioral liberation’.

 However, knowledge can also become power to others. When people can predict your behavior, they can manipulate you. Humans don’t like to be controlled, they strive for autonomy. Therefore, the stronger a theory is (the more it is able to predict behaviour), the more people will try to invalidate the theory. This is a disadvantage for social psychology, because there is a high drive for people to invalidate theories that predict human behaviour.

What can we do about it?

Since humans try to invalidate strong social psychological theories, it seems that the science might be removed from the public. However, another way is to find out how people react to theories. This might help the science to predict and control the effects of knowledge. This is called a psychology of ‘enlightenment’. However, this is hard. This theory may also be rejected because of its threats to feelings of autonomy.

What about psychological theory and cultural change?

Another argument for that social psychology is based on historical circumstances is resembled in the fact that many predictors change over time. For example, variables that successfully predict political activism during the Vietnam war do not predict activism during later periods. Thus, the factors changed over time.

What are the implications for an historical science of social behavior?

The author states that the study of social psychology is an historical undertaking. Scientists are often engaged in describing contemporary affairs. They utilize scientific methodology, but the results are not scientific principles. The author describes five alterations to the field that could help scientists.

Alteration 1: Toward an Integration of the Pure and Applied

Academic psychologists seem to be prejudiced against applied researchers. It is seen as being limited to solving immediate problems, and pure research is seen as contributing to basic and enduring knowledge. However, as you now know, pure research is also not endurable. However, one shortcoming of applied research is that the terms used are often very specific. In academic psychology, the explanatory language is more general and can be used more broadly. The author suggest to focus on contemporary social issues which are based on the application of scientific methods and tools for broad generality.

Alteration 2: From Prediction to Sensitization

The author describes that the traditional aim in psychology to predict and control behaviour is misleading. Therefore, it should be play more a role as a ‘sensitizing device’, with which it can elucidate the range of factors that potentially influence behaviour under various conditions. Social psychology can sharpen one’s sensitivity to subtle influences and pinpoint assumptions about behaviour which are not proved useful in the past. When a social psychologist is asked for advice, the best thing to do would be to inform the inquirer about a number of possible occurrences, and thus expand the inquirer’s sensitivity.

Alteration 3: Develop Indicators of Psycho-Social Dispositions

The author suggests that concepts in social psychology should not be viewed as basic as in the natural sciences. Therefore, there is a high need for methodologies that tap into the prevalence, strength, and form of psychological dispositions over time. This means that a technology of psychologically sensitive social indicators is required.

Alteration 4: Research on Behavioral Stability

Phenomena vary in the extent to which they are subject to historical change. This signals the need for methods that determine the relative durability of social phenomena. For instance, scientists could employ cross-cultural methods. Content analytic techniques could also be used to study accounts in earlier historical periods.

Alteration 5: Toward an Integrated Social History

The author concludes by stating that there should be more focus on the interrelation of events over extended periods of time. Thus, the history should be studied in a broad context, including political, economic, and institutional. This can help to achieve understanding in an integrated way.

BulletPoints

• Psychology is defined as the science of human behaviour, and social psychology is defined as the branch that deals with human interaction. A goal of science is to establish general laws through systematic observations. Social psychologists thus want to discover causal relationships to establish basic principles that explain the phenomena studied in social psychology. The idea behind this goal was that if general principles of human behaviour could be established, it might be possible to reduce social conflict, mental illness, and to create optimal social conditions that would benefit society. Some even hoped to translate these principles into mathematical forms. However, in contrast to events in the natural sciences, concepts in the social sciences are not stable. Instead, social concepts are nonrepeatable and fluctuate a lot over time. This means that knowledge cannot accumulate. In this paper, the author discusses how social psychology is a historical inquiry.

• Social science can be viewed as a communications system. The scientist receives messages, transmitted by humans. These messages are ‘noise’, which need to be decoded with the use of scientific theories. This were Back’s (1963) ideas. However, his ideas had to be extended by including another task of the scientist: he or she needs to be a communicator. When a scientists discovers ideas, he or she needs to communicate these, so to help others. Thus, science and society are in a feedback loop.

• Based on Rosenthal’s (1966) research that found that even subtle cues about experimenter expectations can alter the behaviour of the subject, it is now common practice to avoid telling the participant about the theory behind an experiment. There are thus ‘naïve subjects’ required. This is also a difference between the natural and social sciences, in which communication in the latter can have a vital impact on behavior. However, when people learn about psychological processes, they might become more knowledgeable in terms of thinking about alternatives. Thus, being conscious of certain behaviors may modify behaviours and thus lead to ‘behavioral liberation’.

What are the twenty difficulties in conducting research in certain areas of psychology?

1. The response-class problem

This difficulty or problem is about that it is difficult to slice up behaviour in measurable units. For example, what is important to getting a degree? Is it sitting an exam, writing a sentence, writing a letter or is the firing of a neuron that is responsible for the movement of the arm and hand important? This problem is already evident in the experiment of the Skinner box and becomes even more problematic when studying human behaviour.

2. Situation-Taxonomy problem

This is kind of the same as the respons-class problem, but then it is applied to stimuli or to situations. For example, every theory of personality defines behaviour as a function of the person and the situation. So, the situation should be defined: is it the whole culture, the region a person lives in, a t town, a house, a room?

3. Unit of Measurement

This is about the questions in rating scales and in psychometrics. This is a problem, because there is often no agreement in whether, for example, an interval or a ratio scale should be used.

4. Individual differences

Individuals differ in their dispositions and even more problematic, in the way their dispositions are shaped and organized. This makes it difficult for researchers to conduct research and compare individuals on the results.

5. Polygenic Heredity

Most of the research done in the soft areas of psychology are things that are influenced by polygenic systems. This means that there are many different influences on for example, introversion. In other words, introversion is probably the result of a confluence of different polygenic contributors such as anxiety, dominance, need for affiliation, etc.

6. Divergent Causality

This difficulty is about that a small event can lead to many different (divergent) outcomes. For example, many students marry other students. But, the way that they get to know these other students vary from individual to individual and it depends on things such as to which classroom they get allocated.

7. Idiographic problem

There are many factors that influence certain things, for example personality. These factors also interact and therefore every individual is quite unique. This is a problem for research, because the research methods should therefore be extremely large and complex. This is the same for the statistical analysis, which also should be sophisticated enough to analyse all the factors and their numerous interactions.

8. Unknown Critical Events

This problem is related to divergent causality and the idiographic understanding. This problem is about that it is hard to know which events in the development of personality have been critical. Sometimes these critical events are unknown to the researchers as well as to the individual. These critical events can also be inner events such as fantasies.

9. Nuisance Variables

For research it is good to sort variables in three classes: variables that we manipulate, variables that we do not manipulate but can control (or keep constant to reduce the influence of them on the results) and variables that are ‘quasirandom’, which means that they only contribute to the measurement error or the standard deviation of a static. But, there are also variables that are not random but that are systematic and that are influenced by other variables. The problem is that it is not always easy to determine the size and the sign of these relationships. One of the biggest of these ‘nuisance variables’ is socioeconomic status, which is very hard to control for, because this variable has a big influence on many aspects of people’s lives.

10. Feedback Loops

In psychology it is known that an individual’s behavior affects the behavior of other person. This fact also makes that conducting research on behavior can be very complex.

11. Autocatalytic Processes

Autocatalysis means that one of the end products leads to catalyzing the process itself. In psychology examples of these processes are depression and anxiety as affects or economic failure as a social impact. It is very difficult to get a clear view of situations in which autocatalytic processes are involved.

12. Random Walk

Sometimes things happen randomly. Then the quantification of this behaviour becomes difficult. For instance, luck is one of the biggest reasons for individual differences. Social scientists seem to neglect this sometimes in their research.

13. Sheer Number of Variables

There are many variables that contribute to a psychological disorder or to a personality trait. Because of this, there are also a lot of nuisance variables and all these variables also lead to idiographic development through divergent causality.

14. The Importance of Cultural Factors

Sometimes researchers forget to include cultural factors that lead to certain disorders or diseases. For example, a physician will treat someone with a stomach ache in the same way, regardless of their social class. This is a problem, because sometimes a problem can be solved or prevented by looking at the cultural factors (like, someone belongs to the lowest class regarding SES and therefore he or she has many deficits in his or her diet which lead to stomach ache).

15. Context-Dependent Stochastotogicals

In Psychology, the term ‘nomological network’ is used to represent the relations between constructs. The constructs are ‘nodes’ which are linked to other constructs through strands. Meehl states that ‘nomological’ in psychology is a synonym for ‘meaning’ and it could even be misleading. He says this, because the relationships are not as strict as suggested by calling it a nomological network. In psychology, instead of strict relationships, there are correlations, tendencies, statistical clusterings, increments of probabilities and stochastic dispositions. The author therefore suggests a “stochastological network”. With this he tries to make clear that the results obtained in psychological experiments are dependent on the context in which the measurements were obtained. Because it is hard to determine what influences what and how big this influence is. So, in soft psychology, the context is very important. But, we do not know the complete list of the contextual influences, the function form of context dependency for the known influences and the numerical values of parameters in those functions and lastly the values of the context variables.

16. Open Concepts

It is often hard to come up with operational definitions for theoretical concepts. Even if researchers come up with operational definitions, these seem to lack theoretical background and technologies to measure them. This means that there are a lot of ‘open concepts’ in psychology, which further complicates research.

17. Intentionality, Purpose and Meaning

Humans think, plan and intend way more than any other species. This makes them stand out compared to other species, but again leads to complexity in researching their behaviour.

18. Rule Governance

People or human beings do not only do things out of inner motivation, they also do things because they think that is what is expected from them. They feel obligated to follow certain rules. In research there is not a clear definition of when a rule can be considered as a rule. So when conducting research, it is sometimes difficult to determine whether one performs some kind of behavior because of real, internal motivation or because he or she is following a certain rule.

19. Uniquely Human Events and Powers

Because many of the research in psychology, such as digestion and reproduction, comes from studies with animals, some things are very difficult to research. For example, Richard M. Elliot stated that the reason for the lack of high quality research on humor is that humans are the only species that laugh. So, because some things are uniquely human, it makes it difficult to conduct research. Except for laughing, other examples that are unique to human are seeking revenge for things that happened years earlier or has discussions on discussions.

20. Ethical Constraints on Research

There are many ethical constraints in research in psychology. For example, it is unethical to let a group of children from as old as 10 years smoke and then compare them to children who did never smoke. Meehle names five noble traditions in psychology that have lead to great insights. In the times of these traditions, the ethical constraints were a lot less than that they are now. The five noble traditions are: descriptive clinical psychiatry, psychometric assessment, behavior genetics, behavior modification and psychodynamics. He states that Freud did not perform any t-test, but that he would choose Freud’s clinical observations over most of the researchers’ t-tests. All these traditions have not focused on statistical significance testing.

Most emotions occur in the contexts of social interactions and relationships. However, most psychological studies have focused on emotions in nonsocial situations, such as during physical threats. When you assume that emotions develop in social contexts, then you also assume that emotions are ‘socially costructed’. In this article, it is described how emotion is construed from a multicomponential perspective of emotion. This means that emotions are viewed as emerging from the interplay between different components (cognitive, motivational, and physiological), rather than being unitary entities. Also, a person’s appraisal of the situation arises from other components of emotion. For instance, when someone spills a drink over you and you think that this is on purpose, you might get angry. However, when you see that the person slipped and it happened on accident, you may not be as angry. It is described how emotions differ based on different contexts: interactions, relationships, and the sociocultural context.

How are emotions construed within contexts?

Moment-to-moment interactions

Emotions develop as a function of interactions. For instance, when you are arguing, your emotion depends highly on the other person’s reactions: is he or she accepting or denying your emotions? Thus, the emotions that others have and express, interact with your own emotions. Also, emotions are influenced by the type of relationships. If you know someone for a very long time you may experience different emotions when he or she lies to you, compared to when you just met someone. Emotions are thus shaped by the relationship in which they occur, and they also shape relationships. There are also differences in the way that emotions are construed across sociocultural contexts. For example, some cultures emphasize autonomy, while others emphasize the relatedness between people (collectivism). Therefore, in individualistic cultures, anger may be seen as functional and it may be accepted. In contrast, in collectivistic cultures, the expression of emotions that reflect individual desires (anger) may be discouraged and perceived as immature.

Thus, the social construction of emotion is an iterative and ongoing process that unfolds within interactions, relationships, and which derive their shape and meaning from the large sociocultural context. The process of social construction happens during each emotional episode.

What are the interdependencies between the three different contexts?

There are not many studies conducted on the interplay between interactions, relationships and cultural contexts. One rare example on this topic is a cross-cultural study on mother-child interactions in Germany and Japan. This study showed that German mothers, in response to a disobeying child, infer that the child is acting on purpose. Therefore they react with anger. In response, the child reacts with protest. Then, the mother sees this as affirmation for her idea that her child is acting like that on purpose. This turns into a tantrum by the child. Then the mother feels like she lost control and the conflict is unresolved and may be repeated in the future. In Japan, mothers were more likely to view the disobedience of the child from the child’s perspective (“it is just a child, he or she just wants to play, he or she is just tired”). When their children misbehave, Japanese others repeat their demands in a friendly manner, and feel sorry for the child. The child reacted with irritation to the demand, and the mother feels disappointment and regret. In Japan, ittaikan refers to the desired feeling of harmony and one-ness, and it is a prevailing value. Therefore, both the child and the mother made concessions. The mothers comforted their children, and distracted them. This lead to that the child complied and the conflict was solved. These types of interactions have long-term effects on children’s relational patterns: escalations of interactions in early childhood predict the level of empathy-based altruism 9 years later. This example shows the interplay between the different contexts: mother-child emotional interactions were constructed based on prevalent cultural meanings, these shaped relational contexts immediately and over time, and therefore these culturally prevalent ways of dealing with emotions are maintained over time.

What can be concluded?

There is thus evidence for the social construction of emotion in three different contexts: moment-to-moment interactions, relationships, and cultural contexts. These findings signal that emotional construction should not only be viewed on one level of analysis. Instead, emotions should be studied from a social constructionist perspective.

BulletPoints

• Most emotions occur in the contexts of social interactions and relationships. However, most psychological studies have focused on emotions in nonsocial situations, such as during physical threats. When you assume that emotions develop in social contexts, then you also assume that emotions are ‘socially costructed’. In this article, it is described how emotion is construed from a multicomponential perspective of emotion. This means that emotions are viewed as emerging from the interplay between different components (cognitive, motivational, and physiological), rather than being unitary entities. Also, a person’s appraisal of the situation arises from other components of emotion. For instance, when someone spills a drink over you and you think that this is on purpose, you might get angry. However, when you see that the person slipped and it happened on accident, you may not be as angry. It is described how emotions differ based on different contexts: interactions, relationships, and the sociocultural context.

• There is thus evidence for the social construction of emotion in three different contexts: moment-to-moment interactions, relationships, and cultural contexts. These findings signal that emotional construction should not only be viewed on one level of analysis. Instead, emotions should be studied from a social constructionist perspective.

What is this article about?

In 1948, the WHO introduced a definition of health as “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity”. This definition has been criticised over the past 60 years, and the criticism is increasing. This article describes the limitations of this definition and describes how it can be made more useful.

What are the limitations of the WHO definition?

 Most criticism on the WHO definition is about the word ‘complete’ in relation to wellbeing. He or she states that almost no one is healthy for most of the time. This is a problem, because this has lead medical technologies and drug industries to sell a lot of things that are supposed to increase “health”, and develop drugs for “conditions” that are not actually health problems. Also thresholds for interventions tend to be lower. The author suggest that this is due to the emphasis on complete physical wellbeing. 

The second criticism is that the demography of populations and the nature of diseases have changed considerably. In 1948, acute diseases presented the main burden of illness and chronic diseases led to death. However, there are now different disease patterns. Currently, ageing with chronic illnesses has become the norm. The healthcare system is under pressure, and this is also partly due to the WHO definition which seems to declare people with chronic diseases and disabilities as ill.

The third criticism on the definition is about the operationalisation. It is impracticable because ‘complete’ is not operational (not measurable).

What do we need?

The author suggests that the definition should be changed. The Ottawa Charter is one proposal, and emphasises social and personal resources as well as physical capacity. However, WHO has never accepted any proposal for change of their definition. It is also hard to redefine health, because it is complex; there are many aspects that need to be considered and there are many stakeholders. A discussion between Dutch experts about the definition of health was “the ability to adapt and self manage”. To be able to use this concept, it should be identified for the three domains of health: physical, mental, and social health.

Physical health

When an organism is healthy, the organism is capable of ‘allostasis’: maintaining physiological homeostasis through changing circumstances. When an organism experiences stress, it should be able to create a protective response, to reduce the potential for harm, and restore equilibrium. If this coping strategy is not successful then there is damage, called allostatic load. This eventually leads to illness.

Mental health

Mental health is described as ‘the sense of coherence’, which is a factor that contributes to successful coping, recovering from psychological stress, and prevent post-traumatic stress disorders. It is about understanding, managing and giving meaning to difficult situations. Being able to adapt and manage oneself improves subjective wellbeing and may result in a good interaction between mind and body. For example, patients with chronic fatigue syndrome who receive cognitive behavioural therapy show positive effects in terms of symptoms and wellbeing, as well as an increase in brain grey matter volume.

Social health

In the social domain, health is seen as people’s capacity to fulfil their potential and obligations, the ability to manage their life with some independence, and to be able to participate in social activities including work, despite having a medical condition. Patients with chronic illnesses who learnt to manage their lives better, reported improved self-rated health, less distress, less fatigue, more energy and less perceived limitations. Their healthcare costs also reduced!

How can we measure health?

Health measurements should relate to health as the ability to adapt and to self-manage. Operational tools include existing methods for assessing functional status and measuring quality of life and sense of wellbeing.

BulletPoints

• In 1948, the WHO introduced a definition of health as “a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity”. This definition has been criticised over the past 60 years, and the criticism is increasing. This article describes the limitations of this definition and describes how it can be made more useful.

• The second criticism is that the demography of populations and the nature of diseases have changed considerably. In 1948, acute diseases presented the main burden of illness and chronic diseases led to death. However, there are now different disease patterns. Currently, ageing with chronic illnesses has become the norm. The healthcare system is under pressure, and this is also partly due to the WHO definition which seems to declare people with chronic diseases and disabilities as ill.

• The author suggests that the definition should be changed. The Ottawa Charter is one proposal, and emphasises social and personal resources as well as physical capacity. However, WHO has never accepted any proposal for change of their definition. It is also hard to redefine health, because it is complex; there are many aspects that need to be considered and there are many stakeholders. A discussion between Dutch experts about the definition of health was “the ability to adapt and self manage”. To be able to use this concept, it should be identified for the three domains of health: physical, mental, and social health.

How, and what, can we know?

Epistemology is a part of philosophy and it is called ‘the theory of knowledge’. It answers the question: “how and what can we know?”. It is about thinking about the nature of knowledge itself, the scope of knowledge and about the reliability and validity of knowledge. Research methods have the goal of answering questions. However, before answering a question, one needs to know what the goal is, so: what are the objectives of our research? One also needs to decide on what kinds of things are possible to find out. This means that one should adopt an ‘epistemological position’ before carrying out any research.

What is positivism?

Positivism is one example of an epistemological position. According to positivism, there is a one-to-one relationship between the world (objects, events, phenomena) and our perception and understanding of it. If one takes a positivistic stand, then one believes that what is ‘out there’ is also all there is. It is also called ‘the correspondence theory of truth’. It suggests that phenomena directly determine our perception. There is thus no room for talking about differences in perception based on thoughts or beliefs. According to positivism, the goal of research is to produce objective knowledge: knowledge based on a view from the outside.

What is empiricism?

Related to positivism, there is empiricism. According to empiricism, our knowledge of the world should be derived from ‘facts of experience’. According to empiricism, simple observations are combined and lead to more complex ideas. Theory follows from observations: theory has the goal to make sense of the data collected through observation. Experiments are based on this position. However, contemporary, many researchers do not agree with the idea that perception is always related to facts. For instance, the more one knows about something, the more detail one perceives when observing it. Perception is known to be selective and people can perceive the same phenomenon in different ways. However, modern-day empiricists still argue that knowledge depends on the collection and analysis of data. Only theory is not enough: all knowledge must be complemented with data. There is a difference between the terms ‘empiricist’ and ‘empirical’: ‘empiricist’ refers to the attitude that all knowledge must be grounded in data, and ‘empirical’ refers to research involving the collection and analysis of data.

What is hypothetico-deductivism?

Karl Popper introduced the idea of ‘hypothetico-deductivism’, as a reaction to limitations of positivism, empiricism, and inductivism. It is now the basis of mainstream psychology. Popper explained that a collection of observations does not indicate causality. Even if someone observes the same thing repeatedly, one can not be sure: there is always the possibility that the next observation is off. This is called the problem of induction, and it meant that no scientific theory in that time could be verified because they relied on inductivism. Therefore Popper proposed to, instead of relying on induction and verification, scientists should rely on deduction and falsification. In hypothetico-deductivism, theories are tested by deriving hypotheses from them and then test these in practice. The aim of the research is to put the theory’s ideas to test, by either rejecting the theory of retain from it. Thus, instead of relying on confirmatory evidence, in hypothetico-deductivism researchers look for disconfirmation or falsification. It is about finding out whether a claim is not true. By a process of eliminating claims, one can find the truth.

What are critiques of the ‘scientific method’?

Popper’s method was also criticized, for example:

1. Hypothetico-deductivism limits theory development: relying on hypotheses generated by existing theories, it is hard to generate new theories.
2. Hypothetico-deductivism is elitist: since this position works with existing theories, it excludes people who are not familiar with such theories and systems from its practice.
3. Hypothetico-deductivism is a myth: in this position, scientists test hypotheses to find out which theories can be discarded. Thomas Kuhn (1962, 1970) disagreed. He stated that theories are not put to test in this way: if scientists found out that a theory could not be confirmed, then they assumed that their experiment was wrong.

What are feminist critiques of established epistemologies?

Most of the critiques noted came from females. In the 1960s and 1970s, females remarked that women are invisible in social scientific work. When women were studied, they were often portrayed as being inferior to men in terms of moral development, intelligence, and conversational style. To challenge these inequalities, the feminist scholars questioned the epistemological foundations of these types of knowledge. This lead to the critique of ‘male science’, which includes the following arguments:

1. The male as the norm. Most of the studies carried out were done with male subjects. Therefore, the findings from these studies were generalized to the whole population of males. Then, other members of society were compared to these males, making these males ‘the norm’.
2. The God trick. The ‘male science’ claimed to be objective. Feminists argued that the attempt to be ‘objective’ and the strategies adopted towards this goal were not effective. They stated that it is impossible for a researcher to position themselves ‘outside of’ the subject matter. Attempts to do so were called ‘God’s eye view’.

However, in despite of these feminist critiques, there is no one feminist epistemology or methodology. There were several approaches developed by feminist social scientists and philosophers, called standpoint epistemology (Harding, 1991), ethnomethodology (Stanley & Wise, 1983), and varieties of feminist post-structuralism (Henriques et al., 1984).

What is social constructionism?

Social constructionism is about the fact that human experience, including perception, is mediated by history, culture, and linguistics. What one perceives and experiences is thus not a direct reflection of environmental conditions. Instead, it must be understood within these conditions. It states that there are ‘knowledges’ instead of ‘knowledge’. Research within social constructionism is about how there are various ways of constructing social reality. For example, researchers in this field have studied emotion, prejudice, and psychopathology to describe how these provide a way of construing reality instead of reflecting reality.

What about epistemology and methodology?

To answer what the relationship is between epistemology and methodology, one first needs to be aware of what the difference is between ‘method’ and ‘methodology’. Even though these terms are often used interchangeably, they are different. ‘Methodology’ is defined as ‘a general approach to studying research topics’, while ‘method’ refers to a specific research technique. For example, when a researcher adopts an epistemological position, he or she will adopt a methodology of collecting data. The specific methods used can differ. Hypothetico-deductivism is one approach that offers both an epistemological position and a research method, namely hypothesis-testing through experimentation. A researcher’s epistemological and methodological commitments do limit him or her in which methods can be used. Consider a social constructionist methodology: this methodology is not compatible with methods that are designed to measure variables in a population. This is because it questions the validity of ‘psychological variables’, and is more about exploring different ways in which these variables are construed.

What can be said about qualitative research?

Qualitative research methods can and are used by researchers with different epistemological positions. Empiricist as well as social constructionists use qualitative measures. This means that there are ‘qualitative methodologies’, and not one ‘qualitative methodology’. However, qualitative methods share a number of assumptions, and this can be referred to as ‘qualitative methodology’.

Qualitative methodology

Qualitative research is about meaning. It is about how people make sense of the world and how they experience events. They are thus interested in the quality and texture of experience, and not with identifying causal relationships. Qualitative researchers ask questions such as: “What do people do when they form groups?”, or “How do people manage change in the workplace?”.

Epistemological differences

Epistemologies determine the approach to qualitative data and thus the theory that is used as a framework. If one takes a social constructionist position, then they may approach text using a discourse analytic theoretical framework. When one has a empirical position, then one may use the grounded theory method or content analysis to identify categories of meaning. The different qualitative methodologies can be differentiated based on the extent to which they emphasize reflexivity and by the degree of emphasis on language. Reflexivity refers to that a researcher is aware of his or her contribution in the research, and acknowledging that it is impossible to remain ‘outside of’ the subject matter (think back about the ‘God’s eye’). Reflexivity is then about that the researcher reflects on what their role is in the research. There are two types of reflexivity: personal reflexivity and epistemological reflexivity. Personal reflexivity refers to reflecting on the ways in which our own values, experiences, interests, beliefs, political commitments, and so forth affect the research. It is also about thinking how the research may have affected and possibly changed us. Epistemological reflexivity is about questions such as: “How has the research question defined and limited what can be ‘found’?, and “How has the design of the study and the method of analysis ‘constructed’ the data and the findings?”. Qualitative researchers differ in the emphasis they place on reflexivity in their research: for some, personal and epistemological reflexivity are important and should be reported, while for others they do acknowledge the importance, but do not report it. One part of reflexivity is ‘critical language awareness’. This means that the words we use to describe experiences play a part in the construction of the meanings we attribute to experiences: it has to a constructive dimension. The categories and labels that researchers use will thus shape their findings. For example, when a researchers asks a respondent how she felt during a medical procedure, this invokes the category ‘emotion’. Whatever the response is, emotion will have to be oriented to.

What are ‘small q’ and ‘Big Q’?

There is a distinction between two meanings of ‘qualitative research’, with big Q referring to open-ended, inductive research methodologies. ‘Big’ Q is oriented with theory generation and exploration of meanings, whereas ‘little q’ refers to including non-numerical data into hypothetico-deductive research (thus complementing experiments with qualitative data). For an example of ‘Big Q’, researchers can include an open-ended question in a forced-choice questionnaire, and then use content analysis to ‘score’ the qualitative material. It is thus not about ‘bottom-up’, the goal is not to find ways to gain new insights into the ways in which participants construct meaning and/or experience of their world. Instead, they have a hypothesis and they have defined categories, against which the qualitative data is then checked. In the book, it is mainly about ‘Big Q’ methodology.

What are three epistemological questions?

To evaluate research, one needs to know what the goal of the research is. There are three questions that can help to identify the methodology’s epistemological roots:

1. What kind of knowledge does the methodology aim to produce? The type of knowledge that is aimed to produce depends on the epistemological positions.
2. What kinds of assumptions does the methodology make about the world? This is about ‘ontology’, which is defined as the nature of the world. Compared to epistemology, which asks: “How can we know?”, ontology asks: “What is there to know?”. When one assumes that events are generated by underlying structures such as socioeconomic status, this would constitute a materialist ontology. If one assumes that psychological phenomena are independent from such structures, this is called an idealist position. Ontological positions are divided into ‘realist’ and ‘relativist’. A realist ontology is about that the world exists of structures and objects that have cause-effect relationships. Materialism is a form of realist ontology. A relativist ontology states is about that there are different interpretations of the world. Idealism is form of a relativist ontology.
3. How does the methodology conceptualize the role of the researcher in the research process? Some methodologies in qualitative research view the researcher as a builder who constructs a house: the researcher is the central figure in the research process. Other methodologies in this research acknowledge the importance of the researcher, but do not view the researcher as the author of the findings. Instead, they view the researchers as someone who uses their skills to find evidence.

BulletPoints

• Epistemology is a part of philosophy and it is called ‘the theory of knowledge’. It answers the question: “how and what can we know?”. It is about thinking about the nature of knowledge itself, the scope of knowledge and about the reliability and validity of knowledge. Research methods have the goal of answering questions. However, before answering a question, one needs to know what the goal is, so: what are the objectives of our research? One also needs to decide on what kinds of things are possible to find out. This means that one should adopt an ‘epistemological position’ before carrying out any research.

• Related to positivism, there is empiricism. According to empiricism, our knowledge of the world should be derived from ‘facts of experience’. According to empiricism, simple observations are combined and lead to more complex ideas. Theory follows from observations: theory has the goal to make sense of the data collected through observation. Experiments are based on this position. However, contemporary, many researchers do not agree with the idea that perception is always related to facts. For instance, the more one knows about something, the more detail one perceives when observing it. Perception is known to be selective and people can perceive the same phenomenon in different ways. However, modern-day empiricists still argue that knowledge depends on the collection and analysis of data. Only theory is not enough: all knowledge must be complemented with data. There is a difference between the terms ‘empiricist’ and ‘empirical’: ‘empiricist’ refers to the attitude that all knowledge must be grounded in data, and ‘empirical’ refers to research involving the collection and analysis of data.

• Most of the critiques noted came from females. In the 1960s and 1970s, females remarked that women are invisible in social scientific work. When women were studied, they were often portrayed as being inferior to men in terms of moral development, intelligence, and conversational style. To challenge these inequalities, the feminist scholars questioned the epistemological foundations of these types of knowledge. This lead to the critique of ‘male science’, which includes the following arguments:

  • The male as the norm. Most of the studies carried out were done with male subjects. Therefore, the findings from these studies were generalized to the whole population of males. Then, other members of society were compared to these males, making these males ‘the norm’.
  • The God trick. The ‘male science’ claimed to be objective. Feminists argued that the attempt to be ‘objective’ and the strategies adopted towards this goal were not effective. They stated that it is impossible for a researcher to position themselves ‘outside of’ the subject matter. Attempts to do so were called ‘God’s eye view’.

• To answer what the relationship is between epistemology and methodology, one first needs to be aware of what the difference is between ‘method’ and ‘methodology’. Even though these terms are often used interchangeably, they are different. ‘Methodology’ is defined as ‘a general approach to studying research topics’, while ‘method’ refers to a specific research technique. For example, when a researcher adopts an epistemological position, he or she will adopt a methodology of collecting data. The specific methods used can differ. Hypothetico-deductivism is one approach that offers both an epistemological position and a research method, namely hypothesis-testing through experimentation. A researcher’s epistemological and methodological commitments do limit him or her in which methods can be used. Consider a social constructionist methodology: this methodology is not compatible with methods that are designed to measure variables in a population. This is because it questions the validity of ‘psychological variables’, and is more about exploring different ways in which these variables are construed.

• There is a distinction between two meanings of ‘qualitative research’, with big Q referring to open-ended, inductive research methodologies. ‘Big’ Q is oriented with theory generation and exploration of meanings, whereas ‘little q’ refers to including non-numerical data into hypothetico-deductive research (thus complementing experiments with qualitative data). For an example of ‘Big Q’, researchers can include an open-ended question in a forced-choice questionnaire, and then use content analysis to ‘score’ the qualitative material. It is thus not about ‘bottom-up’, the goal is not to find ways to gain new insights into the ways in which participants construct meaning and/or experience of their world. Instead, they have a hypothesis and they have defined categories, against which the qualitative data is then checked. In the book, it is mainly about ‘Big Q’ methodology.

To study the efficacy of psychotherapeutic treatments, researchers use validated self-report questionnaires to quantify symptoms. Based on randomized controlled study designs (RCTs), evidence for therapeutic efficacy is gathered. In an RCT, symptom levels are assessed for a group of patients pre- and posttreatment, compared to a group of patients who have received an alternative treatment or no treatment at all (a placebo). RCTs are regarded as the best type of evidence in therapeutic research. In meta analyses, multiple RCTs are reported and this serves as a basis for large-scale policy decisions on the organization of evidence-based mental health care. When using RCTs, one requires significance tests to show that the treatment group performed better than the control (alternative/no treatment group). In psychotherapeutic research, evidence is thus based on numbers (differences between the groups and p-values for significance). In this article, it is described how patients who participate in the research translate their story into numbers, so that their story becomes ‘data’ that is used as evidence.

In psychotherapy research, numerical data is collected using standardized self-report questionnaires: participants choose one of several answers. However, this comes with issues. First, the respondents/participants have to be able to understand the questions. This is not always the case: sometimes a question is interpreted differently by someone. Participants may also give a different response, based on what they believe is expected of them. This can lead to over- or underestimation in responses. Second, respondents have to score their behavior on numerical scales. This scoring is affected by the format! For example, participants feel more comfortable to score “extreme” on a -5 to +5 scale compared to a scale going from 0 to 10. The reason for this is that participants feel that a ‘0’ in the -5 to +5 scale reflects ‘neutrality’, while in the scale going from 0 to 10 they feel like it reflects ‘absolute absence’. Third, the type of answers that respondents can provide are limited and do not allow for nuances. Lastly, respondents find it hard to decide which reference point they have to keep in mind while scoring (so: does this question refer to my whole life, the last year, the last week?).

These issues are just part of a broader problem. In psychology, the human beings are both the object and the subject of the science, which means that we have to ask the object (the human) to study their own behaviour (the subject). When humans are asked to reflect on their feelings, thoughts, attitudes and behaviors, the idea of being observed or assessed may invoke a change in their behaviour (which is called the Hawthorne Effect). For example, someone can answer in a socially desirable way or someone could emphasize a set of complaints when they know that they are being observed. It is hard to deal with these issues, because each human is different.

In psychotherapy, these problems are even more prominent because in psychotherapy the objects of interest are behaviours, thoughts and feelings that already deviate from the norm. In the clinical practice, patients often experience a lot of different symptoms. However, in research it is often required to isolate different symptoms, to be able to engage in causal attribution.

Even though there are these kind of epistemic problems, this is hardly talked about in empirical papers. Instead, the numerical approach is still regarded as the most reliable and trustworthy means to obtain evidence in psychotherapy research. The idea is that when RCTs are designed properly, when the used questionnaires/measures are valid, and when the data-analysis is reliable and transparent, then the evidence will be sound. However, trs his reasoning leaves out that the value of the evidence, even if the analysis is perfect, still depends on the input into the analysis. This means that the data that is collected has to be valid in the first place to make any research valid. In this article, the aim is to analyze the validity of the ‘data’ collected in psychotherapeutic research. They look at how patients’ stories are translated into numerical data in psychotherapy research. To do so, they discuss a case of a participants who participated in one of their psychotherapeutic studies (SCS).

What methods and materials were used?

Participants

This study makes use of a ‘case’, which is an example of an in-depth analysis of a person. In this study, June was the case. June is 26 year old and was referred to treatment by her general practitioner. June experiences an intense and overwhelming fear to be sick/dysfunctional. She fears to have different kinds of ‘psychological and physical diseases’ and she is afraid to go crazy. When she hears, reads, talks or even thinks about specific problems that other people have, she becomes obsessed with the fear of having the problem as well. These problems range from depression, paranoia, and substance dependence to cancer and intestinal problems. June was selected because she made a lot of annotations on paper-and-pencil questionnaires, which make her a ‘rich case’.

Therapy

June received 58 psychotherapeutic session. The therapist was a 36-year old male with experience in psychotherapeutic therapy. He works on the principles of supportive-expressive therapy, as defined by Luborsky. This is a form of psychodynamic treatment which focuses on creating a supportive therapeutic environment to pursue cognitive and emotional understanding.

Research team

The case study in this article was conducted by a research team consisting of the first, third, and fourth author. The first author had the task to conduct follow up on the data collection. However, the researcher never spoke to the patient because she was not reachable. All contact with the patient was conducted by the therapist. To avoid interpretation bias because of therapeutic experience over narrative analysis, the therapist was not involved in the analyses of the case data. The findings were discussed with the therapist (the second author) to reach consensus on the interpretations.

Materials

The data in this article were obtained in a naturalistic psychotherapy study (SCS). In this SCS, all therapy sessions were recorded. Furthermore, after each therapy session, the patients completed self-reports and the therapist wrote a clinical report. Examples of the used questionnaires were:

• The General Health Questionnaire (GHQ-21). This is a 12-item self-report measure that assesses general mental wellbeing in the previous week. For example: “Have you recently been able to enjoy your normal day-to-day activities?”. These items are answered on a 4-point Likert scale, in which 0 = Not at all, 1 = Not more than usual, 2 = A bit more than usual, and 3 = A lot more than usual. The GHQ was administered after each therapy session.

• The Beck Depression Inventory-II Dutch Version (BDI-II-NL). This questionnaire measures the presence and severity of depression symptoms with 21 items. In the Dutch version, each item has a title that indicates a symptom, for example “aversion of one-self”. This is answered by a 4-point Likert scale. An answer is scored by circling the number before the sentence. For example, the item “suicidal thoughts or wishes” provides the following options: 0 = I don’t have any thoughts of killing myself, 1 = I have thoughts of killing myself, but I would not carry them out, 2 = I would like to kill myself, and 3 = I would kill myself if I had the chance. In the current study, the BDI was administered following every eighth therapy session.

• The Depressive Experiences Questionnaire (DEQ-nl). This questionnaire assesses personality patterns that are associated with depressive symptoms and it contains 66 items, for example: “It is not ‘who you are’, but ‘what you have achieved’ that counts.” The items are scored on a 7-point Likert scale, with 0 = Not at all to 7 = Extremely. In the current study, the DEQ was administered following every eighth therapy session.

• The Inventory of Interpersonal Problems (IIP). The IIP assesses interpersonal dynamics from the responder’s perspective, specified for the relationship with their parents. It consists of respectively 64, 32, and 32 items. An example is: “I find it hard to feel good about the happiness of others.” The respondent has to answer how strongly this item reflects themselves. Items are scored on a 5-point Likert scale with 0 = Not at all and 4 = Very strongly. The IIP was administered following every eighth therapy session. The shortened 32-item version was administered following every therapy session.

• The Personal Style Inventory (PSI). This is a 48-item that assesses personality styles that are related to depression. For example: “I often put the needs of others before my own”. These items are scored on a 6-point Likert scale, ranging from 1 = Completely disagree to 6 = Completely agree. The PSI was administered following every eighth therapy session.

• The Symptom Checklist (SCL-90). The SCL-90 assesses mental wellbeing and deviation and it has 90 items. Examples are: “To what extent were you hindered by one of the following complaints” during the previous week?”. The items have to be scored for the previous week and are answered based on a 5-point Likert scale, with 0 = Not at all to 5 = Extremely. The SCL was administered after every eighth therapy session.

What was the procedure?

June came to the therapy for a period of three years: weekly, bi-weekly or monthly. After the 53th week, she stopped coming to therapy. After two years, she restarted her treatment. She received 58 sessions of treatment. After three sessions with the therapist, she agreed to participate in a pilot psychotherapy study. She signed an informed consent form. She also agreed to participate in follow-up interviews, but as noted before, she did not show up to these interviews. In the psychotherapy study (the SCS), different methods were used: quantitative, qualitative and biological. All sessions were recorded (audiotaped) so that the sessions could be transcribed. After each therapy session, the paper-and-pencil questionnaires were administered. June requested to complete the questionnaires in the presence of the therapist, and this request was accepted. The therapist did not look at the scores and he brought the paper-and-pencil questionnaires to the university. Then, the first author put the data into SPSS. Doing so, the therapist noticed the amount of visual and textual annotations on the questionnaires and therefore June was purposively selected for a case study.

This article is about the experience of questionnaire administration and not about the results. Therefore, the experiences and visualizations that June reported are discussed, and not the quantitative scores on the questionnaires.

How was the data analyzed?

The narrative data were analyzed based on the principles of ‘interpretative phenomenological analysis’. First, June’s therapeutic narrative was analysed qualitatively in two different research projects. The first project (HU & FT) focused on the narrative expression of complaints and anxious obsessions, and the second project (ED & FT) focused on the narrative experience of questionnaire administration. In both projects, the researchers first familiarized themselves with the data by transcribing and re-reading it. After, they coded the data in a data-driven manner. Both projects made use of iterative analysis. There were no hypotheses proposed, because this could interfere with a data-driven analysis.

What were the findings?

Three main/core themes in June’s complaints are discussed.

First, June is afraid to be ‘crazy’ or ‘ill’.  She fears to be diagnosed with depression, bipolar mood disorder, paranoia, loss of control, substance dependence, panic or anxiety disorder, chronic fatigue, personality issues, HSP, and other things. She does not fear to have a specific disorder or disease, she just fears to have something. June also seems to be susceptible for suggestion. When she hears about a random condition, she fears to have it herself. For example, she reports about suicide:

Like, sui-, the word suicide com-comes up in my mind and then I am [extremely – in dialect] afraid, I don’t want that [breathes heavily]. Like and then, then it is.. Yeah I am here by myself so in theory I could.. dó something like that, there is no one here to control me, like.. (Session 25)

Whenever June speaks, she analyses her words and says things like: “That is a sign of depression, isn’t it?”

Second, June is afraid to be locked up in a psychiatric institution. June searches for signs of abnormality, because she is afraid that others will perceive her as being crazy. She thinks that when others find her crazy, they might lock her up. Consider the following narrative:

Well, that is that ultimate fear, like, yes, you will be imprisoned, or put in psychiatry, like in a small room or something. And no one will come to you, like, no interaction or no communication or, like, not being recognized as a person but as a patient or something. Or as a “lunatic.” Like, like not normal anymore. (Session 2)

In sum, June’s overt phobic fears seem to stem from a fear of deviating from the societal norm and being an outcast.

Third, June is afraid to be “too much and too little”. June’s relation with societal norms is visualized in a drawing on the questionnaire. She explains her drawing by stating:

With this picture. A box, that is society…. You can even see it in micro or like I don’t know, in 2D. And that is me… that.. that smear, and those are the lines of society and sometimes I have hiatuses, and other times I jump beyond them. And I compare that, or I found that on society, and then I see like [gasps for breath ostentatiously]. So what I do is, those smears that go beyond, those I don’t wanna have, those have to go away. I want control over them, and on the other hand it frightens me, those hiatuses, like [gasps for breath ostentatiously], like “Oh no.” So I would really like to be exactly on the box. (Session 58)

To fit in the ‘box’, she tries to control her feelings and her thoughts, so that it is neither too much or too little. She tries to find understanding of her feelings and thoughts. She also feels an urge to visualize, and to make her understanding as concrete as possible. One ambivalence in June’s behaviour is that she searches to fit societal norms, but she does not want to coincide with those norms. Thus, she does everything she can to control herself, but on the other hand she is afraid that she might coincide with the box. Consider the following narrative:

“[I am] afraid to be put in a cage and [I want to] escape from that” (Session 11).

What was June’s questionnaire experience?

First, June is conflicted about the meaning of items. This means that she can not interpret the items in a straightforward manner. She takes a lot of time to decide which interpretation is “right”. Consider the following narrative:

“I am too generous towards others” … Does [this item] mean materialistically or do you mean…? For me that is really… contradicting. For me, that is materialistic…. I am very egoistic, but uhm… Regarding stuff, like borrowing something that they may use or like, with my time that they… Then I am ver-.. a bit too generous. […] Do I have to write that down like that, or not? Or? (Session 1)

Because of this, she scores the item “I am too generous towards others” by writing down “materially” (scored a 0), and writing down “time & services” (scored a 4 out of 4).

Another example refers to her answers on the item “I am a very independent person”. She scores this item both a 7 (yes) and 1 (no). She annotates: “when I am anxious, I feel a need for someone to hold me”. She also sometimes divides items into different parts and provides multiple answers to one item.

She also often changes the instructed time frame, so that it better fits the reference that she has in mind. For example, the GHQ is scored on a 4-point Likert scale with 0 = Not at all, 1 = Not more than usual, and 3 = A lot more than usual. June adds a category called “A bit less than usual”. June is also very ambivalent about scores: she often crosses scores and changes them to another number. She also marks two numbers and connects them with a line, indicating that she prefers a score in the middle of those two numbers.

June’s use of symbols and visual mark-ups indicate that she finds it important to elaborate on her answers.

Second, June is apprehensive about the confrontation with items in and the evaluation by questionnaires. This happens because topics can make her fearful. She also visualizes these fears by writing “from this thought, I get scared”.

Third, June is fueled by the act of questionnaire completion. This refers to that June was apprehensive of the questionnaire items and that she requested to complete them with the therapist. During this process, June regularly asks the therapist about the meaning of items and instructions and scores, and so forth. It seems that asking questions calms her down. At the end of the 53th session, June tells the therapist that she would like to speak with him about how questionnaire items “suggest” things for her or “bring things to her mind”. However, after this session, she does not return for a number of years. In the 54^th session, she explains that she felt that her anxiety was under control and that she did not need therapy anymore. Later she experienced panic attacks and decided to call the therapist for a session. In this session, she explained that the questionnaires brought phobic thoughts to her mind.

How does questionnaire administration interact with June’s primary complaints?

The questionnaire as suggestion

When June hears, reads, sees or even thinks about an illness, she becomes fearful and experiences thoughts that she has an illness or psychological disorder. This means that her fears are provoked and increased by the questionnaires in the study. The questionnaires thus became a catalyst of June’s phobic fears. This means that the questionnaire may become a feared stimulus itself. This explains why she requests the therapist to stay with her during administration: this makes her feel more relaxed.

The questionnaire is not only a primer of June’s fears, it also creates new phobic fears. For example, multiple questions about suicide lead June to wonder whether she would ever be capable of doing something like that. Even though she does not experience any suicidal thoughts, she gets scared she might be able to do something like it. This impacts the way she scores the items. For example, she is afraid that she might have a personality disorder. Therefore, she is inclined to score her alleged narcissistic traits, because she feels that she needs to be “honest” in her scoring. However, this means that the scores based on her fear and not on her actual experience.

The questionnaire as judgmental other

The story that June tries to tell via the questionnaire seems to be addressed to ‘another’, which could be a concrete person or a even broader, society. This is indicated by the amount of details and remarks she puts next to the items. However, June also fears the verdict of others. This means that the questionnaire becomes a substitute for the evaluating other, which scares June because she is afraid that others will perceive her as ‘crazy’. She does not trust herself, so she feels a strong need to keep control of herself and to prevent others of perceiving what she thinks she can perceive herself.

The questionnaire as a cage

On the one hand, June does everything to fit in, and she wants to not fall out of the ‘normal’. She is afraid that she might be locked up by others if she does so. Therefore, she tries to find any signal that shows that she is weird. Thus she tries to fit the ‘box’ of society, but she is very afraid to be locked up in a ‘cage’. This leads to that she is afraid to ‘tick’ specific answers, because she is afraid that these answers may lead her to be locked up. For every answer that she provides, the tries to be as nuanced as possible and tries to avoid leaving an image that is open for interpretation. In terms of the questionnaire, June tries to find all issues that could imply craziness. She then evaluates her own behaviours and thoughts based on this questionnaire. However, every box on the questionnaire increases the risk of being understood within some specific image. This means that the questionnaire is a tool to avoid rejection by others, but it also becomes a warrant to be rejected by others. This makes June feel trapped, and in her attempt to escape, she internalizes all the signs so that she could negate any possible symptoms before they could even appear to others. June ends up controlling herself, and therefore the questionnaire becomes her cage.

What can be concluded?

June shows that questionnaires are not always the best method. She experienced a lot of problems trying to interpret the questionnaire. She felt conflicted about the meaning of items, and she found it hard to interpret items in a straightforward way. She also felt limited by the provided scales, indicated with the amount of visualizations and remarks. This is consistent with other experiences of questionnaire administration in other research settings. For example, Malpass et al. (2016) found that patients often find it hard to ‘fit’ their answers in the scales. Some patients call the scales ‘too simplistic’. This means that the scores that patients provide do not always reflect their experience. This can cause differences in interpretation by different respondents, and this can harm between-subjects comparability and eventually lead to an entirely invalid measurement.

Second, Schwarz (1990) noted the communicative problems of scales. This also played a role for June she appeared to be aware of the audience of the questionnaires. This also fits with other findings, for example the finding that substance abuse patients like to answer the questionnaires in the presence of the administrator, so that they could clarify their thoughts behind their answers. Another study showed that, when participants are asked to ‘think out loud’ during questionnaire completion, they often hesitated, which indicates that they try to fit in the social situation. This means that questionnaires are often used as a tool to communicate with another person.

However, from the perspective of research, June’s tendency to communicate directly to the administrator poses a threat to validity of research. The researchers has to make a difficult decision: should I put June’s data into a standardized format that is often used to analyse data, which means that only one of the answers that June provided is used as data. This means that a lot of information is lost. The researcher could also decide to input every response, but this then makes it hard to compare to other participants. Both options form a threat to the accuracy and legitimacy of the data, because there are only two options: lose information, or lose comparability. This means that comparability is a specific epistemic choice.

June was also aware of the evaluation of others who read her questionnaires. This makes it clear that questionnaire administration in psychotherapeutic research is not only about gaining information from ‘the object of interest’, but that it also involves an active process of meaning-making by the subject who is responding. Questionnaire items have to, in addition to make sense in the common way, also be meaningful within the personal context of the subject. For example, in one study with questionnaire administration after job loss, the participants interacted with the questionnaires to create stories that fit their experiences. Participants use three strategies to deal with questionnaire items: 1) reject it, 2) construct their experiences in a way that matches the questionnaire’s questions, and 3) reformulating the questionnaire’s questions to match their own experiences. This means that the conclusions stemming from questionnaire data are crafted by the respondents from an idiosyncratic point of reference. This means that data do not just represent a report drawn from the respondent, or that it is a general process by which demand characteristics by the researcher impact the respondents answers. Instead, it is dyad between question and answer that has to be made sense of.

McClimans (2011) argued that it is impossible to determine how legitimate a question is before any answers are obtained. This means that it is impossible to determine whether standardized questions are legitimate, before knowing participant’s answers. Instead, to validate the legitimacy of questions, we should be looking into the content of the answer, just like was done in the current study. This means that researchers should genuinely listen and try to learn from the respondents’ answers and ask them what they mean. Another issue is that asking questions can interfere with answering. For example in the case of June. This is referred to as ‘performativity’, which means that descriptions of symptoms are used to describe experiences: based on the description, participants evaluate their own experiences.

What can be said about the validity of data?

June’s case shows that the validity of data is threatened in multiple ways. It lead to cognitive and communicative issues as Schwarz (1990, 2007) described. This means that the act of data collection may interact with the variables that are being studied.Further, it it is vital for the validity of research to scrutinize the validity of data: it is important to understand what the data means, instead of just checking the validity of measures. Currently, psychological research does not seem to be equipped properly to scrutinize the validity of data collection. As Cronbach and Meehl (1995) stated, validity is not a feature of the instrument itself, instead it depends on a valid application in a particular research context. This means that measures should be infused with qualitative questions on the meaning of answers. Also, for statistical analysis to be valid, the input to the analysis should be valid. June’s case shows that validity problems involve more than discussed in the literature. Questionnaire administration can affect the complaints, just because it is measured.

BulletPoints

• In psychotherapy research, numerical data is collected using standardized self-report questionnaires: participants choose one of several answers. However, this comes with issues. First, the respondents/participants have to be able to understand the questions. This is not always the case: sometimes a question is interpreted differently by someone. Participants may also give a different response, based on what they believe is expected of them. This can lead to over- or underestimation in responses. Second, respondents have to score their behavior on numerical scales. This scoring is affected by the format! For example, participants feel more comfortable to score “extreme” on a -5 to +5 scale compared to a scale going from 0 to 10. The reason for this is that participants feel that a ‘0’ in the -5 to +5 scale reflects ‘neutrality’, while in the scale going from 0 to 10 they feel like it reflects ‘absolute absence’. Third, the type of answers that respondents can provide are limited and do not allow for nuances. Lastly, respondents find it hard to decide which reference point they have to keep in mind while scoring (so: does this question refer to my whole life, the last year, the last week?).

• This study makes use of a ‘case’, which is an example of an in-depth analysis of a person. In this study, June was the case. June is 26 year old and was referred to treatment by her general practitioner. June experiences an intense and overwhelming fear to be sick/dysfunctional. She fears to have different kinds of ‘psychological and physical diseases’ and she is afraid to go crazy. When she hears, reads, talks or even thinks about specific problems that other people have, she becomes obsessed with the fear of having the problem as well. These problems range from depression, paranoia, and substance dependence to cancer and intestinal problems. June was selected because she made a lot of annotations on paper-and-pencil questionnaires, which make her a ‘rich case’.

• June was also aware of the evaluation of others who read her questionnaires. This makes it clear that questionnaire administration in psychotherapeutic research is not only about gaining information from ‘the object of interest’, but that it also involves an active process of meaning-making by the subject who is responding. Questionnaire items have to, in addition to make sense in the common way, also be meaningful within the personal context of the subject. For example, in one study with questionnaire administration after job loss, the participants interacted with the questionnaires to create stories that fit their experiences. Participants use three strategies to deal with questionnaire items: 1) reject it, 2) construct their experiences in a way that matches the questionnaire’s questions, and 3) reformulating the questionnaire’s questions to match their own experiences. This means that the conclusions stemming from questionnaire data are crafted by the respondents from an idiosyncratic point of reference. This means that data do not just represent a report drawn from the respondent, or that it is a general process by which demand characteristics by the researcher impact the respondents answers. Instead, it is dyad between question and answer that has to be made sense of.