The contribution of quantitative and qualitative research methods - summary of chapter 11 of Historical and conceptual issues in psychology, by Brysbaert, M and Rastle, K (second edition)

Foundation of psychology
Chapter 11
The contribution of quantitative and qualitative research methods

The essence of quantitative research

Quantitative research methods: research methods based on quantifiable data; are associated with the natural-science approach based on the hypothetico-deductive method.

Assumptions underlying quantitative research methods

There is an outside reality that can be discovered

Quantitative psychologists start from the assumption that phenomena in the world have an existence outside people’s minds.
They defend the idea that humans can discover reality by using the scientific method.
They are well aware of the fact that science is not a linear accumulation of facts but proceeds through trial and error. But are convinced that in the long term the scientific method based on the hypothetico-deductive model leads to an understanding of reality → scientific knowledge is cumulative

The main aim of scientific research is to find universal causal relationships

Researchers are primarily interested in discovering relationships between causes and effects.
How general are principles? And how do humans function?
Ideally they hope the mechanisms they discover will apply to all humans.

Trying to avoid confounds and sources of noise

Users of quantitative research methods are extremely vigilant about the possible intrusion of undesired factors into their designs.
They try to maximally control the circumstances under which they run their studies
They also try to eliminate the impact of random variables called noise.

Suspicion about the researcher’s input

A source of confounding and noise that is of particular interest to quantitative psychology researchers is the researcher him- or herself.
To protect themselves against biases and noise, quantitative researchers make use of standardised measurements and instruments.

Progress through falsification

Researchers constantly try to prove each other wrong.

Research methods are divided into three broad orientations

• Descriptive research
• Relational research
• Experimental research

Descriptive research

Observation of numerical data

Detailed observation is the start of scientific research.
Typical for quantitative research is that the data are gathered in a numerical form, either by collecting measurements or by counting frequencies of occurrence.

Before researchers collect data, they have a good idea of how they will analyse them; what types of measurements they will obtain and what types of statistics they can apply to summarise and evaluate the data.

Large samples and a few data points per participant

The vast majority of descriptive quantitative studies involve the collection of a limited amount of data from a reasonably large group of participants.
Two main reasons to include large groups

• The larger the sample examined, the more representative it becomes for the population
  Researchers usually want to generalise their findings from the sample studied to the population
• Large numbers of observation yield more precise statistics
  The mean of a sample is closer to the population mean when it is based on a large sample than when it is based on a small sample

Descriptive research usually is only the first step of a quantitative research programme, because researchers want to know what caused the data they observe.
To discover cause-effect relations, a fist move is to find out which events (variables) are related.

Relational research

Correlations

The way to find out whether two variables are related according to quantitative psychologists is to collect measures of both variables and to correlate them.
These correlations also provide information about how strong the correlation is.

People do not seem to be good at detecting which variables are correlated.
Problems involved in the intuitive detection of correlations by humans

• The failure to detect genuine correlations
  In particular, negative correlations seem to be difficult to perceive
• Illusory correlations: perception of a correlation between events for which no independent evidence can be found
  People are prone to illusory correlations when
  • Two variable overlap in meaning
  • The correlation is good for the self-esteem

Factor analysis

Statistical correlations make it possible to investigate the pattern of correlations between any number of variables.
Factor analysis: a statistical technique calculating how many factors are needed to account for the correlations between the variables measured and how these variables relate to the factors.

Experimental research

Correlations do not allow researchers to be sure about causes

Relational research does not allow to be certain about the origin of the correlation.

Experiments to determine cause-effect relations

To be able to draw firm conclusions about cause and effect, quantitative psychologists will set up an experiment in which they manipulate the suspected cause and see whether this has an effect on the phenomenon they are examining.

Controlling for confounding variables

Only the suspected causes are manipulated and the resulting changes in the phenomenon registered.
Everything else must be held constant.

Confounding variable: variable that was not taken into account in the study and that may be the origin of the effect observed.
To address the confounds, experimental psychologists introduce an increasing number of controls.
Or they run several experiments on the same topic and look for converging evidence across studies.

Experiments are not always possible

Not all issues in psychology can be addressed experimentally.
This may be one of the reasons why progress in psychology is harder to achieve than in other sciences.

Strengths of quantitative research methods

The application of powerful statistical analysis enables researches to detect nearly every pattern of association in large datasets.
The application of falsification tests prevents wrong ideas and weak theories from thriving for too long.

Limits of quantitative methods

No interest in the person behind the participant

The research is designed in such a way that each participant returns one or a few numbers, which can be used in statistical analysis.
Participants are confronted with researchers who shun close interactions during the data gathering because they fear such interactions might invalidate the study.

The lack of interest in the person behind the participant is of particular concern when the research concerns real-life situations.
In these situations psychologists can learn a lot by listening to the experiences and opinions of the people involved.

Research is too much driven by what can be measured numerically and tested experimentally

Quantitative psychologists have a bias to limit their research to topics that can easily by measured.
Quantitative imperative: a bias only to find measurable topics interesting because quantitative research methods require numerical data.

Much research has been geared towards questions that can be addressed empirically.
The aspects of mental life that cannot be captured by numbers and that cannot be manipulated in an experiment have been considered of secondary importance

The falsification test lends itself better to destroying ideas than to finding practical solutions to specific problems

The falsification test is primarily geared toward erasing wrong theories rather than generating new ones.

Interim summary

The essence of quantitative research

• Quantitative research methods refer to research methods based on quantifiable data and the following assumptions
  • There is an objective reality to be discovered
  • The main aim of scientific research is to find universal cause-effect relations
  • To do this, one has to rely on the hypothetico-deductive method and avoid confounds and sources of noise
• A distinction can be made between descriptive, relational and experimental research
  • Descriptive research: trying to express variables as numbers, usually involves a few measures from a large group of participants
  • Relational research: searching for statistical correlations in order to understand relationships between variables. Use of factor analysis to find the structure in datasets with many variables
  • Experimental research: searching for cause-effect relationships by excluding confounding variables. Experiments are often not possible.
  • Status of the different types of research can be understood by analogy with the hierarchy of evidence in medical science
• Strengths
  • Inherits the strengths of the natural sciences
  • Application of powerful statistical techniques enables researchers to detect every pattern of association in large datasets
• Weaknesses
  • No interest in the person behind the participant
  • Research too much driven by what can be measured numerically and tested experimentally
  • The falsification test is not primarily geared towards the generation of new ideas and finding practical solutions to specific problems

The essence of qualitative research

Qualitative research methods: research methods based on understanding phenomena in their historical and socio-cultural context; are associated with the hermeneutic approach based on understanding the meaning of a situation

Assumptions underlying qualitative research methods

In psychology there is little or no evidence for a reality outside people’s minds

Most qualitative psychologists are not convinced that in psychology there is an objective reality, which can be discovered with the scientific approach.
For them the only reality that matters is the reality as perceived and constructed by people.
There are differences in the degree that the various methods question the existence/importance of an objective reality.
It is more important to understand people’s views rather than their responses to aspects of the environment.

Attempts to control the situation make the setting artificial and impoverished

Quantitative researchers are misguided in their attempts to try to measure ‘reality’ in unbiased ways.
The attempts turn the environment into an artificial setting that robs the participants of their usual ways of interacting and coping with meaningful situations.

The investigator should become an active participant and listen to what the participant has to say.
He should be guided by a constructive desire to understand the meaning of what is going on.

Qualitative researchers acknowledge that the approach they promote entails the danger of the conclusions being influenced by the researcher but argue that:

• This danger is offset by the expected gains due to an understanding of the situation
• All conclusions, even those reached on the basis of quantitative research and falsification tests, are relative (because they depend on the paradigm)
• The most obvious biases can be avoided by being aware of them and by doing the analysis in such a way that it can be repeated and checked by others

Immersion and understanding

The point of departure of qualitative psychology is the immersion of the researcher in the situation that is being studied, so that the meaning of the situation can be understood.
The researcher approaches the situation open-mindedly and sees what comes out.

Ideographic vs. nomothetic

Ideographic approach: the conclusions of a study stay limited to the phenomenon under study
Nomothetic approach: a study is run in search of universal principles that exceed the confines of the study.

Induction rather than deduction

Psychologists should pay more attention to inductive reasoning.
Bracketing: requirement in qualitative research to look at a phenomenon with an open mind and to free oneself from preconceptions.

Qualitative research is evidence-based

In qualitative research, too, a study depends on collecting and analysing empirical findings.
In addition, these data must be gathered and made available in such a way that the conclusions can be verified by others.

The main difference in data with quantitative research is that the findings typically are not coded in a numerical formed.
They comprise an organised set of verbal statements that in the researcher’s eyes summarises the examined situation.

Data collection and analysis

Qualitative research stress the importance of ‘rich information’.
Information that in the participant’s eyes adequately describes the situation.
The participants determine what will be found

Data collection

Semi-structured interview: interview in which each interviewee gets a small set of core questions, but for the rest of the time is encouraged to speak freely; achieved by making use of open-ended, non-directive questions.

Focus groups: technique in which a group of participants freely discuss a limited set of questions.

Transcription

The raw materials of semi-structured interviews usually consists of auditory or visual recordings.
These have to be transcribed in written form, so that they can easily be referred to.

The transcription also contains non-verbal signals.

After the transcription, the written records are numbered.

Data analysis

In a qualitative analysis the investigator rewrites the raw materials as a flow chart of core ideas, based on multiple close readings and guided by the questions emphasised by the different approaches.
The analysis requires an adequate classification of the various statements into a number of (recurring) themes and clear ideas of how the components are interconnected.
The researcher tries to encompass the data as comprehensively as possible.

Grounded theory

Grounded theory: qualitative research method that tries to understand what is going on in a particular situation and which, on the basis of a qualitative analysis and induction, tries to come to a theoretical insight grounded in the data

In a grounded theory analysis, the investigator rewrites the raw material on the basis of questions such as ‘What is going on here?’.
On the basis of these questions, the participants’ answers are recorded into a sequence of themes, which are then grouped into higher-order categories.
This makes it possible for a theory to emerge from the data through inductive reasoning.

Phenomenological analysis

Unease with grounded theory

Limitations of grounded theory

• It largely assumed the existence of an objective reality that was there to be discovered
  The findings that emerged from a grounded theory analysis were meant to describe ‘reality’
• Grounded theory stressed the importance of inductive reasoning and verification
• It did not take into account the fact that the data provided by the participants actually comprised their perceptions and interpretations of what was happening

Inspiration form Husserl

Hermeneutically inspired psychologists stressed that the primary aim of qualitative research was to examine what reality looked like for the participants, leaving open the question whether in psychology there is something of an objective, person-independent reality.

Edmund Husserl (1859-1938)
Phenomenology.
Stressed that psychology should be a reflective study of consciousness as experienced from the first-person point of view.
It is better for psychology to return to the experience itself.
The human experience was not in the first place a matter of lawful responses to events in the environment, but a system of interrelated meanings, which Husserl called a Gestalt or Lebenswelt.

Interpretative phenomenological analysis (IPA): qualitative research method in psychology that tries to understand how a phenomenon is experienced by the people involved.

How does IPA work?

IPA resembles very much the analysis of grounded theory.
The main difference is that IPA is centred on how participants make sense of their personal and social world.
It attempts to explore the personal experience and is concerned with the participant’s personal account, not with an understanding of the phenomenon itself.

IPA acknowledges the input form the researcher, who tries to make sense of the participants trying to make sense of their world.
The researcher’s involvement does not mean that the researcher is allowed to introduce obvious biases. Investigators are advised to bracked as much as possible and to approach the new situation open-mindedly.
IPA accepts that some form of meta-interpretation on the part of the researcher is possible.

Key elements of IPA:

• It is an inductive approach
• Aims to capture and explore the meanings that participants assign to their experiences
• Researchers reduce the complexity of the raw data through rigorous and systematic analysis
• The analysis is primarily focused on what is specific for the phenomenon studied
• The resulting analysis is interpretative
• A successful analysis is transparent and plausible
• Researchers should reflect upon their role in the process

Discourse analysis

The linguistic turn in the philosophy of science and in critical psychology

Discourse analysis: qualitative research method that aims to discover how social relations between people are determined by the language they use.
Language is the only topic worth investigating because it makes the world in which humans live.

The linguistic turn in postmodernist writings influenced the development of qualitative research in three ways:

• Researchers turned their attention to language and examined the possibilities and impossibilities language brought with it.
• Researchers became particularly interested in language use in real-world or naturally occurring events
• There was an enhanced reflexivity on the part of the researchers regarding their own language use.

How are relations between people determined by the language they use, and how do people try to achieve goals by means of their language?

All variants of discourse analysis try to determine how participants use discursive resources and what effects it has.

Strengths of qualitative research

• Directly focused on understanding situations and solving problems
  Qualitative research is directly aimed at understanding a problem and working towards a solution.
• Generation of new ideas and elaboration of theories
  Because qualitative studies involve intensive investigations, they are particularly well suited for finding new ideas.
• More perceptive to the needs of participants
  Because qualitative researchers want to understand events as they are perceived by the participants, they will have a much better feeling for the participant’s needs.

Limits of qualitative research

• Limits of inductive reasoning and verification
• Less well suited to decide between theories
• Qualitative methods are based on introspection
• The researcher’s involvement may be a disadvantage in high-stakes situations

Interim summary

The essence of qualitative research

• Qualitative research methods are directed at understanding phenomena in their historical and socio-cultural context. They are based on the following assumptions
  • In psychology there is little or no evidence for a reality outside people’s perception and experience
  • Attempts to control the situation make the setting artificial and no longer meaningful
  • Researchers must immerse themselves in the situation so that they can understand the meaning of the situation
  • Qualitative research is in the first place meant to understand specific situations (ideographic) and not come to general rules (nomothetic)
  • Induction is more important than deduction
  • Qualitative research must remain evidence-based, starting from a careful and verifiable collection of data
• Data collection usually occurs by means of semi-structured interviews with a limited number of participants; increasingly also focus groups are used
• The data need to be transcribed and analysed up to saturation along the lines proposed by the qualitative method that is used
• Three methods
  • Grounded theory: tries to understand the phenomenon
  • Interpretative phenomenological analysis (IPA): tries to understand the ways in which the phenomenon is perceived and experienced
  • Discourse analysis: tries to understand how language constructs human interactions
• Strengths
  • Directly focuses on understanding situations and solving problems
  • Generation of new ideas and elaboration of theories
  • More attention to the participant’s needs
• Weaknesses
  • Based on induction and verification
  • No external criterion to decide between theories
  • Based on introspection
  • Input from the researcher may be a problem in high-stake situations

How do quantitative and qualitative research methods relate to each other?

The two types of research are incompatible

Incompatibility of the underlying paradigms

Psychologists who stress the incompatibility of the quantitative and qualitative methods emphasise the different world views underlying them.

• Objective reality vs. social construction
• Natural sciences vs. humanities
• Hypothetico-deductive vs. hermeneutics
• Mainstream psychology vs. critical psychology

Quantitative psychology’s arguments against the need for qualitative psychology

• The hermeneutic and postmodernist movements throw away all the progress that has been made
• They reject the existence of an outside reality, which is the reason to exist of science
• Qualitative research methods do not provide researchers with new information and devalue psychological research to pop psychology

Qualitative psychology’s arguments against the need for quantitative psychology

• Mainstream psychology clutch at the scientific method not because it has brought understanding of human functioning, but because it has brought status and money to psychology departments
• Quantitative research is misguided in its search for the ‘objective reality’
• If there is no objective reality, quantitative research methods have nothing to tell us about human functioning

Trying to reconcile quantitative and qualitative research methods are attempts to regain lost ground

Psychologists who consider qualitative and quantitative research to be incompatible often mistrust efforts to unite them, because they see these efforts as disguised attempts to regain lost ground.

Two types of research complement each other

The other view is that qualitative and quantitative research methods can be used in tandem depending on the question one wants to answer.

Quantitative research is more than a positivist search for physical laws

Scientists nowadays make a distinction between

• A deterministic process: a process in which the variability is so small that you can predict the outcome with high accuracy when you know the precursors
• A stochastic process: a process where the variability in the possible determinants and the contribution of the random noise is so big that it becomes impossible to exactly predict the next outcome.

Qualitative research is more than a chat with participants

Qualitative research must include

• Representativeness
• Confirmability
• Credibility
• Comparison of situations that differ on one critical aspect
• Alternative explanations
• Refutability

Interim summary

How do quantitative and qualitative research methods relate to each other?

• Some psychologists see them as incompatible and argue that psychology must make a choice
  • The underlying philosophies are mutually exclusive
  • Attempts to combine both approaches are disguised attempts to improve the standing of the natural-science research line or the hermeneutics oriented research line at the expense of the other
• Other psychologists see both types of research methods are complementary; they focus more on the type of information provided by each method rather than on the philosophies that underlie them
  • Fervent supporters of each approach tend to depict an exaggerated view of the other approach
  • The weakness of one approach are the strengths of the other

Focus on: is too much respect for the philosophy of science bad for morale?

Interim summary

Is philosophy of science useful for psychology?

• Psychology has tried to follow the directives from philosophers of science on how to do ‘proper’ science, but has been confronted with changing and at times conflicting advice
• The problem may be that philosophy of science in vain tries to distil a limited set of rules that would govern a process which is not deterministic
• An alternative view that may be more in line with the stochastic nature of scientific discovery is the evolutionary account. According to this model, the rise and fall of scientific ideas follow Darwinian principles of random variation and natural selection
• Because natural selection depends on the fir of an idea in the environment and because science depends on the wider culture to be financed, the ultimate criterion determining whether an idea will survive may be whether society at large finds the idea interesting and useful
• This may entail a return to the pragmatic criterion