Critical Thinking in Quasi-Experimentation - Shadish - 2008 - Article


Introduction

With experiments we manipulate an assumed cause and then we observe which effects do follow. This is used for all modern scientific experiments, we try to discover the effects that a cause generates. It is also used in quasi-experiments, but in that context we have to think critically about causation. In a quasi-experiment there is no random assignment used to conditions, but they are carefully chosen.

An Example of a quasi-experiment

For example, in a quasi-experiment children were chosen for the control group. They tried to create a control group that is the same, as much as possible, as the treatment group.

Causation

In our daily lives we mostly intuitively recognize causal relationships. Nevertheless, for many years a precise definition to cause and effect hasn’t been made by philosophers. The definitions depend partly on each other.

What is a cause?

When we look at causes, such as lightning or a lighted match, we can see that none of them is sufficient to generate the effects. We also need multiple other conditions. For example, a lighted match alone is not enough to start a fire, we also need oxygen, combustible materials, it has to be dry, and so on. We can call this lighted match an ‘inus condition’. This means that it’s an insufficient but still needed part of an unnecessary but sufficient condition. We need many factors for an effect to occur, but most of the time we don’t know all of them or their relation.

Experimental causes

For experimental causes the critical feature is that they are manipulable, otherwise we cannot deliberately vary them to discover what happens. When we look at quasi-experiments, the cause is the thing that was manipulated. It’s possible that the researcher doesn’t realize all the thing that were manipulated and that there are many more.

What is an effect?

Something that is the reverse to a fact is a counterfactual. With experiments, we observe what did happen, but for the counterfactual we look at what would have happened. And we can never observe this counterfactual. We still try to create an approximation to this unobservable counterfactual in experiments, and then we have to understand how the given source differs from the initial condition. Often, the best approximation is random assignment that forms a control group. Even though that this control group is not perfect, because the persons in the control group are not identical to those in the treatment group.

Counterfactuals in quasi-experiments

In quasi-experiments the differences between treatment and control are normally not random, but systematic. So these nonrandom controls may not tell us much about what would have happened. These quasi-experiments make use of two different tools to do so. The first one is observing the same unit over time and the second tool is to try to make the nonrandom control groups as identical as possible to the treatment group. But we do not know all the variables, there are always unknown differences. So this creates the problem of being not as good an estimate of the counterfactual as are random controls. 

Causal relationship

According to John Stuart Mill, we have a causal relationship if the cause happened before the effect, the cause related is to the effect, and we cannot find a plausible alternative explanation for the effect. But in most studies it is impossible to know which of the two variables came first. Quasi-experiments have two ways to improve this. First, they force the cause to come before the effect by first manipulating the presumed cause and then observing an outcome afterward. Second, they allow the researcher to control some of the third-variable alternative explanations. Nevertheless, the researches almost never knows what those third variables are.

Campbell’s threats to valid causal inference

Campbell (1957) has provided a tool to identify differences between the control and treatment groups. He codified some of the most commonly encountered group differences which give reasons why researchers might make a mistake in causation. But this list is very general, even though that the threats are often context specific. Part of the critical thinking used in quasi-experimentation is to identify the alternative explanations, to see if they are plausible, and then show whether or not these alternative explanations occurred and could explain the effect.

Critical thinking in quasi-experiments means showing alternative explanations are unlikely

Falsification, created by Popper, is to deliberately try to falsify the conclusion that you wish to draw, rather than only seek information corroborating them. These conclusions are plausible until shown otherwise. Quasi-experimentation also follows this logic. Experimenters have to identify a causal claim, and then they have to generate and examine plausible alternative explanations. But there are a two problems. The first is that the causal claim is never completely clear and detailed. So mostly the claim is just changed slightly when being falsified. Second,our observations are never perfect. They always reflect our wishes, so they can never provide definitive results. So we neither confirm or disconfirm the causal claim.

Check page access:
Public
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

How to use more summaries?


Online access to all summaries, study notes en practice exams

Using and finding summaries, study notes en practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Starting Pages: for some fields of study and some university curricula editors have created (start) magazines where customised selections of summaries are put together to smoothen navigation. When you have found a magazine of your likings, add that page to your favorites so you can easily go to that starting point directly from your profile during future visits. Below you will find some start magazines per field of study
  2. Use the menu above every page to go to one of the main starting pages
  3. Tags & Taxonomy: gives you insight in the amount of summaries that are tagged by authors on specific subjects. This type of navigation can help find summaries that you could have missed when just using the search tools. Tags are organised per field of study and per study institution. Note: not all content is tagged thoroughly, so when this approach doesn't give the results you were looking for, please check the search tool as back up
  4. Follow authors or (study) organizations: by following individual users, authors and your study organizations you are likely to discover more relevant study materials.
  5. Search tool : 'quick & dirty'- not very elegant but the fastest way to find a specific summary of a book or study assistance with a specific course or subject. The search tool is also available at the bottom of most pages

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study (main tags and taxonomy terms)

Field of study

Access level of this page
  • Public
  • WorldSupporters only
  • JoHo members
  • Private
Statistics
507 1
Comments, Compliments & Kudos:

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.