Summary Evaluating Methodologies in International Studies (Harvey)

Deze samenvatting is gebaseerd op het studiejaar 2013-2014.

Chapter 1


Many classical scholars, who, in general, prefer qualitative research (for example case studies), have been taking a skeptical position towards quantitative and scientific research (for example empirical research) in International Politics. One clear example is Henry Bull. His fear was that historical background, diplomacy and philosophy would be ignored and hence the quality of education would decline. Moreover, he claimed that scientific research in IR was impossible, since research would not be able to both control all the variables and do experiments.

In this article, both the classical and scientific positions on international studies are confronted, using the claims of Henry Bull and scientific scholar J. David Singer. Three main claims of Bull are countered by the arguments of Singer.

Classical approach vs. scientific approach

The three claims of Bull countered by arguments of scientific scholars:

  1. Classical scholars were afraid that scientific researchers would not be able to use interesting subjects and construct daring, “out of the box” research questions. Moreover, they argued that the scientific approach in its essence would never get into the real International Politics. However, scientific scholars were capable to do this. Nevertheless, due to the fact they had little ambition or imagination, scientific research did not cross many borders in this area. Scientific scholars, on the other hand, pointed out problems on the classical manner of research.
    1. One was the fact that it can be hard for classical scholars to get good descriptions of subjects (descriptions are often used with qualitative research). This problem is worsened because of the fact that many variables in IR are abstract and not tangible.
    2. In connection with this, a central question remains that of the possibility to trust one’s own impression.
    3. Even in modern times, it is not always clear how one gets their information and which sources are used. Hence, it is hard to extract the context and side on which the writer was positioned. Information on these factors is needed in order to do reliable research. In this area, the scientific approach is more explicit. Furthermore, the scientific approach is replicable (a research can entirely be reproduced).
    4. Another source of critic on this matter comes from postmodernists. They claim that people are not able to be objective. Following from this, in order to do correct scientific research, one should diminish the difference between objectivity and findings that can possible by biased.

For these points, scientific research can be the solution. It dissolves many of the uncertainties in the area of descriptions and the context. Still, the scientific approach on research in IR has some downsides. It will not be sufficient for normative or legal investigations. Here, research that is more descriptive is needed. Even so, scientific investigations can help the qualitative findings of classical research.

  1. The second point is about the scientific approach and cumulative knowledge.  Cumulative knowledge is all the knowledge a group of researchers has that can be useful for solving the bigger problem.

    1. On an object as for example war, there are many findings. All of these findings are relevant for the big research questions. Moreover, to get to these findings, it is necessary to employ scientific research. Qualitative investigations only will not suffice.
    2. These findings are now nothing more than just findings. They are part of a larger thing. The findings do not yet function as a theory. Every finding must be accepted with a challenge, so only the best are used. Now, we are at the frontier of scientific research.
    3. When we would only be counting on qualitative research, we would risk relying on correlation evidence only (and hence we could be drawing conclusions too early). Hence, qualitative data only would be too simplistic and can even make for wrong assumptions.
    4. Quantitative research then can look for the different effects of the different variables. Now, oversimplification would be prevented. However, this is the hardest part of doing research: there are so many variables influencing phenomena. Moreover, these variables can also be mixed. Therefore, we can say that we are still at the frontier of scientific research in IR.
    5. Eventually, these variables will lead to a new theory. Those theories that are widely excepted can be added to the core of cumulative knowledge. However, at this moment in IR, there is not yet core knowledge that would give us evidence that is impossible to deny.
  1. The last subject of clash between proponents of the classical approach and those of the scientific approach is the importance of methodological training and the importance of more traditional subjects. Classical theorists were afraid that the latter would be substituted for the former (we can argue, however, that this has not been the case, since methodological training happens not very often). It remains important to emphasize history and world politics. Students should also know about things as normative theory and philosophy, for it is important to consider your philosophical position before doing research. For example, realism has been unconsciously adopted as a framework for research. This realist paradigm can be overcome by accepting scientific research, which challenges the realist framework.


When weighing the arguments of Bull and Singer, we can say that scientific research does add valuable things to study of International Politics. It has crossed that border of interesting subjects, sometimes in combination with qualitative research and sometimes on its own. Second, on cumulative knowledge, one can argue that international studies are close to the establishment of core knowledge. Here too, qualitative and quantitative researches have been combined. Qualitative investigation on its own could easy make for wrong assumptions. When combined, we can do much more research on the different variables. If this is expended, core knowledge within IR will be established. Finally, we have to look for the combination of quantitative research and its methodology, and the knowledge of history, politics and normative theory. Moreover, to cure theoretical dogmatism (such as the realist paradigm) we can use findings from scientific investigations.

For years, the relationship between supporters of the classical approach and those of the scientific approach can be described a “Cold War”. Lately, however, those two approaches have been combined and even integrated in several studies. For example, qualitative investigations strengthen quantitative findings to add more depth.

Chapter 2


Sartori, an Italian scientific scholar, has been criticizing qualitative research (case studies) in International Relations.

  1. Qualitative research, according to him, has an idiographic and not a nomothetic focus. Idiographic is an approach to knowledge that emphasizes individual cases, as nomothetic is its opposite: it focuses on the generalization and the establishment of laws.
  2. Another complaint on qualitative research has been that history occupies a more important position than theory.
  3. Classical scholars have been focusing too little on the logic of inference (the ability to derive logical conclusions of research) and on the ability to generalize. These are on the contrary methodological points that scientific scholars regard as very important. Hence, many proponents of the scientific approach regarded qualitative investigations as subjective.
  4. Moreover, qualitative research in its essence is nonreplicable (the research cannot be overproduced) and non-falsifiable (the hypothesis of a research cannot be tested).   

This has been the case in the 1950’s and 1960’s. Nowadays, more is known on the methodology of qualitative research. In addition, qualitative research now includes more than only case studies. Qualitative research and scientific research have been integrating (qualitative methods have been applied to empirical research), leading to more interest in the qualitative methodology. Currently, qualitative scholars are more nomothetic, using theories and generalization. Moreover, proper case selection has become more important.


There is no consensus on the definition of a case study yet; this still depends on the purposes of the investigation and on the topic. We can make a distinction between case studies (small-N investigations) and comparative studies (large-N investigations). There has been shift from an emphasis on history to an emphasis of theory. Nowadays, cases are regarded as an “instance” of a series of events, part of a broader thing, observed at a certain point in time (a unit, a phenomenon observed at a certain time). A set of theories is used to help the investigation of the case study and decide which variables are relevant. There are always many observations on the same variable.


There are different types of case studies. These mentioned below are ideal types, in most case studies we find combinations of certain types.

  1. Atheoretial or configurative-idiographic. This type focuses on a single case. Case studies of this kind are mostly just descriptive; they do not aim for a broader understanding or a broader theory. The aim is to make a holistic picture of a historical event, called “total history”.
  2. Interpretive or disciplined-configurative. This type is theoretically structured, but it still focuses on explaining a single case rather than developing a theory. This type too does not aim at establishing a general theory.
  3. Hypothesis-generating or heuristic. This type of case study is more nomothetic, since here a case is used to construct general theory. In addition, the choice for a certain case here is made by broader theoretical considerations and not by the value of the case specifically.
  4. Plausibility probes. This type can be used for both the construction and the testing of hypotheses. It functions as a quantitative investigation, but is more efficient.
  5. Deviant case studies. These search for events that do not fit the expected pattern of a theory. These are used for theory refinement and even theory alteration.
  6. Theory-confirming and theory- infirming (Lijphart). This type can be used to test theories and hypotheses. However, here, the problem is that the balance between the amount variables and the number of cases is not correct; there are too many variables. This difficulty can be overcome by increasing the number of cases or by decreasing the variables, or by using comparable cases with different variables.

The first two types of case studies are regarded as less valuable than the other four. The latter are more guided by theory, which then have explicit analyses, normative biases and causal propositions. This makes for fewer contradictions; moreover, these types of case studies can be verified empirically.

Case studies can thus generate and refine hypotheses. Often, a fierce investigation of a historical event is part of the case study. This examination of an event can add more variables and hence refine the hypothesis. Moreover, the examination contributes to the process of theory creation, since case studies help with the understanding of key variables and their legitimacy. For example, a political scientist uses a theory as a start to interpret a case on the one hand. On the other hand, the same case can then point out things in the theory that have to refined or even altered. On their turn, other cases are used to test these refinements. Hence, we can argue that case studies with a great theoretical structure can be helpful in refining existing hypotheses.


There are various strategies for case selection.  

  1. Comparable-cases strategy. This is almost similar to the method of difference and focuses on cases with different values on the dependent variable and similar values on all except for one of the independent variables. This makes for an easy identification of causal factors.
  2. The second strategy is based on the method of agreement, which emphasizes cases that are different on all except for one of the independent variables and are similar on the dependent variables. Here, all factors that vary across cases are left out.
  3. Most-different design looks for cases that differ on many explanatory variables and that do not differ on the dependent variable.
  4. Most-similar design is just the other way around; it finds cases that are similar on many explanatory variables but that differ on the dependent variable.
  5. The best strategy for case selection is often a combination of most-similar and most-different designs. Here, however, can it be hard to find cases that are truly comparable. Moreover, when not used in combination with a within-case method, this strategy can lead to the same outcome.
  6. Powerful are within-case comparisons. Here, for example, relationships are measured at different points in time within the same case. Such long-term cases look like most-similar systems designs because of the fact that within this type, there are many constant variables (as geography, for example). These constant variables do not change much over time. Some variables do change, and these are now easy to identify.
  7. Another powerful design is the combination of across-case and within-case comparisons.
  8. The strategy of random selection of cases, which can be useful in investigations with a large number of cases, often lead to biases in research with a small number of cases. Still, with nonrandom selection, we should take in account the danger of overrepresentation of cases with the key variable.
  9. Selecting on the independent variable makes for that cases cannot be compared. This can lead to the “dogs that didn’t bark”, which is problematic, except when studying deviant cases.
  10. Sometimes there is more than one necessary condition. This means that there are more causes, which can lead to the same outcome. Then we must examine the combinations of different sets of factors too.
  11. Another helpful strategy is crucial case studies, with most-likely and least-likely case research designs. A most-likely case is a case that perfectly fits a theory, and a least-likely case is just the other way around. When the latter case does have a theory that is valid, it is much more valuable than when the former case fits.
  12. The same goes for the existence of an alternative theory: when there is an alternative theory that actually fits the case better than the used theory, and when observations are consistent with the used theory, it will have stronger support.


Another way to within-case selection is process tracing (much data is used to develop a theory). Process tracing can show causes between conditions and outcomes. This is done by a big investigation. Here, many different points in the development of a historical event (the case) are used. Process tracing can be useful when studying causes and causal mechanisms. Moreover, process tracing can be used handy when testing nonlinear propositions or path-dependent historical processes. Here, case studies have some advantage over statistical methods since not only correlations but also causes can be found. However, statistics should not be underestimated.


Despite the fact that we can argue that case studies are essential for the understanding of historical events and that they are useful for the development of general theories, case studies also have some serious limitations.

  1. Often there are a large number of variables in relation with a small number of cases.
  2. In some instances statistics have a serious advantage over case studies, since sometimes a large number of cases is needed (relating to point 1). This is especially important for theories with a great “probabilistic” factor (cases with very likely outcomes).
  3. Case studies cannot measure the precise magnitude of a certain factor that influences the outcome as well as statistical investigations.


Because of the fact that quantitative research on their turn also has its limitations, it would be a good thing to combine it with the case study method. One way, for example, is the combination of case studies and statistical research. First, the case study is used for theory or hypothesis refinement and then statistical investigation can test these refinements. However, there is not yet many academic work on this combination. 


We should focus on building better theories. The better a theory, the smaller becomes the necessity for a rival theory. The better a theory’s predictions, the lesser observations are needed to test the theory. When reaching this point of lesser observations, the case study method can become very useful. When a bigger number of observations is needed (when theories are weaker), quantitative investigations as empirical tests or statistics have the advantage.

Chapter 3


Case study is the most flexible methodology used in International Relations. A profound discussion of this approach and its methodologies in IR, however, does not exist. Moreover, case studies are not commonly integrated with empirical research. This article consists of a general strategy for the use of case study investigation for confirmation.

Advantages of Case Studies

  1. Case studies are cost-effective for exploratory investigations
  2. Case studies have an advantage when dealing dynamic descriptive hypothesis (this is what we call process tracing). Moreover, case studies will uncover differences between the hypothesis and the outcome of the research faster and factors outside the theory are easier found.
  3. Case studies can detect nonevents or non-behavior (for example deterrence), and their characteristics.
  4. A variable can be correlated with another variable in several cases, but the process on how these variables are related can differ across these cases. Case studies are able to compare these processes and hence can show the unspecified features of a theory. This is done by case studies that are mixing exploratory and confirmatory outcomes of the same research.

Case selection

In practice:

  1. Hypotheses: the understanding of the theory and hypothesis is very important.
  2. Sufficient and a necessary condition: the necessary condition makes a hypothesis true, the sufficient condition ties this “truth” to a consequence.
  3. Variability: the greater the variability, the better.
  4. Confounding variables (variables that correlate with dependent and independent variables): identify those that affect the dependent variable but that are not mentioned explicitly in the theory.
  5. Population (the group from which the case is drawn): make a clear definition and mention those included or excluded. Moreover, determine how the population is divided on the (confounding) variables.

Compared to scientific research strategies there is not much documentation on case selection and other methodological considerations. On the one hand, a scientist chooses one case specifically to work with, but on the other hand, we often find generalizations derived from a single case.

An author should always know some basics on the characteristics of the population to which the case applies. This starts with an unbiased definition of the population, since a biased definition often leads to a biased choice for a certain case. Another problem can be caused by a too briefly formulated definition of the research question. This, too, can lead to a biased selection of a case.

To overcome these difficulties, the first thing to do is to acknowledge the fact that within case selection, there will always be the risk of a selection bias. Secondly, the context for a choice for a certain case should be clear. When considering the context, it is important to divide cases based on the existence of an independent variable rather than based on the dependent variable (this stems from the logic of experimental design). Moreover, when one wants to adopt a certain part of the population in its hypothesis, one should make sure that this is proportional to the existence of these variables in the population (systematic sampling). Hence, you cannot just randomly choose to include a certain case, first you have to check that the variables are also included in the population.

The difference between exploratory and confirmatory case studies

When starting with a case study, the basic details of the story are clear, as are the outcome and the background.  Exploratory case studies can be seen as the first stage of a murder investigation when there are many suspects, whereas confirmatory case studies are more like preparing the case for court when there is a main suspect.

Confirmatory case studies:

  1. Advantage: it can, compared to other forms of research, reveal easier why things are as they are. They show the actual facts and the difference between these and the expected process.
  2. A problem of design within confirmatory case studies is the backward reasoning process; we start at an outcome and then reason back with an explanation.
  3. The case design is often theory driven. Before starting the research, the elements from the theory have to be known.
  4. There are two types of elements: confirmatory and disconfirmatory (since refutation of a theory has generally more value). Hence, there must be evidence in favor of the theory and evidence against it.
  5. There can also be two competing views. Here, evidence on both sides has to be identified.

Exploratory case studies: there is mostly a general way, a general strategy on how to employ research. However, at a certain point, this general way has to be adapted to the specific case, and vice versa.

As for deterrence theory, it is hard to overcome the “the dog that didn’t bark” problem. Here, mostly only deterrence failures are studied, since these are easy to identify. However, also important is testing evidence for a theory against competing explanations (not only its failures, but also its successes).

Another problem with case studies in IR is the fact that often both explanations can be meaningful. More than one hypothesis can be valid. This can hinder the process of deriving

Cross case comparisons

The design for the comparison of cases is very important, since many scholars argue that case studies can be practical only when they are generalized.

  1. To do a proper cross case comparison, we need a “typical” structure; otherwise, generalization and inference are impossible.
  2. Necessary for good cross case comparisons is an underlying hypothesis that guides the collection of data and the actual research. However, often, different case studies are based on different hypotheses. Sometimes it can be even so that each different researcher goes on and on about their own methodologies. Here, too, we will find problems with generalization and inference.
  3. When only one single author does cross case comparisons, there is often a more clear structure and hence these are then more reliable.
  4. Replication (the preservation of some important qualities of the original design when one characteristic is changed) is inherently linked to cross case comparisons.
  5. When doing cross case comparisons, we must keep in mind that all cases consist of important differences and similarities. Important is to find out which elements fit which category.
  6. Transparency is the most important feature of comparative case studies.
  7. In the cross-case comparative design should the features that make for inferences and generalization be emphasized.
  8. There is never one theory that supports a case entirely.
  9. Confounding factors should be known, since if they return in different cases, there might exist variable bias. Then, theory refinement is necessary.


There are some general principles that we have to keep in mind when doing case studies.

  1. Transparency: everything must be clear to the reader. Hence, the underlying assumptions become clear. Moreover, the important process of replication is facilitated.
  2. Adequacy: not all methods are applicable to all cases, or there is more than method applicable to a certain problem. When the latter is the case, we should choose the most effective method.
  3. Experimentation is often seen as being the most perfect methodology. Even here, there are two things to keep in mind. First, replication is needed for experimentation to be useful. Second, improvement is needed for experimentation (and all other forms of quantitative research forms) to remain good.
Join World Supporter
Join World Supporter
Log in or create your free account

Why create an account?

  • Your WorldSupporter account gives you access to all functionalities of the platform
  • Once you are logged in, you can:
    • Save pages to your favorites
    • Give feedback or share contributions
    • participate in discussions
    • share your own contributions through the 7 WorldSupporter tools
Follow the author: Vintage Supporter

JoHo kan jouw hulp goed gebruiken! Check hier de diverse bijbanen die aansluiten bij je studie, je competenties verbeteren, je cv versterken en je een bijdrage laten leveren aan een mooiere wereld

Access level of this page
  • Public
  • WorldSupporters only
  • JoHo members
  • Private
Content categories
Comments, Compliments & Kudos

Add new contribution

This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Enter the characters shown in the image.