Selection - Meltzoff & Cooper - 2008 - Article

The Critical Thinking Community defined the ability to think critically as follows:

Critical thinking is self-directed, self-disciplined, self-monitored, and self-corrective thinking. It presupposes assent to rigorous standards of excellence and mindful command of their use. It entails effective communication and problem-solving abilities, as well as commitment to overcome our native egocentrism and sociocentrism.

This text focuses on critical thinking in the context of evaluating scientific research, in the hopes of:

• exhorting you to adopt a critical mental set when reading research literature.

• Trying to increase your understanding of the principles and methods of research that are used to produce evidence.

• Alerting you to booby traps that can compromise the research and cloud the evidence.

• Providing you with materials that can be used for practice, with the goal of improving and honing your critical reading skills when you are exposed to research in the sciences.

Psychologists often spend their time in one of the following ways or in combination:

• Generating knowledge by doing research.

• By teaching of by directing the research of others, transmitting knowledge to others.

• Applying psychological knowledge in clinical of consulting settings.

Therefore, psychologists always read research literature. They should be able to evaluate research critically. Invaluable in this pursuit are:

• Having a good grasp of the scientific method.

• Having a good grasp of the principles of research design.

• Knowing what to look for.

What does critical reading require?

The way a scientific study does or does not meet the scientific standards for evidence and proof is adressed in any critique of such a study.

What is involved in a mental set for critical reading is a kind of general open skepticism. When reading scientific documents, every thought that is written, every conclusion that is stated, must be screened through your internal skeptical scanner.

In the relatively recent past there has been an explosion in the amount of scientific research. To accommodate this, researchers have developed a set of rules for accumulating evidence on the same hypothesis in a complete, transparent, and unbiased manner: systematic review, research synthesis or meta-analysis.

What is critical reading?

The critical reader is interactive, actively anticipating what is to come. In scientific reading, this anticipation involves the use of rules for conducting valid research. The critical reader then discovers whether the expectancies have een met. The following include the kinds of expectancies that flow naturally from given antecedents:

• The research question guides the review of previous literature that sets the context for the scientific report.

• The literature review and the statement of the problem relate clearly to the hypotheses.

• The hypotheses set up research design expectancies and suggest what variables should be manipulated, measured and/or controlled.

• The hypotheses, appropriate design, and type of data dictate the method of data analysis.

• The analysis of the data influences the kinds of conclusions, inferences and generalizations that can be made.

The critical reader is constantly engaged, checking his or her evolving expectancies against the actual way the study was carried out.

Good ‘critical’ interactive reading does not require the uncovering of crucial flaws. When you critically read a flawed study, you cannot accept any of it as true.

Key characteristics for critical reading according to the authors of the book are:

• Actively anticipate what you expect to read.

• Make use of explicit rules to judge whether your expectations are met.

• Constantly monitor whether your expectations are being met.

• When you make an evaluative judgement about the strengths and limitations of an article, have clear and justifiable reasons.

Knowledge of research design is necessary to know what to anticipate and to expect. People who lack specialized knowledge can always hire an expert to advise them.

What is peer review?

In some articles in some journals, the articles have been prescreened by refereers and an editor. Scientist call this peer review. The article describes this as ‘a formal system whereby a piece of acadamic work is scrutinized by people who were not involved in its creation but are considered knowledgeable about the subject’. It is meant to ensure the quality of reports that appear in the scientific record.

Your first obligation as a critical reader is to discover whether the journal conducts peer review, because there are many journals who do not do this. Among journals there is also a difference in the strictness of their peer review criteria before publishing an article. No article that succeeds in being published is accompanied by a guarantee of excellence. Flaws and weaknesses appear in all published articles. The best-established scientific conclusions occur when evidence accumulates, when multiple studies, none perfect, lead to the same answer.

A research article consists of three components:

• The context and purpose of the research, statement of the problem, and formulation of hypothesis.

• The reserach itself.

• The interpretation and discussion of the results.

A reader who knows much about a topic, but little about research design principles is not in a good position to evaluate a research article critically. A reader who knows little about the topic, but much about research design, can nonetheless evaluate the design and execution of the research. The principles of research design transcend content areas. The effectiveness of a reader is therefore dependent on knowledge of research design and on skillful application of the knowledge.