*These notes do not include the introduction to R*
HARKing: changing or creating the hypothesis after seeing the results.
P-hacking: researchers collect or select data or statistical analyses until nonsignificant results become significant.
Selective reporting: only reporting significant results, and not reporting the non-significant results.
Publication bias: only articles with significant effects are published.
So the problem is: the literature is not representative for the population.
What can we do to avoid these biases/to make ‘great results’ less important? --> Registered reports: when you use pre-registration it is on forehand clear what you’re going to do and what you’re interested in, in a manner that is verifiable by others. Four central aspects of the Registered Reports Model are:
- Researchers decide hypotheses, study procedures and main analyses before data collection
- Part of the peer review process takes place before studies are conducted
- Passing this stage of review virtually guarantees publication
- Original studies and high-value replications are welcome
Methods of pre-registration include:
- Sample (size)
- Design, variables
- Measures
- Exclusion criteria
- Analysis plan
How does it work? Authors submit the STAGE 1 manuscript, stage 1 peer review takes place and if the reviews are positive, the journal offers in-principle acceptance (IPA) regardless of the study outcome.
The advantages of Registered Reports:
--> For the scientific community:
- Rigorous review of theory and methods
- Eliminates publication bias and reporting bias
- Increases the reproducibility of science
--> For scientists
- Keeping track of what you did and why
- Peer review when it is most helpful
- Publication guaranteed regardless of the results
Common misconceptions:
1. ‘Pre-registration prevents the exploration of your data/creativity’. This is not the case. It allows for exploratory science, it simply prevents reporting exploratory analyses as confirmatory.
2. ‘Pre-registration does not allow for making changes and I cannot predict what will happen’. This is not correct. It allows for these things, as long as you report your deviation from your pre-registered plan.
Questions? Let me know in the contribution section!
Follow me for more summaries on statistics!
