Dienes (2011). Bayesian versus orthodox statistics: Which side are you on?” – Article summary

Probabilities are long-run relative frequencies for the collective, rather than an individual. Probabilities do not apply to theories, as individual theories are not collectives. Therefore, the null hypothesis cannot be assigned a probability. A p-value does not indicate the probability of the null hypothesis being true.

Power or a p-value is not necessary in Bayesian statistics, as a degree of plausibility can be assigned to theories and the data tells us how to adjust these plausibilities. It is only needed to determine a factor by which we should change the probability of different theories given the data.

The probability of a hypothesis being true is the prior probability (P(H)). The probability of a hypothesis given the data is the posterior probability (P(H|D)). The probability of obtaining the exact data given the hypothesis is the likelihood (P(D|H)). Therefore, the posterior probability is the likelihood times the prior probability.

The likelihood principle states that all information relevant to inference contained in data is provided by the likelihood. In a distribution, the p-value is the area under the curve at a certain point. The likelihood is the height of the distribution at a certain point.

The p-value is influenced by the stopping rule (1), whether or not the test is post-hoc (2) and how many other tests have been conducted (3). These things do not influence the likelihood.

The Bayes factor is the ratio of the likelihoods. The Bayes factor is driven to 0 if the null hypothesis is true, whereas the p-values fluctuate randomly if the null hypothesis is true and data-collection continues. The Bayes factor is slowly driven towards the ‘truth’. Therefore, the Bayes factor gives a notion of sensitivity. It distinguishes evidence that there is no relevant effect from no evidence of a relevant effect. It can be used to determine the practical significance of an effect.

Adjusting conclusions according to when the hypothesis was thought of would introduce irrelevancies in inference and therefore, the timing of the hypothesis is irrelevant in Bayesian statistics. In assessing evidence for or against a theory, all relevant evidence should be taken into account and the evidence should not be cherry picked.

Rationality refers to having sufficient justification for one’s beliefs. Critical rationalism is a matter of having one’s beliefs subjected to critical scrutiny. Irrational beliefs are beliefs not subjected to sufficient criticism.

It is possible to have a uniform (1), normal (2) and half-normal (3) distribution. In a uniform distribution, all values are equally likely. In a normal distribution, one value is most likely given the theory and a half-normal distribution is a normal distribution centred on zero with only one tail. It predicts a theory into one direction but smaller effects are more likely than larger effects.

There are several weaknesses of the Bayesian approach:

1. Bayesian analyses force people to specify predictions in detail
2. Bayesian analyses do not control for Type I and Type II errors
3. Bayesian analyses use arbitrary and subjective prior analyses

There are several rules of thumb to determine what a theory predicts:

1. If positive values are seen as more likely (for example because of determinable limits), a uniform distribution can be used for the positive range.
2. If small positive values are seen as more likely, then a half-normal distribution for positive values can be used.
3. When in doubt about what a theory predicts, it is wise to spread out previous results. This means rounding a previous finding up and using that as a mean and using half of the mean as the standard deviation to use as a prior.
4. If it is thought that smaller values are more likely than large values, a mode of zero can be used with a standard deviation of a half.
5. The standardized effect size (Cohen’s d) can be used as an estimate for the relevant standard deviation if there is no past research with the same dependent variable.