Adjerdid & Kelly (2018). Big data in psychology: A framework research advancement. - Article summary

Most contemporary research uses a sample size as small as reasonable to address the specific question of interest. However, the samples are often too small and not always representative of the population of interest. These issues, as well as questionable research practices, contribute to the replication crisis.

Cattell’s databox states that methods can be classified based on the structure of the data. Data can be organized along the dimensions persons (1), variables (2) and occasions (3). Big data will increase all of these dimensions.

The theory of data states that in search of insight, formal statistical methods leverage observations that are selected from a universe of potential observations and parsed into usable information for use in statistical methods.

Big data consists of three parts:

1. Big ‘n’
   Big data allows access to a lot of participants for data on real-world behaviour. These larger samples allow for the detection of specific effects with a high degree of precision and allows estimation of complex statistical models. It also has the potential to facilitate research samples that are more representative than current samples.
2. Big ‘v’
   Big data allows access to a plethora of variables. This allows for many more measures of individual behaviour. However, these variables are captured in a less structured environment than regular research practices.
3. Big ‘t’
   Big data allows access to a lot of participants observed over a lot of periods of time. This allows for easier longitudinal research.

There are several challenges of big data:

1. Getting access
   This is a challenge of big data as the companies that collect data are not always eager to share the data. However, a partnership with the company is often only necessary when changes need to be made to the data (e.g. randomized treatment). It is also possible to purchase the data from a third-party or recruit people directly at a social media website. The use of crowd-source samples is also possible. Access to a lot of variables can be done in the same way or by using questionnaires such as Qualtrics with custom scripts. The costs of obtaining a lot of data is usually not very high.
2. Technical challenges
   The technical challenges emerge because data needs to be formatted in a way to fit research. Technical expertise is required to administer, manage and clean the data. It is also possible to make use of existing software packages.
3. Making sense of big data
   Translating the data collected by websites, which is collected without research in mind, needs to be translated into constructs and outcomes of interest to psychology. One way of doing this is measure the construct of interest for several people on the website and then translate the raw data of these people into models that can fit all the other data.
4. Statistical challenges
   Traditional statistical techniques need to be scrutinized when the data sets are sufficiently large (i.e. big ‘n’) because results will almost always be statistically significant. Effect sizes need to be taken into account. Big data may allow for the development of new statistical techniques but simultaneously, well-known statistical issues are exacerbated.
5. Theoretical and research value
   Big data research needs to be theory-driven and cannot always be a-theoretical. It has the potential to rigorously test and validate psychological theories.

A lot of big data research’s effect sizes are significant. However, this does not imply that the effects that are found have any practical significance.