by Daniel Balliet, Giuliana Spadaro, Benny Markovitch, & Wouter Beek
The Cooperation Databank (CoDa) includes an annotated history of empirical studies on human cooperation in social dilemmas. Social dilemmas are situations that involve a conflict of interests and people must choose between a behavior that is best for themselves and a behavior that is best for the collective. Cooperation is choosing to do a behavior that is best for the collective. CoDa contains six decades of studies on individual decision making in dyads and groups facing social dilemmas.
The studies have been annotated by domain experts according to differences in the samples, study characteristics, and for the variables used to predict cooperation within the studies. Here we report an overview of the history of research on cooperation and how studies vary according to each of these variables. Effect sizes have also been documented on the relation between different variables predicting cooperation. When using the CoDa platform to search the Databank and conduct meta-analyses, you can use these variables to search and select studies and to predict variance in effect sizes.
One benefit of CoDa is that we can write living reviews on the history of research on cooperation, which are directly connected to the data, become updated when new studies are added to the Databank, and that allow people to interact with the data. Our first living review (i.e., data story) using CoDa describes the data in the Databank. In this data story we address:
- How many papers, studies, effects and treatments are annotated in the Databank?
- How did the field of cooperation research grow over time?
- How do studies vary according to sample and study characteristics?
- How do effect sizes and standard errors vary across studies?
- Which variables are used to predict cooperation within the studies?
- Learn how CoDa can be used to study trends in research over time.
- Learn about the meta-study data in CoDa, such as publication status, prolific authors, and country.
Each of these topics provide interactive figures that allow you to efficiently acquire information for own your research, casually browse the information in the Databank, and explore for yourself the trends in research on cooperation.
In this data story we do not define all of the variables used in cooperation research. You can learn about the definitions and values of these variables at the CoDa website and using the ontology explorer in the CoDa platform.
How many papers, studies, effects and treatments are annotated in the Databank?
CoDa contains academic papers about human cooperation in social dilemmas. Each paper reports on one or more studies. A study reports one or more effect sizes. These effects can be correlations (as a single treatment) or involve a comparison of two treatments. A treatment occurs when observations of a phenomenon occur in different contexts, such as multiple levels of a manipulated independent variable.
Figure 1 gives a quantitative overview of CoDa. It shows the main types of data that are stored in the Databank, together with the number of instances for each type.
Figure 1 ― High-level overview of the main types of data that appear in CoDa.
How did the field of cooperation research grow over time?
Figure 2 shows the number of papers, studies, effects, and treatments for each year for which the Databank contains information. Notice the spike in the 1970s with respect to papers in the Databank. After 1977, the number of papers, studies, effects, and treatments drops to below the values before 1963 only to be recovered in the 1990s.
Below, you can choose whether the figure displays the number of studies, effects, and treatments.
Figure 2 ― Papers, studies, effects, and treatments plotted by year the study was conducted.
How do studies vary according to sample and study characteristics?
All studies in CoDa have studied cooperation using social dilemma paradigms. That said, the studies still contain heterogeneity in the implementation of the paradigms and the samples. For example, CoDa documents a wide range of sample characteristics. These are features of the samples that can vary across studies, such as the percent of males, mean age, whether the study had a student sample, and the academic discipline of the students. Studies can also vary according to how the social dilemma paradigm was implemented, such as group size, iterations, and the degree of conflicting interests. When using the CoDa platform, you can select studies based on these characteristics and you can use these characteristics as moderators to predict variation in effect sizes.
Interacting with the figure below, you can learn about the variation that exists in studies on cooperation. You can also learn about typical practices in this area of research. For example, most studies on human cooperation (a) do not use deception, (b) involve monetary incentives, (c) have student samples, and (d) the sample have on average 49 percent male participants.
Below, you can select the specific sample or study characteristic that you are interested in learning more about.
Figure 3a displays the definition of the selected characteristic, including information about the properties and values.
Figure 3a ― An overview of information about a selected sample or study characteristic for studies included in CoDa.
Sample and study characteristics have either cardinal or nominal values:
-
Nominal characteristics have a set of predefined labels as their possible values. Examples include the game type (e.g., public goods dilemma or resource dilemma), whether or not the study was published, and the country in which a study was conducted. All nominal study characteristics can be visualized in Figure 3b by selecting a specific characteristic from the dropdown list.
-
Cardinal characteristics quantify some aspect of the study's sample or social dilemma paradigm. Examples include the mean age of the participants, the year in which the study was conducted, and the degree of conflicting interests (k value). All cardinal study characteristics can be visualized in Figure 3c by selecting a specific characteristic from the dropdown list.
Figure 3b ― Displays the number of studies, effects and treatments for each value for the selected nominal study characteristic.
Figure 3c ― Displays the number of studies for each value of the selected cardinal characteristic. Uses Outlier percentile of 0.25.
How do effect sizes and standard errors vary across studies?
Studies in CoDa include effect sizes. Effect sizes describe the strength and direction of a relation between two variables (e.g., an independent variable and a dependent variable). Cooperation is always the dependent variable for all of the effect sizes in CoDa. You can see the kinds of independent variables that were annotated in Figure 5. CoDa contains two kinds of standardized effect sizes; (1) standardized mean differences (Cohen's d), and (2) the correlation coefficient (r). Below, we provide information about the effect sizes in CoDa, including their corresponding quantitative information (e.g., standard errors).
Figure 4a displays for each type of effect size the number of effects, the range of values, and the average (unweighted) effect size. You can then select to observe additional quantitative information associated with the effect sizes.
Importantly, for the data presented below, the majority of effect sizes have a randomly determined direction (e.g., a random comparison of treatment 1 versus treatment 2), but when using the CoDa platform you can specify the direction of the effect size.
Figure 4a ― Displays the quantitative information about effect sizes, according to both the standardized mean differences (d) and correlation coefficient (r)
In the figures below, you can select to view information about the effect sizes and their quantitative information.
All nominal effect size information can be visualized in Figure 4b. For example, the default display option displays what information was used to calculate effect sizes. As you can see, the vast majority of effect sizes were calculated based on either means or proportions.
All cardinal effect size information can be visualized in Figure 4c. For example, you can select to display the distribution of standard errors of the effect sizes or the sample sizes used to calculate the effect sizes. In this figure you can specify whether you want information to be displayed about either the r values or d values.
Figure 4b ― Displays the number of effect sizes for the type of effect and how the effects were computed using an algorithm.
Figure 4c ― Displays the number of effect sizes for each value of the selected cardinal characteristic. Uses Outlier percentile of 0.25.
Which variables are used to predict cooperation within the studies?
CoDa has an ontology representing the variables that have been measured or manipulated and then used to predict cooperation within studies. Each study has been annotated as for reporting these variables and their relation with cooperation. Below is an overview of the variables that have been annotated across studies. The size of the cell corresponds to the relative number of treatments that have been coded for variable. For example, individual differences measures, such as personality variables, have been some of the most frequently studied variables. Punishment is also studied more frequently than reward. You can navigate Figure 5 to learn about the many different topics that have been studied in cooperation research. You can also go to the CoDa platform and use the ontology explorer to also learn more about these variables, including their definition, possible values, and inter-rater agreement.
Figure 5 ― Hierarchy of the independent variables (IV) that appear in the CoDa Databank.
CoDa can be used to study trends in research over time.
Researchers have devoted more or less attention to different topics (e.g., independent variables) over time in the study of human cooperation. The following figures give you a sense for how you can use CoDa to identify trends in the study of cooperation.
In Figure 6, you can select to display up to 3 of 14 different personality variables according to the number of studies that reported a correlation between that variable and cooperation, per year. For example, you can see that trust propensity has been studied for a longer period than social value orientation, both of which have been studied for a longer period than self-control. You can also use the figure to compare the relative number of studies that measured each variable. Therefore, CoDa can be used to identify relatively more popular topics in the study of cooperation and to compare histories of different topics.
Figure 6 ― Overview of the number of studies per year for the most researched individual differences variables in CoDa.
CoDa can also be used to compare the effect sizes of different topics and to study how these effect sizes may have changed over time. This can be achieved in the CoDa platform, but here we provide a few examples to give you a sense of the value of this perspective on a literature. Figures 7a to 7c display the effect sizes for different measured variables predicting cooperation. In these figures, you can plot these effect sizes over time. Each figure displays a slope that depicts the association between the effect size and the year in which the study was conducted.
Figure 7a allows you to display effect sizes for up to 4 different personality variables.
Figure 7b allows you to display effect sizes for up to 4 different Schwartz values variables.
Figure 7c allows you to display effect sizes for up to 5 different variables related to partner perceptions, including expectations of partner behavior, in the social dilemma.
For each of these figures, the correlation represents the direction and magnitude between the selected variable and cooperation. For example, in Figure 7a social value orientation has mostly positive correlations, these correlations are larger and more diverse than the correlations for trust propensity, and the correlations for each of these variables has not really changed much over time.
Figure 7a ― Effect sizes (r) for up to 4 individual differences variables, plotted according to their effect size and the year when the study was conducted
Figure 7b ― Effect sizes (r) for up to 4 of the Schwartz values in relation to cooperation, plotted according to the effect size and the year the study was conducted.
Figure 7c ― Effect sizes (r) for participants' expectations of partner behavior, including some partner perception variables across the years.
CoDa contains meta-study data
Each study is associated with a publication status (e.g., published article versus doctoral dissertation), and if published, may have an associated DOI. Each study was reported by any number of Authors and has been conducted within a specific country. Below, we offer a several figures the describe the number of papers, studies, effects, and treatments per publications status, DOI status, top prolific authors, and the country where the study was conducted. You can quickly see that most studies were published, have an associated DOI, and were conducted in the United States.
Figure 8a ― Displays the number of papers, studies, effects and treatments according to publication status.
Figure 8b ― Displays the number of papers, studies, effects and treatments associated with a DOI.
Figure 8c ― The most prolific authors in CoDa (including the range of years their studies were conducted).
Figure 8d ― The overall number of papers, studies, and effects in CoDa reported for each country where the studies were conducted.
That concludes our first living review of research on human cooperation using social dilemmas. The data story provides an overview of the data included in CoDa, and in the process the data story delivers insights into the history of research on this topic. The data story will be automatically updated with the entry of new studies to CoDa, and therefore will keep pace with a growing literature. The data story also provided an opportunity to learn about how the CoDa platform can be used to compare different topics of research on cooperation. For a more in-depth analysis about how research on cooperation has changed over time, you can visit the cooperation databank website (https://cooperationdatabank.org/) to view our second data story on that topic.
How to cite this as an online report:
Balliet, D., Spadaro, G., Markovitch, B., & Beek, W. (2020, date). What's in the databank? Cooperation Databank. https://cooperationdatabank.org/data-stories/what-is-contained-in-the-databank/
