Stage 2 Registered Report Personal factors and group creative outcomes: A correlational meta-analysis

Open Science Disclosures

We shared all procedures, materials, datasets, and analysis code on the Open Science Framework (https://osf.io/xwph9/). The pre-registration and additional information about decisions are available in the supplementary materials of the extended data. We deviated from Stage 1 with the following: We did not use Scopus because we were not able to access it. We modified keywords during the literature search to have more results, as our first search was too restrictive. Finally, we have not conducted all publication bias tests mentioned, and the cumulative test because of the lack of studies to conduct them.

Design

Personal factors are explained in Table 1. They consist of personality traits: 1) openness, 2) conscientiousness, 3) extraversion, 4) agreeableness, 5) neuroticism; emotion: 1) social anxiety and 2) emotional intelligence; cognition: 1) Cognitive style, 2) epistemic motivation, 3) self-efficacy, 4) need for closure, 5) Need for Cognition. The three categories of creative outcomes are 1) number of ideas generated, 2) originality of these ideas and 3) usefulness of the ideas.

Eligibility criteria

Studies including personal factors (see Table 1) and measuring creative outcomes in group settings are included in our analysis.

Search strategy

Database searches. To identify articles that are potentially relevant to our topic of investigation, we conducted searches using Google Scholar, Psychinfo, Web of Science - social science citation index, Proquest- dissertations and theses (for suitability, see Gehanno et al., 2013; Martin-Martin et al., 2019; Moreau & Gamble, 2020).

For personal factors, the keywords were personality trait*, openness, extraversion, introversion, conscientiousness, agreeability, neuroticism, anxiety, social anxiety, thinking style, convergent thinking, divergent thinking, Need for Closure, creative self-efficacy, epistemic motivation, Need for Cognition, emotional intelligence. For creative outcomes, the keyword was simply “creativity.” Initially, we had planned to use more specific search terms (such as “number of ideas”); however, this led to very few results which prompted us to switch to a broader search strategy. All search patterns included the following operators: “group*” OR “team” AND “correlation”.

During the search, keywords related to constructs were linked with the Boolean logic operators “OR” and keywords between construct 1 and construct 2 with “AND”. Variations of the keywords were included in the search with the original keywords if search results yielded fewer than 100 results, linked with “OR”. (e.g., “personality trait*” AND “useful*”). More information on the search pattern process can be found in the coding sheet under the tab “search pattern.” Database searches for each search pattern were terminated after combing through 30 records consecutively without potentially relevant papers for the inclusion criteria.

The search included papers listed under the “related articles” and “cited by” features in Google Scholar to identify papers that are similar or have cited the identified articles. We looked at other articles that were published by identified authors in the field to check if there were relevant papers that we may have missed. We systematically contacted the authors of the identified articles (see the pre-registered email template in the supplement) and issued a call for unpublished findings on ResearchGate and Twitter to find relevant unpublished data. For all the articles, titles, abstracts, tables, and methods sections were scanned to identify the relevance of a source.

Inclusion and exclusion criteria

Correlational meta-analyses typically exclude studies that manipulated the target variable prior to measurement (e.g., Chevance et al., 2019), or alternatively conduct a separate meta-analysis for studies with manipulations or interventions (e.g., Schmitt et al., 2014; van Kleeck et al., 2010). First, we decided to restrict our meta-analysis to correlational studies that measure personal factors in creative contexts. Studies were excluded if they 1) experimentally manipulated an independent variable related to personal factors (e.g., manipulated motivation, social climate and anxiety, information and Need for Closure, etc.), 2) failed to report the crucial statistics necessary for a meta-analysis (i.e., correlation coefficient or other effect sizes that can be transformed into correlation coefficient, sample size), or 3) were not written in English or French unless all necessary information was provided in English or could be obtained from the authors.

Screening

Studies that met our criteria were coded into the “Searched articles” tab within the coding sheet. Articles were scanned to determine whether they should be included into the main coding sheet or not. Reasons for exclusion were documented. Authors of studies with missing statistics were contacted for relevant datasets/information through the “contacting author” tab and the corresponding mail template (see supplementary material). If the dataset was provided, we included the article in the main coding tab. Finally, the PRISMA diagram in Figure 1 and the included studies in Table 2 summarize how and which studies were included.

Figure 1. Meta-analysis flow diagram in accordance with PRISMA 2020 (Page et al., 2020).

Coding

Extracted data from the included studies were recorded in the “main coding sheet” tab. When available, the main correlations between personal factors and creative outcomes were recorded, along with intercorrelations between creative outcomes, the type of scale used, sample demographics, and publication information. Moderator variables were coded in a separate column in the main coding sheet.

Analysis

We developed a Rmarkdown Script for the statistical analyses. The packages used are indicated in the supplementary material. Our main package for meta-analysis is Psychmeta (Dahlke & Wiernik, 2019). We used Pearson’s correlation coefficient r as the main indicator of effect size. Whenever available, we used correlations obtained directly from authors of original papers. If only regression results were available, we converted to correlation by using a transformation provided in the supplement. If not possible, we asked the authors to provide either the correlation or raw data. Correlations were corrected for attenuation by using the formula (Dahlke & Wiernik, 2020):

Given the range of different outcomes for each category, we expected the heterogeneity in the sample to be relatively high. Thus, a random effect model was used for all relationships. Split conditions due to moderators were collapsed to allow for comparison of the relationships. All conversions and coding decisions were documented. A meaningful association is expected as having a correlation of at least r = .10 (Cohen, 1988; Gignac & Szodorai, 2016; Schäfer & Schwarz, 2019). We documented all conversions and coding decisions. We included the original quotes and/or table/page numbers from the original articles into the coding sheet to facilitate reproducibility. We plotted forest plots presenting the correlation for every relationship. We presented the relation with confidence intervals and the sample size of each study.

Statistical heterogeneity between studies was determined using an 80% credibility interval (as used in Borenstein et al., 2009; Wiernik et al., 2017). Wiernik and Kostal (2019) explained how the credibility interval performed better than the most used Q significance test. The main reasons are that the Q test is underpowered in most situations and that it confounds the sample size of studies and the magnitude of effects found in the studies. Finally, Paterson et al. (2016) suggested thresholds for credibility intervals as an indication for moderation (e.g., corrected correlations < .15 as negligible, .15–.24 as small, .25–.39 as moderate, and ≥ .40 as large). If there was indeed meaningful heterogeneity, we explored potential moderators. Our design for the analysis is shown in Table 3.

Exploratory analyses

Initially, we had planned to include additional possible moderators that emerged during the literature search. Due to the low number of studies, such additional exploratory analyses were not possible.

Publication bias

To address possible biases, we corrected for sampling error and measurement error, as indicated in the guidelines for psychometric meta-analyses (Schmidt & Hunter, 2015). Reliability was corrected using internal consistency artifact distributions (alpha or composite reliability) compiled from studies included in the present meta-analysis. A summary of weighted mean internal consistency can be found in the supplementary materials. We had originally planned to conduct a sensitivity analysis (Mathur & VanderWeele, 2020) with the use of cumulative meta-analysis but could not do so due to the lack of studies.

Finally, we conducted a PET-PEESE analysis (Stanley & Doucouliagos, 2014) and a p-curve analysis (Simonsohn et al., 2014).

Moderator analyses

Due to the extremely small number of effect sizes, it was not possible to analyze all moderators. Table 5 summarizes the moderator data.

In particular, only one moderator is significant, the effect of time (β = 0.01, p = .01). However, the β estimator is small and indicates that each minute spent in the creativity session increases the relationship between personal factors and creative outcomes by r = .01. Additionally, the residual heterogeneity test is significant, indicating that other uncaptured moderators are at work (Q = 382.19, p < .001).

Inter-correlations

The inter-correlations between the creative outcomes provide no evidence for a relationship between the number of ideas and originality (7 studies, ρ = -.01, [-.30, .29]), a positive relationship between the number of ideas and usefulness (2 studies, ρ = .67, [.670, .674]), and originality and usefulness (2 studies, ρ = .47 [.10, .85]). For the personality traits, 4 studies found a weak relationship between openness and extraversion ρ = .18, [.02, .33] (see Table 6).

Power analysis

We created a sunset plot in Figure 2 to display the statistical power of studies included in the meta-analysis. The average power is 13.5% and the replicability index 0% which means that we have no chance to reject H0 when there is a true effect, and no chance at all to replicate one study given the median average power of studies included in the meta-analysis and success rates of these studies (see Motyl et al., 2017 for R-index).

Figure 2. Power test of all studies.

Reliability

Table 7 presents the mean reliability scale of all variables included. The reliability was high across all variables, between.75 and.91.

Publication bias

The PET-PEESE analysis was not significant (PET: t = 1.95, p = .06; PEESE: t = 1.85, p = .07). In particular, the average correlation found by the PET test is r = -.17 [-.40, .06] and by the PEESE test r = -.06 [-.18, .06] which indicates an effect size close to 0 with a tendency to be rather negative.

The p-curve test indicated no signs of unpublished non-significant results (see Figure 3). It also indicated a statistical power of 93% [0.88, 0.97]. However, in the case where there are few effect sizes and strong heterogeneity, the p-curve test is not reliable (van Aert et al., 2016).

Figure 3. P-curve analysis.

Regarding the cumulative analysis, it was not possible to perform it because of the too small number of studies included in the analysis (< 20).

Personal factors related to creativity in group setting

This meta-analysis investigated the links between personal factors and group creativity. The results are limited, given the very few studies and thus the very few effect sizes we could include. We found support for only one relationship, positive, between self-efficacy and all three creativity outcomes: the number, originality, and utility of ideas.

These results, although disappointing, are not so surprising. In 2018, Coursey and colleagues wrote (p. 2-3): “Whereas many studies have examined the role of individual differences in individual creativity, relatively few have examined the role of individual differences in group and team creativity.” A little further on, they concluded (Coursey et al., 2018, p. 26): “We have provided an overview of how individual difference factors may influence group creativity. Much of this is based on the literature concerning individual differences in individual creativity. It is presumed that there will be some degree of similarity between the effects observed for individuals and groups.”

Our systematic review confirms that the majority of studies are based on individual creativity, as we found only 11 studies to include in our meta-analysis. Focusing on the creativity side, out of the 758 articles selected—with the criterion requiring the exact term “group” or the exact term “team” to be mentioned—160 studies concerned individual creativity, creative personality, perception of being creative, creative potential, or creative intelligence. Furthermore, when selecting the most used indicators in the creativity literature (number, originality, and usefulness of ideas), we found that the majority of studies did not measure group creativity with these indicators.

Seventy-two studies measured perceived creativity, whether by other team members or by the manager. In most of these studies, creativity was measured by Zhou and George’s (2001) 13-item scale, an example of which is “the team/team member suggests new ways to achieve goals or objectives” or “the team/team member often has new or innovative ideas”. The scale has not been psychometrically evaluated and there seems to be no evidence of a link between the scores on this scale and the actual creativity of individuals. We have not found any studies that measured both perceived creativity with this scale and group creative outcomes. In addition to this measure, about 20 studies measured ‘general team performance’, asking a question about creativity in this performance, or about creative decision-making in a team, but then measured only the choice made by the team. In contrast, direct measures of creativity were much rarer. This could be because such measures require experts in the creative area who can code ideas according to the indicators used (numbers, originality, and usefulness of ideas). Finally, despite the selection criteria, 181 results were not about creativity or did not measure it, with the word “creativity” being mentioned in the introduction or discussion of the study.

Furthermore, there was a small number of studies that manipulated individual factors, for example by measuring personality traits prior to experimentation, and by creating groups by separating individuals according to their level (low or high) on a personality trait. Others measured, for example, optimism, individualism, temperament, depression, envy, critical thinking, or self-constructs. In total, these exclusion criteria applied to 136 studies. It would be possible to conduct an experimental meta-analysis on the subject, which would not measure the relationship between individual factors and group creativity but the impact of a particular personality trait on group creativity. However, the results could be as disparate and inconclusive as in this meta-analysis, as other exclusion criteria would apply. Indeed, many of these studies also included only measures of perceived creativity, measures of individual creativity, and some were qualitative in nature. Thus, it is not clear that a meta-analysis of this type would provide enough effect sizes to conduct an informative analysis.

Out of the 11 studies included, 1 is from 2019, and 3 are from 2020, a sign that personality factors and group creativity are currently being investigated. In this sense, it is possible that this meta-analysis came a bit too early compared to the primary studies on the subject.

Time limit

The moderator analysis revealed a possible weak effect of time on the relationship between personal factors and creativity outcomes, as each minute spent in the creativity session increases the relationship between personal factors and creative outcomes by r = .01. However, given the limitation of the data, we could not test the hypothesis on cognitive closure. Most of the studies included in the analysis had a time constraint between 5 and 10 minutes, and for these studies, the moderation analysis revealed no effect. The effect of time limit was found for the studies comparing 10 and 20 minutes; the relationships were systematically stronger for the 20 minutes than for the 10 minutes condition. This finding suggests that a longer session leads to a stronger possibility for participants to express their individuality in group creative tasks, at least up to 20 minutes. We are limited in the generalizability of our findings, as the relationships were only found for 5 associations relating self-efficacy to originality/usefulness.

Future research directions

The results of this meta-analysis may help to improve the field of research linking creativity and personality.

First, we built on the conjecture of Coursey et al. (2018) and improved current knowledge by systematically quantifying the data on the subject. We found that the available data was limited. Very few studies measure group creativity. Most studies measure perceived creativity, through questionnaires and surveys, and some studies make up their own measures of creativity, according to their own criteria. It would be possible to improve our knowledge of group creativity by using standardized creativity methods and protocols and by using direct measures, especially as in the majority of the studies included, inter-expert reliability was high, indicating that the perception of creativity is shared among experts.

Only one personal factor had a positive relationship with the three group creative outcomes: self-efficacy. This is not surprising, as self-efficacy is one of the most widely used individual factors in psychology. For example, the book Self-efficacy: The exercise of control (Bandura, 1997) is cited 108 314 times on Google scholar (December 2022). The other social, cognitive, and emotional factors seem to be less studied or not studied in correlational studies. Surprisingly, many studies reported some personality traits, but not all. In the same way that some researchers use their own way of measuring creativity, others only partially measure personality (sometimes measuring types and not traits, or a single trait). This treatment of personal factors limits the possibilities of aggregating data in meta-analysis. The field could be improved by using standardized personality questionnaires and reporting all data points necessary.

Several studies measured the relationship between individual factors and group creativity, but as they did not report information essential for a meta-analysis, we could not include them. Of the 11 studies included, we requested the information for 6 of them. 8 additional studies could not be included because we were unable to contact the authors. Thus, we could have nearly doubled the number of studies included if information had been made available. The field of the research could be improved by systematically sharing the raw data and the analysis conducted. In addition to the benefit of the aggregation into a meta-analysis, this would allow for the conduct of individual participant data (IPD) meta-analyses (van Aert, 2022).

A traditional meta-analysis is conducted at the group level - in other words, by asking the question “How do average group personal factors correlate with average group creativity?” An IPD meta-analysis can answer a more precise question: “How does an individual’s personality trait in this group correlate with his or her creativity in the group?” This question cannot be answered without data at the individual level.

The power of the studies was very low, around 13.5%. The number of participants is quite high (see Table 2), but the correlations seem to be extremely low, which requires an extremely high number of participants to indicate an effect if it exists. Again, reporting the raw data at the individual level would increase the precision of the estimates, improving the power of the study without the need to dramatically increase the number of participants.

The studies included were conducted in the USA, UK, China, Germany, Singapore, the Netherlands, and Romania. More importantly, many of the studies were performed on students doing group projects. It would be preferable to conduct studies in other contexts, including non-WEIRD, non-student samples. The many moderators that we tried to highlight were not investigated enough or were not investigated at all. We recommend that researchers methodically record all the data relating to the context of the creative activity, in particular the time allocated to participants, the type of task, the number of participants per group, and the individual characteristics of the participants (demographics, type of recruitment, randomization of the participants in groups or not, etc.).

Interpretations and implications

One should be cautious in drawing a relationship between perceived creativity and actual creativity. For example, Pretz and McCollum (2014) did not find evidence for a relationship between self-rated and expert-rated creativity (r = -.07). Professionals wishing to improve group creativity should use objective measures of creativity, or expert observations, rather than relying on personal or peer measures. This could be done through a common definition of originality and usefulness, followed by a pre-test rating by peers. To this objective, it is possible to define an intra-class correlation threshold for which the correlation is high enough to indicate that observers are homogeneously rating the originality and usefulness of ideas (typical thresholds are between 0.75 and 0.90, Koo & Li, 2016). Intraclass correlation is a type of correlation performed on groups of data, unlike Pearson correlation which operates on pairs of data (it is also called interclass correlation); however, the formulas used for both types of correlation are similar.

If the intra-class correlation is sufficient, then observers can continue to evaluate ideas. If the correlation is not sufficient, then it is necessary to find the cause of this lack of relationship, which may be related to a misunderstanding of the objective, difficulty in understanding the originality or usefulness of the ideas in relation to the creativity theme, or a difference in the appreciation of the ideas. After the exchange, the experts will carry out the analysis on a new sample until the intra-class correlation is sufficient to score all the ideas.

It seems important to increase the attention of researchers and professionals to individual differences in group creativity. Although it is difficult to make recommendations based on the meta-analysis results, the hypotheses stated remain possible. Fostering openness to new ideas, paying attention to factors promoting group dynamics (extraversion, agreeableness, low neuroticism, high emotional intelligence, low social anxiety, and low need for closure), and making sense of the common goal (motivation, need for cognition) remain possible avenues for improving creative outcomes in groups.

In particular, a conclusion can be made about creative self-efficacy. Self-efficacy was the only factor to show a positive relationship with group creative outcomes in the meta-analysis. One limitation of this relationship is that creative self-efficacy is not easy to develop. Several recent studies have failed to improve it (Capron Puozzo & Audrin, 2021; Ohly et al., 2017). However, these studies had extremely small sample sizes (n = 69 and n = 23 respectively).

It is also possible that there are several types of creative self-efficacy. Hughes et al. (2018) suggested that there are creative leaders and individuals who are more focused on a creativity support function. This leader/supporter distinction could be related to personality traits, but their relationship remains to be investigated.

The final finding concerns the environment associated with creativity. In this meta-analysis, only one moderating effect appears to be different from chance, namely time. This is a moderator that has been the subject of many studies on individual and team creativity. In particular, almost a third of the best ideas in terms of originality are generated in the first 5 minutes, and after 15 minutes, less than 10% of the ideas are of very good quality, while 50% of them are judged as bad (Reinig & Briggs, 2008). It is possible that personal factors that promote persistence on task (e.g., motivation, conscientiousness, sense of self-efficacy) also promote the possibility of having original ideas longer, a hypothesis that remains to be tested. Furthermore, in terms of the relationship between constructs, one study (Jung et al., 2015) indicated that people who generate the most ideas also come up with better quality ideas, with quality defined as an average between originality and usefulness. Another application of this type of study for professionals is to build on the most creative individuals to improve the overall creativity of all participants, regardless of their personal factors.