Using literature-based discovery to develop hypotheses for the moderating effect of massively multiplayer online games

Step 1: Literature search

In this study, our objective was to identify factors (B terms) that may influence the specific relationships between our A (i.e., “flow”) and C (i.e., “game addiction”) concepts. We identified the possible pathways between A and C concepts in two sets of studies: (1) studies focusing on MMO games, and (2) studies not focusing on MMO games. Domain experts then evaluated the path differences between the two sets of studies to identify possible explanations for the moderating effect of MMO games on flow and game addiction.

We conducted six searches from October 2 to October 26, 2020, three for MMOG studies and three for non-MMOG studies. We searched the following databases: (1) Association for Computing Machinery Digital Library, (2) Academic Search Premier, (3) Web of Science, (4) ScienceDirect, (5) Scopus, (6) Wiley Online Library, (7) Business Source Complete, (8) Springer’s Behavioral Science journals, and (9) Business Source Premier. We limited the search to research article abstracts. We used the advanced search option for each database, which enabled us to use Boolean operators. For example, we performed the following search in each database for articles related to the concept of flow:

We added the search term “experience” to the keywords to ensure that our search results were relevant to the psychological flow state. After the search for papers that studied flow and MMO games yielded few results, we used “immersion” in the search criteria because it has been used as an alternative term to “flow” in many relevant articles.

We combined the search results from (1) and (3) above and removed duplicates using an R script to obtain a final set of flow-related MMO studies. Similarly, the search results from (2) and (4) above were combined and duplicates were removed to generate a final set of flow-related non-MMO studies.

We used the following keywords for addiction-related studies:

We combined the search results from all nine databases again and removed all duplicates to generate the final sets of addiction related MMO and non-MMO studies. We added the search term “games” to ensure that the search results were relevant to video games.

Accordingly, these searches generated four sets of documents. Table I shows the total number of non-duplicate abstracts in each set after the combined search of all databases.

Step 2: Generation of B terms

We analyzed the text data using the tm and SnowballC packages in R. We first changed all texts to lowercase. We then removed all numbers, white spaces, and English stop words from the tm package. Finally, we stemmed the words. For each document set, we created a term–document matrix for the unigrams and bigrams. We identified the common unigrams and bigrams between the “flow” and “addiction” sets for both MMO and non-MMO studies. We considered the terms in these sets as the B terms that may suggest how MMO games weaken the relationship between flow and addiction.

We obtained four lists of B terms, as follows: (1) common unigrams for MMO studies, (2) common bigrams for MMO studies, (3) common unigrams for non-MMO studies, and (4) common bigrams for non-MMO studies. The lengths of these lists were 988, 1,346, 4,868, and 10,986, respectively. As our data set included more non-MMO studies, the corresponding lists of B terms were longer than those for MMO studies.

We used lexical statistics to rank the generated B terms. Specifically, for each B term in a keyword list, we computed the average term frequency-inverse document frequency (tf-idf) for both the flow and addiction sets. We ranked the B terms in each list by the sum of the average tf-idf of the “flow” set and the average tf-idf of the “addiction” set. Two of the authors then independently eliminated words they considered too general (e.g., games, Internet, systems, match) or irrelevant (e.g., experiment, questionnaire, evaluate, study), to obtain approximately 30 to 40 B terms in each list. The generation of this intermediate shortlist relied on the individual authors’ subjective judgment. The two authors then came together to discuss the findings and further reduced the length of each list to the 20 most promising B terms based on their consensus. Disagreements were resolved by consulting the third author. Table II presents the final lists of the top 20 common B terms (both unigrams and bigrams) for MMO and non-MMO studies, while Figure 3 illustrates the flowchart for our implementation of LBD.

Figure 3. LBD implementation of flowchart.

Step 3: Domain experts’ evaluation of the B terms

Our research included interviews with domain experts. Our study application was approved by the Sub-Committee on Research Ethics and Safety of the Research Committee of Lingnan University, Hong Kong (ref. EC059/2021). Each domain expert signed a written consent for prior to the interview. After finalizing the lists of B terms, we interviewed three domain experts, all of whom were active game addiction researchers, for one hour using a Web conferencing software tool. The appendix (Figshare:Appendices, https://doi.org/10.6084/m9.figshare.21719993.v1) presents the interview questions. To make the lists more manageable, we divided the B terms into (1) unigrams and bigrams that were common to MMO and non-MMO studies, (2) unigrams and bigrams that appeared only in MMO studies, and (3) unigrams and bigrams that appeared only in non-MMO studies. Each list included 15 to 20 keywords. We presented these keywords to the domain experts and asked them to select the terms that they felt may help explain the moderating effect of MMO games on the relationship between flow and game addiction. We identified three explanations.

Explanation #1: Social interaction in MMO games may “break the flow” or induce positive peer influence

The critical role of social interactions in the game experience is a key difference between MMO games and non-MMO games. Using B terms such as “social environment,” “teamwork,” and “social setting,” the domain experts commented that social interactions in MMO games (both in-game and in real-life) may distract users’ attention during gameplay and prevent their progression from flow to game addiction. Considering the multiplayer nature of MMOGs, Expert 1 said:

Expert 2 made a similar comment.

Considering real-life social interactions, Expert 3 noted that some players prefer to play with their real-life friends. This peer influence may prevent their progression to game addiction:

Explanation #2: Non-MMOGs offer extrinsic motivation in line with flow theory

Based on the B term “extrinsic motivation,” the interview data suggested that MMO games and non-MMO games offer their players different types of extrinsic motivation. On the one hand, non-MMO games with a score- or point-based system provide a greater sense of achievement, which can strengthen the relationship between flow and game addiction. On the other hand, this sense of achievement may be missing in MMO games because they are often open-ended and game performance depends more on rankings relative to others. According to Expert 2,

As discussed in the Literature Review section, the balance between challenging gameplay and player skills is central to the experience of flow. In non-MMO games, players often begin with simple tasks and are challenged by increasingly complex tasks as their skills improve. This type of game progression is more in line with flow theory than the game progression in open-ended MMOGs.

Explanation #3: Intrinsic motivation and escapism may be more important drivers of MMO game addiction

Using the B terms “intrinsic motivation” and “escapism,” Expert 1 highlighted the relationship between escapism and intrinsic motivation and the importance of intrinsic motivation in MMOGs:

In accordance with this comment, it should be noted that the virtual worlds in MMO games can provide players with an experience that is unavailable in the real world, which can divert their minds from real-life unpleasantness. The players’ avatars may also allow them to identify with their MMO game characters and motivate them to continue playing the game even in the absence of flow. Therefore, “escapism” may represent an important alternative pathway to MMO game addiction because players may use gameplay as a maladaptive coping strategy (Billieux et al. 2015).

Summary

We focused on the interview data that explained the weakened relationship between flow and game addiction observed in MMO games. Three explanations were identified. First, even if a game player really enjoys playing an MMO game, they cannot continue to play if their team members leave in the middle of the game. Social interactions can break their flow experience and prevent addiction. Positive peer influence may also help prevent addiction among players who prefer to play MMO games with their real-life friends. Second, compared with the open-ended nature of MMO games, the reward systems of non-MMO games are typically more in line with flow theory. Third, intrinsic motivation and escapism may provide important alternative pathways to game addiction in MMO games.

Domain experts’ evaluation of the LBD method

During their interviews, the domain experts were asked to comment on the advantages and disadvantages of using computer-generated keywords to identify potential moderating factors. They were also asked about their overall experience with LBD. Overall, they found that LBD was a useful research tool. Expert 3 observed the following:

Expert 2 made a similar comment:

However, not all of the generated terms may be useful; therefore, domain expertise is needed for LBD to be useful. In addition, the semantic meaning of a word in the social sciences may sometimes be ambiguous or depend on the context of use. LBD removes the contexts of use of keywords; therefore, the literature may need to be revisited. Expert 1 commented accordingly:

Expert 2 raised another limitation of the LBD method. When we search for keywords in the literature and use term frequency to identify promising terms, we may miss some relatively new concepts related to MMO games:

In terms of the presentation of potential moderators among the keywords, the domain experts found that it was appropriate to divide the keywords into (1) those that are common to both MMO and non-MMO studies and (2) those that are not common to both sets of studies. The domain experts also agreed on 20 keywords as being roughly the optimal length because a longer list would become too difficult to evaluate:

Finally, all of the domain experts would consider using LBD if a free software tool were available; however, one domain expert pointed out that useful guidelines should also be provided.

<Note: The quotes of the experts have been edited to correct grammatical errors because English is not their first language.>

Theoretical contributions

Nowadays, the academic community produces publications at an accelerated rate, which makes it difficult to connect and integrate the necessary knowledge from the disjointed literature. This problem of knowledge overspecialization is common in multidisciplinary research topics, such as game addiction. In this study, we used domain expert interview data with the LBD method to explain the moderating effect of MMO games on the relationship between flow and game addiction.

The domain expert interview data suggested that the relationship between flow and game addiction may be weakened in MMO games for three reasons. First, the need to play as a team may sometimes prevent players’ progression from flow to game addiction. Positive peer influence may also help prevent game addiction. Second, achieving flow depends not only on the game being challenging but also on the players’ skills (Csikszentmihályi 1990). Unlike most non-MMO games that use a point-based system to track players achievements, the open-ended progression in MMO games may not be appealing to more goal-oriented players. Third, intrinsic motivation and escapism may provide important alternative pathways to MMO game addiction.

The underlying cognitive mechanisms behind the formation of game addiction differ according to different game genres; therefore, our findings suggest that different game genres require dedicated research attention (e.g., You et al. 2017, Blasi et al. 2019, Lee et al. 2021).

Practical implications

Our results suggest that the underlying motivational factors for game addiction differ for different game genres. Specifically, compared with non-MMO games, the achievement motive for MMO game players may be smaller, but they may be more likely to use their gameplay as a maladaptive strategy to cope with negative real-life experiences. This perspective is consistent with PsychGuides.com (n.d.), which divides game addiction into two categories: (1) standard video games that typically have a clear goal, such as saving a princess, and (2) MMO games that typically have no ending. Game addiction in the first category is suggested to be mostly driven by completing missions or beating high scores or preset standards. In contrast, addicted players in the second category form virtual social networks with other players and use their gameplay to escape from reality and gain in-game social acceptance. Therefore, game genres must be considered in effective treatments for game addiction.

Room for improvement in implementing LBD

Although we received generally positive feedback from the domain experts, we found three issues that must be addressed to improve our implementation of LBD.

First, we relied on subjective judgment to remove irrelevant items from the B terms. Although we reached consensus on the final lists only after two researchers independently produced an intermediate shortlist, we believe there is room to improve the objectivity of generating B terms with LBD. In future, we will explore the possibility of automatic ontology generation (Bedini and Nguyen 2007) to replace the use of lexical statistics in the generation of keywords.

Second, some keywords have similar meanings. The term vectors should be clustered to enable an accurate reflection of the importance of keywords in the lexical statistics. The final lists after keyword consolidation will be a more comprehensive representation of the related concepts. Therefore, they will become more useful for domain experts.

Third, we asked the domain experts to freely discuss any moderating factors that they found from the presented keyword lists, which allowed us to extract rich and in-depth knowledge. However, it is quite time-consuming to conduct interviews. An alternative approach to collect feedback from domain experts asynchronously or in response to a long list of B terms would be to ask them to evaluate the usefulness of each keyword using a Likert scale individually and independently. The keywords could then be ranked using the average usefulness scores. In this way, rating each keyword individually will be less cognitively demanding than devising a flow theory based on a complete keyword list.