<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.4536.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Software Tool Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                    <subj-group>
                        <subject>Bioinformatics</subject>
                    </subj-group>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Enrichment Map &#x2013; a Cytoscape app to visualize and explore OMICs pathway enrichment results</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Isserlin</surname>
                        <given-names>Ruth</given-names>
                    </name>
                    <uri content-type="orcid">https://orcid.org/0000-0002-6805-2080</uri>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Merico</surname>
                        <given-names>Daniele</given-names>
                    </name>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Voisin</surname>
                        <given-names>Veronique</given-names>
                    </name>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Bader</surname>
                        <given-names>Gary D.</given-names>
                    </name>
                    <uri content-type="orcid">https://orcid.org/0000-0003-0185-8861</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>The Donnelly Centre, University of Toronto, Toronto, ON M5S 3E1, Canada</aff>
                <aff id="a2">
                    <label>2</label>The Centre for Applied Genomics, Program in Genetics and Genome Biology, The Hospital for Sick Children, Toronto, ON M5G 1X8, Canada</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:gary.bader@utoronto.ca">gary.bader@utoronto.ca</email>
                </corresp>
                <fn fn-type="con">
                    <p>DM initiated and designed the project. RI wrote the manuscript and the software. RI, VV, and DM analyzed and modified existing design. GDB supervised the project.</p>
                </fn>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>1</day>
                <month>7</month>
                <year>2014</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2014</year>
            </pub-date>
            <volume>3</volume>
            <elocation-id>141</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>20</day>
                    <month>6</month>
                    <year>2014</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Isserlin R et al.</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/3-141/pdf"/>
            <abstract>
                <p>High-throughput OMICs experiments generate signals for millions of entities (i.e. genes, proteins, metabolites or any measurable biological entity) in the cell. In an effort to summarize and explore these signals, expression results are examined in the context of known pathways and processes, through enrichment analysis to generate a set of pathways and processes that is significantly enriched. Due to the high redundancy in annotation resources this often results in hundreds of sets. To facilitate the analysis of these results, we have developed the Enrichment Map app to visualize enrichments as a network. We have updated Enrichment Map to support Cytoscape 3, and have added additional features including new data formats and command line access.</p>
            </abstract>
            <funding-group>
                <funding-statement>This work was supported by a NRNB grant (U.S. National Institutes of Health, National Center for Research Resources grant number P41 GM103504) to Gary D. Bader.</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>With the expansion and accessibility of a wide range of experimental techniques to accurately identify and measure any known genomics feature ranging from proteins, transcripts, genes, microRNAs, copy number variations, or DNA methylation in a high-throughput manner, signals for thousands of entities are often generated for an individual OMICs experiment. In efforts to interpret these results in the context of perturbed cellular mechanisms, the entities are often scored and examined for enrichment in known pathways and processes.</p>
            <p>Pathway enrichment analysis helps to uncover general trends or themes present in the data, instead of focusing on one or a few favorite differential genes. Available tools are abundant, designed for varying data types and implemented using a range of different statistical tests: given a set of biological entities, these OMICs signals are then translated into a set of significant pathways and processes (reviewed in Khatri 
                <italic toggle="yes">et al.</italic>
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>
                </sup>, Huang 
                <italic toggle="yes">et al.</italic>
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>
                </sup>). Due to the high redundancy that exists between pathway databases coming from multiple functional annotations of gene products, pathway enrichment often results in a long list of potentially interesting pathways. To help analyze the set of differential pathways, we created the Enrichment Map app to display enrichment results as a network, where pathways are nodes in the network and edges represent known pathway cross-talk defined by the number of genes shared between the pair of pathways and where the network layout organizes the map into functional modules
                <sup>
                    <xref ref-type="bibr" rid="ref-3">3</xref>
                </sup>.</p>
            <p>In this paper, we present the recent implementation of the Enrichment Map app for Cytoscape 3 as well as new features.</p>
        </sec>
        <sec>
            <title>Implementation</title>
            <p>Although originally designed to support Gene Set Enrichment Analysis (GSEA)
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup> the current Enrichment Map app supports multiple enrichment results from tools such as DAVID
                <sup>
                    <xref ref-type="bibr" rid="ref-5">5</xref>
                </sup>, BiNGO
                <sup>
                    <xref ref-type="bibr" rid="ref-6">6</xref>
                </sup>, and GREAT
                <sup>
                    <xref ref-type="bibr" rid="ref-7">7</xref>
                </sup> as well as simplified generic input files which one can easily create from your own enrichment results. Tools like g:Profiler
                <sup>
                    <xref ref-type="bibr" rid="ref-8">8</xref>
                </sup> allow users to download results in an Enrichment Map compatible generic format.</p>
            <p>With the ongoing effort to populate gene annotation and pathway databases, it is difficult for standalone enrichment tools to keep databases up to date. For convenience, we compile gene set files or GMT files, a format created for the GSEA software, to describe all the genes contained in a specified gene set, monthly, from a comprehensive set of annotation and Pathway databases (
                <ext-link ext-link-type="uri" xlink:href="http://download.baderlab.org/EM_Genesets/">http://download.baderlab.org/EM_Genesets/</ext-link>), including standard sources, like MSigDB
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>
                </sup>. Although originally GMT files were specific to GSEA, with the expansion of R and Bioconductor it is now straightforward to load
GMT files into data structures in R using packages like GSA (
                <ext-link ext-link-type="uri" xlink:href="http://statweb.stanford.edu/~tibs/ftp/GSA.pdf">http://statweb.stanford.edu/~tibs/ftp/GSA.pdf</ext-link>) and analyze your OMICs expression data with one of the many different gene set enrichment algorithms such as geneSetTest in the Limma package
                <sup>
                    <xref ref-type="bibr" rid="ref-9">9</xref>
                </sup>, global test
                <sup>
                    <xref ref-type="bibr" rid="ref-10">10</xref>
                </sup>, or Camera
                <sup>
                    <xref ref-type="bibr" rid="ref-11">11</xref>
                </sup>. Visualizing the resulting enrichments is straightforward by exporting to our generic format which minimally consists of the geneset name, description and associated enrichment p-value. Through this mechanism, no matter what the dataset of interest is, gene, protein or metabolite expression, the resulting enrichment analysis can be displayed as an enrichment map.</p>
            <p>There are two main ways to input data into Enrichment Map, through the user interface (
                <xref ref-type="fig" rid="f1">Figure 1</xref>) or the command tool (
                <xref ref-type="table" rid="T1">Table 1</xref>). The user interface is an interactive way to specify all the required files and parameters based on the analysis type chosen. The command tool allows users to automatically create maps directly from the command line, other Cytoscape apps or other programs which can include in-house enrichment tools.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>Enrichment Map app user interface</title>
                    <p>Illustration of Enrichment Map user interface which consists of four main parts: analysis type, file specifications, node and edge filtering. For each analysis type there is a different set of required files. For added functionality there are a set of optional files that can be included to help annotate and explore results. Tuning parameters such as p-value and q-value helps control the number of nodes while tuning the similarity coefficient helps control the number of edges.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/4852/39cbdb14-c58f-4a6a-b4d9-33017ab96beb_figure1.gif"/>
            </fig>
            <table-wrap id="T1" orientation="portrait" position="anchor">
                <label>Table 1. </label>
                <caption>
                    <title>Command tool specification outlined for each of the analysis types.</title>
                    <p>There is an additional command optimized for GSEA inputs only.</p>
                </caption>
                <table content-type="article-table" frame="hsides">
                    <thead>
                        <tr>
                            <th align="left" colspan="1" rowspan="1">Command</th>
                            <th align="left" colspan="1" rowspan="1">Required Arguments</th>
                            <th align="left" colspan="1" rowspan="1">Optional Arguments</th>
                        </tr>
                    </thead>
                    <tbody>
                        <tr>
                            <td colspan="1" rowspan="1">enrichment map build
                                <break/>analysistype="GSEA"</td>
                            <td colspan="1" rowspan="1">
                                <bold>gmtFile</bold>=filepath to geneset file
                                <break/>
                                <bold>enrichmentsDataset1</bold>=filepath to enrichments
                                <break/>
                                <bold>enrichments2Dataset1</bold>=filepath to enrichments
                                <break/>
                                <bold>pvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.05}</styled-content>
                                <break/>
                                <bold>qvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.1}</styled-content>
                                <break/>
                                <bold>coefficients</bold>=one of the following
                                <break/>[OVERLAP, JACCARD, COMBINED],
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default:OVERLAP}</styled-content>
                                <break/>
                                <bold>similaritycutoff</bold>=numerical cutoff,
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default : 0.5}</styled-content>
                            </td>
                            <td colspan="1" rowspan="1">
                                <bold>expressionDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to expression file
                                <break/>
                                <bold>ranksDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to rank file
                                <break/>
                                <bold>classDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to class file
                                <break/>
                                <bold>phenotype1Dataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=Text representing Phenotype
                                <break/>
                                <bold>phenotype2Dataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=Text representing Phenotype2
                                <break/>
                                <bold>enrichmentsDataset2</bold>=filepath to enrichments
                                <break/>
                                <break/>
                                <bold>enrichments2Dataset2</bold>=filepath to enrichments
                                <break/>
                                <break/>
                                <styled-content style="#FE0000" style-type="color">(Replace 1 for 2 to specify which dataset the file is)</styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">enrichmentmap build
                                <break/>analysistype="generic"</td>
                            <td colspan="1" rowspan="1">
                                <bold>gmtFile</bold>=filepath to geneset file
                                <break/>
                                <bold>enrichmentsDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to enrichments
                                <break/>
                                <bold>pvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.05}</styled-content>
                                <break/>
                                <bold>qvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.1}</styled-content>
                                <break/>
                                <bold>coefficients</bold>=one of the following
                                <break/>[OVERLAP, JACCARD, COMBINED],
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default:OVERLAP}</styled-content>
                                <break/>
                                <bold>similaritycutoff</bold>=numerical cutoff,
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default : 0.5}</styled-content>
                            </td>
                            <td colspan="1" rowspan="1">
                                <bold>expressionDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to expression file
                                <break/>
                                <bold>ranksDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to rank file
                                <break/>
                                <bold>classDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to class file
                                <break/>
                                <bold>phenotype1Dataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=Text representing Phenotype
                                <break/>
                                <bold>phenotype2Dataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=Text representing Phenotype2
                                <break/>
                                <break/>
                                <bold>enrichmentsDataset2</bold>=filepath to enrichments
                                <break/>
                                <break/>
                                <styled-content style="#FE0000" style-type="color">(Replace 1 for 2 to specify which dataset the file is)</styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">enrichmentmap build
                                <break/>analysistype=
                                <break/>"David/BiNGO/Great"</td>
                            <td colspan="1" rowspan="1">
                                <bold>enrichmentsDataset1</bold>=filepath to enrichments
                                <break/>
                                <bold>pvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.05}</styled-content>
                                <break/>
                                <bold>qvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.1}</styled-content>
                                <break/>
                                <bold>coefficients</bold>=one of the following
                                <break/>[OVERLAP, JACCARD, COMBINED],
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default:OVERLAP}</styled-content>
                                <break/>
                                <bold>similaritycutoff</bold>=numerical cutoff,
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default : 0.5}</styled-content>
                            </td>
                            <td colspan="1" rowspan="1" valign="top">
                                <bold>expressionDataset</bold>
                                <styled-content style="#FE0000" style-type="color">1</styled-content>=filepath to expression file
                                <break/>
                                <bold>enrichmentsDataset2</bold>=filepath to enrichments
                                <break/>
                                <styled-content style="#FE0000" style-type="color">(Replace 1 for 2 to specify which dataset the file is)</styled-content>
                            </td>
                        </tr>
                        <tr>
                            <td colspan="1" rowspan="1">enrichmentmap
                                <break/>gseabuild</td>
                            <td colspan="1" rowspan="1">
                                <bold>edb</bold>=filepath to GSEA results edb directory
                                <break/>
                                <bold>pvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.05}</styled-content>
                                <break/>
                                <bold>qvalue</bold>=numerical cutoff, 
                                <styled-content style="#6434FC" style-type="color">{default : 0.1}</styled-content>
                                <break/>
                                <bold>coefficients</bold>=one of the following
                                <break/>[OVERLAP, JACCARD, COMBINED],
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default:OVERLAP}</styled-content>
                                <break/>
                                <bold>similaritycutoff</bold>=numerical cutoff,
                                <break/>
                                <styled-content style="#6434FC" style-type="color">{default : 0.5}</styled-content>
                            </td>
                            <td colspan="1" rowspan="1" valign="top">
                                <bold>expression</bold>=filepath to expression file
                                <break/>
                                <bold>expression2</bold>=filepath to expression file
                                <break/>
                                <bold>edbdir2</bold>=filepath to edb directory</td>
                        </tr>
                    </tbody>
                </table>
            </table-wrap>
            <p>Once files and parameters have been specified, the Enrichment Map can be created. Unlike a traditional biological network, nodes in an Enrichment Map represent a set of genes (e.g. a pathway) and their connections the set of genes that two nodes have in common (e.g. pathway cross-talk). Every Enrichment Map is associated with a set of files, parameters, and a number of datasets (currently limited to two) (
                <xref ref-type="fig" rid="f2">Figure 2</xref>). Datasets contain gene sets, enrichments, and expression all of which is needed to interactively update the map through cutoff adjustment sliders found in the legend panel or display the genes contained in a given node or edge selection as a heatmap.</p>
            <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                <label>Figure 2. </label>
                <caption>
                    <title>Enrichment Map build process overview.</title>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/4852/39cbdb14-c58f-4a6a-b4d9-33017ab96beb_figure2.gif"/>
            </fig>
            <p>Enrichment Map app was ported to Cytoscape 3 as a bundle app using Open Service Gateway initiative (OSGi) services provided through the extensive Cytoscape API (version 3.1). The look and feel of the app remains similar to the original implementation for Cytoscape 2 with user input interfaces and view panels including expression heatmap and legend being a direct port from the original source. Given the new framework, each panel implements the CytoPanelComponent and is a registered service associated with the Enrichment Map app. The main enrichment map input panel is registered only once a user opens the app. The remaining view panels are only registered once an enrichment map is created. Enrichment Map consists of one main taskFactory that given an Enrichment Map object populated with a set of input files will construct the appropriate task iterator. Depending on the files specified different parsing tasks can be added to the iterator. Additionally, multiple files of the same type can also be added to the queue with distinct instantiations of a parsing task (with different files specified on task creation). All parsed files populate fields contained in the Enrichment Map object which is then passed to and updated by each of the subsequent tasks (
                <xref ref-type="fig" rid="f2">Figure 2</xref>).</p>
            <p>The BuildEnrichmentMapTaskFactory is used by both the user interface and command tool to construct an enrichment map. Command tool functionality for Enrichment Map requires the given task to define its variables as tunables. Tunables are user supplied information needed by the task. User interfaces can be automatically generated for such tasks based on the set of tunable definitions. When implementing the Enrichment Map tunable task it was our intention to replace our current user interface with the one automatically generated by the task. Given the varied data required from the user as well as the interactive nature of our current user interface the generated tunable interface although functional lacked features that our users are accustomed to. For instance, to specify the analysis type or similarity cutoff our interface has two sets of radio buttons where all the options are visible and only one is selectable. In the tunable interface the same choice can only be represented as a single selection list, a drop down list the user can choose one option from. Both representations are functional but we preferred the radio button implementation therefore, we decided to keep our original interface and add the tunable task solely for the command tool functionality.</p>
        </sec>
        <sec sec-type="results">
            <title>Results</title>
            <p>To illustrate the functionality of Enrichment Map we analyzed and visualized an expression dataset from the Gene Expression Omnibus (GEO)
                <sup>
                    <xref ref-type="bibr" rid="ref-12">12</xref>
                </sup> for mouse fibroblast cells. The experiment was designed to compare gene expression in fibroblast cells in the heart to those in the tail to highlight genes that are uniquely expressed in heart fibroblasts
                <sup>
                    <xref ref-type="bibr" rid="ref-13">13</xref>
                </sup> (GSE50531). Raw expression data was scored using the GEO2R tool available on the GEO website. These expression data were input to GSEA along with a recent compilation of mouse pathway gene sets (May 14, 2014;

                <ext-link ext-link-type="uri" xlink:href="http://download.baderlab.org/EM_Genesets/May_14_2014/">http://download.baderlab.org/EM_Genesets/May_14_2014/</ext-link>) to calculate enrichments. GSEA output files were given to
the app with the cutoffs p-value &lt; 0.005, q-value &lt; 0.05 and overlap similarity coefficient &gt; 0.3. The Enrichment Map generated had roughly the same number of enriched gene sets specific to heart as to tail with cardiac specific sets associated only with the heart phenotype (
                <xref ref-type="fig" rid="f3">Figure 3</xref>, red nodes).</p>
            <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                <label>Figure 3. </label>
                <caption>
                    <title>Enrichment Map of heart fibroblast versus tail fibroblast expression.</title>
                    <p>Using the search field you can enter any text to search all attributes of the given network. Highlighted nodes, (shown as yellow nodes with red edges just left of center) are genesets that contain the gene TBX20.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/4852/39cbdb14-c58f-4a6a-b4d9-33017ab96beb_figure3.gif"/>
            </fig>
            <p>One of the main genes mentioned in the paper associated with this dataset was TBX20 as a specific cardiogenic fibroblast gene found to be important for both normal cardiac development and postinfarct repair
                <sup>
                    <xref ref-type="bibr" rid="ref-13">13</xref>
                </sup>. In Enrichment Map it is easy to find all gene sets that contain it by entering the term TBX20 into the search box (
                <xref ref-type="fig" rid="f3">Figure 3</xref>) (this will also highlight any gene sets that have TBX20 in the name or any other attribute). Built-in search functionality in Cytoscape 3 has improved from Cytoscape 2. All attributes associated with a given network are indexed so there is no longer the need to specify which attribute you would like to search through. Selection of individual or sets of nodes and edges creates a view of the genes contained within the selection as a heat map (
                <xref ref-type="fig" rid="f4">Figure 4</xref>).</p>
            <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                <label>Figure 4. </label>
                <caption>
                    <title>Node Heat Map Panel (contained in the Cytoscape table panel) displayed on selection of &#x201c;Pericardium development (GO:0060039)&#x201d; gene set.</title>
                    <p>If GSEA results are loaded into Enrichment Map, GSEA leading edge genes, defined as the set of genes that contribute most to the enrichment, are highlighted in yellow.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/4852/39cbdb14-c58f-4a6a-b4d9-33017ab96beb_figure4.gif"/>
            </fig>
            <p>Often one of the main challenges after creating an Enrichment Map is going from a network in Cytoscape to publication quality figures. We format the labels so they are more readable and don&#x2019;t extend across the whole screen, but as a result modules often contain overlapping labels that are difficult to read and require hours of manual formatting to create networks that can be used for figures. Using the Cytoscape 3 built-in scaling feature (Layout&gt;Scale), the visualization of clusters and networks can be improved.</p>
        </sec>
        <sec sec-type="conclusions">
            <title>Conclusions</title>
            <p>The Enrichment Map app allows users to translate large sets of enrichment results to a network where highly similar terms cluster together to better highlight overall trends and themes of the underlying data. The details behind the enrichment can be further investigated within the Enrichment Map app using the built-in expression viewer to see all the entities associated with a selected pathway.</p>
        </sec>
        <sec>
            <title>Software availability</title>
            <p>Software available from: 
                <ext-link ext-link-type="uri" xlink:href="http://apps.cytoscape.org/apps/enrichmentmap">http://apps.cytoscape.org/apps/enrichmentmap</ext-link>
			</p>
            <p>Latest source code: 
                <ext-link ext-link-type="uri" xlink:href="https://github.com/BaderLab/EnrichmentMapApp">https://github.com/BaderLab/EnrichmentMapApp</ext-link>
			</p>
            <p>Source code as at the time of publication: 
                <ext-link ext-link-type="uri" xlink:href="https://github.com/F1000Research/EnrichmentMapApp/releases/tag/V1.0">https://github.com/F1000Research/EnrichmentMapApp/releases/tag/V1.0</ext-link>
			</p>
            <p>Archived source code as at the time of publication: 
                <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.5281/zenodo.10542">http://dx.doi.org/10.5281/zenodo.10542</ext-link>
                <sup>
                    <xref ref-type="bibr" rid="ref-14">14</xref>
                </sup>
			</p>
            <p>License: Lesser GNU Public License 2.1: 
                <ext-link ext-link-type="uri" xlink:href="https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html">https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html</ext-link>
			</p>
            <p>Tutorials 
                <ext-link ext-link-type="uri" xlink:href="http://baderlab.org/Software/EnrichmentMap#Tutorials_and_Examples">http://baderlab.org/Software/EnrichmentMap#Tutorials_and_Examples</ext-link>
			</p>
        </sec>
    </body>
    <back>
        <ref-list>
            <ref id="ref-1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Khatri</surname>
                            <given-names>P</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Sirota</surname>
                            <given-names>M</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Butte</surname>
                            <given-names>AJ</given-names>
                        </name>
					</person-group>:
                    <article-title>Ten years of pathway analysis: current approaches and outstanding challenges.</article-title>
                    <source>
						
                        <italic toggle="yes">PLoS Comput Biol.</italic>
					</source>
                    <year>2012</year>;<volume>8</volume>(<issue>2</issue>):<fpage>e1002375</fpage>.
                    <pub-id pub-id-type="pmid">22383865</pub-id>
                    <pub-id pub-id-type="doi">10.1371/journal.pcbi.1002375</pub-id>
                    <pub-id pub-id-type="pmcid">3285573</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <label>2</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Huang da</surname>
                            <given-names>W</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Sherman</surname>
                            <given-names>BT</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Lempicki</surname>
                            <given-names>RA</given-names>
                        </name>
					</person-group>:
                    <article-title>Bioinformatics enrichment tools: paths toward the comprehensive functional analysis of large gene lists.</article-title>
                    <source>
						
                        <italic toggle="yes">Nucleic Acids Res.</italic>
					</source>
                    <year>2009</year>;<volume>37</volume>(<issue>1</issue>):<fpage>1</fpage>&#x2013;<lpage>13</lpage>.
                    <pub-id pub-id-type="pmid">19033363</pub-id>
                    <pub-id pub-id-type="doi">10.1093/nar/gkn923</pub-id>
                    <pub-id pub-id-type="pmcid">2615629</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <label>3</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Merico</surname>
                            <given-names>D</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Isserlin</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Stueker</surname>
                            <given-names>O</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Enrichment map: a networkbased method for gene-set enrichment visualization and interpretation.</article-title>
                    <source>
						
                        <italic toggle="yes">PLoS One.</italic>
					</source>
                    <year>2010</year>;<volume>5</volume>(<issue>11</issue>):<fpage>e13984</fpage>.
                    <pub-id pub-id-type="pmid">21085593</pub-id>
                    <pub-id pub-id-type="doi">10.1371/journal.pone.0013984</pub-id>
                    <pub-id pub-id-type="pmcid">2981572</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Subramanian</surname>
                            <given-names>A</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Tamayo</surname>
                            <given-names>P</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Mootha</surname>
                            <given-names>VK</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Gene set enrichment analysis: a knowledgebased approach for interpreting genome-wide expression profiles.</article-title>
                    <source>
						
                        <italic toggle="yes">Proc Natl Acad Sci U S A.</italic>
					</source>
                    <year>2005</year>;<volume>102</volume>(<issue>43</issue>):<fpage>15545</fpage>&#x2013;<lpage>15550</lpage>.
                    <pub-id pub-id-type="pmid">16199517</pub-id>
                    <pub-id pub-id-type="doi">10.1073/pnas.0506580102</pub-id>
                    <pub-id pub-id-type="pmcid">1239896</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-5">
                <label>5</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Huang da</surname>
                            <given-names>W</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Sherman</surname>
                            <given-names>BT</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Lempicki</surname>
                            <given-names>RA</given-names>
                        </name>
					</person-group>:
                    <article-title>Systematic and integrative analysis of large gene lists using DAVID bioinformatics resources.</article-title>
                    <source>
						
                        <italic toggle="yes">Nat Protoc.</italic>
					</source>
                    <year>2009</year>;<volume>4</volume>(<issue>1</issue>):<fpage>44</fpage>&#x2013;<lpage>57</lpage>.
                    <pub-id pub-id-type="pmid">19131956</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nprot.2008.211</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <label>6</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Maere</surname>
                            <given-names>S</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Heymans</surname>
                            <given-names>K</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Kuiper</surname>
                            <given-names>M</given-names>
                        </name>
					</person-group>:
                    <article-title>BiNGO: a Cytoscape plugin to assess overrepresentation of gene ontology categories in biological networks.</article-title>
                    <source>
						
                        <italic toggle="yes">Bioinformatics.</italic>
					</source>
                    <year>2005</year>;<volume>21</volume>(<issue>16</issue>):<fpage>3448</fpage>&#x2013;<lpage>3449</lpage>.
                    <pub-id pub-id-type="pmid">15972284</pub-id>
                    <pub-id pub-id-type="doi">10.1093/bioinformatics/bti551</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <label>7</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>McLean</surname>
                            <given-names>CY</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Bristor</surname>
                            <given-names>D</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Hiller</surname>
                            <given-names>M</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>GREAT improves functional interpretation of cis-regulatory regions.</article-title>
                    <source>
						
                        <italic toggle="yes">Nat Biotechnol.</italic>
					</source>
                    <year>2010</year>;<volume>28</volume>(<issue>5</issue>):<fpage>495</fpage>&#x2013;<lpage>501</lpage>.
                    <pub-id pub-id-type="pmid">20436461</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nbt.1630</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-8">
                <label>8</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Reimand</surname>
                            <given-names>J</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Arak</surname>
                            <given-names>T</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Vilo</surname>
                            <given-names>J</given-names>
                        </name>
					</person-group>:
                    <article-title>g:Profiler--a web server for functional interpretation of gene lists (2011 update).</article-title>
                    <source>
						
                        <italic toggle="yes">Nucleic Acids Res.</italic>
					</source>
                    <year>2011</year>;<volume>39</volume>(<issue>Web Server issue</issue>):<fpage>W307</fpage>&#x2013;<lpage>W315</lpage>.
                    <pub-id pub-id-type="pmid">21646343</pub-id>
                    <pub-id pub-id-type="doi">10.1093/nar/gkr378</pub-id>
                    <pub-id pub-id-type="pmcid">3125778</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-9">
                <label>9</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Gentleman</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Carey</surname>
                            <given-names>V</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Huber</surname>
                            <given-names>W</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Bioinformatics and computational biology solutions using R and Bioconductor</article-title>, volume 746718470. Springer,<year>2005</year>.
                    <pub-id pub-id-type="doi">10.1007/0-387-29362-0</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-10">
                <label>10</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Goeman</surname>
                            <given-names>JJ</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Van De Geer</surname>
                            <given-names>SA</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>De Kort</surname>
                            <given-names>F</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>A global test for groups of genes: testing association with a clinical outcome.</article-title>
                    <source>
						
                        <italic toggle="yes">Bioinformatics.</italic>
					</source>
                    <year>2004</year>;<volume>20</volume>(<issue>1</issue>):<fpage>93</fpage>&#x2013;<lpage>99</lpage>.
                    <pub-id pub-id-type="pmid">14693814</pub-id>
                    <pub-id pub-id-type="doi">10.1093/bioinformatics/btg382</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-11">
                <label>11</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Wu</surname>
                            <given-names>D</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Smyth</surname>
                            <given-names>GK</given-names>
                        </name>
					</person-group>:
                    <article-title>Camera: a competitive gene set test accounting for inter-gene correlation.</article-title>
                    <source>
						
                        <italic toggle="yes">Nucleic Acids Res.</italic>
					</source>
                    <year>2012</year>;<volume>40</volume>(<issue>17</issue>):<fpage>e133</fpage>.
                    <pub-id pub-id-type="pmid">22638577</pub-id>
                    <pub-id pub-id-type="doi">10.1093/nar/gks461</pub-id>
                    <pub-id pub-id-type="pmcid">3458527</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-12">
                <label>12</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Barrett</surname>
                            <given-names>T</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Wilhite</surname>
                            <given-names>SE</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Ledoux</surname>
                            <given-names>P</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>NCBI GEO: archive for functional genomics data sets--update.</article-title>
                    <source>
						
                        <italic toggle="yes">Nucleic Acids Res.</italic>
					</source>
                    <year>2013</year>;<volume>41</volume>(<issue>Database issue</issue>):<fpage>D991</fpage>&#x2013;<lpage>D995</lpage>.
                    <pub-id pub-id-type="pmid">23193258</pub-id>
                    <pub-id pub-id-type="doi">10.1093/nar/gks1193</pub-id>
                    <pub-id pub-id-type="pmcid">3531084</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-13">
                <label>13</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Furtado</surname>
                            <given-names>MB</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Costa</surname>
                            <given-names>MW</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Pranoto</surname>
                            <given-names>EA</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>Cardiogenic genes expressed in cardiac fibroblasts contribute to heart development and repair.</article-title>
                    <source>
						
                        <italic toggle="yes">Circ Res.</italic>
					</source>
                    <year>2014</year>;<volume>114</volume>(<issue>9</issue>):<fpage>1422</fpage>&#x2013;<lpage>1434</lpage>.
                    <pub-id pub-id-type="pmid">24650916</pub-id>
                    <pub-id pub-id-type="doi">10.1161/CIRCRESAHA.114.302530</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-14">
                <label>14</label>
                <mixed-citation publication-type="data">
                    <person-group person-group-type="author">
						
                        <name name-style="western">
                            <surname>Isserlin</surname>
                            <given-names>R</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Merico</surname>
                            <given-names>D</given-names>
                        </name>
						
                        <name name-style="western">
                            <surname>Voisin</surname>
                            <given-names>V</given-names>
                        </name>
						
                        <etal/>
					</person-group>:
                    <article-title>F1000Research/EnrichmentMapApp.</article-title>
                    <source>
						
                        <italic toggle="yes">ZENODO.</italic>
					</source>
                    <year>2014</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.5281/zenodo.10542">Data Source</ext-link>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report5298">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.4852.r5298</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Markowetz</surname>
                        <given-names>Florian</given-names>
                    </name>
                    <xref ref-type="aff" rid="r5298a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r5298a1">
                    <label>1</label>Department of Oncology, University of Cambridge, Cambridge, UK</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>15</day>
                <month>7</month>
                <year>2014</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Markowetz F</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport5298" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.4536.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>This paper is an update on the Enrichment Map introduced in 2010 by the same authors. They have extended the methodology and made it available in the newest version of Cytoscape.</p>
            <p>Gene set enrichment methods of various forms are one of the most widely used first steps to gain a global picture of which pathways or other functional units are involved in some molecular phenotype. However, it is generally very hard to make sense of the results - mostly because lists of enriched gene sets can be very long and might be due to a small set of genes that appear in many of them.</p>
            <p>This is where the Enrichment Map comes in: By making the overlap between gene sets explicit it allows users to visually explore 'enrichment themes' (clusters of overlapping gene sets). Because it is simple and informative I believe this will become the standard way to present enrichment results.</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report5301">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.4852.r5301</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Salomonis</surname>
                        <given-names>Nathan</given-names>
                    </name>
                    <xref ref-type="aff" rid="r5301a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-9689-2469</uri>
                </contrib>
                <aff id="r5301a1">
                    <label>1</label>Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center Research Foundation, Cincinnati, OH, USA</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>14</day>
                <month>7</month>
                <year>2014</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2014 Salomonis N</copyright-statement>
                <copyright-year>2014</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport5301" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.4536.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The authors present an already highly used and very useful approach for making sense of highly redundant biological enrichment results that arise from the analysis of transcriptome and genomics datasets. The added additional utility of accessing the software by command line and using alternative input formats makes this a highly accessible plugin for Cytoscape, that will clearly be widely adopted by the Cytoscape community through its availability in Cytoscape 3.0.</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.</p>
        </body>
    </sub-article>
</article>
