<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.22139.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Software Tool Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>scRepertoire: An R-based toolkit for single-cell immune receptor analysis</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved with reservations]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Borcherding</surname>
                        <given-names>Nicholas</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-1427-6342</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                    <xref ref-type="aff" rid="a2">2</xref>
                    <xref ref-type="aff" rid="a3">3</xref>
                    <xref ref-type="aff" rid="a4">4</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Bormann</surname>
                        <given-names>Nicholas L.</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-8847-8727</uri>
                    <xref ref-type="aff" rid="a5">5</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Department of Pathology, University of Iowa, Iowa City, IA, USA</aff>
                <aff id="a2">
                    <label>2</label>Medical Scientist Training Program, University of Iowa, Iowa City, IA, USA</aff>
                <aff id="a3">
                    <label>3</label>Cancer Biology Graduate Program, University of Iowa, Iowa City, IA, USA</aff>
                <aff id="a4">
                    <label>4</label>Holden Comprehensive Cancer Center, University of Iowa, Iowa City, IA, USA</aff>
                <aff id="a5">
                    <label>5</label>Department of Psychiatry, University of Iowa, Iowa ity, IA, USA</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:nicholas-borcherding@uiowa.edu">nicholas-borcherding@uiowa.edu</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>27</day>
                <month>1</month>
                <year>2020</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2020</year>
            </pub-date>
            <volume>9</volume>
            <elocation-id>47</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>22</day>
                    <month>1</month>
                    <year>2020</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2020 Borcherding N and Bormann NL</copyright-statement>
                <copyright-year>2020</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/9-47/pdf"/>
            <abstract>
                <p>Single-cell sequencing is an emerging technology in the field of immunology and oncology that allows researchers to couple RNA quantification and other modalities, like immune cell receptor profiling at the level of an individual cell. A number of workflows and software packages have been created to process and analyze single-cell transcriptomic data. These packages allow users to take the vast dimensionality of the data generated in single-cell-based experiments and distill the data into novel insights. Unlike the transcriptomic field, there is a lack of options for software that allow for single-cell immune receptor profiling. Enabling users to easily combine mRNA and immune profiling, scRepertoire was built to process data derived from 10x Genomics Chromium Immune Profiling for both T-cell receptor (TCR) and immunoglobulin (Ig) enrichment workflows and subsequently interacts with the popular Seurat R package. The scRepertoire R package and processed data are open source and available on 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/ncborcherding/scRepertoire">GitHub</ext-link> and provides in-depth tutorials on the capability of the package.</p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>Single-cell RNA sequencing</kwd>
                <kwd>immune receptor profiling</kwd>
                <kwd>R</kwd>
                <kwd>clonotypic analysis</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="http://dx.doi.org/10.13039/100000054">
                    <funding-source>National Cancer Institute</funding-source>
                    <award-id>CA206255</award-id>
                </award-group>
                <funding-statement>Funding for this project was provided from National Institute of Health F30 fellowship CA206255. </funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec sec-type="intro">
            <title>Introduction</title>
            <p>The molecular resolution offered by single-cell sequencing (SCS) technologies has led to extensive investigations in the realms of developmental biology, oncology, and immunology. In terms of the latter field, SCS offers the ability to couple the exploration of transcriptomic heterogeneity in immune cells along a disease process with clonality
                <sup>
                    <xref ref-type="bibr" rid="ref-1">1</xref>
                </sup>. A number of methods exist for dimensional reduction of mRNA data, reviewed by Chen 
                <italic toggle="yes">et al.</italic>
                <sup>
                    <xref ref-type="bibr" rid="ref-2">2</xref>
                </sup> that have been implemented into R packages to assist in processing and analysis of SCS experiments. However, a gap exists in the processing of V(D)J sequencing, descriptive statistics, clonal comparisons, and repertoire diversity with the current SCS R packages.</p>
            <p>With these limitations in mind, scRepertoire
                <sup>
                    <xref ref-type="bibr" rid="ref-3">3</xref>
                </sup> was generated (
                <xref ref-type="fig" rid="f1">Figure 1</xref>). Built using R, scRepertoire is a toolkit to assist in the analysis of immune profiles for both B and T cells, while interacting with the popular Seurat pipeline
                <sup>
                    <xref ref-type="bibr" rid="ref-4">4</xref>&#x2013;
                    <xref ref-type="bibr" rid="ref-6">6</xref>
                </sup>. scRepertoire also includes processed single-cell mRNA and V(D)J sequencing data of 12,911 tumor-infiltrating and peripheral-blood T cells derived from three renal clear cell carcinoma patient, which is characterized below to demonstrate the capabilities of the package.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>A general workflow for single-cell data analysis involving scRepertoire.</title>
                    <p>The analysis starts with the single-cell immune and mRNA sequencing and Cell Ranger-based alignment with the 10x Genomics pipeline. With the TCR or Ig sequencing, scRepertoire can import the filtered overlapping DNA segments, or contigs. The alignments are filtered by cell type of interest and combined using the individual cell barcodes. Clonotypes can be called using the gene sequence of the immune receptor loci, CDR3 nucleotide sequence or CDR3 amino acid sequence. After clonotype assignment, more extensive clonotypic analysis can be performed at the individual sample level or across all samples. General outputs from scRepertoire can be imported into Seurat objects to visualize clonotype data overlaid onto the cell clustering. Likewise, metadata from the Seurat objects can be imported into scRepertoire to analyze clonotypes by assigned clusters.</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/24415/cfd77d18-d2bf-40c1-8126-efd0b8999cac_figure1.gif"/>
            </fig>
        </sec>
        <sec sec-type="methods">
            <title>Methods</title>
            <sec>
                <title>Operation</title>
                <p>System requirements for running scRepertoire
                    <sup>
                        <xref ref-type="bibr" rid="ref-3">3</xref>
                    </sup> include the installation of 
                    <ext-link ext-link-type="uri" xlink:href="https://www.r-project.org/">R</ext-link> v3.5.1 and the the 
                    <ext-link ext-link-type="uri" xlink:href="https://satijalab.org/seurat/">Seurat</ext-link> R package (v3.1.2). Utilization of scRepertoire is dependent on the total number of single-cells being processed, with a base estimate of 1 Gb of random-access memory and a modern CPU.</p>
            </sec>
            <sec>
                <title>Data</title>
                <p>The isolation and processing of the 10x-Genomics-based single-cell mRNA and V(D)J Chromium sequencing data for immune cells has previously been described
                    <sup>
                        <xref ref-type="bibr" rid="ref-7">7</xref>,
                        <xref ref-type="bibr" rid="ref-8">8</xref>
                    </sup>. In addition, T cells were identified using expression values for canonical T cell markers: 
                    <italic toggle="yes">CD3D</italic>, 
                    <italic toggle="yes">CD4</italic>, 
                    <italic toggle="yes">CD8A</italic>, 
                    <italic toggle="yes">CD8B1</italic> and previous clustering. T cells were isolated and reclustered using the integration method from the 
                    <ext-link ext-link-type="uri" xlink:href="https://satijalab.org/seurat/">Seurat</ext-link> R package (v3.1.2) with 20 principal components and a resolution of 0.5
                    <sup>
                        <xref ref-type="bibr" rid="ref-4">4</xref>
                    </sup>. All code used to generate the figures appearing in the manuscript is available at 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/ncborcherding/scRepertoire">https://github.com/ncborcherding/scRepertoire</ext-link>.</p>
            </sec>
            <sec>
                <title>Implementation</title>
                <p>The scRepertoire was built and tested in 
                    <ext-link ext-link-type="uri" xlink:href="https://www.r-project.org/">R</ext-link> v3.5.1. Analysis for scRepertoire was inspired from the bulk immune profiling 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/tcR/index.html">tcR</ext-link> (v2.2.4) R package without derivations in code
                    <sup>
                        <xref ref-type="bibr" rid="ref-9">9</xref>
                    </sup>. Clonotypes can be called using the combination of immune loci genes, a more sensitive approach, or the nucleotide/amino acid sequence of the complementary-determining region 3 (CDR3). In addition to the base functions in R, data processing was performed using the 
                    <ext-link ext-link-type="uri" xlink:href="https://dplyr.tidyverse.org/">dplyr</ext-link> (v0.8.3) and 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/reshape2/index.html">reshape2</ext-link> (v1.4.3) R packages. Visualizations are generated using the 
                    <ext-link ext-link-type="uri" xlink:href="https://ggplot2.tidyverse.org/">ggplot2</ext-link> (v3.2.1) and 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/ggalluvial/index.html">ggalluvial</ext-link> (v0.11.1) R packages with color pallets derived from the use of 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/colorRamps/index.html">colorRamps</ext-link> (v2.3) and 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/RColorBrewer/index.html">RColorBrewer</ext-link> (v1.1.2) R packages. Diversity metrics are calculated using the 
                    <ext-link ext-link-type="uri" xlink:href="https://cran.r-project.org/web/packages/vegan/index.html">vegan</ext-link> (v2.5-6) R package. Visual outputs of functions are stored as layers of geometric or statistical ggplot layering, allowing users to easily modify presentation.</p>
            </sec>
        </sec>
        <sec sec-type="results">
            <title>Results</title>
            <sec>
                <title>Clonal analysis</title>
                <p>scRepertoire
                    <sup>
                        <xref ref-type="bibr" rid="ref-3">3</xref>
                    </sup> can be used to call clonotypes using the CDR3 amino acid/nucleotide sequences, by gene usage, or by the combination of CDR3 nucleotide sequences and genes. Using the 
                    <italic toggle="yes">quantContig</italic> function, unique clonotypes can be visualized as raw values or scaled to the size of the library for samples or by type (
                    <xref ref-type="fig" rid="f2">Figure 2A</xref>). The total abundance of clonotypes can also be visualized calling 
                    <italic toggle="yes">abundanceContig</italic> (
                    <xref ref-type="fig" rid="f2">Figure 2B</xref>) or relative abundance of clonotypes (
                    <xref ref-type="fig" rid="f2">Figure 2C</xref>). Additionally, the distribution of CDR3 nucleotide or amino acid sequences for clonotypes can be visualized with 
                    <italic toggle="yes">lengthContig</italic> (
                    <xref ref-type="fig" rid="f2">Figure 2D</xref>).</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>Figure 2. </label>
                    <caption>
                        <title>Basic clonotypic analysis functions in scRepertoire.</title>
                        <p>(
                            <bold>A</bold>) Scaled unique clonotypes by total number of TCRs sequenced by patient and type of sample (peripheral, P; tumor, T), using the 
                            <italic toggle="yes">quantContig</italic> function. (
                            <bold>B</bold>) Total abundance of clonotypes by sample and type using the 
                            <italic toggle="yes">abundanceContig</italic> function. (
                            <bold>C</bold>) Relative abundance of clonotypes using density comparing peripheral blood to tumor samples. (
                            <bold>D</bold>) CDR3 nucleotide length analysis by sample using the 
                            <italic toggle="yes">lengthContig</italic> function. The bimodal nature of the curve is a function of calling clonotypes for cells with both one and two immune receptors sequenced.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/24415/cfd77d18-d2bf-40c1-8126-efd0b8999cac_figure2.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Proportional analysis and diversity measures</title>
                <p>More in depth analysis of clonal architecture is available. Within the framework of scRepertoire, analysis of clonal homeostasis, or the clonal space occupied by clonotypes of specific proportions, can be visualized by 
                    <italic toggle="yes">clonalHomeostasis</italic> function (
                    <xref ref-type="fig" rid="f3">Figure 3A</xref>). Similarly, 
                    <italic toggle="yes">clonalProportion</italic> can be called to look at the proportion of clonal space occupied by specific clonotypes (
                    <xref ref-type="fig" rid="f3">Figure 3B</xref>). Overlap between the samples can be calculated and visualized with 
                    <italic toggle="yes">clonalOverlap</italic>, using either the overlap coefficient or Morisita index methods (
                    <xref ref-type="fig" rid="f3">Figure 3C</xref>). Measured of diversity across samples or groups can be quantified with the 
                    <italic toggle="yes">clonalDiversity</italic> function, demonstrating an overall reduction in clonal diversity in tumor samples (
                    <xref ref-type="fig" rid="f3">Figure 3D</xref>).</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>Figure 3. </label>
                    <caption>
                        <title>Advanced clonal measures between samples.</title>
                        <p>(
                            <bold>A</bold>) Clonal homeostatic space representations across all six samples using the gene and CDR3 AA sequence for clonotype calling. (
                            <bold>B</bold>) Relative proportional space occupied by specific clonotypes across all six samples using the gene and CDR3 AA sequence for clonotype calling. (
                            <bold>C</bold>) Morisita overlap quantifications for clonotypes across all six samples. (
                            <bold>D</bold>) Diversity measures based on clonotypes by sample type using Shannon, Inverse Simpson, Chao, and abundance-based coverage estimator (ACE) indices.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/24415/cfd77d18-d2bf-40c1-8126-efd0b8999cac_figure3.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Seurat interaction</title>
                <p>After the processing and analysis of the TCR repertoire with the base features, the next step is using scRepertoire to interact with the single-cell mRNA data. The expression data for the 12,911 cells built into the package have already been clusters (
                    <xref ref-type="fig" rid="f4">Figure 4A</xref>), with a clear distribution of the clusters into peripheral-blood- versus tumor-predominant (
                    <xref ref-type="fig" rid="f4">Figure 4B</xref>). Using the 
                    <italic toggle="yes">combineSeurat</italic> function in scRepertoire, we can look at the clonotypic frequencies of cells that comprise the UMAP-based clusters (
                    <xref ref-type="fig" rid="f4">Figure 4C</xref>), with notable expansion in the C2, C3, and C6 clusters (
                    <xref ref-type="fig" rid="f4">Figure 4D</xref>). The C7 and C8 clusters also have a relatively high frequency. In addition to clonal distribution, we can also use 
                    <italic toggle="yes">highlightClonotypes</italic> to set specific sequences of clonotypes to be visualized (
                    <xref ref-type="fig" rid="f4">Figure 4E</xref>), with clonotype 1 referring to the amino acid sequence &#x201c;CAVNGGSQGNLIF_CSAEREDTDTQYF&#x201d; and clonotype 2 for the amino acid sequence "NA_CATSATLRVVAEKLFF". Interesting clonotype 2 is restricted to a subcluster of the C6 cluster (
                    <xref ref-type="fig" rid="f4">Figure 4E</xref>). After combining both the clonotype and expression data, interaction between categories, such as cluster label and clonotype frequency can be visualized with the 
                    <italic toggle="yes">alluvialGraph</italic> function.</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Interaction of scRepertoire with the Seurat R package.</title>
                        <p>(
                            <bold>A</bold>) UMAP projection of the ccRCC T cells (n=12,911) into 12 distinct clusters. (
                            <bold>B</bold>) UMAP projection with peripheral blood (red) and tumor (blued) populations highlighted and an accompanying relative proportion composition of each cluster, scaled by the total number of peripheral blood and tumor cells, respectively. (
                            <bold>C</bold>) Using the 
                            <italic toggle="yes">combineSeurat</italic> function places individual cells into groups by the number of clonotypes, which then can be displayed overlaid with the UMAP projection. (
                            <bold>D</bold>) 
                            <italic toggle="yes">combineSeurat</italic> also calculates the frequency of clonotypes, which can be used to more closely examine cluster composition, such as the indicate boxplot. (
                            <bold>E</bold>) After combing the clonotype information with the Seurat object, 
                            <italic toggle="yes">highlightClonotypes</italic> can be used to specifically highlight the individual clonotypes of interest using the sequence information. (
                            <bold>F</bold>) Interaction of clonotypes between multiple categories can be examined using the 
                            <italic toggle="yes">alluvialGraph</italic> function.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/24415/cfd77d18-d2bf-40c1-8126-efd0b8999cac_figure4.gif"/>
                </fig>
            </sec>
        </sec>
        <sec sec-type="conclusions">
            <title>Conclusions</title>
            <p>scRepertoire (9) is a R-based toolkit for the analysis of single-cell immune receptor profiling. The package is able to take the annotated filtered outputs from the 10x Genomics Cell Ranger platform and provide analysis a number of modalities, including calling clonotypes, clonal space/homeostasis, clonal diversity, and repertoire overlap between samples. Outputs from scRepertoire can combined with dimensional reduction strategies for single-cell RNA quantifications, allowing users to analyze mRNA and immune profiles together. Under the creative commons v4.0 license, the scRepertoire package is freely available from the GitHub repository and is extensively annotated to assist in implementation and modification.</p>
        </sec>
        <sec>
            <title>Data availability</title>
            <sec>
                <title>Source data</title>
                <p>Zenodo: scRepertoire. 
                    <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5281/zenodo.3612216">https://doi.org/10.5281/zenodo.3612216</ext-link>
                    <sup>
                        <xref ref-type="bibr" rid="ref-3">3</xref>
                    </sup>.</p>
                <p>Folder &#x2018;Data&#x2019; contains all data required to run the vignettes described in the 
                    <italic toggle="yes">Results</italic>. This is also available on 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/ncborcherding/scRepertoire/tree/master/data">GitHub</ext-link>.</p>
                <p>Data are available under the terms of the 
                    <ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/legalcode">Creative Commons Attribution 4.0 International license</ext-link> (CC-BY 4.0).</p>
            </sec>
        </sec>
        <sec>
            <title>Software availability</title>
            <p>Source code is available from GitHub: 
                <ext-link ext-link-type="uri" xlink:href="https://github.com/ncborcherding/scRepertoire">https://github.com/ncborcherding/scRepertoire</ext-link>.</p>
            <p>Archived source code at the time of publication: 
                <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5281/zenodo.3612216">https://doi.org/10.5281/zenodo.3612216</ext-link>
                <sup>
                    <xref ref-type="bibr" rid="ref-3">3</xref>
                </sup>.</p>
            <p>License: 
                <ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/4.0/legalcode">Creative Commons Attribution 4.0 International</ext-link>.</p>
        </sec>
    </body>
    <back>
        <ref-list>
            <ref id="ref-1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Papalexi</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Satija</surname>
                            <given-names>R</given-names>
                        </name>
</person-group>:
                    <article-title>Single-cell RNA sequencing to explore immune cell heterogeneity.</article-title>
                    <source>

                        <italic toggle="yes">Nat Rev Immunol.</italic>
</source>
                    <year>2018</year>;<volume>18</volume>(<issue>1</issue>):<fpage>35</fpage>&#x2013;<lpage>45</lpage>.
                    <pub-id pub-id-type="pmid">28787399</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nri.2017.76</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <label>2</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Chen</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ning</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Shi</surname>
                            <given-names>T</given-names>
                        </name>
</person-group>:
                    <article-title>Single-Cell RNA-Seq Technologies and Related Computational Data Analysis.</article-title>
                    <source>

                        <italic toggle="yes">Front Genet.</italic>
</source>
                    <year>2019</year>;<volume>10</volume>:<fpage>317</fpage>.
                    <pub-id pub-id-type="pmid">31024627</pub-id>
                    <pub-id pub-id-type="doi">10.3389/fgene.2019.00317</pub-id>
                    <pub-id pub-id-type="pmcid">6460256</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <label>3</label>
                <mixed-citation publication-type="data">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Borcherding</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bormann</surname>
                            <given-names>NL</given-names>
                        </name>
</person-group>:
                    <article-title>scRepertoire (Version 0.0.1).</article-title>
                    <source>

                        <italic toggle="yes">Zenodo.</italic>
</source>
                    <year>2020</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://www.doi.org/10.5281/zenodo.3612216">http://www.doi.org/10.5281/zenodo.3612216</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Stuart</surname>
                            <given-names>T</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Butler</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hoffman</surname>
                            <given-names>P</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Comprehensive Integration of Single-Cell Data.</article-title>
                    <source>

                        <italic toggle="yes">Cell.</italic>
</source>
                    <year>2019</year>;<volume>177</volume>(<issue>7</issue>):<fpage>1888</fpage>&#x2013;<lpage>1902.e21</lpage>.
                    <pub-id pub-id-type="pmid">31178118</pub-id>
                    <pub-id pub-id-type="doi">10.1016/j.cell.2019.05.031</pub-id>
                    <pub-id pub-id-type="pmcid">6687398</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-5">
                <label>5</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Macosko</surname>
                            <given-names>EZ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Basu</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Satija</surname>
                            <given-names>R</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Highly Parallel Genome-wide Expression Profiling of Individual Cells Using Nanoliter Droplets.</article-title>
                    <source>

                        <italic toggle="yes">Cell.</italic>
</source>
                    <year>2015</year>;<volume>161</volume>(<issue>5</issue>):<fpage>1202</fpage>&#x2013;<lpage>14</lpage>.
                    <pub-id pub-id-type="pmid">26000488</pub-id>
                    <pub-id pub-id-type="doi">10.1016/j.cell.2015.05.002</pub-id>
                    <pub-id pub-id-type="pmcid">4481139</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <label>6</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Butler</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hoffman</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Smibert</surname>
                            <given-names>P</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Integrating single-cell transcriptomic data across different conditions, technologies, and species.</article-title>
                    <source>

                        <italic toggle="yes">Nat Biotechnol.</italic>
</source>
                    <year>2018</year>;<volume>36</volume>(<issue>5</issue>):<fpage>411</fpage>&#x2013;<lpage>420</lpage>.
                    <pub-id pub-id-type="pmid">29608179</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nbt.4096</pub-id>
                    <pub-id pub-id-type="pmcid">6700744</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <label>7</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Borcherding</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ahmed</surname>
                            <given-names>KK</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Voigt</surname>
                            <given-names>AP</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Transcriptional heterogeneity in cancer-associated regulatory T cells is predictive of survival.</article-title>
                    <source>

                        <italic toggle="yes">bioRxiv.</italic>
</source>
                    <year>2018</year>; 478628.
                    <pub-id pub-id-type="doi">10.1101/478628</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-8">
                <label>8</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Vishwakarma</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bocherding</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Chimenti</surname>
                            <given-names>MS</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Mapping the Immune Landscape of Clear Cell Renal Cell Carcinoma by Single-Cell RNA-seq.</article-title>
                    <source>

                        <italic toggle="yes">bioRxiv.</italic>
</source>
                    <year>2019</year>; 824482.
                    <pub-id pub-id-type="doi">10.1101/824482</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-9">
                <label>9</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Nazarov</surname>
                            <given-names>VI</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Pogorelyy</surname>
                            <given-names>MV</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Komech</surname>
                            <given-names>EA</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>tcR: an R package for T cell receptor repertoire advanced data analysis.</article-title>
                    <source>

                        <italic toggle="yes">BMC Bioinformatics.</italic>
</source>
                    <year>2015</year>;<volume>16</volume>:<fpage>175</fpage>.
                    <pub-id pub-id-type="pmid">26017500</pub-id>
                    <pub-id pub-id-type="doi">10.1186/s12859-015-0613-1</pub-id>
                    <pub-id pub-id-type="pmcid">4445501</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report60873">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.24415.r60873</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Stuart</surname>
                        <given-names>Tim</given-names>
                    </name>
                    <xref ref-type="aff" rid="r60873a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r60873a1">
                    <label>1</label>New York Genome Center, New York City, NY, USA</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>9</day>
                <month>3</month>
                <year>2020</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2020 Stuart T</copyright-statement>
                <copyright-year>2020</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport60873" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.22139.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>This paper presents a new R package for the analysis of T-cell and B-cell clonotypes inferred from V(D)J recombination events. The package is designed for the analysis of data generated using the 10x Genomics V(D)J kit that allows clonotypes and transcriptomes to be co-assayed in the same individual cells.</p>
            <p> </p>
            <p> Overall the package appears to fill an unmet need in the analysis of single-cell V(D)J data, and works nicely with the existing Seurat package for single-cell analysis.</p>
            <p> </p>
            <p> While the focus of the paper is on the software, a more detailed explanation of the biology of V(D)J recombination and how this information is detected in single cells would improve the clarity of the manuscript. A description of the major existing challenges and goals of immune clonotype analysis could also be added.</p>
            <p> </p>
            <p> I do not have extensive expertise in the immunology and so my review focuses on the software implementation. I leave discussion of the value and impact of the software to other reviewers. However I will reiterate comments from other reviewers in saying that there should be a distinction made between T and B cell clonotypes, due to the Ig class switching that occurs in B cells.&#x00a0;</p>
            <p> </p>
            <p> The paper also states that &#x201c;The expression data for the 12,911 cells built into the package&#x201d;. I believe the clonotype information is included in the package in the data directory, but I don&#x2019;t think the expression data is included. Including large gene expression matrix within the R package is not advisable anyway, and the authors should make it available through other means (NCBI GEO, for example).</p>
            <p> </p>
            <p> 
                <bold>Code review</bold>
            </p>
            <p> </p>
            <p> I would encourage the authors to submit the package to CRAN or another R repository. This assist in the distribution of the package, and the process of getting the package accepted on CRAN will likely greatly improve the code quality. At a minimum, the authors should try to get the package to pass R CMD check with no notes, warnings, or errors.</p>
            <p> </p>
            <p> Currently the package includes all rendered figures in the vignettes folder. I would recommend removing these files, and also removing the `ggsave` function calls within the vignette to prevent these files being written while compiling the vignette. This will reduce the size of the package.</p>
            <p> </p>
            <p> Several functions in the package use scoping assignment to assign variables in the global environment. This is considered bad practice and should be avoided in all cases.</p>
            <p> </p>
            <p> Extra files such as .DS_Store should be removed from the git repository and from the package. Use the .gitignore and .Rbuildignore files for this.</p>
            <p> </p>
            <p> Avoid importing code within R functions, for example `require(ggplot2)` calls. Instead, document the dependencies using roxygen2, for example `@importFrom ggplot2 ggplot`.&#x00a0;</p>
            <p> </p>
            <p> To access data in a Seurat object, I highly recommend using the functions defined in Seurat for this purpose rather than accessing the slots directly. For example, use obj[[]] to access metadata rather than obj@meta.data and Idents(obj) rather than obj@active.ident</p>
            <p> </p>
            <p> In general the documentation of functions can be greatly improved. Try to include a text description of each function, a detailed description of the parameters, document the returned values, and include an executable example.</p>
            <p> </p>
            <p> It is generally not advisable to overwrite functions in base R or other packages with variable names, for example the `call` variable in `clonalDiversity` overwrites the base R `call` function.</p>
            <p> </p>
            <p> Replace code like `class(df)[1] == &#x201c;Seurat&#x201d;` with `inherits(x = df, what = "Seurat&#x201d;)`</p>
            <p> </p>
            <p> In plotting functions such as clonalOverlap, consider returning the ggplot object rather than printing the object. For example, replace `suppressWarnings(print(plot))` with `return(plot)`. This will allow users to modify the plot that is generated.&#x00a0;</p>
            <p> </p>
            <p> Some code sections are duplicated, for example L91:104 and L131:144 in seuratFunctions.R. Consider putting duplicated code into functions.</p>
            <p> </p>
            <p> Imported functions should be added to the namespace. Documenting the imports using roxygen2 (as has been done for parameters and exports) will take care of this.</p>
            <p> </p>
            <p> The highlightClonotypes function is a bit redundant with existing functions in Seurat, ie DimPlot function with the cells.highlight parameter.</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Yes</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Partly</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Yes</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Yes</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Single-cell genomics, bioinformatics, epigenomics</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
        <sub-article article-type="response" id="comment5593-60873">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Borcherding</surname>
                            <given-names>Nicholas</given-names>
                        </name>
                        <aff>University of Iowa, USA</aff>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>None</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>9</day>
                    <month>6</month>
                    <year>2020</year>
                </pub-date>
            </front-stub>
            <body>
                <p>This paper presents a new R package for the analysis of T-cell and B-cell clonotypes inferred from V(D)J recombination events. The package is designed for the analysis of data generated using the 10x Genomics V(D)J kit that allows clonotypes and transcriptomes to be co-assayed in the same individual cells.</p>
                <p> </p>
                <p> Overall the package appears to fill an unmet need in the analysis of single-cell V(D)J data, and works nicely with the existing Seurat package for single-cell analysis.</p>
                <p> </p>
                <p> 
                    <bold>While the focus of the paper is on the software, a more detailed explanation of the biology of V(D)J recombination and how this information is detected in single cells would improve the clarity of the manuscript. A description of the major existing challenges and goals of immune clonotype analysis could also be added.</bold>
                </p>
                <p> The authors agree that a more extensive explanation of VDJ biology would be a benefit, however, due to the limitations of less than 1000 words and 10 citations, we are limited by the structure of the F1000Research. We have added citations to the readme file to help users.</p>
                <p> </p>
                <p> I do not have extensive expertise in the immunology and so my review focuses on the software implementation. I leave discussion of the value and impact of the software to other reviewers. However I will reiterate comments from other reviewers in saying that there should be a distinction made between T and B cell clonotypes, due to the Ig class switching that occurs in B cells.&#x00a0;</p>
                <p> </p>
                <p> 
                    <bold>The paper also states that &#x201c;The expression data for the 12,911 cells built into the package&#x201d;. I believe the clonotype information is included in the package in the data directory, but I don&#x2019;t think the expression data is included. Including large gene expression matrix within the R package is not advisable anyway, and the authors should make it available through other means (NCBI GEO, for example).</bold>
                </p>
                <p> The seurat object with the package is available in the zenodo repository and linked in the github repository. The raw data has been deposited in GEO and more information can be found in the two preprints cited.</p>
                <p> </p>
                <p> </p>
                <p> Code review</p>
                <p> </p>
                <p> 
                    <bold>I would encourage the authors to submit the package to CRAN or another R repository. This assist in the distribution of the package, and the process of getting the package accepted on CRAN will likely greatly improve the code quality. At a minimum, the authors should try to get the package to pass R CMD check with no notes, warnings, or errors.</bold>
                </p>
                <p> Thank you for the suggestions, the authors believe this point helped the structure of the package immensely. We have used the devtools check function and as of the version in the github repository and zenodo at resubmission, there is no warnings, errors, or notes. We have begun the process of submitting the package to Bioconductor and have passed the build phase of the process.</p>
                <p> </p>
                <p> 
                    <bold>Currently the package includes all rendered figures in the vignettes folder. I would recommend removing these files, and also removing the `ggsave` function calls within the vignette to prevent these files being written while compiling the vignette. This will reduce the size of the package.</bold>
                </p>
                <p> Thank you for the suggestion, this has been completed.</p>
                <p> </p>
                <p> 
                    <bold>Several functions in the package use scoping assignment to assign variables in the global environment. This is considered bad practice and should be avoided in all cases.</bold>
                </p>
                <p> Scoping assignments in these instances are to give users the options of exporting the tables or matrices that form the visualization. We have modified the activity of the exportTable parameter to return a data frame used if TRUE or visualization if FALSE (default).</p>
                <p> </p>
                <p> 
                    <bold>Extra files such as .DS_Store should be removed from the git repository and from the package. Use the .gitignore and .Rbuildignore files for this.</bold>
                </p>
                <p> Thank you for the suggestion, we have removed .DS_Store.</p>
                <p> </p>
                <p> 
                    <bold>Avoid importing code within R functions, for example `require(ggplot2)` calls. Instead, document the dependencies using roxygen2, for example `@importFrom ggplot2 ggplot`.</bold>&#x00a0;</p>
                <p> Thank you for the suggestion, we have made this change.</p>
                <p> </p>
                <p> 
                    <bold>To access data in a Seurat object, I highly recommend using the functions defined in Seurat for this purpose rather than accessing the slots directly. For example, use obj[[]] to access metadata rather than obj@meta.data and Idents(obj) rather than </bold>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident">
                        <bold>obj@active.ident</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Thanks for the suggestion, we have changed this in the new version of the R package.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>In general the documentation of functions can be greatly improved. Try to include a text description of each function, a detailed description of the parameters, document the returned values, and include an executable example.</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Both reviewers have made this excellent suggestion, the new version of scRepertoire has more comprehensive documentation, including descriptions and working examples.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>It is generally not advisable to overwrite functions in base R or other packages with variable names, for example the `call` variable in `clonalDiversity` overwrites the base R `call` function.</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Thank you for the suggestion, we have modified this for all functions.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>Replace code like `class(df)[1] == &#x201c;Seurat&#x201d;` with `inherits(x = df, what = "Seurat&#x201d;)`</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> We have made this change and also replace all class() evaluations.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>In plotting functions such as clonalOverlap, consider returning the ggplot object rather than printing the object. For example, replace `suppressWarnings(print(plot))` with `return(plot)`. This will allow users to modify the plot that is generated.&#x00a0;</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Excellent suggestion and this has been fixed.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>Some code sections are duplicated, for example L91:104 and L131:144 in seuratFunctions.R. Consider putting duplicated code into functions.</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Thank you for the suggestion, part of the change in the &#x201c;call&#x201d; to cloneCall variable in the functions also including creating a function to deal with the user selection. These lines were consistently repeated across multiple functions, but now have been reduced. We have also extensively modularized the code and added these to utils.R .</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>Imported functions should be added to the namespace. Documenting the imports using roxygen2 (as has been done for parameters and exports) will take care of this.</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> This has been completed.</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> 
                        <bold>The highlightClonotypes function is a bit redundant with existing functions in Seurat, ie DimPlot function with the cells.highlight parameter.</bold>
                    </ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> &#x00a0;</ext-link>
                </p>
                <p>
                    <ext-link ext-link-type="uri" xlink:href="mailto:obj@active.ident"> Agreed, although redundant, this was a request from an early beta tester, we plan to keep the function in.</ext-link>
                </p>
            </body>
        </sub-article>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report59209">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.24415.r59209</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Pagani</surname>
                        <given-names>Massimiliano</given-names>
                    </name>
                    <xref ref-type="aff" rid="r59209a2">2</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-7017-9304</uri>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Drufuca</surname>
                        <given-names>Lorenzo</given-names>
                    </name>
                    <xref ref-type="aff" rid="r59209a1">1</xref>
                    <role>Co-referee</role>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Bonnal</surname>
                        <given-names>Raoul Jean Pierre</given-names>
                    </name>
                    <xref ref-type="aff" rid="r59209a2">2</xref>
                    <role>Co-referee</role>
                </contrib>
                <aff id="r59209a1">
                    <label>1</label>School of Medicine and Surgery, Universit&#x00e0; degli Studi di Milano Bicocca, Milan, Italy</aff>
                <aff id="r59209a2">
                    <label>2</label>National Institute of Molecular Genetics, Milan, Italy</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>3</day>
                <month>3</month>
                <year>2020</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2020 Drufuca L et al.</copyright-statement>
                <copyright-year>2020</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport59209" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.22139.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The paper describes a new package (scRepertoire) of functions for the analysis of clonality in single cell experiments in the R analytical environment.</p>
            <p> </p>
            <p> While it is true, as it is claimed, that 'a gap exists in the processing of V(D)J sequencing, descriptive statistics, clonal comparisons, and repertoire diversity with the current SCS R packages', nonetheless few packages already exist for the analysis of clonality in bulk samples (such as the inspiring package tcR). From the text it is not immediately clear why there is the need for a brand new package rather than a simpler 'wrapper' to bridge the gap between single cell based data and existing analytical packages.</p>
            <p> </p>
            <p> Although package functionalities are clearly illustrated in the paper, software documentation could be improved: behaviour of some functions is not immediately clear without studying the code directly (e.g. the way multi-chains clonotypes are treated by combineContigs()).</p>
            <p> </p>
            <p> Similar to its inspiring package (tcR) and leveraging on the vegan package, scRepertoire implements different measures for diversity. Nonetheless, it has been suggested that conventional diversity measures fail to describe properly clonotype distributions and several complementary methods have been proposed like the Recon&#x00a0;package by Kaplinsky
                <sup>
                    <xref ref-type="bibr" rid="rep-ref-59209-1">1</xref>
                </sup> and Arnaout
                <sup>
                    <xref ref-type="bibr" rid="rep-ref-59209-1">1</xref>
                </sup> or Startrac&#x00a0;package by Zhang and colleagues
                <sup>
                    <xref ref-type="bibr" rid="rep-ref-59209-2">2</xref>
                </sup>. Interaction with such packages would greatly increase scRepertoire analytical effectiveness.</p>
            <p> </p>
            <p> Similarly, the function abundanceContig(), especially in the unscaled form, is of little use per se. Interestingly, recent methods have been proposed to for the comparative analysis of clone size distributions
                <sup>
                    <xref ref-type="bibr" rid="rep-ref-59209-3">3</xref>
                </sup>&#x00a0;and could be easily incorporated into scRepertoire adding considerable power to it (though assessment of required numerosity should be introduced).</p>
            <p> </p>
            <p> The package claims to be designed both for TCR and BCR analysis but definition of clonality in B cells is slightly different than in T cells due to isotype switch and somatic hypermutation phenomena following activation. Therefore clonotype identity between two cells should be defined differently between BCR and TCR analysis.</p>
            <p> </p>
            <p> Concerning overlap, beside the nice representation as a heatmap, it could be useful to have the chance to output the matrix itself rather than the plot only.</p>
            <p> </p>
            <p> 10x vdj methods occasionally fails to reconstruct complete clonotypes or it reconstructs putatively aberrant clonotypes (clonotypes with multiple beta chains). Currently scRepertoire does not allow to filter for specific chain compositions but such feature would be worth adding, together with a graphical visualization of relative frequencies of chain composition across clonotypes.</p>
            <p> </p>
            <p> Paired 10x gene expression profile and vdj scoped analysis are not guaranteed to reconstruct the information for the exact same pool of barcodes, thus the combineSeurat() function could be improved by allowing to specify whether an inner or Seurat-sided joining is to be performed and ensuring that the joining is performed correctly.</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Yes</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Partly</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Partly</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Yes</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Integrative Biology, Cancer Immunology</p>
            <p>We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.</p>
        </body>
        <back>
            <ref-list>
                <title>References</title>
                <ref id="rep-ref-59209-1">
                    <label>1</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Robust estimates of overall immune-repertoire diversity from high-throughput measurements on samples</article-title>.
                        <source>
                            <italic>Nature Communications</italic>
                        </source>.<year>2016</year>;<volume>7</volume>(<issue>1</issue>) :
                        <elocation-id>10.1038/ncomms11881</elocation-id>
                        <pub-id pub-id-type="doi">10.1038/ncomms11881</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-59209-2">
                    <label>2</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Lineage tracking reveals dynamic relationships of T cells in colorectal cancer.</article-title>
                        <source>
                            <italic>Nature</italic>
                        </source>.<volume>564</volume>(<issue>7735</issue>) :
                        <elocation-id>10.1038/s41586-018-0694-x</elocation-id>
                        <fpage>268</fpage>-<lpage>272</lpage>
                        <pub-id pub-id-type="pmid">30479382</pub-id>
                        <pub-id pub-id-type="doi">10.1038/s41586-018-0694-x</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-59209-3">
                    <label>3</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>powerTCR: A model-based approach to comparative analysis of the clone size distribution of the T cell receptor repertoire.</article-title>
                        <source>
                            <italic>PLoS Comput Biol</italic>
                        </source>.<volume>14</volume>(<issue>11</issue>) :
                        <elocation-id>10.1371/journal.pcbi.1006571</elocation-id>
                        <fpage>e1006571</fpage>
                        <pub-id pub-id-type="pmid">30485278</pub-id>
                        <pub-id pub-id-type="doi">10.1371/journal.pcbi.1006571</pub-id>
                    </mixed-citation>
                </ref>
            </ref-list>
        </back>
        <sub-article article-type="response" id="comment5592-59209">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Borcherding</surname>
                            <given-names>Nicholas</given-names>
                        </name>
                        <aff>University of Iowa, USA</aff>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>None</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>9</day>
                    <month>6</month>
                    <year>2020</year>
                </pub-date>
            </front-stub>
            <body>
                <p>The paper describes a new package (scRepertoire) of functions for the analysis of clonality in single cell experiments in the R analytical environment.</p>
                <p> </p>
                <p> 
                    <bold>While it is true, as it is claimed, that 'a gap exists in the processing of V(D)J sequencing, descriptive statistics, clonal comparisons, and repertoire diversity with the current SCS R packages', nonetheless few packages already exist for the analysis of clonality in bulk samples (such as the inspiring package tcR). From the text it is not immediately clear why there is the need for a brand new package rather than a simpler 'wrapper' to bridge the gap between single cell based data and existing analytical packages.</bold>
                </p>
                <p> Thank you for your input. To clarify, indeed a gap exists. The vast majority of packages available for V(D)J single-cell analysis is python-based reconstructions of TCR/BCR sequences (see, scTCRseq , TRAPeS and TraCeR). The tcR package that the reviewer indicates is in fact deprecated and only functional for bulk sequencing, there is no barcoded-based method. However, the authors of tcR have a newer R package, immunarch, that is freely available and produced by the company immunomind. This newer package offers single-cell support, but the workflow within the package does not allow for the combination of both TCR/BCR chains, and will call clonotypes with a single chain. In addition, as the reviewers keenly mention in the accompanying comments, single-cell technologies have unique pitfalls that need to be addressed as opposed to just creating wrapper functions. These concepts have been added to a newer version of the manuscript.</p>
                <p> </p>
                <p> 
                    <bold>Although package functionalities are clearly illustrated in the paper, software documentation could be improved: behaviour of some functions is not immediately clear without studying the code directly (e.g. the way multi-chains clonotypes are treated by combineContigs()).</bold>
                </p>
                <p> We have added extensive documentation to the package.</p>
                <p> </p>
                <p> 
                    <bold>Similar to its inspiring package (tcR) and leveraging on the vegan package, scRepertoire implements different measures for diversity. Nonetheless, it has been suggested that conventional diversity measures fail to describe properly clonotype distributions and several complementary methods have been proposed like the Recon&#x00a0;package by Kaplinsky&#x00a0;and Arnaout</bold> 
                    <bold>or Startrac&#x00a0;package by Zhang and colleagues. Interaction with such packages would greatly increase scRepertoire analytical effectiveness.</bold>
                </p>
                <p> We have added the function StartracDiversity() to the development version of the package to allow for the migr, exp, and tans indices to be calculated from the Seurat object after the addition of the clonotype information. This function is detailed in the vignette due to limited word count of this article.</p>
                <p> </p>
                <p> 
                    <bold>Similarly, the function abundanceContig(), especially in the unscaled form, is of little use per se. Interestingly, recent methods have been proposed to for the comparative analysis of clone size distributions and could be easily incorporated into scRepertoire adding considerable power to it (though assessment of required numerosity should be introduced).</bold>
                </p>
                <p> We have added the function clonesizeDistribution() to allow users to examine the hierarchical&#x00a0; clustering based on clone size distributions of samples to the scRepertoire package along with a comprehensive description and covered the usage in the vignette accompanying the package. Due to the length limitation for the manuscript, we only reference the function in the manuscript with the appropriate citation.</p>
                <p> </p>
                <p> 
                    <bold>The package claims to be designed both for TCR and BCR analysis but definition of clonality in B cells is slightly different than in T cells due to isotype switch and somatic hypermutation phenomena following activation. Therefore clonotype identity between two cells should be defined differently between BCR and TCR analysis.</bold>
                </p>
                <p> We agree with the need to add thresholding for clonotype assignment for the BCR in the context of somatic hypermutation. We have separated the combineContig() function into combineTCR() and combineBCR() functions, the latter organizes the data similarly, but defines nucleotide sequences by the normalized Hamming Distance, with sequences greater than &gt;0.85 assigned to the same group. In the strict definition of the clonotypes in the combineBCR() the vgene is also added to both the light and heavy chains. We have kept the ability to visualize and analyze clonotypes at the amino acid, nucleotide sequence and gene level as well, to all users more choices.</p>
                <p> </p>
                <p> 
                    <bold>Concerning overlap, beside the nice representation as a heatmap, it could be useful to have the chance to output the matrix itself rather than the plot only.</bold>
                </p>
                <p> When appropriate, the visualization functions have the exportTable variable that allows a user to get the table or matrix that enables the visualization. Although this feature is already present, we will add discussion to the manuscript and accompanying vignette.</p>
                <p> </p>
                <p> 
                    <bold>10x vdj methods occasionally fails to reconstruct complete clonotypes or it reconstructs putatively aberrant clonotypes (clonotypes with multiple beta chains). Currently scRepertoire does not allow to filter for specific chain compositions but such feature would be worth adding, together with a graphical visualization of relative frequencies of chain composition across clonotypes.</bold>
                </p>
                <p> If the authors understand the suggestion, the reviewers are asking for additional filtering options for combineTCR() and combineBCR(), two separate functions created from the reviewers earlier points. The function has the ability to remove barcodes with NA values or with multiple chains. More specifically to the combineTCR() function we have added removeMulti (removes all barcodes with &gt;= 3 chains) and filterMulti (selects the 2 highest expressing chains). In terms of the chain filtering and visualizations, we are more than happy to add additional filtering and visualization tools for users. If the reviewer could clarify more on the how they envisioned the chain composition visualization, we would appreciate more insight.</p>
                <p> </p>
                <p> </p>
                <p> 
                    <bold>Paired 10x gene expression profile and vdj scoped analysis are not guaranteed to reconstruct the information for the exact same pool of barcodes, thus the combineSeurat() function could be improved by allowing to specify whether an inner or Seurat-sided joining is to be performed and ensuring that the joining is performed correctly.</bold>
                </p>
                <p> We have added the variable NA into the new combineBCR() and combineTCR(), which functions by removing cells without clonotype information. If we understand the reviewer correctly, the differential joining function would also need to address the seurat object as a whole instead of just merging the meta.data.</p>
            </body>
        </sub-article>
    </sub-article>
</article>
