<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.164583.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Software Tool Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Assembly and quantification of transcripts from noisy long reads with NIFFLR</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 1 approved with reservations, 2 not approved]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Guo</surname>
                        <given-names>Alina</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-6664-0145</uri>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Pertea</surname>
                        <given-names>Mihaela</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-0762-8637</uri>
                    <xref ref-type="aff" rid="a1">1</xref>
                    <xref ref-type="aff" rid="a2">2</xref>
                    <xref ref-type="aff" rid="a3">3</xref>
                </contrib>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Zimin</surname>
                        <given-names>Aleksey V</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Resources</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Visualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-5091-3092</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Center for Computational Biology, Johns Hopkins University, Baltimore, MD, 21205, USA</aff>
                <aff id="a2">
                    <label>2</label>Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland, 21205, USA</aff>
                <aff id="a3">
                    <label>3</label>Department of Computer Science, Johns Hopkins University, Baltimore, Maryland, 21205, USA</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:aleksey.zimin@gmail.com">aleksey.zimin@gmail.com</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>20</day>
                <month>6</month>
                <year>2025</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2025</year>
            </pub-date>
            <volume>14</volume>
            <elocation-id>608</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>13</day>
                    <month>6</month>
                    <year>2025</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Guo A et al.</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/14-608/pdf"/>
            <abstract>
                <sec>
                    <title>Background</title>
                    <p>Long-read RNA sequencing technologies can produce complete or near-complete transcript sequences. Recently introduced methods for direct RNA and cDNA sequencing can provide a high-throughput strategy for the discovery of novel and rare gene isoforms. However, the high error rates in ONT sequences limit the ability to exactly pinpoint splice site boundaries when aligning reads to the genome.</p>
                </sec>
                <sec>
                    <title>Methods</title>
                    <p>In this paper, we present a novel tool called NIFFLR (Novel IsoForm Finder using Long Reads) that identifies and quantifies both known and novel isoforms using long-read RNA sequencing data. NIFFLR recovers known transcripts and assembles novel transcripts present in the data by aligning exons from a reference annotation to the long reads.</p>
                </sec>
                <sec>
                    <title>Results</title>
                    <p>NIFFLR effectively recovers correct transcripts from simulated reads based on known transcript annotations, achieving higher sensitivity and precision compared to several previously-published tools. On real data, NIFFLR shows the high accuracy as measured by concordance of isoform counts to the counts computed from Illumina data for the same sample. We applied NIFFLR to a set of 92 GTEx long-read samples and produced transcript counts for both novel and known isoforms. In total, we identified and quantified 121,155 isoforms present in the RefSeq annotation of GRCh38 and 106,667 high-confidence novel isoforms across 32,875 genes present in two or more samples in these data, more than previous studies identified in this data set.</p>
                </sec>
                <sec>
                    <title>Conclusions</title>
                    <p>NIFFLR is an effective tool aimed at assembly and quantification of transcripts present in the long high error transcriptome reads. NIFFLR is released under an open-source license (GPL 3.0) and is available on GitHub at 
                        <uri xlink:href="https://github.com/alguoo314/NIFFLR/releases">https://github.com/alguoo314/NIFFLR/releases</uri>.</p>
                </sec>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>transcriptome</kwd>
                <kwd>quantification</kwd>
                <kwd>assembly</kwd>
                <kwd>discovery</kwd>
                <kwd>annotation</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="https://doi.org/10.13039/100000076">
                    <funding-source>Directorate for Biological Sciences</funding-source>
                    <award-id>IOS-2432298</award-id>
                </award-group>
                <award-group id="fund-2">
                    <funding-source>NIH</funding-source>
                    <award-id>R35-GM130151</award-id>
                </award-group>
                <award-group id="fund-3">
                    <funding-source>NIH</funding-source>
                    <award-id>R01-HG006677</award-id>
                </award-group>
                <funding-statement>This work was supported by National Science Foundation grant IOS-2432298 to Johns Hopkins University (PI Zimin, Co-PI Salzberg), and by National Institutes of Health grants to Johns Hopkins University R01-HG006677 (PI Salzberg) and R35-GM130151 (PI Salzberg). Zimin is a member of the Salzberg lab at JHU. </funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec id="sec5" sec-type="intro">
            <title>Introduction</title>
            <p>Direct RNA and cDNA sequencing technologies from Oxford Nanopore Technologies (ONT) produce long transcriptome reads with high yields at relatively low cost. However, the per-base error rates of ONT reads are still much higher than those of Illumina reads. Several computational tools have recently been developed to assemble transcripts and quantify isoforms in samples sequenced using ONT reads, including FLAIR (
                <xref ref-type="bibr" rid="ref4">Tang AD et al., 2020</xref>), ESPRESSO (
                <xref ref-type="bibr" rid="ref1">Gao Y et al., 2023</xref>), and IsoQuant (
                <xref ref-type="bibr" rid="ref3">Prjibelski AD et al., 2023</xref>). All these tools begin by mapping the long reads to the genome using the Minimap2 (
                <xref ref-type="bibr" rid="ref2">Li H, 2018</xref>) aligner in spliced alignment mode. However, the high error rate of ONT reads makes it challenging to precisely identify splice sites through spliced alignment alone. Therefore, these tools incorporate additional information to locate the splice sites accurately. FLAIR can correctly identify splice sites by either using alignments of short-read RNA-seq data or by using a reference annotation. ESPRESSO accepts novel splice junctions only if at least one read aligns perfectly to the reference genome within 10 nucleotides (nt) of the splice site, a stringent criterion that limits its ability to discover novel junctions. IsoQuant replaces novel splice sites with nearby annotated sites within a user-defined distance and restores short, skipped exons according to the reference annotation. For all these programs, misalignments can lead to incorrect identification of splice junctions, which may subsequently result in inaccurate transcript reconstruction.</p>
            <p>Here, we present NIFFLR (Novel IsoForm Finder using Long Reads), a tool designed to construct and quantify both annotated and novel isoforms using a reference annotation and long RNA sequencing reads. Unlike other isoform identification tools, NIFFLR does not rely on a spliced aligner to map reads onto the reference genome. Instead, it extracts exons from the given annotation and aligns them directly to the long reads. NIFFLR then constructs transcripts by identifying an optimal path through the mapped exons for each long read, removes redundant transcripts that are contained within others, filters out transcripts with low read support, compares the predicted transcripts to the reference annotation, and finally quantifies both annotated and novel isoforms. For efficient exon-to-read alignment, NIFFLR uses a custom aligner based on a partial suffix array adapted from the MaSuRCA assembler (
                <xref ref-type="bibr" rid="ref5">Zimin et al., 2013</xref>).</p>
        </sec>
        <sec id="sec6" sec-type="methods">
            <title>Methods</title>
            <sec id="sec7">
                <title>Implementation</title>
                <p>We designed the NIFFLR algorithm to build transcripts (i.e., sequences of exons) by computing the optimal tiling of every long read using exons and transcripts provided as input. We require the following inputs: long RNA sequencing reads in FASTQ format, a reference genome sequence file in FASTA format, and a reference annotation file in GTF format.</p>
                <p>First, we extract the exon sequences from the reference genome using the annotation and output them into a FASTA file. The name of each exon encodes the chromosome name, start and end position on the chromosome, the name of the gene to which the exon belongs, and its orientation. We reverse complement all exon sequences that are on the reverse strand.</p>
                <p>We then use a version of a technique first utilized in the MaSuRCA assembler (
                    <xref ref-type="bibr" rid="ref5">Zimin et al., 2013</xref>) to efficiently compute approximate alignments of exons to the long reads. This alignment technique, which we refer to as psa_aligner, is based on a partial suffix array (PSA). The PSA is designed to efficiently compute approximate alignments, or alignment intervals between two sets of DNA sequences. The psa_aligner first builds a partial suffix array from a concatenated string S containing the sequences of all exons, separated by the letter &#x2018;N&#x2019; (note that no &#x2018;N&#x2019; characters are allowed in the reference sequence). We also record the starting position of each exon in S. Unlike a traditional suffix array, the PSA limits the suffix size to a predefined value K. The suffix array allows us to quickly locate all occurrences of a given subsequence of length K (or a K-mer) within S, and thus identify all exons and positions where a particular K-mer occurs. We then examine each K-mer in a given long read and compute all the longest common sub-sequences (LCS) of K-mers between the read and the exons, using a default value of K = 12. The approximate alignment coordinates are then determined by calculating the best linear fit between the positions of K-mers belonging to the LCS in the read and on the exon. We only retain alignments where matching K-mers cover at least 35% of the bases within the match interval. Each alignment provides alignment start and end positions, along with the exon and read overhangs, as shown in 
                    <xref ref-type="fig" rid="f1">
Figure 1</xref>. For each exon, we record the number of K-mers in the LCS, the alignment start and end positions, and the implied start and end on the read. The implied start is calculated as alignment_start-a_overhang, and the implied alignment end is alignment_stop+b_overhang.</p>
                <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                    <label>
Figure 1. </label>
                    <caption>
                        <title>Definitions of alignment coordinates.</title>
                    </caption>
                    <graphic id="gr1" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/181114/23262718-c4b2-4221-9835-2cf8f451a741_figure1.gif"/>
                </fig>
                <p>After building the alignments, we assign each long read to a gene locus using a &#x201c;majority vote&#x201d; approach. Specifically, for each read, we compute the total number of K-mers in all LCSs for all matching exons from different gene loci and assign the read to the locus L whose exons have the highest total number of matching K-mers. Alignments of any exons that belong to different gene loci are then discarded. Next, we build the transcript matching the read by finding the best tiling of read using exons that belong to locus L. The best sequence maximizes coverage of the read while minimizing gaps or overlaps in the implied alignment coordinates. The long read defines a 5&#x2019; to 3&#x2019; forward direction, specifying a topological order. We sort the aligning exons in the order of their &#x201c;alignment start&#x201d; coordinates if aligned in the forward direction, or &#x201c;alignment end&#x201d; coordinates if aligned in the reverse direction. Since we only kept alignments of exons that all belong to a single gene locus L, the exons must all align either in forward or reverse direction. For simplicity, below we describe the algorithm assuming all exons are aligned in the forward direction; the reverse case is treated the same, by reversing the long read.</p>
                <p>We represent the exon tiling problem as a graph, where nodes represent exons and edges are defined by gaps or overlaps of 20 bases or less between the implied end of an exon and the implied start of the following exon in the topological order. Next, we choose the &#x201c;starting&#x201d; nodes that are not connected on the left. A starting node must be connected on the right and have an &#x201c;alignment start&#x201d; closest to the 5&#x2019; end of the long read or fully cover it. If multiple exons share the same &#x201c;alignment start&#x201d; coordinate due to alternative splicing, we select the exon with the smallest 5&#x2019; &#x201c;overhang&#x201d;. If the 5&#x2019; overhang is the same for more than one exon, we use all such exons as alternative start nodes. We solve the exon tiling problem by finding the longest path through the graph, starting from any start node that minimizes the penalty, defined as the average gap/overlap size between connected exons in the path. In case of a tie, we select the path that maximizes the sum of exon matching lengths minus the sum of the overhangs of the first and last exons. 
                    <xref ref-type="fig" rid="f2">
Figure 2</xref> illustrates an example of such a path. Once the longest path is identified, we examine the genomic coordinates of the exons, which are encoded in their sequence IDs. We eliminate the path if there is an overlap between the genomic coordinates of the exons in the path, which could indicate that the long read is chimeric or that there is a significant local genome rearrangement that NIFFLR cannot handle.</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>
Figure 2. </label>
                    <caption>
                        <title>An illustration of the optimal path of exons through a long transcriptomic read (shown in green).</title>
                        <p>Shading shows the alignment regions. Arrows indicate links. The best path shown in red is the longest path that minimizes the gap/overlap/overhang penalty. Exon1 is chosen as the start exon because exon1+ exon3 have a longer alignment than exon2. Exon5 is alternatively spliced compared to exon6 and exon7, and its longest match is the same as exon6&#x2019;s, shorter than exon6 and exon7 combined, and hence not selected for the optimal path. Exon2 is alternatively spliced as well.</p>
                    </caption>
                    <graphic id="gr2" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/181114/23262718-c4b2-4221-9835-2cf8f451a741_figure2.gif"/>
                </fig>
                <p>We convert the best path of exons for each read into a plausible transcript and then group reads that yield the same transcript. For each transcript, we record the reads contributing to it, along with the minimum of the average gap/overlap penalty (A
                    <sub>min</sub>) and the minimum of the maximum gap/overlap penalty (G
                    <sub>min</sub>) across all paths of reads that yielded the transcript. In subsequent steps, we use only those transcripts where A
                    <sub>min</sub> &lt; 5 and G
                    <sub>min</sub> &lt; 15. These values are empirically obtained parameters and they yielded the best performance in our experiments with simulated reads.</p>
                <p>We then use the GffCompare tool to create a set of maximal transcripts by removing those whose intron chains are contained in longer assembled transcripts. We call this set of transcripts &#x201c;non-redundant&#x201d;. Next, we perform the first round of transcript quantification, using all originally assembled transcripts to assign reads to the non-redundant transcripts based on containment. Reads from assembled transcripts, which are contained in multiple maximal transcripts, are distributed proportionally to the size of the container transcripts. For each maximal transcript, we calculate the following:
                    <list list-type="order">
                        <list-item>
                            <label>1.</label>
                            <p>The number of reads supporting the transcript.</p>
                        </list-item>
                        <list-item>
                            <label>2.</label>
                            <p>The minimum read coverage across all intron junctions.</p>
                        </list-item>
                        <list-item>
                            <label>3.</label>
                            <p>The total number of junctions covered by at least one read.</p>
                        </list-item>
                        <list-item>
                            <label>4.</label>
                            <p>The portion of the transcript covered by reads.</p>
                        </list-item>
                    </list>
                </p>
                <p>By design, all intron junctions are covered in the maximal set. After quantification, we perform a transcript recovery step where we attempt to recover reference annotation transcripts that are likely present in the sample, but their intron chains are not completely covered by any long read. If a maximal transcript is contained within a reference transcript, we tentatively replace the contained transcript with the containing transcript from the reference. We then perform quantification again and eliminate multi-exon reference transcripts where none of the intron junctions are spanned by long reads, which means that only one exon had reads aligned to it. These reference transcripts are unlikely to be present in the sample. This procedure is designed to eliminate computed isoforms whose intron chains are contained in the reference transcripts, as these are unlikely to represent genuinely novel isoforms and are likely sequenced from known transcripts. Next, we identify novel transcripts (i.e., those not present in the reference) and apply stricter filtering criteria, requiring the minimum average gap in the exon paths to be less than 2 and the minimum of the maximum gap to be less than 5. This yields the final set of transcripts, containing both novel and known transcripts, which we then again quantify to produce the final set of quantified transcripts.</p>
            </sec>
            <sec id="sec8">
                <title>Operation</title>
                <p>NIFFLR is designed to run under 64-bit Linux operating system. NIFFLR requires at least 16Gb of RAM and supports multi-core multi-threaded hardware environment. NIFFLR code consists of shell and Python scipts and C++ code. We provide installation instructions for NIFFLR on github: 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/alguoo314/NIFFLR">https://github.com/alguoo314/NIFFLR</ext-link>. Basic usage of NIFFLR is as follows: /path/nifflr.sh -r genome.fasta -f reads.fastq -g genome.gtf.</p>
            </sec>
        </sec>
        <sec id="sec9" sec-type="results">
            <title>Results</title>
            <p>In this section, we compare NIFFLR to other similar methods such as FLAIR2, IsoQuant, and ESPRESSO, and discuss the results of applying NIFFLR to ONT data from the Genotype-Tissue Expression (GTEx) project (
                <xref ref-type="bibr" rid="ref6">Glinos et al., 2022</xref>). We performed two evaluations to compare NIFFLR to the existing methods. First, we assessed the performance of each program on a set of simulated ONT direct RNA sequencing reads. Next, we tested all programs on a sample from the GTEx project that was sequenced using both Illumina and ONT technologies.</p>
            <sec id="sec10">
                <title>Comparison on simulated long reads</title>
                <p>We simulated reads using NanoSim software (
                    <xref ref-type="bibr" rid="ref7">Yang et al., 2017</xref>) from the human reference genome GRCh38.p14 and its corresponding RefSeq genome annotation (RS_2024_08). We derived read error profiles from ONT reads of GTEx sample 1192X, which was sequenced with both Illumina RNA-seq and ONT technologies. We used the Illumina reads from the same sample to generate an expression profile for the simulation. Our simulated data set contained approximately 7.8 million reads with an average error rate of 8.7% and an N50 read length of 944 bp. According to Nanosim output, the simulated set had 50,748 unique transcripts expressed.</p>
                <p>All programs in this comparison allow the use of a reference annotation to identify and correct splice junctions, and we provided such annotation in all our experiments. Note that FLAIR and IsoQuant have options allowing them to run without annotation, but their accuracy is higher if annotation is provided. To make the evaluation more realistic, we split the reference annotation into a &#x201c;core&#x201d; set of transcripts, which is the set with the smallest number of transcripts where each exon was present at least once (referred to as the 
                    <italic toggle="yes">known</italic> set), and the rest of the transcripts (referred to as the 
                    <italic toggle="yes">novel</italic> set). By design, the core set contained every reference donor and acceptor splice site at least once. We provided the core set but not the novel set to all programs. This way we ensured that some portion of the expressed transcripts were not present in the input set of the reference transcripts, enabling us to measure the programs&#x2019; ability to discover and quantify novel transcripts in addition to the known transcripts. Our simulated set consisted of reads simulated from 50,748 transcripts, of which 33,686 comprised the core set and the remaining 17,062 comprised the novel set. In our experiments, we measured the number of novel and known transcripts correctly recovered by the programs, as well as the number of false positive transcripts, using the GffCompare tool (
                    <xref ref-type="bibr" rid="ref9">Pertea &amp; Pertea, 2020</xref>). False positives were defined as any transcripts output by the programs that did not have a complete intron chain match to a transcript in the known or novel set. 
                    <xref ref-type="table" rid="T1">
Table 1</xref> shows the comparison of the programs on the simulated data. NIFFLR has the best sensitivity in recovering known, novel, and all isoforms, and the best overall F1 score, while only losing to IsoQuant in precision. NIFFLR recovers the most isoforms from both the known and novel sets while keeping the number of spurious isoforms relatively low. This result demonstrates that when novel isoform discovery and quantification are the primary goals, NIFFLR is the best tool.</p>
                <table-wrap id="T1" orientation="portrait" position="float">
                    <label>
Table 1. </label>
                    <caption>
                        <title>Performance of the assembly and quantification pipelines on simulated data.</title>
                        <p>The best values are in bold. NIFFLR recovers the most novel isoforms and the most isoforms total (32,711) while keeping the number of erroneous isoforms lower than FLAIR2 and ESPRESSO, resulting in the best sensitivity and F1 score for isoform recovery. Isoquant is the most conservative and the least sensitive, both on novel and known isoform discovery.</p>
                    </caption>
                    <table content-type="article-table" frame="hsides">
                        <thead>
                            <tr>
                                <th align="left" colspan="1" rowspan="1" valign="top"/>
                                <th align="left" colspan="1" rowspan="1" valign="top"># of novel isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Sn for novel isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top"># of known isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Sn for known isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top"># of all correct isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Sn for all isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Pr for all isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">F1 for all isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">
# of spurious isoforms</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">All simulated transcripts</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">17062</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">100.0%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">33686</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">100.0%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">50748</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">100.0%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">100.0%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">100</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">0</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">FLAIR2</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">4988</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">29.2%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">15529</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">46.1%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">20517</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">40.4%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">54.8%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">46.5</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">41777</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">IsoQuant</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">1926</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">11.3%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">19629</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">58.3%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">21555</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">42.5%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>98.1%</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">59.3</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>964</bold>
</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">NIFFLR</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>5153</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>30.2%</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>27558</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>81.8%</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>32711</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>64.5%</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">73.5%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">
                                    <bold>68.7</bold>
</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">7961</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">ESPRESSO</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">1490</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">8.7%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">20750</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">61.6%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">22240</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">43.8%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">67.7%</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">53.2</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">24198</td>
                            </tr>
                        </tbody>
                    </table>
                </table-wrap>
                <p>We compared the read counts computed by each program for every transcript to the actual counts from the simulation. 
                    <xref ref-type="fig" rid="f3">
Figure 3a</xref> presents box-and-whisker plots of the ratios (expressed as base-2-logarithms) of the actual and computed counts for each transcript. The box spans the upper and lower quartile of the ratios and the whiskers represent the range for 95% of the values, with individual outliers outside of the 95% interval shown as dots. NIFFLR has a tighter distribution than FLAIR and ESPRESSO, though it is slightly outperformed by IsoQuant. ESPRESSO shows the worst overall performance, both in terms of the distribution&#x2019;s tightness and bias. 
                    <xref ref-type="fig" rid="f3">
Figure 3b</xref> shows a more detailed comparison of the ratios between the computed counts from NIFFLR and IsoQuant, compared to the actual counts for the subset of 18,686 isoforms quantified by both tools. We observe that in this comparison the accuracy is nearly identical, with NIFFLR counts showing less overall bias. This figure suggests that the reason for the slightly lower accuracy (wider whiskers) of NIFFLR compared to IsoQuant in panel (a) is the inclusion of counts for many more transcripts by NIFFLR, capturing less reliable lower-count transcripts, which IsoQuant discards. In the simulated data comparison, NIFFLR demonstrates superior quantification accuracy and sensitivity overall.</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>
Figure 3. </label>
                    <caption>
                        <title>

                            <bold>(a)</bold> Box and whisker plots of the log2 ratios (y-axis) of the actual and computed read counts for each transcript for simulated reads.</title>
                        <p>The box spans the upper and lower quartile of the log2 ratios, and the whiskers represent 95% of the values, with individual outliers outside of the 95% interval shown as dots. IsoQuant and NIFFLR show the least variation from the true counts in the simulated data. 
                            <bold>

                                <bold>(b)</bold> Box and whisker plots of the log2 ratios of the actual and computed read counts for each transcript from the set of 18,686 simulated transcripts quantified by both NIFFLR and IsoQuant.</bold> IsoQuant and NIFFLR show the same accuracy (the height of the box and whiskers are the same size) on this set of transcripts, however, NIFFLR counts have smaller bias (the mean and the median for NIFFLR are closer to zero) and fewer outliers.</p>
                    </caption>
                    <graphic id="gr3" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/181114/23262718-c4b2-4221-9835-2cf8f451a741_figure3.gif"/>
                </fig>
            </sec>
            <sec id="sec11">
                <title>Comparison on a real data sample sequenced with both Illumina and ONT technologies</title>
                <p>For this experiment, we selected the GTEX-1192X sample, which was sequenced with both Illumina and Oxford Nanopore instruments. The ONT data contained 7.6 million long reads with an N50 of 872 bp and a total sequence of 5.3 Gbps. In this dataset, the exact expression of existing and novel transcripts is unknown. However, we can estimate the number and abundances of the known transcripts from the Illumina RNA-seq data, which provides much deeper coverage of the sample. We used StringTie2 (
                    <xref ref-type="bibr" rid="ref8">Kovaka et al., 2019</xref>) in reference-guided mode to assemble the Illumina data, and this yielded 51,909 distinct transcript variants. The reference-guided mode of StringTie does not output any novel isoforms. 
                    <xref ref-type="table" rid="T2">
Table 2</xref> shows the number of total isoforms and known isoforms found by the four long-read quantification programs when using the ONT data. NIFFLR identified and quantified 43,093 transcripts that matched the reference, which was more than twice as many as any of the other pipelines. To evaluate the accuracy of the quantification, we compared the read counts computed by the programs to the transcript coverage values computed by StringTie on the Illumina data from the same sample. To adjust for the overall coverage difference, we multiplied the coverage values for the Illumina data by 1.59, corresponding to the ratio of the number of bases in the Illumina reads (8.5B bp) divided by the number of bases in the ONT reads (5.33B bp). 
                    <xref ref-type="fig" rid="f4">
Figure 4</xref> presents box-and-whisker plots of the ratios (expressed as base-2-logarithms) of the scaled transcript coverages computed with StringTie from Illumina RNA-seq reads and the read counts computed with long-read pipelines from Oxford Nanopore reads for the same sample. The box spans the upper and lower quartile of the ratios and the whiskers represent the range for 95% of the values, with individual outliers outside of the 95% interval shown as dots. The quantification estimates produced by NIFFLR were the second most-consistent to StringTie, outperformed slightly by Isoquant. NIFFLR was the most sensitive, quantifying 26,312 isoforms found in the Illumina RNA-seq data by StringTie.</p>
                <table-wrap id="T2" orientation="portrait" position="float">
                    <label>
Table 2. </label>
                    <caption>
                        <title>Performance of long-read transcriptome assembly and quantification methods on GTEx ONT data. NIFFLR recovers the largest number of reference isoforms.</title>
                    </caption>
                    <table content-type="article-table" frame="hsides">
                        <thead>
                            <tr>
                                <th align="left" colspan="1" rowspan="1" valign="top"/>
                                <th align="left" colspan="1" rowspan="1" valign="top"># of reference isoforms</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">
# of total isoforms</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">FLAIR2</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">14,957</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">75,557</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">IsoQuant</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">17,183</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">17,183</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">NIFFLR</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">43,093</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">58,377</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">ESPRESSO</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">21,026</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">26,222</td>
                            </tr>
                        </tbody>
                    </table>
                </table-wrap>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>
Figure 4. </label>
                    <caption>
                        <title>Comparison of scaled transcript coverages computed with StringTie from Illumina RNA-seq reads and the read counts computed with long-read pipelines from Oxford Nanopore reads for the same sample.</title>
                        <p>NIFFLR quantified 26,312 reference transcripts that were also quantified with StringTie, far more than the competing pipelines. IsoQuant counts are the most consistent with StringTie counts derived from Illumina data for the same sample, and NIFFLR counts are the second closest.</p>
                    </caption>
                    <graphic id="gr4" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/181114/23262718-c4b2-4221-9835-2cf8f451a741_figure4.gif"/>
                </fig>
            </sec>
            <sec id="sec12">
                <title>Isoform discovery with NIFFLR on 92 GTEx samples</title>
                <p>We applied NIFFLR to identify and quantify isoforms in 92 ONT GTEx samples described in (
                    <xref ref-type="bibr" rid="ref6">Glinos et al., 2022</xref>), using the RefSeq annotation of GRCh38.p14 as the reference. Across all samples, we identified 135,343 known isoforms and 316,284 novel isoforms in 35,686 genes. Our high confidence set included isoforms identified in two or more sequence samples, and it includes 106,667 novel isoforms and 121,155 known isoforms across 32,875 genes. Number of isoforms identified by NIFFLR far exceeds the number reported by FLAIR (
                    <xref ref-type="bibr" rid="ref6">Glinos et al., 2022</xref>), which identified 93,718 transcripts across 21,067 genes, of which 77% were novel. 
                    <xref ref-type="fig" rid="f5">
Figure 5</xref> illustrates the distribution of counts of novel isoforms across all samples. Interestingly, NIFFLR identified 13 novel isoforms that were present in all 92 samples. Three of these 13 isoforms are annotated in the CHESS annotation version 3.0.1 (
                    <xref ref-type="bibr" rid="ref10">Varabyou et al., 2023</xref>), or in the GENCODE annotation release 47, with one isoform present in both annotations. 
                    <xref ref-type="table" rid="T3">
Table 3</xref> shows the breakdown of novel and known transcripts found by NIFFLR in GTEx long-read data by tissue. As expected, the percentage of novel isoforms increases with increase in the number of samples for a given tissue, as rare isoforms become more abundant.</p>
                <fig fig-type="figure" id="f5" orientation="portrait" position="float">
                    <label>
Figure 5. </label>
                    <caption>
                        <title>The number of novel isoforms discovered by NIFFLR vs. the number of samples these isoforms were found.</title>
                        <p>The total number of novel isoforms identified by NIFFLR in the 92 GTEX samples was 451,627. Of these, 223,805 were only seen in a single sample and 13 isoforms were identified in all 92 samples.</p>
                    </caption>
                    <graphic id="gr5" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/181114/23262718-c4b2-4221-9835-2cf8f451a741_figure5.gif"/>
                </fig>
                <table-wrap id="T3" orientation="portrait" position="float">
                    <label>
Table 3. </label>
                    <caption>
                        <title>Breakdown of novel and known transcripts found by NIFFLR in GTEx long-read data by tissue.</title>
                        <p>The share of novel isoforms increases with the increase in the number of samples for a given tissue. We used all isoforms identified by NIFFLR for the counts shown in this table.</p>
                    </caption>
                    <table content-type="article-table" frame="hsides">
                        <thead>
                            <tr>
                                <th align="left" colspan="1" rowspan="1" valign="top">Tissue</th>
                                <th align="left" colspan="1" rowspan="1" valign="top"># Samples</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Novel Transcripts</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">Known Transcripts</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">
Percent Novel Transcripts</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Adipose</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">1</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">9,273</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">30,159</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">23.5</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Brain</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">22</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">113,294</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">103,644</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">52.2</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Breast</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">1</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">8,391</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">32,940</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">20.3</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Cultured Fibroblasts</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">22</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">156,097</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">103,354</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">60.2</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Heart</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">16</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">71,024</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">86,431</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">45.1</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">K562 (Human Chronic Myelogenous Leukemia cell line)</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">4</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">22,056</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">33,677</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">39.6</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Liver</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">8</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">46,781</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">68,622</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">40.5</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Lung</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">8</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">73,414</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">84,373</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">46.5</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Muscle</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">9</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">76,409</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">75,407</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">50.3</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">Pancreas</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">1</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">9,313</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">32,101</td>
                                <td align="left" colspan="1" rowspan="1" valign="bottom">22.5</td>
                            </tr>
                        </tbody>
                    </table>
                </table-wrap>
            </sec>
        </sec>
        <sec id="sec13" sec-type="discussion">
            <title>Discussion</title>
            <p>In this manuscript, we describe a novel approach for the discovery and quantification of isoforms from long-read RNA sequencing data produced by Oxford Nanopore sequencing technology. The key difference between NIFFLR and other published programs with similar functionality is that NIFFLR aligns exons from the reference annotation directly to the reads, rather than performing spliced alignment of the reads to the genome. This approach works best for well-annotated genomes, such as the human genome, offering superior sensitivity in this case. However, NIFFLR can still be applied to genomes where their annotation is less reliable, after inferring potential exons from the Illumina RNA-seq data using transcriptome assemblers such as StringTie.</p>
            <sec id="sec14">
                <title>Timings comparison</title>
                <p>NIFFLR is generally fast enough for research use. As shown in 
                    <xref ref-type="table" rid="T4">
Table 4</xref>, NIFFLR was slower than FLAIR2 and IsoQuant, but much faster than ESPRESSO on both simulated and real datasets. Most of the runtime for NIFFLR was spent on aligning exons to the long reads.</p>
                <table-wrap id="T4" orientation="portrait" position="float">
                    <label>
Table 4. </label>
                    <caption>
                        <title>Timings for the quantification software measured on the simulated and real data.</title>
                        <p>We ran all experiments on a 24-core Intel Xeon Gold server with 1TB or RAM, using 24 threads. Time is in hours.</p>
                    </caption>
                    <table content-type="article-table" frame="hsides">
                        <thead>
                            <tr>
                                <th align="left" colspan="1" rowspan="1" valign="top"/>
                                <th align="left" colspan="1" rowspan="1" valign="top">IsoQuant</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">FLAIR2</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">NIFFLR</th>
                                <th align="left" colspan="1" rowspan="1" valign="top">
ESPRESSO</th>
                            </tr>
                        </thead>
                        <tbody>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">Simulated reads</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">0.7</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">1.3</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">1.9</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">45</td>
                            </tr>
                            <tr>
                                <td align="left" colspan="1" rowspan="1" valign="top">GTEx sample</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">1.2</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">2.1</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">3.2</td>
                                <td align="left" colspan="1" rowspan="1" valign="top">106</td>
                            </tr>
                        </tbody>
                    </table>
                </table-wrap>
                <p>NIFFLR is written in shell script, Python, and C++ (the psa_aligner code). To simplify installation, we provide an install script that performs system checks and compiles all necessary executables. We have tested the installation on several popular Linux distributions including RedHat 7, 8, and 9, as well as Ubuntu 18, 20, and 22 LTS.</p>
            </sec>
        </sec>
        <sec id="sec15">
            <title>Software availability
</title>
            <p>

                <list list-type="bullet">
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Software available from: 
                            <ext-link ext-link-type="uri" xlink:href="https://github.com/alguoo314/NIFFLR">https://github.com/alguoo314/NIFFLR</ext-link>
                        </p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Source code available from: 
                            <ext-link ext-link-type="uri" xlink:href="https://github.com/alguoo314/NIFFLR">https://github.com/alguoo314/NIFFLR</ext-link>
                        </p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Archived source code at time of publication: Zenodo doi 
                            <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5281/zenodo.15585584">10.5281/zenodo.15585584</ext-link>
                        </p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>License: GNU General Public License v3.0</p>
                    </list-item>
                </list>
            </p>
        </sec>
        <sec id="sec16">
            <title>Ethical considerations</title>
            <p>Ethics and consent are not required.</p>
        </sec>
    </body>
    <back>
        <sec id="sec19" sec-type="data-availability">
            <title>Data availability
</title>
            <p>

                <list list-type="bullet">
                    <list-item>
                        <label>&#x2022;</label>
                        <p>The supplementary materials, transcript assembly and quantification results computed by NIFFLR from GTEx data are available on Zenodo.</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>[Zenodo]. [Supplementary information and transcripts assembled by NIFFLR software for 92 GTEx long-read transcriptome sequencing samples]. [
                            <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.5281/zenodo.15585443">10.5281/zenodo.15585443</ext-link>].</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>The project contains the following underlying data: Transcripts assembled by NIFFLR software for 92 GTEx long-read transcriptome sequencing samples along with the number of samples the transcripts were observed in. Supplementary materials: commands we used to run NIFFLR and competing software for comparisons are listed in the Supplementary Information.</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>combined92.combined.chr.gtf &#x2013; GTF format file (9-column tab separated text) containing assembled transcripts on human GRCh38 assembly, chromosomes identified with chromosome names.</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>combined92.combined.gtf &#x2013; GTF format file (9-column tab separated text) containing assembled transcripts on human GRCh38 assembly, chromosomes identified with NCBI RefSeq chromosome IDs.</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>combined92.combined.min2sampl.gtf &#x2013; GTF format file (9-column tab separated text) containing assembled transcripts found in at least two samples, on human GRCh38 assembly, chromosomes identified with NCBI RefSeq chromosome IDs.</p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Supplementary materials.pdf &#x2013; Supplementary materials for the manuscript titled &#x201c;Assembly and quantification of transcripts from noisy long reads with NIFFLR.&#x201d;</p>
                    </list-item>
                </list>
            </p>
            <p>Data are available under the terms of the 
                <ext-link ext-link-type="uri" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International license</ext-link> (CC-BY 4.0).</p>
        </sec>
        <ack>
            <title>Acknowledgements</title>
            <p>We thank Steven L. Salzberg, Bloomberg Distinguished Professor of Biomedical Engineering, Computer Science and Biostatistics at Johns Hopkins University for help with editing the manuscript and obtaining funding for this project.</p>
        </ack>
        <ref-list>
            <title>References</title>
            <ref id="ref1">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Gao</surname>
                            <given-names>Y</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wang</surname>
                            <given-names>F</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wang</surname>
                            <given-names>R</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>ESPRESSO: robust discovery and quantification of transcript isoforms from error-prone long-read RNA-seq data.</article-title>
                    <source>

                        <italic toggle="yes">Science Advances.</italic>
</source>
                    <year>2023 Jan 20</year>;<volume>9</volume>(<issue>3</issue>):<fpage>eabq5072</fpage>.
                    <pub-id pub-id-type="pmid">36662851</pub-id>
                    <pub-id pub-id-type="doi">10.1126/sciadv.abq5072</pub-id>
                    <pub-id pub-id-type="pmcid">PMC9858503</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref6">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Glinos</surname>
                            <given-names>DA</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Garborcauskas</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hoffman</surname>
                            <given-names>P</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Transcriptome variation in human tissues revealed by long-read sequencing.</article-title>
                    <source>

                        <italic toggle="yes">Nature.</italic>
</source>
                    <year>2022 Aug 11</year>;<volume>608</volume>(<issue>7922</issue>):<fpage>353</fpage>&#x2013;<lpage>359</lpage>.
                    <pub-id pub-id-type="pmid">35922509</pub-id>
                    <pub-id pub-id-type="doi">10.1038/s41586-022-05035-y</pub-id>
                    <pub-id pub-id-type="pmcid">PMC10337767</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref8">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Kovaka</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zimin</surname>
                            <given-names>AV</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Pertea</surname>
                            <given-names>GM</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Transcriptome assembly from long-read RNA-seq alignments with StringTie2.</article-title>
                    <source>

                        <italic toggle="yes">Genome Biol.</italic>
</source>
                    <year>2019 Dec</year>;<volume>20</volume>:<fpage>273</fpage>&#x2013;<lpage>278</lpage>.
                    <pub-id pub-id-type="pmid">31842956</pub-id>
                    <pub-id pub-id-type="doi">10.1186/s13059-019-1910-1</pub-id>
                    <pub-id pub-id-type="pmcid">PMC6912988</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref2">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Li</surname>
                            <given-names>H</given-names>
                        </name>
</person-group>:
                    <article-title>Minimap2: pairwise alignment for nucleotide sequences.</article-title>
                    <source>

                        <italic toggle="yes">Bioinformatics.</italic>
</source>
                    <year>2018 Sep 15</year>;<volume>34</volume>(<issue>18</issue>):<fpage>3094</fpage>&#x2013;<lpage>3100</lpage>.
                    <pub-id pub-id-type="pmid">29750242</pub-id>
                    <pub-id pub-id-type="doi">10.1093/bioinformatics/bty191</pub-id>
                    <pub-id pub-id-type="pmcid">PMC6137996</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref9">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Pertea</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Pertea</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <article-title>GFF utilities: GffRead and GffCompare.</article-title>
                    <source>

                        <italic toggle="yes">F1000Res.</italic>
</source>
                    <year>2020</year>;<volume>9</volume>.</mixed-citation>
            </ref>
            <ref id="ref3">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Prjibelski</surname>
                            <given-names>AD</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Mikheenko</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Joglekar</surname>
                            <given-names>A</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Accurate isoform discovery with IsoQuant using long reads.</article-title>
                    <source>

                        <italic toggle="yes">Nat. Biotechnol.</italic>
</source>
                    <year>2023 Jul</year>;<volume>41</volume>(<issue>7</issue>):<fpage>915</fpage>&#x2013;<lpage>918</lpage>.
                    <pub-id pub-id-type="pmid">36593406</pub-id>
                    <pub-id pub-id-type="doi">10.1038/s41587-022-01565-y</pub-id>
                    <pub-id pub-id-type="pmcid">PMC10344776</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref4">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Tang</surname>
                            <given-names>AD</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Soulette</surname>
                            <given-names>CM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Baren</surname>
                            <given-names>MJ</given-names>
                            <prefix>van</prefix>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Full-length transcript characterization of SF3B1 mutation in chronic lymphocytic leukemia reveals downregulation of retained introns.</article-title>
                    <source>

                        <italic toggle="yes">Nat. Commun.</italic>
</source>
                    <year>2020 Mar 18</year>;<volume>11</volume>(<issue>1</issue>):<fpage>1438</fpage>.
                    <pub-id pub-id-type="pmid">32188845</pub-id>
                    <pub-id pub-id-type="doi">10.1038/s41467-020-15171-6</pub-id>
                    <pub-id pub-id-type="pmcid">PMC7080807</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref10">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Varabyou</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sommer</surname>
                            <given-names>MJ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Erdogdu</surname>
                            <given-names>B</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>CHESS 3: an improved, comprehensive catalog of human genes and transcripts based on large-scale expression data, phylogenetic analysis, and protein structure.</article-title>
                    <source>

                        <italic toggle="yes">Genome Biol.</italic>
</source>
                    <year>2023 Oct 30</year>;<volume>24</volume>(<issue>1</issue>):<fpage>249</fpage>.
                    <pub-id pub-id-type="pmid">37904256</pub-id>
                    <pub-id pub-id-type="doi">10.1186/s13059-023-03088-4</pub-id>
                    <pub-id pub-id-type="pmcid">PMC10614308</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref7">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Yang</surname>
                            <given-names>C</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Chu</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Warren</surname>
                            <given-names>RL</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>NanoSim: nanopore sequence read simulator based on statistical characterization.</article-title>
                    <source>

                        <italic toggle="yes">GigaScience.</italic>
</source>
                    <year>2017 Apr</year>;<volume>6</volume>(<issue>4</issue>):<fpage>1</fpage>&#x2013;<lpage>6</lpage>.
                    <pub-id pub-id-type="pmid">28327957</pub-id>
                    <pub-id pub-id-type="doi">10.1093/gigascience/gix010</pub-id>
                    <pub-id pub-id-type="pmcid">PMC5530317</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref5">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Zimin</surname>
                            <given-names>AV</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Mar&#x00e7;ais</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Puiu</surname>
                            <given-names>D</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>The MaSuRCA genome assembler.</article-title>
                    <source>

                        <italic toggle="yes">Bioinformatics.</italic>
</source>
                    <year>2013 Nov</year>;<volume>29</volume>(<issue>21</issue>):<fpage>2669</fpage>&#x2013;<lpage>2677</lpage>.
                    <pub-id pub-id-type="pmid">23990416</pub-id>
                    <pub-id pub-id-type="doi">10.1093/bioinformatics/btt476</pub-id>
                    <pub-id pub-id-type="pmcid">PMC3799473</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report395536">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.181114.r395536</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Gao</surname>
                        <given-names>Yuan</given-names>
                    </name>
                    <xref ref-type="aff" rid="r395536a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r395536a1">
                    <label>1</label>Beijing Institute of Genomics, Chinese Academy of Sciences, Beijing, Beijing, China</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>4</day>
                <month>8</month>
                <year>2025</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Gao Y</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport395536" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.164583.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>reject</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>Guo et al. proposed NIFFLR, a tool for assembling and quantifying transcripts using long-read RNA-seq data. However, the current manuscript does not provide sufficient evidence to demonstrate the novelty or efficiency of NIFFLR in analyzing long-read data. Their evaluation and conclusions are not convincing.</p>
            <p> 1 Most readers would be confused about the novelty or advances of NIFFLR. It&#x2019;s based on the alignment of constructed exon-exon junction sequences, a strategy used by many tools long ago. Does this strategy work for mini-exons that are shorter than 10 nt or 20 nt? This strategy also heavily depends on annotated exons. Can it identify any novel splice donors or acceptors? For tissues or organisms with incomplete annotation, how will this strategy be affected? All of above need be carefully evaluated and described in details.</p>
            <p> 2 The aligner (psa_aligner) used by the authors needs comprehensively assessment, e.g. whether psa_aligner is suitable for noisy long-read data. An evaluation and comparison with the commonly used long-read aligner Minimap2 should be included.</p>
            <p> 3 The NIFFLR programming script is poorly written and difficult to install and use for analysis. For example, when I tried to run NIFFLR, I received an error message from python (not NIFFLR) and could not finished the analysis. The error, &#x201c;IndexError: list index out of range&#x201d;, occurred during the &#x201c;performing filtering and quantification&#x201d; step.</p>
            <p> 4 The authors used simulated data to evaluate NIFFLR and other tools. However, this evaluation may result in biased results. Many reports have noted that existing simulators are not suitable for evaluation. For example, according to a recent paper published by Dr. Hagen Tilgner and colleagues (Mikheenko A, et al., 2022 [Ref 1]), NanoSim randomly selects a starting position in a transcript to simulate truncation based on a uniform distribution. However, in real long-read data, a uniform distribution cannot be observed.</p>
            <p> 5 A direct comparison of the tools for transcript identification and quantification using real long-read data with ground truth need be included. Many previous studies used SIRV E2 for evaluation, which contains 69 synthesized transcript isoforms with different abundances. I actually tried to run NIFFLR to analyze SIRV data, but it failed. NIFFLR produced an "exon extraction failed" error when processing the GTF annotation of SIRV, while I did not encounter any errors when using other tools. This is another example of poor programming of NIFFLR.</p>
            <p> 6 I am stunned that the authors used transcript abundance from short-read data to evaluate the performance of long-read tools. The bias of short-read data has been so widely reported. For example, a Nature Methods paper published by Chen et al. 2025 [Ref 2] provided important evidence on this. In addition to the bias, the novel transcript isoforms in the data that are similar to annotated isoforms would also confuse short-read quantification.</p>
            <p> 7 The authors need to provide sufficient evidence before arbitrarily judging existing tools. For example, they described ESPRESSO&#x2019;s strategy for identifying novel splice junctions as &#x201c;a stringent criterion that limits its ability to discover novel junctions&#x201d;. However, they did not provide any evidence regarding the sensitivity of detecting novel splice junctions. Novel splice junctions can be further divided into junctions with novel splice sites and junctions as novel combination of annotated splice sites. Both types need to be compared between ESPRESSO and NIFFLR before the authors can draw such a conclusion.</p>
            <p> 8 Why was BamBu (Chen Y, et al., 2023 [Ref 3]) not included in the evaluation and comparison? BamBu was published online more than two years ago and has been widely used for long-read data analysis. To demonstrate the advantages of their tool, the authors will need to compare it with BamBu.</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>No</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Partly</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>No</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Partly</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Partly</p>
            <p>Reviewer Expertise:</p>
            <p>Bioinformatics, Computational Biology</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.</p>
        </body>
        <back>
            <ref-list>
                <title>References</title>
                <ref id="rep-ref-395536-1">
                    <label>1</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Sequencing of individual barcoded cDNAs using Pacific Biosciences and Oxford Nanopore Technologies reveals platform-specific error patterns</article-title>.
                        <source>
                            <italic>Genome Research</italic>
                        </source>.<year>2022</year>;<volume>32</volume>(<issue>4</issue>) :
                        <elocation-id>10.1101/gr.276405.121</elocation-id>
                        <fpage>726</fpage>-<lpage>737</lpage>
                        <pub-id pub-id-type="doi">10.1101/gr.276405.121</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-395536-2">
                    <label>2</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>A systematic benchmark of Nanopore long-read RNA sequencing for transcript-level analysis in human cell lines</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2025</year>;<volume>22</volume>(<issue>4</issue>) :
                        <elocation-id>10.1038/s41592-025-02623-4</elocation-id>
                        <fpage>801</fpage>-<lpage>812</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-025-02623-4</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-395536-3">
                    <label>3</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Context-aware transcript quantification from long-read RNA-seq data with Bambu</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2023</year>;<volume>20</volume>(<issue>8</issue>) :
                        <elocation-id>10.1038/s41592-023-01908-w</elocation-id>
                        <fpage>1187</fpage>-<lpage>1195</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-023-01908-w</pub-id>
                    </mixed-citation>
                </ref>
            </ref-list>
        </back>
        <sub-article article-type="response" id="comment14842-395536">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Zimin</surname>
                            <given-names>Aleksey</given-names>
                        </name>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>CONFLICT OF INTEREST NOTE: This reviewer, Yuan Gao, Chinese Academy of Sciences, Beijing, China, is in direct conflict with this publication. The reviewer is the lead author of ESPRESSO software, and NIFFLR is a direct competitor to that software. We cite the ESPRESSO publication in this manuscript.</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>24</day>
                    <month>10</month>
                    <year>2025</year>
                </pub-date>
            </front-stub>
            <body>
                <p>1. Most readers would be confused about the novelty or advances of NIFFLR. It&#x2019;s based on the alignment of constructed exon-exon junction sequences, a strategy used by many tools long ago. Does this strategy work for mini-exons that are shorter than 10 nt or 20 nt? This strategy also heavily depends on annotated exons. Can it identify any novel splice donors or acceptors? For tissues or organisms with incomplete annotation, how will this strategy be affected? All of above need be carefully evaluated and described in details.</p>
                <p> 
                    <bold>Response:</bold> The novelty of NIFFLR lies in its approach of constructing transcripts from existing exons guided by long reads. NIFFLR shows significant improvements in sensitivity and quantification accuracy compared to existing tools. Since NIFFLR relies on existing exons, it cannot identify novel donor or acceptor sites; however, it can detect novel introns, i.e. novel donor-acceptor pairs. As we state in the manuscript, NIFFLR is designed for genomes with high-quality annotations, such as the human genome. NIFFLR strategy works well for exons that are longer than the minimum alignment K-mer size (default value is 12).&#x00a0; Any exon shorter than this value will not be detected in the alignment and will likely be skipped. Fortunately, the percentage of transcripts containing such mini-exons is small.&#x00a0; Human RefSeq annotation (mail chromosomes only) contains 379021 exons, and only 598 are shorter than 12 bp.</p>
                <p> </p>
                <p> 2. The aligner (psa_aligner) used by the authors needs comprehensively assessment, e.g. whether psa_aligner is suitable for noisy long-read data. An evaluation and comparison with the commonly used long-read aligner Minimap2 should be included.</p>
                <p> 
                    <bold>Response:&#x00a0;</bold>psa_aligner is not a novel tool. It is used in the MaSuRCA assembler [REF] to map super-reads built from Illumina reads to noisy long reads. Evaluating or comparing alignment methods was not the focus of our study.&#x00a0; In our tests we found that Minimap2 lacked sufficient sensitivity, particularly by missing alignments of short exons. We decided to use instead psa_aligner which only produces pseudo-alignments.</p>
                <p> </p>
                <p> 3. The NIFFLR programming script is poorly written and difficult to install and use for analysis. For example, when I tried to run NIFFLR, I received an error message from python (not NIFFLR) and could not finished the analysis. The error, &#x201c;IndexError: list index out of range&#x201d;, occurred during the &#x201c;performing filtering and quantification&#x201d; step.</p>
                <p> 
                    <bold>Response:</bold> Thank you for raising this concern and for sharing the error message. We reviewed the relevant part of the NIFFLR pipeline but we were unable to replicate the issue. Based on the error message reported &#x201c;
                    <italic>IndexError: list index out of range</italic>&#x201d; during the 
                    <italic>filtering and quantification</italic> step, the problem is likely due to unexpected formatting in the reference GFF/GTF file. The quantification.py script assumes that each line in the reference file follows the standard GFF/GTF structure, containing 
                    <bold>at least 9 tab-separated fields</bold>, with the final (ninth) field being a semicolon-delimited string of attributes (e.g., with the transcript ID as the first entry). If any line deviates from this structure, it can trigger an IndexError.</p>
                <p> To improve stability and usability, we have significantly reworked the pipeline and released a new version, v2.0.0, which is described in the revised manuscript.&#x00a0; This new version is easier to install and use and provides improved compatibility with input files that may slightly deviate from standard GTF/GFF specifications.</p>
                <p> </p>
                <p> 4. The authors used simulated data to evaluate NIFFLR and other tools. However, this evaluation may result in biased results. Many reports have noted that existing simulators are not suitable for evaluation. For example, according to a recent paper published by Dr. Hagen Tilgner and colleagues (Mikheenko A, et al., 2022 [Ref 1]), NanoSim randomly selects a starting position in a transcript to simulate truncation based on a uniform distribution. However, in real long-read data, a uniform distribution cannot be observed.</p>
                <p> 
                    <bold>Response:</bold> We agree that simulated data have limitations and may not perfectly capture all features of real long-read sequencing. However, simulated data is the only way to evaluate software on datasets where the ground truth is fully known.&#x00a0; Nanosim is a peer-reviewed, widely used tool for generating simulated long reads, and its use is standard practice for benchmarking transcriptome assembly methods. Therefore, we believe it is appropriate to use NanoSim to produce simulated reads for the comparisons presented in this study.</p>
                <p> </p>
                <p> 5. A direct comparison of the tools for transcript identification and quantification using real long-read data with ground truth need be included. Many previous studies used SIRV E2 for evaluation, which contains 69 synthesized transcript isoforms with different abundances. I actually tried to run NIFFLR to analyze SIRV data, but it failed. NIFFLR produced an "exon extraction failed" error when processing the GTF annotation of SIRV, while I did not encounter any errors when using other tools. This is another example of poor programming of NIFFLR.</p>
                <p> 
                    <bold>Response:</bold> We thank the reviewer for the suggestion. We have added an evaluation of the ability of different programs to identify isoforms in the SIRV E2 dataset to the Results section. The new NIFFLR version 2.0.0 now works correctly with the SIRV GTF file, resolving the previous &#x201c;exon extraction failed&#x201d; error.</p>
                <p> </p>
                <p> 6. I am stunned that the authors used transcript abundance from short-read data to evaluate the performance of long-read tools. The bias of short-read data has been so widely reported. For example, a Nature Methods paper published by Chen et al. 2025 [Ref 2] provided important evidence on this. In addition to the bias, the novel transcript isoforms in the data that are similar to annotated isoforms would also confuse short-read quantification.</p>
                <p> 
                    <bold>Response:</bold> We understand why the reviewer is raising this objection, as indeed biases of short-read data have been widely reported. However, transcript quantification using short-read data has long been a standard in the field, and thousands of studies have successfully used short reads to quantify transcript expression (e.g. Romeo-Cardeillac et al., BMC Genomics, 2024; Kuehl et al., Nature, 2025; D'Sa et al., Sci. Adv., 2025). Widely used tools such as StringTie2 (Kovaka et al., Genome Biology, 2019), Salmon ( Patro et al., Nat Methods, 2017) and Kallisto (Bray et al., Nat Biotechnol, 2016) all rely on short reads for quantification. While it is true that short-read data can exhibit biases and that novel isoforms similar to annotated transcripts may complicate quantification, these limitations notwithstanding, short-read quantification still provides a reliable reference that can be used to assess the accuracy of long-read quantifications, as has been done in previous studies (Pardo-Palacios et al., 2023, Tang et al., 2020). &#x00a0;We therefore consider it appropriate to use short-read transcript quantification as a benchmark for evaluating long-read tools. This approach provides a practical and widely accepted reference for assessing quantification performance, while acknowledging that no method is entirely free of bias.</p>
                <p> </p>
                <p> 7. The authors need to provide sufficient evidence before arbitrarily judging existing tools. For example, they described ESPRESSO&#x2019;s strategy for identifying novel splice junctions as &#x201c;a stringent criterion that limits its ability to discover novel junctions&#x201d;. However, they did not provide any evidence regarding the sensitivity of detecting novel splice junctions. Novel splice junctions can be further divided into junctions with novel splice sites and junctions as novel combination of annotated splice sites. Both types need to be compared between ESPRESSO and NIFFLR before the authors can draw such a conclusion.</p>
                <p> 
                    <bold>Response:</bold>&#x00a0; We have removed the statement that ESPRESSO uses &#x201c;a stringent criterion that limits its ability to discover novel junctions&#x201d; from the introduction. For context, our statement that ESPRESSO uses &#x201c;a stringent criterion that limits its ability to discover novel junctions&#x201d; was not based on empirical benchmarking but rather on the algorithmic description provided in the ESPRESSO manuscript. As described in the manuscript, ESPRESSO classifies novel junctions into two types: junctions with novel splice sites (Novel Not in Catalog, NNC) and novel combinations of annotated splice sites (Novel In Catalog, NIC). According to the ESPRESSO manuscript, a novel transcript isoform is reported only if (i) each splice junction is supported by at least two perfectly aligned reads and (ii) the combination of junctions is not a substring of any other novel isoform. Perfectly aligned reads are defined as having no mismatches or indels within 10 nt of splice sites.&#x00a0; &#x00a0;</p>
                <p> In contrast, NIFFLR does not rely on spliced alignments and does not impose strict local alignment quality requirements. Instead, it uses a custom approximate aligner to map reference exons directly to long reads and assembles transcript models by optimizing exon tilings. This makes NIFFLR more tolerant of sequencing errors and more flexible in capturing novel combinations of annotated exons. Subsequent filtering and quantification steps in NIFFLR help exclude spurious transcripts. We note, however, that NIFFLR does not detect novel splice sites, as its transcript models are built from a predefined set of reference exons.</p>
                <p> </p>
                <p> 8. Why was BamBu (Chen Y, et al., 2023 [Ref 3]) not included in the evaluation and comparison? BamBu was published online more than two years ago and has been widely used for long-read data analysis. To demonstrate the advantages of their tool, the authors will need to compare it with BamBu.</p>
                <p> 
                    <bold>Response:</bold> We added Bambu to all evaluations.</p>
            </body>
        </sub-article>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report393925">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.181114.r393925</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Reese</surname>
                        <given-names>Fairlie</given-names>
                    </name>
                    <xref ref-type="aff" rid="r393925a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-9240-0102</uri>
                </contrib>
                <aff id="r393925a1">
                    <label>1</label>University of California, Irvine, California, USA</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>4</day>
                <month>8</month>
                <year>2025</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Reese F</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport393925" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.164583.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>reject</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The authors present NIFFLR, a minimap2-free tool for the assembly and quantification of known and novel transcripts from long-read RNA-seq data. In the paper, they describe NIFFLR, the developed method, which works using partial suffix arrays and kmer-matching of annotated exons to the long reads themselves; thus bypassing the potential error-prone choices that are made when aligning noisy long reads at splice junctions. Overall, this seems like a really interesting and novel approach to a problem (errors in minimap2 alignments) that I&#x2019;ve seen several times now in the field. However, I have several major concerns about the quality of the benchmarking and how the paper integrates into the overall field. I elaborate on my concerns below.</p>
            <p> </p>
            <p> Major concerns: 
                <list list-type="order">
                    <list-item>
                        <p>It&#x2019;s unclear why certain parameter choices were made for the implementation, such as k = 12 or coverage &gt;= 35%, at the level of the exon to read alignment or Amin / Gmin at the level of choosing the most optimal path. The authors mention that they performed optimization but the results of these experiments are not included. Adding these results showing that this performance is optimal would increase the confidence in the tool and these decisions taken.</p>
                    </list-item>
                    <list-item>
                        <p>One of my main concerns is that, based on my understanding of the method, the only novel transcripts that can be discovered by NIFFLR are those that use already-annotated exons. This could substantially limit its ability to identify biologically relevant novel isoforms, which frequently involve novel splice sites or exons and are described in previous long-read RNA-seq studies. The authors should consider alternative methods to include novel splicing as part of their novel isoform discovery.</p>
                    </list-item>
                    <list-item>
                        <p>Related to point 2, it is hard to assess the performance of the tool relative to other reported findings in the field because it only contains novel-in-catalog novel transcripts according to the very commonly-used SQANTI classification (Pardo-Palacios F,&#x00a0;et al., 2024 [Ref 1]). It is common in the field to examine the proportions of transcripts from each of these categories as a quality control metric.</p>
                    </list-item>
                    <list-item>
                        <p>My main concern in this paper however is the quality of the benchmarking. There have been many papers that have performed long-read RNA-seq benchmarking in the past and have employed various metrics to evaluate the results: (Dong X,&#x00a0;et al., 2023 [Ref 2], Chen Y,&#x00a0;et al., 2022 [Ref 3], Pardo-Palacios F,&#x00a0;&#x00a0;et al., 2024 [Ref 4]),</p>
                        <p> The manuscript would benefit from adopting such standard benchmarking strategies that have become widely-accepted. As it stands, it is very tough to contextualize its results in the field as a whole. Some more point-by-point recommendations would be at least the following: 
                            <list list-type="order">
                                <list-item>
                                    <p>For the simulations, as referenced in point 2, it is unrealistic to expect that novel transcripts will only arise from novel combinations of known exons (NICs). The authors should consider using existing simulated novel transcript ground truth datasets that exist, such as the one from LRGASP.&#x00a0;</p>
                                </list-item>
                                <list-item>
                                    <p>For the quantification benchmarking, it is more common to report a correlation metric between the ground truth and the estimated quantification from the tool. This would make these results more interpretable and comparable to other benchmarking efforts in the field.&#x00a0;</p>
                                </list-item>
                                <list-item>
                                    <p>Also related to the quantification benchmarking, there are no significance values reported for any of the pairwise comparisons; just written speculation on the visual appearance of the plots. Statistical analyses would increase the confidence of these results.</p>
                                </list-item>
                                <list-item>
                                    <p>Finding more isoforms is not necessarily a metric of a &#x201c;better&#x201d; tool, but is referenced as if it is in the text related to Figure 5. In fact, the percentage of reported &#x201c;novel&#x201d; transcripts for various GTEx tissues is surprisingly high and therefore the high number of transcripts could be indicative of over-calling novel transcripts and therefore poor specificity. Instead, the authors should additionally overlap the discovered isoforms with those discovered by Glinos et al. in their original paper (or other external datasets) to see how well their method recapitulates what others have already said about the dataset.&#x00a0;</p>
                                </list-item>
                            </list> </p>
                    </list-item>
                    <list-item>
                        <p>Would the authors be able to speculate on any specific use case (a specific cohort, a specific technology, etc) where the exons-to-reads alignment approach might be especially beneficial compared to traditional minimap2-based approaches? This might add some insight to the discussion.</p>
                    </list-item>
                </list> </p>
            <p> Minor concerns:&#x00a0; 
                <list list-type="order">
                    <list-item>
                        <p>As the development of the method is of central importance to the paper, the implementation could be expanded on or explained a bit more. In particular, the concepts of the &#x201c;implied&#x201d; starts / ends of the alignments were confusing in Figure 1 and related text. Similarly, Figure 2 was a bit overly-complicated, and perhaps the authors could consider presenting the possible transcript paths separately as alternative transcripts in genome browser format (ie IGV or UCSC).</p>
                    </list-item>
                    <list-item>
                        <p>At the end of the first paragraph of main text on page 7, the authors state &#x201c;This result demonstrates that when novel isoform discovery and quantification are the primary goals, NIFFLR is the best tool&#x201d; when referring to the simulation experiment where they measured isoform detection / assembly only. This means this result has no relevance for quantification.&#x00a0;</p>
                    </list-item>
                    <list-item>
                        <p>If the authors mean &#x201c;false positive transcripts&#x201d; by &#x201c;spurious transcripts&#x201d;, they should simply refer to them as the latter as there is no definition for &#x201c;spurious&#x201d; transcripts in the text.&#x00a0;</p>
                    </list-item>
                    <list-item>
                        <p>In table 2, IsoQuant has the same number of known and total isoforms; implying it found no novel isoforms. I am highly doubtful this is correct and is probably a typo.</p>
                    </list-item>
                    <list-item>
                        <p>Figure 3 is missing y axis labels. Furthermore, the meaning of the box and whiskers are elaborated on in the results, which should just be in the legend. Additionally, Figure 3b is just a zoomed in duplicate of two plots from 3a and is unnecessary.&#x00a0;</p>
                    </list-item>
                    <list-item>
                        <p>For Figure 4 the authors describe a strange method of depth normalization to compare the long-read RNA-seq to short-read RNA-seq transcript quantification estimates. They should use normal TPM / CPM normalization. This figure also has unnecessary details about the plot in the results section which should be in the legend (same as in Figure 3).</p>
                    </list-item>
                    <list-item>
                        <p>The authors make no references to their supplementary material PDF in the main body of the text. They should include references so that readers know where to find the calls made to perform the benchmarking etc.&#x00a0;</p>
                    </list-item>
                </list> 
                <bold> </bold>
            </p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Partly</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Yes</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Yes</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Partly</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>No</p>
            <p>Reviewer Expertise:</p>
            <p>Long-read transcriptomics, including development of tools and their benchmarking for long-read transcriptomics analysis. I am less experienced in the field of alignment algorithms and cannot judge this implementation as thoroughly.</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.</p>
        </body>
        <back>
            <ref-list>
                <title>References</title>
                <ref id="rep-ref-393925-1">
                    <label>1</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>SQANTI3: curation of long-read transcriptomes for accurate identification of known and novel isoforms</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2024</year>;<volume>21</volume>(<issue>5</issue>) :
                        <elocation-id>10.1038/s41592-024-02229-2</elocation-id>
                        <fpage>793</fpage>-<lpage>797</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-024-02229-2</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-393925-2">
                    <label>2</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Benchmarking long-read RNA-sequencing analysis tools using in silico mixtures</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2023</year>;<volume>20</volume>(<issue>11</issue>) :
                        <elocation-id>10.1038/s41592-023-02026-3</elocation-id>
                        <fpage>1810</fpage>-<lpage>1821</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-023-02026-3</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-393925-3">
                    <label>3</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>A systematic benchmark of Nanopore long-read RNA sequencing for transcript-level analysis in human cell lines</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2025</year>;<volume>22</volume>(<issue>4</issue>) :
                        <elocation-id>10.1038/s41592-025-02623-4</elocation-id>
                        <fpage>801</fpage>-<lpage>812</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-025-02623-4</pub-id>
                    </mixed-citation>
                </ref>
                <ref id="rep-ref-393925-4">
                    <label>4</label>
                    <mixed-citation publication-type="journal">
                        <person-group person-group-type="author"/>:
                        <article-title>Systematic assessment of long-read RNA-seq methods for transcript identification and quantification</article-title>.
                        <source>
                            <italic>Nature Methods</italic>
                        </source>.<year>2024</year>;<volume>21</volume>(<issue>7</issue>) :
                        <elocation-id>10.1038/s41592-024-02298-3</elocation-id>
                        <fpage>1349</fpage>-<lpage>1363</lpage>
                        <pub-id pub-id-type="doi">10.1038/s41592-024-02298-3</pub-id>
                    </mixed-citation>
                </ref>
            </ref-list>
        </back>
        <sub-article article-type="response" id="comment14841-393925">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Zimin</surname>
                            <given-names>Aleksey</given-names>
                        </name>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>24</day>
                    <month>10</month>
                    <year>2025</year>
                </pub-date>
            </front-stub>
            <body>
                <p>
                    <bold>Major concerns:</bold>
                </p>
                <p> </p>
                <p> 1.&#x00a0;It&#x2019;s unclear why certain parameter choices were made for the implementation, such as k = 12 or coverage &gt;= 35%, at the level of the exon to read alignment or Amin / Gmin at the level of choosing the most optimal path. The authors mention that they performed optimization but the results of these experiments are not included. Adding these results showing that this performance is optimal would increase the confidence in the tool and these decisions taken. 
                    <bold>Response: </bold>The default value of K=12 was empirically chosen as it provided the best balance between sensitivity and precision of the &#x00a0;psa_aligner on multiple data sets. Values of 11 or lower introduced numerous false positive alignments and slowed down NIFFLR without improving sensitivity, whereas values above 12 resulted in missed alignments of short exons.&#x00a0; In general, the optimal value of K depends on the read error rate and the minimum exon length in the annotation. For lower-error rate transcriptome sequencing data, K can be increased to 15&#x2013;17 to achieve faster alignment. This information has been now added to our manuscript in section&#x2026;.</p>
                <p> </p>
                <p> 2.&#x00a0;One of my main concerns is that, based on my understanding of the method, the only novel transcripts that can be discovered by NIFFLR are those that use already-annotated exons. This could substantially limit its ability to identify biologically relevant novel isoforms, which frequently involve novel splice sites or exons and are described in previous long-read RNA-seq studies. The authors should consider alternative methods to include novel splicing as part of their novel isoform discovery.</p>
                <p> 
                    <bold>Response: </bold>We agree with the reviewer&#x2019;s observation that using a known set of exons is a key limitation of NIFFLR.&#x00a0; NIFFLR is designed to be used with well-annotated genomes, such as the human genome, where nearly all exons are known and most novel transcripts arise from alternative splicing. We have clearly stated this limitation in the manuscript.&#x00a0; Despite this constraint, we believe NIFFLR provides tremendous value through its ability to quantify known isoforms and to detect and quantify novel alternatively spliced isoforms with high sensitivity and accuracy, rivalling the best published tools.</p>
                <p> </p>
                <p> 3.&#x00a0;Related to point 2, it is hard to assess the performance of the tool relative to other reported findings in the field because it only contains novel-in-catalog novel transcripts according to the very commonly-used SQANTI classification (Pardo-Palacios F, et al., 2024 [Ref 1]). It is common in the field to examine the proportions of transcripts from each of these categories as a quality control metric.</p>
                <p> 
                    <bold>Response:&#x00a0;</bold>We designed our comparison of tools using simulated data so that not all reference transcripts present in the data were provided to the programs, including NIFFLR.&#x00a0; This way we can evaluate each program&#x2019;s ability to recover novel transcripts that are present in the sample, but absent from the supplied reference annotation. To measure precision, we report the number of false positive transcripts as well as the precision values in Table 1.&#x00a0;</p>
                <p> </p>
                <p> 4.&#x00a0;My main concern in this paper however is the quality of the benchmarking. There have been many papers that have performed long-read RNA-seq benchmarking in the past and have employed various metrics to evaluate the results: (Dong X, et al., 2023 [Ref 2], Chen Y, et al., 2022 [Ref 3], Pardo-Palacios F,&#x00a0; et al., 2024 [Ref 4]),&#x00a0;</p>
                <p> The manuscript would benefit from adopting such standard benchmarking strategies that have become widely-accepted. As it stands, it is very tough to contextualize its results in the field as a whole. Some more point-by-point recommendations would be at least the following:</p>
                <p> </p>
                <p> 4a. For the simulations, as referenced in point 2, it is unrealistic to expect that novel transcripts will only arise from novel combinations of known exons (NICs). The authors should consider using existing simulated novel transcript ground truth datasets that exist, such as the one from LRGASP.</p>
                <p> 
                    <bold>Response: </bold>We appreciate the reviewer&#x2019;s suggestion. For this study, we chose to use transcripts produced by Nanosim, a publicly available and peer-reviewed tool specifically designed to generate realistic long-read transcriptome data. Our goal was to ensure a controlled and reproducible comparison under well-defined simulation parameters. We agree that other simulated datasets, such as those from LRGASP, provide valuable complementary benchmarks and will consider incorporating them in future evaluations.</p>
                <p> </p>
                <p> 4b.&#x00a0;For the quantification benchmarking, it is more common to report a correlation metric between the ground truth and the estimated quantification from the tool. This would make these results more interpretable and comparable to other benchmarking efforts in the field. Please see our answer to point 3 below.</p>
                <p> </p>
                <p> 4c.&#x00a0;Also related to the quantification benchmarking, there are no significance values reported for any of the pairwise comparisons; just written speculation on the visual appearance of the plots. Statistical analyses would increase the confidence of these results.</p>
                <p> 
                    <bold>Response:</bold> The reviewer makes a good point. To address it, we have added values of Pearson correlation coefficient (PCC) values to the box-and-whisker plots in the manuscript. These values provide a quantitative measure of concordance between methods and allow readers to assess the significance of the pairwise comparisons more rigorously.</p>
                <p> </p>
                <p> 4d.&#x00a0;Finding more isoforms is not necessarily a metric of a &#x201c;better&#x201d; tool, but is referenced as if it is in the text related to Figure 5. In fact, the percentage of reported &#x201c;novel&#x201d; transcripts for various GTEx tissues is surprisingly high and therefore the high number of transcripts could be indicative of over-calling novel transcripts and therefore poor specificity. Instead, the authors should additionally overlap the discovered isoforms with those discovered by Glinos et al. in their original paper (or other external datasets) to see how well their method recapitulates what others have already said about the dataset.</p>
                <p> 
                    <bold>Response: </bold>We completely agree that the sheer number of novel isoforms is not, by itself, a measure of a &#x201c;better&#x201d; tool.&#x00a0; However, in light of our other benchmarks on both simulated and real data, we believe that the novel isoforms identified by NIFFLR are likely to be highly reliable, and the high percentage of novel transcripts observed in GTEx tissues could indeed reflect incomplete annotation rather than overcalling. Importantly, many independent studies (such as Glinos et al.) have reported that existing annotations capture only a subset of true transcripts, and numerous bona fide isoforms remain unannotated. In this context, NIFFLR&#x2019;s detection of additional isoforms is consistent with these observations.&#x00a0; We have updated the Results section to make this more clear:</p>
                <p> &#x201c;&#x2026;The number of isoforms identified by NIFFLR far exceeds the number reported by FLAIR ( Glinos et al., 2022), which identified 93,718 transcripts across 21,067 genes, of which 77% were novel. 34,876 transcripts in 11,840 gene loci were in common between the set of transcripts identified by ( Glinos et al., 2022) and by this study&#x2026;&#x201d; &#x00a0;</p>
                <p> </p>
                <p> 5. Would the authors be able to speculate on any specific use case (a specific cohort, a specific technology, etc) where the exons-to-reads alignment approach might be especially beneficial compared to traditional minimap2-based approaches? This might add some insight to the discussion.</p>
                <p> 
                    <bold>Response:</bold> NIFFLR is designed to be used with well-annotated genomes, such as the human genome, where almost all exons are known and most of (or all) novel transcripts are due to alternative splicing. We stated the limitation in the manuscript.&#x00a0; Using exons from the reference annotation allows us to avoid performing spliced alignment to high-error transcriptomic reads, thereby achieving higher sensitivity and more accurate quantification of reference transcripts.</p>
                <p> </p>
                <p> 
                    <bold>Minor concerns:</bold>
                </p>
                <p> 1.&#x00a0;As the development of the method is of central importance to the paper, the implementation could be expanded on or explained a bit more. In particular, the concepts of the &#x201c;implied&#x201d; starts / ends of the alignments were confusing in Figure 1 and related text. Similarly, Figure 2 was a bit overly-complicated, and perhaps the authors could consider presenting the possible transcript paths separately as alternative transcripts in genome browser format (ie IGV or UCSC).</p>
                <p> 
                    <bold>Response: </bold>&#x00a0;We have completely revised the Methods section to improve clarity and added a workflow diagram of the algorithm to the Supplementary materials as Figure S1.&#x00a0; Here is the updated algorithm description from the Methods:</p>
                <p> After building the alignments, we assign each long read to a gene locus using a majority vote approach. Specifically, for each read, we compute the total number of K-mers in all LCSs for matching exons across different gene loci and assign the read to the locus L whose exons collectively have the highest total number of matching K-mers. Alignments of exons that belong to different gene loci are then discarded. Next, we build the transcript matching the read by finding the best tiling of the read using exons that belong to locus L&#x2014;that is, the sequence of exons that maximizes read coverage while minimizing gaps or overlaps in the implied alignment coordinates. The long read defines a 5&#x2019; to 3&#x2019; forward direction, which provides a natural topological order. We therefore sort the aligned exons by their alignment start coordinates if they align in the forward direction, or by their alignment end coordinates if they align in the reverse direction. Since we only kept alignments of exons that all belong to a single gene locus L, the exons must all align either in the forward or reverse direction. For simplicity, we describe the algorithm below assuming all exons are aligned in the forward direction; the reverse case is handled the same by reversing the read.</p>
                <p> We represent the exon tiling problem as a directed graph, where nodes correspond to exons, and node weights are exon lengths.&#x00a0; An edge connects the 3&#x2019; end of an exon A to the 5&#x2019; end of exon B if the absolute value of the distance between their aligned positions on the read is less than 20bp. The weight of the edge between exons A and B is defined as this distance.&#x00a0; Next, we choose the &#x201c;start&#x201d; nodes.&#x00a0; A valid start node is an exon that is not connected on the 5&#x2019; end and whose 5&#x2019; end lies upstream of the read start (i.e., has a negative coordinate relative to the read origin), indicating an overhang. If no such exon exists, we select the exon(s) with the smallest positive coordinate among all exons aligned to the read. If multiple exons share the same alignment start due to alternative splicing, we use all such exons as alternative start nodes. We select &#x201c;end&#x201d; nodes in a similar manner: an end node is an exon not connected on the 3&#x2019; end and whose 3&#x2019; ends extends beyond the end of the read (i.e., has an overhang). If none do, we choose the exon(s) whose 3&#x2019; end coordinate is closest to the 3&#x2019; end of the read.</p>
                <p> We solve the exon tiling problem by finding the path through the graph that starts from any start node and ends at an end node and minimizes a penalty function. The penalty is defined as the sum of edge weights along the path plus an overhang penalty, calculated as defined as 0.1&#x00d7; (|5&#x2019; overhang of the start exon| + |3&#x2019; overhang of the end exon|), where |a| denotes the absolute value of a. Ideally, there should be no gaps or overlaps between aligned exons in the transcript, resulting in a perfect path with zero weight.&#x00a0; However, because psa_aligner computes approximate alignments, exon start and end alignment coordinates are imprecise estimates. In case of a tie, we select the path with the larger total node weight (the sum of the lengths of the exons on the path). If a read can be spanned by a single exon &#x2014; either because only one exon maps to the read, or because that exon simultaneously has the start closest to the 5&#x2032; end and the end closest to the 3&#x2032; end of the read&#x2014; we report that single exon as the path. If multiple exons individually span the read, we select the exon that satisfies the condition of being a valid start exon or end exon and has the smallest total overhang length. Finally, if no valid path is found, we report the exon with the longest alignment to the read. Figure 2 illustrates an example of a valid exon path. Once the best path is identified, we examine the genomic coordinates of the exons, which are encoded in their sequence IDs. We discard the path if any exons in it overlap in genomic coordinates, as this likely indicates a long read is chimeric a substantial local genome rearrangement that NIFFLR cannot handle.</p>
                <p> </p>
                <p> 2. At the end of the first paragraph of main text on page 7, the authors state &#x201c;This result demonstrates that when novel isoform discovery and quantification are the primary goals, NIFFLR is the best tool&#x201d; when referring to the simulation experiment where they measured isoform detection / assembly only. This means this result has no relevance for quantification. 
                    <bold>Response: </bold>We thank the reviewer for this observation. We agree that the original placement of the statement implied a conclusion about quantification based solely on isoform detection/assembly, which could be misleading. To address this, we have moved the statement to the end of the last paragraph in the &#x201c;Comparison on simulated long reads&#x201d; section, immediately following the discussion of the quantification benchmarking results. This placement more accurately reflects the context of the data and avoids conflating isoform discovery with quantification performance.</p>
                <p> </p>
                <p> 3.&#x00a0;If the authors mean &#x201c;false positive transcripts&#x201d; by &#x201c;spurious transcripts&#x201d;, they should simply refer to them as the latter as there is no definition for &#x201c;spurious&#x201d; transcripts in the text.</p>
                <p> 
                    <bold>Response: </bold>&#x00a0;We have changed the wording from &#x201c;spurious transcripts&#x201d; to &#x201c;false positive transcripts&#x201d;.</p>
                <p> </p>
                <p> 4. In table 2, IsoQuant has the same number of known and total isoforms; implying it found no novel isoforms. I am highly doubtful this is correct and is probably a typo.</p>
                <p> 
                    <bold>Response: </bold>Thank you for pointing this out, indeed this was a typo and we corrected it.</p>
                <p> </p>
                <p> 5. Figure 3 is missing y axis labels. Furthermore, the meaning of the box and whiskers are elaborated on in the results, which should just be in the legend. Additionally, Figure 3b is just a zoomed in duplicate of two plots from 3a and is unnecessary.</p>
                <p> 
                    <bold>Response: </bold>&#x00a0;We have added the y-axis label to Figure 3. We also wish to clarify that Figure 3b is not a zoomed-in duplicate of plots from 3a. While Figure 3a shows quantification error across all transcripts detected by each tool, Figure 3b focuses specifically on the shared subset of transcripts quantified by both NIFFLR and IsoQuant. This allows for a fair, direct comparison between the two programs on the transcripts shared by both, highlighting differences in quantification performance that could be obscured when considering all transcripts.</p>
                <p> </p>
                <p> 6. For Figure 4 the authors describe a strange method of depth normalization to compare the long-read RNA-seq to short-read RNA-seq transcript quantification estimates. They should use normal TPM / CPM normalization. This figure also has unnecessary details about the plot in the results section which should be in the legend (same as in Figure 3).</p>
                <p> 
                    <bold>Response: </bold>&#x00a0;TPM and transcript coverage are linearly related, and many long-read transcriptome quantification programs report read counts rather than TPMs.&#x00a0; Transcript coverage computed from the Illumina data is directly comparable to the number of long reads covering each transcript. By normalizing the Illumina coverage by the total sequencing depth, we made Illumina read coverage into a metric equivalent to long read counts, enabling a fair comparison between long- and short-read quantifications.&#x00a0;&#x00a0;&#x00a0;</p>
                <p> </p>
                <p> 7.&#x00a0;The authors make no references to their supplementary material PDF in the main body of the text. They should include references so that readers know where to find the calls made to perform the benchmarking etc.</p>
                <p> 
                    <bold>Response: </bold> Thank you for noting that, we added the missing references to Supplementary Materials.</p>
            </body>
        </sub-article>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report393921">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.181114.r393921</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Dewey</surname>
                        <given-names>Colin</given-names>
                    </name>
                    <xref ref-type="aff" rid="r393921a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0003-1498-9254</uri>
                </contrib>
                <aff id="r393921a1">
                    <label>1</label>University of Wisconsin-Madison, Wisconsin, USA</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>4</day>
                <month>8</month>
                <year>2025</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Dewey C</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport393921" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.164583.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The authors describe a novel method and associated software, NIFFLR, for identifying and quantifying expressed transcript structures (both known and novel) from long, noisy RNA sequencing data, such as that produced by Oxford Nanopore Technologies (ONT). &#x00a0;A key challenge for methods addressing this task is the difficulty in identifying the precise locations of exon boundaries in the presence of frequent sequencing errors, particularly indels. &#x00a0;The novel approach taken by NIFFLR is the converse of that of other methods: instead of aligning reads to the genome, NIFFLR aligns annotated exons within the genome to the read. &#x00a0; NIFFLR uses a partial suffix array approach to efficiently align exons to the reads and a graph-based algorithm to identify the likely transcript structure for each read, followed by a series of heuristics to filter and quantify the transcripts. &#x00a0;With both simulated and real data, NIFFLR's accuracy is compared to that of three other methods: FLAIR2, IsoQuant, and ESPRESSO. &#x00a0;These experiments suggest that NIFFLR has high sensitivity and quantification accuracy comparable to the next best method. &#x00a0;Runtime measurements show that NIFFLR runs in time comparable to the fastest methods.</p>
            <p> </p>
            <p> Major comments:</p>
            <p> </p>
            <p> 1. A key limitation of the method is its reliance on a known set of exons: all predicted transcripts must be combinations of known exons. &#x00a0;It appears that even slight variations in the boundaries of exons must be previously annotated for the method to predict transcripts with those variant exons. This general issue is brought up in the last two sentences of the discussion. &#x00a0;The authors suggest that when the full exon set is not known, short read data could be used to delineate the exons. &#x00a0;That may be a feasible strategy, but it is not implemented or evaluated in this work, nor do the evaluations consider novel exons. &#x00a0;This work would be strengthened by evaluations that include the more realistic scenario of novel exons (including 5' and 3' end variants of known exons). &#x00a0;By definition, NIFFLR will not be able to detect or quantify transcripts including these exons, but the impact of these exons on the precision and quantification accuracy of other transcripts should be measured. &#x00a0;Alternatively, the authors could implement the exon delineation strategy that they mentioned and examine NIFFLR's performance in combination with this strategy. &#x00a0;Admittedly, the evaluation on the real data set potentially includes reads from transcripts with novel exons, but it is difficult to discern from the results presented how any novel exons impacted the method's performance.</p>
            <p> </p>
            <p> 2. A number of details of the method are omitted or unclear. &#x00a0;A more formal and detailed presentation of the algorithm is needed. &#x00a0;For example, (A) what algorithm is used to solve the "exon tiling problem"? (B) what, precisely, is the objective function? (C) is the algorithm guaranteed to find the optimal solution? (D) how is an edge allowed between overlapping exons? (E) what is the intuition behind distributing reads "proportionally to the size of the container transcripts"? &#x00a0;Personally, I would like to see a more mathematical presentation and a figure depicting the steps of the entire procedure, particularly those detailed in the paragraph beginning "By design, all intron junctions...".</p>
            <p> </p>
            <p> 3. With respect to the software package, I was ultimately able to compile the software after installing the Boost and zlib development libraries. &#x00a0;It would be helpful to have these compilation dependencies noted in the README. &#x00a0;For ease of use by the community, it would also be helpful to have the package available via conda and/or Docker. &#x00a0;Finally, please provide a small test dataset with the software such that users can make sure that their installation is working and can see what the inputs and outputs look like.</p>
            <p> </p>
            <p> Minor comments:</p>
            <p> </p>
            <p> 4. From the filtering steps of the algorithm, could there be an output provided that might indicate the presence of novel exons at a given locus? &#x00a0;For example, could the user be provided the number of reads that mapped to a locus but that were not ultimately assigned to a transcript?</p>
            <p> </p>
            <p> 5. For quantification evaluations, the box and whisker plots of log2 fold changes are helpful, but I would have also liked to have seen scatterplots of true vs. predicted counts (on log-scaled axes) along with correlation values, as has been common for short-read quantification methods. &#x00a0;Such scatterplots help to visualize potential subsets of transcripts that have biased predictions and trends relative to the magnitude of expression.</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Partly</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Yes</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Yes</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Partly</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Computational biology, bioinformatics, transcriptomics</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
        <sub-article article-type="response" id="comment14840-393921">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Zimin</surname>
                            <given-names>Aleksey</given-names>
                        </name>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>24</day>
                    <month>10</month>
                    <year>2025</year>
                </pub-date>
            </front-stub>
            <body>
                <p>
                    <bold>Major comments:</bold>
                </p>
                <p> </p>
                <p> 1. A key limitation of the method is its reliance on a known set of exons: all predicted transcripts must be combinations of known exons.&#x00a0; It appears that even slight variations in the boundaries of exons must be previously annotated for the method to predict transcripts with those variant exons. This general issue is brought up in the last two sentences of the discussion.&#x00a0; The authors suggest that when the full exon set is not known, short read data could be used to delineate the exons.&#x00a0; That may be a feasible strategy, but it is not implemented or evaluated in this work, nor do the evaluations consider novel exons.&#x00a0; This work would be strengthened by evaluations that include the more realistic scenario of novel exons (including 5' and 3' end variants of known exons).&#x00a0; By definition, NIFFLR will not be able to detect or quantify transcripts including these exons, but the impact of these exons on the precision and quantification accuracy of other transcripts should be measured.&#x00a0; Alternatively, the authors could implement the exon delineation strategy that they mentioned and examine NIFFLR's performance in combination with this strategy.&#x00a0; Admittedly, the evaluation on the real data set potentially includes reads from transcripts with novel exons, but it is difficult to discern from the results presented how any novel exons impacted the method's performance.</p>
                <p> 
                    <bold>Response: </bold>We agree with the reviewer&#x2019;s observation that using a known set of exons is a key limitation of NIFFLR.&#x00a0; NIFFLR is designed to be used with well-annotated genomes, such as the human genome, where almost all exons are known and most novel transcripts are due to alternative splicing. We have added an analysis of the synthetic SIRV E2 data, which includes alternative exons whose boundaries differ by only a few bases, and we show that NIFFLR and ESPRESSO were able to find all but one of the isoforms that are present in the sample.&#x00a0;&#x00a0;&#x00a0;</p>
                <p> </p>
                <p> </p>
                <p> 2. A number of details of the method are omitted or unclear.&#x00a0; A more formal and detailed presentation of the algorithm is needed.&#x00a0; For example, (A) what algorithm is used to solve the "exon tiling problem"? (B) what, precisely, is the objective function? (C) is the algorithm guaranteed to find the optimal solution? (D) how is an edge allowed between overlapping exons? (E) what is the intuition behind distributing reads "proportionally to the size of the container transcripts"?&#x00a0; Personally, I would like to see a more mathematical presentation and a figure depicting the steps of the entire procedure, particularly those detailed in the paragraph beginning "By design, all intron junctions...".</p>
                <p> 
                    <bold>Response: </bold> We added a flowchart of the method as Supplementary Figure S1 and substantially revised the Methods section to improve clarity. The revised text is below:</p>
                <p> </p>
                <p> We represent the exon tiling problem as a directed graph, where nodes correspond to exons, and node weights are exon lengths.&#x00a0; An edge connects the 3&#x2019; end of an exon A to the 5&#x2019; end of exon B if the absolute value of the distance between their aligned positions on the read is less than 20bp. The weight of the edge between exons A and B is defined as this distance.&#x00a0; Next, we choose the &#x201c;start&#x201d; nodes.&#x00a0; A valid start node is an exon that is not connected on the 5&#x2019; end and whose 5&#x2019; end lies upstream of the read start (i.e., has a negative coordinate relative to the read origin), indicating an overhang. If no such exon exists, we select the exon(s) with the smallest positive coordinate among all exons aligned to the read. If multiple exons share the same alignment start due to alternative splicing, we use all such exons as alternative start nodes. We select &#x201c;end&#x201d; nodes in a similar manner: an end node is an exon not connected on the 3&#x2019; end and whose 3&#x2019; ends extends beyond the end of the read (i.e., has an overhang). If none do, we choose the exon(s) whose 3&#x2019; end coordinate is closest to the 3&#x2019; end of the read.</p>
                <p> </p>
                <p> We solve the exon tiling problem by finding the path through the graph that starts from any start node and ends at an end node and minimizes a penalty function. The penalty is defined as the sum of edge weights along the path plus an overhang penalty, calculated as defined as 0.1&#x00d7; (|5&#x2019; overhang of the start exon| + |3&#x2019; overhang of the end exon|), where |a| denotes the absolute value of a. Ideally, there should be no gaps or overlaps between aligned exons in the transcript, resulting in a perfect path with zero weight.&#x00a0; However, because psa_aligner computes approximate alignments, exon start and end alignment coordinates are imprecise estimates. In case of a tie, we select the path with the larger total node weight (the sum of the lengths of the exons on the path). If a read can be spanned by a single exon &#x2014; either because only one exon maps to the read, or because that exon simultaneously has the start closest to the 5&#x2032; end and the end closest to the 3&#x2032; end of the read&#x2014; we report that single exon as the path. If multiple exons individually span the read, we select the exon that satisfies the condition of being a valid start exon or end exon and has the smallest total overhang length. Finally, if no valid path is found, we report the exon with the longest alignment to the read. Figure 2 illustrates an example of a valid exon path. Once the best path is identified, we examine the genomic coordinates of the exons, which are encoded in their sequence IDs. We discard the path if any exons in it overlap in genomic coordinates, as this likely indicates a long read is chimeric a substantial local genome rearrangement that NIFFLR cannot handle.</p>
                <p> </p>
                <p> 3. With respect to the software package, I was ultimately able to compile the software after installing the Boost and zlib development libraries.&#x00a0; It would be helpful to have these compilation dependencies noted in the README.&#x00a0; For ease of use by the community, it would also be helpful to have the package available via conda and/or Docker.&#x00a0; Finally, please provide a small test dataset with the software such that users can make sure that their installation is working and can see what the inputs and outputs look like.</p>
                <p> 
                    <bold>Response: </bold>We added information about software dependencies to the README file.&#x00a0; We also included a script that runs NIFFLR on the SIRV E2 dataset, that downloads the reads from SRA using fastq-dump. In addition, we have made NIFFLR available on Bioconda, where it can now be installed as &#x2018;nifflr&#x2019;.</p>
                <p> </p>
                <p> </p>
                <p> 
                    <bold>Minor comments:</bold>
                </p>
                <p> </p>
                <p> 4. From the filtering steps of the algorithm, could there be an output provided that might indicate the presence of novel exons at a given locus?&#x00a0; For example, could the user be provided the number of reads that mapped to a locus but that were not ultimately assigned to a transcript?</p>
                <p> 
                    <bold>Response: </bold> At this time, we do not provide this directly, but this information is available in an intermediate GTF file that lists all reads contributing to each transcript.&#x00a0; The file is named .gtf. If a transcript listed in that file is not present among the final output transcripts, it indicates that the reads supporting that transcript were ultimately discarded.</p>
                <p> </p>
                <p> </p>
                <p> 5. For quantification evaluations, the box and whisker plots of log2 fold changes are helpful, but I would have also liked to have seen scatterplots of true vs. predicted counts (on log-scaled axes) along with correlation values, as has been common for short-read quantification methods.&#x00a0; Such scatterplots help to visualize potential subsets of transcripts that have biased predictions and trends relative to the magnitude of expression.</p>
                <p> 
                    <bold>Response:&#x00a0;</bold> We added the scatter plots to the Supplementary Materials as Figures S2-S6.&#x00a0; We also included the Pearson correlation coefficient values in the plots shown in the main manuscript.</p>
            </body>
        </sub-article>
    </sub-article>
</article>
