Metrics
Views
22611
Downloads
1748
Get PDF
Get XML
Cite
Export
Track
Research Article
A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations]
Yoav GiladOrna Mizrahi-Man
Yoav GiladOrna Mizrahi-Man
PUBLISHED 19 May 2015
Author affiliations Author Affiliations
Grant information Grant Information
OPEN PEER REVIEW
REFEREE STATUS
Refutation study
This article is included in the Preclinical Reproducibility and Robustness channel.

Abstract

Recently, the Mouse ENCODE Consortium reported that comparative gene expression data from human and mouse tend to cluster more by species rather than by tissue. This observation was surprising, as it contradicted much of the comparative gene regulatory data collected previously, as well as the common notion that major developmental pathways are highly conserved across a wide range of species, in particular across mammals. Here we show that the Mouse ENCODE gene expression data were collected using a flawed study design, which confounded sequencing batch (namely, the assignment of samples to sequencing flowcells and lanes) with species. When we account for the batch effect, the corrected comparative gene expression data from human and mouse tend to cluster by tissue, not by species.

Recently, the Mouse ENCODE Consortium reported that comparative gene expression data from human and mouse tend to cluster more by species rather than by tissue. This observation was surprising, as it contradicted much of the comparative gene regulatory data collected previously, as well as the common notion that major developmental pathways are highly conserved across a wide range of species, in particular across mammals. Here we show that the Mouse ENCODE gene expression data were collected using a flawed study design, which confounded sequencing batch (namely, the assignment of samples to sequencing flowcells and lanes) with species. When we account for the batch effect, the corrected comparative gene expression data from human and mouse tend to cluster by tissue, not by species.

Introduction

The mouse ENCODE Consortium has collected multiple types of genomic and functional data in order to better understand the potential utility of the mouse as a model system for biomedical research. To study gene expression levels, the Consortium collected RNA sequencing data from multiple tissues from human and mouse. Their comparative analysis revealed that gene expression patterns tend to support clustering of the data by species, rather than by tissue (Figure 2a in reference 1).

This pattern was confirmed and discussed in greater detail in a companion paper by Lin et al.2, which also acknowledged that this observation is somewhat unexpected. Indeed, previous comparative studies reported that gene expression data from human and mouse (and across other species more generally) tend to cluster by tissues, not by species. Lin et al. proposed that previous studies might have been biased in their focus on a few ‘specialized’ tissues that tend to express the largest number of ‘tissue-specific genes’, while the overall pattern supports less tissue specificity.

The implications of the observation that human and mouse gene expression data may be clustering by species more than by tissues can be profound. To a large degree, modern biology is built upon the empirical observation that homologous gene regulatory networks establish the identities of homologous cell-types, tissues, and organs across species – the results of Lin et al., if true, challenge these observations and the biological basis of homology. From a more practical perspective, the mouse is arguably the most important animal model for biomedical research. If gene regulation in any mouse tissue is markedly more representative of a general mouse regulatory network than the regulatory network of a corresponding human tissue, this would call into question the utility of the mouse, and perhaps any other non-human animal, as a useful model system for biomedical research.

Here, we present a reanalysis of the mouse ENCODE Consortium comparative RNA sequencing data. We argue that a flaw in their study design raises doubt regarding their conclusions.

Methods

RNA-Seq data, genome and gene annotation files

In December 2014 we asked and were kindly provided by the authors of Lin et al.2 the names of the sequence files used in their comparative analysis. Based on this information we obtained sequence files in FASTQ format (Supplementary Table 1) from the ENCODE project1 site (https://www.encodeproject.org/; some of the files were only available from early January 2015).

For our analysis, we used the same genome build and gene annotation files as in Lin et al.2. The ENSEMBL3 genome build Mus musculus GRCm38.68 was downloaded from ftp://ftp.ensembl.org/pub/release-68/fasta/mus_musculus/dna/Mus_musculus.GRCm38.68.dna_sm.toplevel.fa.gz; the corresponding transcript annotation file was downloaded from ftp://ftp.ensembl.org/pub/release-68/gtf/mus_musculus/Mus_musculus.GRCm38.68.gtf.gz. The Homo sapiens genome build provided by ENSEMBL3 contains haplotypic regions that are not part of the primary assembly. To avoid these regions, genome build Homo sapiens GRCh37 was downloaded from the Illumina iGenomes page: (http://support.illumina.com/sequencing/sequencing_software/igenome.html). The GENCODE4 Release 14 transcript annotation file for human was downloaded from ftp://ftp.sanger.ac.uk/pub/gencode/release_14/gencode.v14.annotation.gtf.gz. The chromosome names in the GENCODE gtf file did not match those in the genome sequence file, and were thus modified.

Sequencing study design

Based on the sequence identifiers found in the FASTQ files, we reconstructed the sequencing study design used to collect the gene expression data in Lin et al.2. The sequence identifier line in a FASTQ file generated from an Illumina sequencing run can take two formats, depending on the version of the Consensus Assessment of Sequence and Variation (CASAVA) pipeline used to generate it. Prior to version 1.8 of this pipeline the sequence identifier line was of the following format (CASAVA v1.7 user guide p.88; downloaded from: http://support.illumina.com/downloads/casava_software_version_17_user_guide_(15011196_a).html

@<machine_id>:<lane>:<tile>:<x_coord>:<y_coord>#<index>/<read_#>

Starting from version 1.8 the sequence identifier line is of the format http://support.illumina.com/help/SequencingAnalysisWorkflow/Content/Vault/Informatics/Sequencing_Analysis/CASAVA/swSEQ_mCA_FASTQFiles.htm

@<machine_id>:<run number>:<flowcell ID>:<lane>:<tile>:<x-pos>:<y-pos> <read>:<is filtered>:<control number>:<index sequence>

Below is a sequence identifier line from the mouse pancreas read1 FASTQ file (sequence identifier lines from the remaining FASTQ files were of similar format):

@D4LHBFN1:276:C2HKJACXX:4:1101:3448:12374 1:N:0:AGTTCC

Based on this information we inferred that the FASTQ files were generated by CASAVA version 1.8 or higher. Thus, we could extract from the sequence identifiers the following details that pertain to the sequencing study design: machine identifier, run number, flowcell identifier, and flowcell lane number. We found that the sequencing was performed in five batches, each consisting of a multiplexed single run on a single lane on one of four sequencers (Figure 1; note that two of the batches, composed of human samples only, differed only in their lane number). The design was such that only one batch contained samples from both species. The remaining four batches could be divided into pairs where each of the two batches had a nearly identical tissue composition, but a different species.

9f5f4330-d81d-46b8-9a3f-d8cb7aaf577e_figure1.gif

Figure 1. Study design.

Sequencing batches as inferred based on the sequence identifiers of the RNA-Seq reads.

Ortholog annotation

Following Lin et al.2, we used the protein-coding ortholog list generated by the modENCODE and mouse ENCODE consortia5. A file containing all orthologs from human, mouse, fly and worm was downloaded from http://compbio.mit.edu/modencode/orthologs/modencode.common.orth.txt.gz. From this list we extracted 14,744 human-mouse one-to-one ortholog pairs, for which both members were included in the transcript annotation files we used. We note that this number is lower than the ~15,106 ortholog pairs reported in Lin et al. We are not certain of the meaning of the ‘~’ in the report of the number of ortholog pairs analyzed by Lin et al. Nevertheless, we believe that a possible explanation for this disparity is a parsing error. The last two columns of the ‘modENCODE ortholog file’ represent the number of genes from each species in the ortholog group. One of the steps required to obtain the subset of ortholog groups for analysis is to select those records where the two last columns have a value of 1 (i.e. one-to-one ortholog pairs). We found that if this selection is done through a command line search that does not require that the value in the last column be exactly “1”, but rather just begins with “1”, then the result is 15,104 putative human-mouse ortholog pairs.

Quality assessment of RNA-Seq data

We used the FastQC software v0.10.0 (http://www.bioinformatics.babraham.ac.uk/projects/fastqc/) to assess the quality of the individual FASTQ files (Supplementary Table 2Supplementary Table 6). We were concerned by evidence for GC content bias and overrepresented sequences. To examine the latter in greater detail, we mapped the sequences overrepresented in at least one sample to the genome of the respective species, using BLAT searches6 against the hg19 (human) and mm10 (mouse) assemblies at the UCSC genome browser site (http://genome.ucsc.edu/)6. We found that in both species many of the overrepresented sequences mapped perfectly to the mitochondrial genome (Supplementary Table 3Supplementary Table 6). For the mouse pancreas sample only, we also found many overrepresented sequences mapped to regions with rRNA repeats from the SSU-rRNA_Hsa and LSU-rRNA_Hsa families.

Mapping RNA-Seq reads to genome sequences

We mapped the RNA-Seq reads to their respective genomes using Tophat v2.0.117 with the following options: “--mate-inner-dist 200” (i.e. inner mate distance is 200nt, based on paired-end reads with length 100nt each and an insert size of 350-450nt ); “--bowtie-n” (i.e. the “-n” option will be used in Bowtie8 in the initial read mapping stage); “-g 1” (i.e. multi-mapping reads will be excluded from alignment); “-m 1” (i.e. one mismatch is allowed in the anchor region of a spliced alignment); “--library-type fr-firststrand” (the libraries had been constructed using the Illumina TruSeq Stranded mRNA LT Sample Prep Kit2). An exception was the mouse pancreas sample, for which the mapping process stalled consistently at the same stage. For this sample we used Tophat v1.4.18 with the same options as above. Tophat requires a Bowtie8 index. For human we used the Bowtie index that was packaged with the genome sequence in the file downloaded from the Illumina iGenomes page (http://support.illumina.com/sequencing/sequencing_software/igenome.html). For mouse we built an index using the bowtie-build utility from Bowtie v2.2.1 (v 0.12.7 for the index used with Tophat v1.4.1).

Calculating gene GC content

For each of the two species we used the appropriate GTF file to generate a table, which contains for each gene its ENSEMBL gene identifier its common name, and the GC content of the sequence covered by the union of the gene’s transcripts. To this end, we first generated a GTF file where overlapping exons from different transcripts of the same gene were merged into a single “exon” with the same sequence coverage, retaining the association with the gene identifier. Next, we computed the nucleotide content of the exons in this new GTF file using the ‘nuc’ utility from bedtools v2.17.09. Finally, we computed the GC content for each gene identifier by summing the number of ‘G’ and ‘C’ nucleotides in its merged exons and dividing by the sum of counts of unambiguous nucleotides in these exons.

Per-gene FPKM values

We used Cufflinks v2.2.110 to compute fragments per kilo base of transcript per million (FPKM) values and aggregate them per gene. The only option used was “--library-type fr-firststrand”. For the required transcript annotation file (“-G” parameter) we used the GTF file for the respective species described in the “Genome and gene annotation files” section. We then generated a matrix of 14,744 by 26 FPKM values for each gene (in the ortholog table) and sample. While generating this table we noticed that some of the common gene names were associated with more than one ENSEMBL gene identifier. In some cases we determined that this was due to gene identifiers that have been retired from the ENSEMBL database3 but were retained in the GTF file (27 and 64 retired identifiers for human and mouse, respectively). These retired identifiers were ignored when constructing the FPKM matrix. For the remaining such cases we incorporated the value from the first appearance of the common name.

Per-gene raw fragment counts

To compute per gene raw counts from the alignment files produced by Tophat7, we used the program featureCounts v1.4.411 with the respective species’ GTF file specified in the “Genome and gene annotation files” section. For all runs we used the following options: “-p” - indicates that fragments rather than reads should be counted; “-C” - indicates that chimeric fragments will not be included in the summarization process; and “-s 2” - indicates that the paired-ends are reversely stranded. We next generated a matrix of 14,744 by 26 raw counts for each gene (in the ortholog table) and sample. Since the output from featureCounts identifies genes by their gene identifier (the ENSEMBL identifier in our case), whereas the ortholog table uses the gene’s common name to identify it, we used the GC content table, which contains both these identifiers to match counts to the correct row in the ortholog table. As we did when generating the FPKM matrix, we ignored the values from retired ENSEMBL identifiers, and if there were still multiple identifiers for the same common name, we used the value from the identifier that appeared first.

Results

In this reanalysis effort, we focused solely on the RNA sequencing data that can be mapped to coding regions. Lin et al.2 reported additional results, related to data on the expression of non-coding transcripts and histone marks. We did not reanalyze these additional data types.

Lin et al.2 analyzed both previously published and newly collected human and mouse gene expression data. The previously published data consist of RNA sequencing from ENCODE, the Illumina Human BodyMap 2.0, and the Roadmap Epigenomics Mapping Consortium. In these previously collected data sets, human and mouse samples were analyzed by different labs at different times, such that there is a clear batch effect that is confounded with species. Lin et al.2 clearly explains this limitation of the previously published data. They state that in order to address this issue they focus on the analysis of only the newly collected data – RNA sequencing data of samples from 13 human and mouse tissues that were collected by the same lab, using the same sample processing protocol. We focus our reanalysis study on the same newly collected data set (see Methods).

Replication of sample clustering by species

As a first step of our study we set out to replicate the analysis of Lin et al.2. To do so, we started with the matrix of FPKM values (computed, using Cufflinks10, based on the read alignments to the genome). This analysis was done within R environment v 3.1.3 GUI 1.65 Snow Leopard build (6912)12. See Supplementary Text 1 for detailed commands, and a supplement zip file for the R input (available in Zenodo: http://dx.doi.org/10.5281/zenodo.17606).

We log2-transformed the FPKM matrix (after adding 1 to avoid undefined values). To visualize the data, we used an approach that is similar in principle to that used by the ENCODE mouse consortium and Lin et al. Specifically, we used the function ‘prcomp’ (with the ‘scale’ and ‘center’ options set to TRUE) to perform principal component analysis (PCA) of the transposed FPKM matrix (so that samples were now in rows and genes in columns), after removal of invariant columns (genes). Scatter plots of the PCA results were generated using the ggplot2 package13. In agreement with the findings of Lin et al.2 the samples cluster mostly by species (Figures 2a, Figure S1 and Figure S2). We also plotted the heatmap of the matrix of Pearson correlations between the 26 samples, using the pheatmap function from the pheatmap package v1.0.214 with default settings (i.e. complete linkage hierarchical clustering using the Euclidean distances). Again, samples from the same species tend to cluster together (Figure 2b).

9f5f4330-d81d-46b8-9a3f-d8cb7aaf577e_figure2.gif

Figure 2. Recapitulating the patterns reported by the mouse ENCODE papers.

a. Two-dimensional plots of principal components calculated by performing PCA of the transposed log-transformed FPKM values (from 14,744 orthologous gene pairs) for the 26 samples, after removal of invariant columns (genes). b. Heatmap based on pairwise Pearson correlation of expression data used in panel a. We used Euclidean distance and complete linkage as distance measure and clustering method, respectively.

Analysis of normalized data after accounting for batch effects yields clustering by tissue

A previous evaluation of normalization methods for RNA-Seq data15 suggested that FPKM values were not optimal for clustering analysis. Therefore, as a basis for our reanalysis, we used the matrix of per-gene raw fragment counts. The entire analysis was done within R environment v 3.1.3 GUI 1.65 Snow Leopard build (6912)12. See Supplementary Text 2 for detailed commands, and a supplement zip file for the R input (available in Zenodo: http://dx.doi.org/10.5281/zenodo.17606).

Following Li et al.16, we removed the 30% of genes with the lowest expression as determined by the sum of fragment counts across all samples. Next, due to the presence of mitochondrial genes among the overrepresented sequences in the data, we also removed reads that map to the 12 mitochondrial genes. This left us with expression data from 10,309 genes for analysis. We note that merely limiting the analysis to this subset of genes does not have a marked effect on the patterns reported by Lin et al. (Figure S3; detailed commands in Supplementary Text 3, and a supplement zip file for the R input (available in Zenodo: http://dx.doi.org/10.5281/zenodo.17606)). We performed within-column normalization to remove the GC bias in the data, indicated by the initial quality assessment. To this end, we applied the ‘withinLaneNormalization’ function from the EDASeq package v2.0.017 to each column in the matrix, using the gene GC values for the species associated with the column. Next, we used the ‘calcNormFactors’ from the edgeR package v3.8.618, with the trimmed mean of M-values (TMM) method19, to calculate normalization factors for the library sizes for the samples. We used these normalization factors in the depth normalization of the columns (using the column sums of the original, unfiltered, counts matrix as a proxy for library sizes). The normalized data were log2-transformed (after adding ‘1’ to each value in the matrix to avoid undefined values).

We then considered how to account for the fact that the assignment of samples to sequencing flowcells and lanes was nearly completely confounded with the species annotations of the samples (Figure 1). The consideration of ‘batch effect’ was the most important difference between the analysis that recapitulated the patterns reported by the mouse ENCODE papers (the previous ‘Results’ section) and the current reanalysis effort. Specifically, we accounted for the sequencing study design batch effects using the ‘ComBat’ function from the sva package v3.12.020, with a model that includes effects for batch, species and tissue. For this purpose the samples were classified into five batches, based on the sequencing study design (see methods and Figure 1).

To visualize the data, we used the function ‘prcomp’ (with the ‘scale’ and ‘center’ options set to TRUE) to perform principal component analysis (PCA) of the transposed log-transformed matrix of ‘clean’ values (after removal of invariant columns, i.e. genes), and the ggplot2 package13 to generate scatter plots of the PCA results. None of the first five principal components (accounting together for 56% of the variability in the data) support the clustering of the gene expression data by species (Figure 3a and Figure S4Figure S5). However, the sixth principal component, which accounts for 6% of the variability in the data, does support such a clustering, suggesting that even though the ‘species’ and ‘batch’ variables are confounded, accounting for ‘batch’ does not remove completely the variability due to ‘species’ (Figure S5). We also plotted a heatmap of the matrix of Pearson correlations between the 26 samples, using the pheatmap function from the pheatmap package v1.0.214 with default settings (i.e. complete linkage hierarchical clustering using the Euclidean distances). This time the heatmap shows considerable clustering of the comparaive gene expression data by tissue (Figure 3b).

9f5f4330-d81d-46b8-9a3f-d8cb7aaf577e_figure3.gif

Figure 3. Clustering of data once batch effects are accounted for.

a. Two-dimensional plots of principal components calculated by applying PCA to the transposed matrix of batch-corrected log-transformed normalized fragment counts (from 10,309 orthologous gene pairs that remained after the exclusion steps described in the results) for the 26 samples, after removal of invariant columns (genes). b. Heatmap based on pairwise Pearson correlation of the expression data used in panel a. We used Euclidean distance and complete linkage as distance measure and clustering method, respectively.

Discussion

In our reanalysis we have made a number of specific choices, including the exclusion of a certain subset of lowly expressed genes, the specific approach we chose to summarize the count data, the standardization and normalization methods we used (for example, we chose to standardize by the total count of reads that mapped to the ortholog gene pairs), the approach we used to account for the GC content bias, and the method we used to account for the sequencing design batch effect. Moreover, we excluded the sequencing data from 12 mitochondrial genes from both species, a step that – to the best of our ability to determine – was not taken by the original studies. In addition, our definition of ortholog gene pairs differs slightly from that of the original study, as we discussed in the methods. In practice, only the correction for the sequencing design batch effect had a drastic impact on the results. For example, without accounting for batch, using per-gene raw fragment counts instead of FPKM values does not seem to impact the degree to which the uncorrected data support clustering by species (Figure S6).

Visualizing or plotting the data is another important area where different choices can sometime lead to quite distinct conclusions. We chose to display, in addition to the PCA plots, heatmaps based on the correlations among the samples. We note that if the actual data (not pairwise correlations) are clustered, the observed patterns (by species or by tissues, in the respective analyses), seem practically identical (Figure S7). The heatmaps shown in the main figures are based on Pearson pairwise correlations, which provide the highest level of clustering by tissue in the analysis that takes into account batch effects. Alternative heatmap plots based on either Spearman pairwise correlations or other distance measures and clustering methods look similar in principle (Figure S8 to Figure S10), but the clustering by tissue is somewhat less pronounced (clustering by species, when batch is not accounted for, is more pronounced).

It is important to note that most of the analysis and plotting decisions we have made contributed to a somewhat better clustering of the expression data by tissue, both visually and empirically. We have made these – mostly standard - analysis and plotting choices regardless of the end result (namely, we believe that these are objectively reasonable choices). Importantly, we made identical choices for the clustering analysis and plot types for the data with and without batch correction, and our conclusions are robust with respect to a wide range of possible alternative approaches (Figure S7Figure S10).

That said, we do acknowledge that we find the clustering of the data by tissue to be a more intuitive pattern. In other words, we believe that the clustering of comparative gene expression data by species – a result that contradicts previous observations – is a surprising outcome. Hence, we would have intuitively accepted as more correct most reasonable choices of analysis pipelines and data visualizations that supported a greater degree of clustering by tissue.

As we mentioned above, most of the choices we made resulted in little difference to the overall pattern. It was only the correction for the sequencing design batch effect that had a profound impact. Once we accounted for the batch effect by using ComBat, the comparative gene expression data no longer clustered by species, and instead, we observed a clear tendency for clustering by tissue. This is not surprising, as the sequencing batch, which we corrected for, was nearly entirely confounded with species. It stands to reason that some individual gene expression levels do cluster by species and some by tissue (see for example, Figure S5). While previous data sets strongly support a general clustering of gene regulatory phenotypes by tissue21, we expect the degree of clustering of the gene expression data to differ somewhat across tissues. Yet, in this particular case, by removing the confounding sequencing batch effect we also removed most of the species effect on gene expression levels (a similar case of confounding batch and main effect of interest was discussed a few years ago, with respect to gene expression differences between human populations22).

One could potentially employ more sophisticated modeling approaches to try and estimate separately the batch and species effects. One idea would be to rely on the fact that there are five sequencing batches, but only two species. This, however, is complicated by the fact that the two sequence batches specific to the human samples share the same run and flowcell (potentially a smaller batch effect), while the two sequence batches specific to the mouse samples are extend over different instruments (potentially a larger batch effect). In any case, we feel that such modeling is beyond the scope of this reanalysis effort. Instead, we conclude that the study design used by the mouse ENCODE consortium was flawed with respect to the questions they set out to address.

In summary, we believe that our reanalysis indicates that the conclusions of the Mouse ENCODE Consortium papers pertaining to the clustering of the comparative gene expression data are unwarranted. In the narrow context of our reanalysis effort, we state that their conclusions are unwarranted, not wrong, because the study design was simply not suitable for addressing the question of ‘tissue’ vs. ‘species’ clustering of the gene expression data. That said, a large body of independent previous work supports general clustering of comparative gene expression data by tissue.

Finally, we note that in this reanalysis effort, we have only focused on the RNA sequencing data collected by the mouse ENCODE consortium. We have not considered information with respect to the study design used to collect the many other types of data reported by this consortium. Given our findings, we believe that it is appropriate to call for a careful review of these other data sets as well.

Data availability

All data are available from the Mouse ENCODE consortium; see Table S1 for specific source URLs and accession numbers.

Software availability

We provide supplementary files of the python codes used to process and prepare the data for analysis with R, and the data files for the python codes. We also provide the R codes we used to perform the different analyses as supplementary files, as well as the input for the R codes.

Archived software files as at the time of publication

Zenodo. Data files and codes used in the reanalysis of the mouse encode comparative gene expression data. DOI: 10.5281/zenodo.17606

License

These codes are provided under the MIT license.

Author affiliations Author Affiliations
Article Versions (1)
Copyright
Download
 
Export To
metrics
VIEWS
22611
 
downloads
1748
Citations
0
 
0
CITE
how to cite this article
Gilad Y and Mizrahi-Man O. A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations] F1000Research 2015, 4:121 (doi: 10.12688/f1000research.6536.1)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Referee Status: ?
Key to Referee Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservationsKey revisions are required to address specific details and make the paper fully scientifically sound
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Read more about the unique F1000Research publication and peer review model here.

Version 1
VERSION 1
PUBLISHED 19 May 2015
Views
701
Cite
Referee Report 30 Jun 2015
Lior Pachter, Department of Mathematics, University of California, Berkeley, Berkeley, CA, USA 
Approved
VIEWS 701
The article "A reanalysis of mouse ENCODE comparative gene expression data" by Gilad and Mizrahi-Man examines a claim, recently published in the pair of papers
  1. Yue F, Cheng Y, Breschi A, et al.: A comparative encyclopedia of DNA elements in the
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Pachter L. Referee Report For: A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations]. F1000Research 2015, 4:121 (doi: 10.5256/f1000research.7019.r8710)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
673
Cite
Referee Report 25 Jun 2015
Mick Watson, The Roslin Institute, University of Edinburgh, Edinburgh, UK 
Approved
VIEWS 673
The study is carried out well and the results support the conclusion.

The paper would benefit from including some of the discussion points made in the comments made to v1 of the paper. Lin et al. have re-sequenced the samples and ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Watson M. Referee Report For: A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations]. F1000Research 2015, 4:121 (doi: 10.5256/f1000research.7019.r8832)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
714
Cite
Referee Report 22 Jun 2015
Michael Eisen, Howard Hughes Medical Institute, University of California, Berkeley, Berkeley, CA, USA 
Approved
VIEWS 714
In this paper Gilad and Mizrahi-Man reanalyze a high-profile dataset from the ENCODE consortium that was used to argue that gene expression levels are more similar for different tissues from the same species than the same tissues from different species, ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Eisen M. Referee Report For: A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations]. F1000Research 2015, 4:121 (doi: 10.5256/f1000research.7019.r8942)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
1268
Cite
Referee Report 26 May 2015
Rafael Irizarry, Department of Biostatistics, Harvard School of Public Health, Boston, USA 
Approved with Reservations
VIEWS 1268
In this PNAS paper is is found that the first three principal components obtained from mouse and human gene expression data correlate with species and not with tissue. This is interpreted to imply that "tissues appear more similar to one another within ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Irizarry R. Referee Report For: A reanalysis of mouse ENCODE comparative gene expression data [version 1; referees: 3 approved, 1 approved with reservations]. F1000Research 2015, 4:121 (doi: 10.5256/f1000research.7019.r8732)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
  • Author Response 26 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    26 May 2015
    Author Response
    Dr. Irizarry,
     
    Thank you for spending the time to provide a review of our work. We agree with you that given the study design used by the mouse ENCODE consortium, applying ... Continue reading
COMMENTS ON THIS REPORT
  • Author Response 26 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    26 May 2015
    Author Response
    Dr. Irizarry,
     
    Thank you for spending the time to provide a review of our work. We agree with you that given the study design used by the mouse ENCODE consortium, applying ... Continue reading

Comments on this article Comments (33)

Version 1
VERSION 1 PUBLISHED 19 May 2015
  • Reader Comment 10 Aug 2015
    Lenny Teytelman, Protocols.io, USA
    10 Aug 2015
    Reader Comment
    There continues to be a lot of discussion about the proper way to review and comment on research post-publication. Many believe that raising questions in public without contacting the journal ... Continue reading
  • Author Response 07 Aug 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    07 Aug 2015
    Author Response
    Shin and co authors,
     
    Thank you for the thoughtful comment. Based on your comment, and recent discussions I have had with Mike (at Banff), I believe that there may be a ... Continue reading
  • Reader Comment 06 Aug 2015
    Shin Lin, Department of Genetics, Stanford University, USA
    06 Aug 2015
    Reader Comment
    Since our last posting, we would like to respond to the following points which have been raised by various colleagues in the field regarding Lin et al.1 and Gilad et ... Continue reading
  • Author Response 09 Jul 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    09 Jul 2015
    Author Response
    Anshul,
     
    Thank you for your thoughtful comment (and for the discussion over Twitter). We appreciate the time and effort you put into this.
     
    You raise several issues, but the most important one ... Continue reading
  • Reader Comment 08 Jul 2015
    Anshul Kundaje, Stanford University School of Medicine, USA
    08 Jul 2015
    Reader Comment
    These comments are specifically wrt. to the slides that Yoav posted a few weeks ago on twitter goo.gl/YPNQ4H. The slides perform several interesting analyses on the "old" (sequencer confounded) and ... Continue reading
  • Author Response 25 Jun 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    25 Jun 2015
    Author Response
    For those with little time to read the entire comment, and those who are not invested enough in this area to study all the details of the papers and back ... Continue reading
  • Author Response 08 Jun 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    08 Jun 2015
    Author Response
    It should be noted that on June 7th, ENCODE authors have fixed the sample annotation with respect to sex (details available on the ENCODE website, or on twitter...). Unfortunately the ... Continue reading
  • Reader Comment 05 Jun 2015
    Nicolas Robine, New York Genome Center, USA
    05 Jun 2015
    Reader Comment
    We downloaded the original dataset supporting the claims in Shin et al (files listed here in Table S1) and processed them through our standard QC pipeline. This include mapping to ... Continue reading
  • Author Response 05 Jun 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    05 Jun 2015
    Author Response
    While the genders of the donors were not reported in the original paper, we found that information in the biosample file on the ENCODE site. From this file, we learned ... Continue reading
  • Reader Comment 04 Jun 2015
    Lenny Teytelman, Protocols.io, USA
    04 Jun 2015
    Reader Comment
    As an advocate of post-publication peer review and discussion, I find this exchange between the Gilad and Snyder groups fascinating. It is precisely the kind of discussion that is illuminating ... Continue reading
  • Reader Comment 03 Jun 2015
    J Michael Cherry, Department of Genetics, Stanford University, USA
    03 Jun 2015
    Reader Comment
    Data for the Lin et al. experiments were added to the ENCODE Portal last week. See http://goo.gl/C3yUwg.  Use the ‘Download’ link to retrieve URLs to retreive metadata and data files.

    We, ... Continue reading
  • Author Response 02 Jun 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    02 Jun 2015
    Author Response
    Shin et al. wrote on May 21st that they collected new data, using a different sequencing design, and that the new data still support their original claim. Regardless of our ... Continue reading
  • Reader Comment 31 May 2015
    Mick Watson, Division of Genetics and Genomics, The Roslin Institute, University of Edinburgh, UK
    31 May 2015
    Reader Comment
    In Lin et al, the paper mentions RNA-Seq carried out at both Stanford and Salk.

    The 26 datasets analysed here (13 mouse and 13 human), listed in Table S1, are they ... Continue reading
  • Reader Comment 27 May 2015
    Michele Busby, Broad Institute, USA
    27 May 2015
    Reader Comment
    My last comment regarding the gene ontology analysis may not hold water. From the description of the gene ontology analysis in the original paper, it appears that they did not ... Continue reading
  • Reader Comment 26 May 2015
    Christopher Mason, Department of Physiology and Biophysics, Weill Cornell Medical College, USA
    26 May 2015
    Reader Comment
    Instead of just re-sequencing the same libraries, a better idea would be to try and ribo-deplete their original RNAs to remove the potentially confounding effect of RNA degradation on the ... Continue reading
  • Reader Comment 26 May 2015
    David Lovell, QUT, Australia
    26 May 2015
    Reader Comment
    Since this study makes use of correlation as a measure of association between the logged gene expression levels of different tissues, I think it’s important to point out that correlation ... Continue reading
  • Reader Comment 23 May 2015
    Uri David Akavia, Department of Biochemistry, McGill University, Canada
    23 May 2015
    Reader Comment
    Dear Yoav,

    If you look at Table 1 of the GTEx pilot article in Science, May 8th (DOI: 10.1126/science.1262110), you can see RIN values of the relevant human tissues. I'm guessing that ... Continue reading
  • Author Response 22 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    22 May 2015
    Author Response
    Shin and ENCODE co-authors,
     
    You have made an extraordinary claim in your papers. It was extraordinary because it was counter-intuitive, because it challenged a strong paradigm in biomedical research (and more ... Continue reading
  • Reader Comment (Member of the F1000 Faculty) 22 May 2015
    Steven Salzberg, McKusick-Nathans Institute of Genetic Medicine, Johns Hopkins University, USA
    22 May 2015
    Reader Comment Member of the F1000 Faculty
    Shin Lin et al: your additional experiment does help, in that you seem to have controlled for machine effects this time, where (as you write) you "have re-generated in a ... Continue reading
  • Author Response 22 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    22 May 2015
    Author Response
    Shin et al.,

    Thank you for posting the new data and analyses. For completion of records, can you please provide additional details on the tissue collection protocols, RNA extraction protocols, and ... Continue reading
  • Reader Comment 21 May 2015
    Shin Lin, Department of Genetics, Stanford University, USA
    21 May 2015
    Reader Comment
    We continue our comment from May 19, 2015, in which we agreed to show new data to the scientific community. We have re-generated in a single experimental batch and re-sequenced ... Continue reading
  • Author Response 21 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    21 May 2015
    Author Response
    Michele, this is a good point; thank you for your thoughts and for taking the time to write a comment.
     
    While I'd argue that there are easy ways to avoid confounding ... Continue reading
  • Reader Comment 21 May 2015
    Michele Busby, Broad Institute, USA
    21 May 2015
    Reader Comment
    I am growing more suspicious that the sequencing batches are a red herring. 

    As pointed out below, in the original paper’s supplemental tables 1 and 2 there is enrichment in dozens ... Continue reading
  • Author Response 21 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    21 May 2015
    Author Response
    Thanks for your kind note, Mike. To your question: Only one tissue was in common across the two species in the mixed batch (brain). The other two samples were from ... Continue reading
  • Reader Comment 21 May 2015
    Joe Foley, McGill University, Canada
    21 May 2015
    Reader Comment
    Shin Lin: "Because batch effects assuredly occur, we sought to minimize biases generally.  First, for 10 of 13 tissues, the corresponding mouse and human samples had matching indexing/barcode primers. ... It ... Continue reading
  • Reader Comment 21 May 2015
    Mike White, Washington University in St. Louis School of Medicine, USA
    21 May 2015
    Reader Comment
    Yoav & Orna, kudos to you for raising this issue and documenting your procedure in detail.

    I don't know if this is useful, but here's my question: there were two tissues ... Continue reading
  • Reader Comment 21 May 2015
    Lee Elizabeth Edsall, Duke University, USA
    21 May 2015
    Reader Comment
    I find it interesting that CASAVA v1.7 doesn't include the run number in the sequence identifier in a fastq file. The run number is included in the other file types ... Continue reading
  • Author Response 20 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    20 May 2015
    Author Response
    I appreciate the author’s intention to recollect some of their data using a different study design. Unfortunately, if data were to be collected from the same set of original samples, ... Continue reading
  • Reader Comment 20 May 2015
    Rafael Irizarry, Harvard / Dana Farber Cancer Institute, USA
    20 May 2015
    Reader Comment
    I wrote this post for those interested in learning more about the mathematical and statistical considerations related to confounding in the context of this discussion: http://simplystatistics.org/2015/05/20/is-it-species-or-is-it-batch-they-are-confounded-so-we-cant-know/
    Competing Interests: No competing interests were disclosed.
  • Reader Comment 20 May 2015
    Michele Busby, Broad Institute, USA
    20 May 2015
    Reader Comment
    This is a nice discussion of important topics!

    I am interested in whether the RIN scores differed substantially between the mouse and the human samples. I would expect the human samples ... Continue reading
  • Reader Comment (Member of the F1000 Faculty) 20 May 2015
    Steven Salzberg, McKusick-Nathans Institute of Genetic Medicine, Johns Hopkins University, USA
    20 May 2015
    Reader Comment Member of the F1000 Faculty
    Y. Gilad makes a salient point - what the mouse ENCODE authors say in their response is misleading in that they use the term "lane effect" when Gilad and Mizrahi-Man ... Continue reading
  • Author Response 20 May 2015
    Yoav Gilad, Human Genetics, University of Chicago, USA
    20 May 2015
    Author Response
    Why are you calling these 'lane effects'? The samples were sequenced on different instruments (different sequencers) and by default - different flow cells. Can you explicitly acknowledge that fact please?
    Competing Interests: No competing interests were disclosed.
  • Reader Comment 20 May 2015
    Shin Lin, Department of Genetics, Stanford University, USA
    20 May 2015
    Reader Comment
    In our analysis comparing the various tissue transcriptomes between human and mouse using datasets collected from a number of laboratories, we reported that significant differences existed between matched tissues across ... Continue reading
CHANNEL
This article is included in the following channel:
DOI: 10.12688/f1000research.4996.1
Research Article
SkateBase, an elasmobranch genome project and collection of molecular resources for chondrichthyan fishes [version 1; referees: 2 approved]
DOI: 10.12688/f1000research.1-16.v1
Opinion Article
Is the pan-genome also a pan-selectome? [version 1; referees: 2 approved]
DOI: 10.12688/f1000research.7223.1
Review
Single-cell transcriptome sequencing: recent advances and remaining challenges [version 1; referees: 2 approved]
Articles that may interest you
Browse by related topics
Alongside their report, referees assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - key revisions are required to address specific details and make the paper fully scientifically sound
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email info@f1000.com for further assistance.
Server error, please try again.