Introducing high school students to the Gene Ontology classification system

We present a tutorial that introduces high school students to the Gene Ontology classification system which is widely used in genomics and systems biology studies to characterize large sets of genes based on functional and structural information. This classification system is a valuable and standardized method used to identify genes that act in similar processes and pathways and also provides insight into the overall architecture and distribution of genes and gene families associated with a particular tissue or disease. By means of this tutorial, students learn how the classification system works through analyzing a gene set using DAVID the Database for Annotation, Visualization and Integrated Discovery that incorporates the Gene Ontology system into its suite of analysis tools. This method of analyzing genes is used by our high school student interns to categorize gene expression data related to behavioral neuroscience. Students will get a feel for working with genes and gene sets, acquire vocabulary, obtain an understanding of how a database is structured and gain an awareness of the vast amount of information that is known about genes as well as the online analysis tools to manage this information that is nowadays available. Based on survey responses, students intellectually benefit from learning about the Gene Ontology System and using the DAVID tools, they are better prepared for future database use and they also find it enjoyable.


Introduction
Genomics is the branch of biology concerned with the study of genes and their functions (see the National Institutes of Health Frequently Asked Questions about Genetic and Genomic Science). Genomics arose from the acceleration of genetic research which was fueled by the development of rapid and affordable DNA sequencing technologies (Shendure et al., 2017). This opened the door to the sequencing of entire genomes. Presently, the DNA codes for thousands of genomes from diverse species have been sequenced and studied (see the National Center for Biotechnology Information Genome database).
The goals in genomics research are to address all genes and their inter-relationships in order to understand the combined influence on the function of an organism. With this newfound knowledge of the staggering number of genes that make up an organism, the Gene Ontology (GO) classification system was created by the Gene Ontology Consortium to organize genes by their similarities and differences (see Gene Ontology Consortium 'About' page). "Ontology" is not a commonly encountered term and there are several definitions that are related to philosophical concepts.
In the context of information science, as described here, "ontology" is concerned with the representation, formal naming and classification system with the purpose of describing the relationship categories and properties of the data. This is similar to Wikipedia which is also based on a controlled vocabulary, categories to group material by like subject matter, and parent-child terms.
The Gene Ontology information is curated, collected, validated, and annotated by the Gene Ontology Consortium in collaboration with their partners which consist of research groups, research communities and other databases (see http://geneontology. org/docs/go-consortium/ This GO classification system provides the scientific community with a structured vocabulary for defining genes (Ashburner et al., . GO terms describe how a gene functions at the molecular level, its location within the cell, and what biological programs it is involved with. Each GO annotation is associated with an evidence code that is comprised of six categories: experimental evidence, phylogenetic evidence, computational evidence, author statements, curatorial statements, and automatically generated annotations (http://geneontology.org/docs/guide-go-evidence-codes/) The importance of the GO term system becomes apparent when analyzing the organization of genomes and coding regions, the distribution of genes involved in specific processes and the conservation of genes across species (Gaudet et al., 2017; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5821137/). This classification system is also quite powerful when analyzing data from large scale gene expression studies (du Plessis et al., 2011) that consider co-expression data from specific tissues obtained under defined circumstances such as treatment with pharmaceutical agents, or with neurodevelopmental disorders, cancer, or diabetes as examples. GO terms are instrumental for understanding the functions of these genes.
Introducing GO terms and the gene classification system to high school students will bring them up to speed on a commonly used research tool in current genomics methods and expose them to the vast amounts of data that have been derived from genomics and systems biology studies.
In the subsequent sections we show an example of how to extract information about a gene from its associated GO terms and then provide instruction for a practical exercise which will enable students to profile a list of genes using GO terms in the bioinformatics resource DAVID, The Database for Annotation, Visualization and Integrated Discovery. This is a protocol that we teach to our high school student interns when they are evaluating gene expression data for their summer projects (Crusio et al., 2017, see BioScience Project student posters). The student research internship projects are in the context of behavioral neuroscience. Students typically work with gene expression data associated with a specific brain region or brain disorder. As an example for projects related to learning and memory, gene expression data for the hippocampus would be used. Concerning a neurodevelopmental disorder like Schizophrenia, gene expression data for the prefrontal cortex would be considered. There are many online databases that have freely available gene expression data and this could be a way to expand the scope of this tutorial. We use the Allen Brain Atlas for our primary source of gene expression data in the student internship projects.

Procedure
Overview of gene function with GO The overall structure of GO is hierarchical and is based on parentchild terms where the parent term is broader and child term is more specialized.
GO terms group genes according to 3 categories, each of which are considered a distinct ontology: Molecular Function (MF, molecular-level activities performed by gene products), Biological Process (BP, the larger processes, or biological programs accomplished by multiple molecular activities), and Cellular Component (CC, the locations relative to cellular structures in which a gene product performs a function).
As an example, consider the GO term classification for the RAB5A gene ( Figure 1). RAB5A belongs to a family of genes called

Amendments from Version 3
In this 4th version of the manuscript, we have added an optional exercise for a gene enrichment analysis with a step by step instructional video and a corresponding figure. There are also changes to the introduction which have been added for clarification based upon reviewer comments.

REVISED
RAB GTPases that are key regulators of intracellular membrane trafficking. Rabs are involved in the formation of transport vesicles and their fusion with membranes. They are enzymes and mediate their function by cycling between a GDP bound inactive and a GTP bound active state. Because of their fundamental and ubiquitous role, this family of genes are associated with many biological processes and diseases.
GOTERM_MF: GTPase activity, protein binding, GTP binding, GDP binding From the RAB5A related GO terms, we get the overall impression that this gene encodes an enzyme that is involved in signaling, transport and vesicle dynamics and is associated with cell membranes. How do we arrive at this description?
In this example, the information obtained from the MF category is that the protein product of the RAB5A gene binds to guanine nucleotides: GTP and GDP (Guanosine tri and di phosphate, respectively) and that it is an enzyme. This is evident by the "GTPase activity" term. Whenever the suffix "ase" is used in the context of a gene or protein, it refers to an enzyme, something that catalyzes a chemical reaction. For the BP category, there are several terms associated with intracellular transport, signaling, and endocytosis. Finally, the terms associated with CC include endosome and endosome-like organelles (melanosomes, synaptic vesicles, phagocytic vesicles), as well as membrane structures (ruffles, rafts).

Gene Analysis in DAVID
DAVID is a database with a suite of analysis tools that groups genes based on different criteria related to GO terms. DAVID also links to other databases that contain primary source information like The Gene Ontology as well as complementary information related to pathways and human disease. In this exercise, students will use the sample gene lists (DEMOLIST1 or DEMOLIST2) that are accessible from the DAVID database to see how the Gene Ontology classification partitions a set of genes based on GO Terms. Screenshots and videos are provided for step by step instruction. We also provide a video to instruct students on analyzing a gene list in DAVID obtained from a random gene list generator.   The "start analysis" button is circled in red and is located at the top left side of the page. This is the first step in submitting a geneset for analysis with GO Terms in DAVID. Figure 3. Submitting a geneset. Screenshot of the page where a geneset can be submitted "Step1: Enter Gene List", there are options to copy and paste a geneset or upload a file. There is also an option to use either of two sample lists provided by the DAVID site. Under the pull down menu, "Step 2 Select Identifier", there are many types of identification designations for the same gene. For the sample lists, the identifier and species will be recognized automatically. When submitting a gene list from the geneset generator, the identifier is "Official Gene Symbol" which is used in most cases. If the identifier that you choose does not match the identifiers in the submitted gene set, you will receive a message stating this and an option to convert to the correct identifier. Here it is necessary to select the species. Again, note that in this instance for the sample genesets provided by DAVID, "Homo sapiens" is already highlighted along with the number of genes that are recognized by the site and for which information is available. Once the species is highlighted, it is necessary to click the "Select Species" button in order to limit the output to just the desired species.
Screenshot 3 (Figure 4). Species selection. You will see a notice: "Multiple Species, have been Detected", Highlight "Homo Sapiens" in the window, Select "Homo Sapiens" below the window (Example -DEMOLIST 1: 149 genes, highlighted in grey, left panel). Next, you will see the message "Submission Successful" (Video 1; Delprato et al., 2019c). Screenshot 4 ( Figure 5). Obtaining the results. Select "Functional Annotation Tool", beneath the blue arrow. Next, select "Functional Annotation Table" Screenshot 6 ( Figure 7). Keyword search. When selecting terms for a keyword search, a more complete outcome is achieved if just a few letters are specified. For example, -"neur" will capture terms both starting with neuro and neural (Video 1, Delprato et al., 2019c). DAVID output can be searched for genes related to other process and diseases as well. Have students evaluate the gene list based on their interest. They can identify genes related to a particular process. Students may work individually or in groups.

Optional exercise
Students may wish to try this with their own gene lists. This online gene list generator will enable students to generate a random list of genes for evaluation. (See also Video 2 for instruction; Delprato et al., 2019c)

Protocol
Step 1. Specify species: Human is the default Step 2. Specify list length: 200-500 is a good representative number. Note that DAVID will not evaluate lists with more than 2000 genes. An error message stating this will be received.
Step 3. Select "Generate" Step 4. Copy the gene list using the "Select All" option and paste the list directly into DAVID for evaluation as described above. Make sure to select "Official Gene Symbol" as the identifier when submitting the gene list.

Optional exercise 2
Students may perform a gene enrichment analysis exercise based on GO annotation of their gene list directly at the Gene Ontology site. (Video 3). The results can be viewed as a pie chart which makes for a great visual representation of the GO categories ( Figure 8). Clicking on the individual sections of the pie will result in more inclusive and specific annotation. Clicking on the legends will redirect to a table that contains the gene names associated with a specific GO category (For a   Screenshot of a keyword search of the DAVID output to identify genes that are relevant to a given biological function. In this case the search term is "neur" (highlighted in yellow), to identify genes related to neurological processes. The keyword search is done via the general search function on your computer. For larger gene sets, a computer program may be used. more in depth explanation of the results page, see http://geneontology.org/docs/go-enrichment-analysis/).

Protocol
Step 1. Paste a gene list in the window of the Gene Ontology landing page (http://geneontology.org/).
Step 2. Specify the ontology (MF, CC, BP) that you would like to analyze from the drop down menu. This can be specified later as well on the results page. (Biological Process is the default).
Step 3. Select species (Homo sapiens is the default).
Step 5: Select the clickable number that represents your list of genes relative to the reference list to which is compared.
Step 6. Click on the pie chart above the table and specify the desired ontology from the drop down menu. Pie chart personalization and options for extended information are described in the text above the pie chart. Pie chart and legends may be saved as screenshots.

Learning assessment
We polled 12 student interns from our summer program for feedback on their experience working with the GO system and DAVID. The survey consists of 5 direct yes or no response questions and two open ended questions. The responses are show in Table 1, Table 2, and Table 3.
For the direct response questions, 100% of the students had not worked or heard of the DAVID database (Ques1, Table 1.) 83% of the students had not heard of the GO system. Two students had heard of the GO system in an advanced placement biology class but had not explored it further (Ques2, Table1). 100% of the students responded that they benefited intellectually from working with the GO system and DAVID tools (Ques3, Table1) and also that they had enjoyed the experience (Ques 4, Table 1). 92% of the students thought that the experience would better prepare them for future database use. One student indicated that this was not applicable (Quest 5, Table 1).
For the open ended survey questions 3a and 4a (Table 2 and  Table 3), students were asked to explain how they benefited from working with the databases and tools and what about the experience they had enjoyed. A two step process was used to analyze their answers. First, the responses to each question were grouped and an online text analyzer text analyzer was used to assess the words occurring with the highest frequency (Workbooks 1 and 2)   Through this internship using DAVID, I was exposed for the first time to the field of bioinformatics and the gene ontology system. I learned more about the role genes play in neurological disorders in humans. I also got a glimpse of what biomedical researchers and neuroscientists actually do and the type of resources that they work with.
I learned about current methods used by researchers and was able to apply those techniques to my own project. The exposure and experience I gained with bioinformatics gave me a better understanding of gene interactions and how researchers analyze them. The tools I used during the internship furthered my understanding of gene profiling and provided me with insight on the importance of research.
Rather than just learning about genetics as I had in biology classes in the past, I had the opportunity to partake in hands-on learning with the DAVID tools. Through using DAVID and the Gene Ontology system, I gained a broader understanding of the biological functions of genes by seeing and being exposed to such a large variety. Being able to apply what I learned from DAVID and the results I gathered to my own independent project helped to further my understanding of genetics while exposing me to the field of bioinformatics.
I learned different ways genes are classified and analyzed (kegg pathway, etc.). I can use this database for future research projects to get a sense of different pathways these genes are involved in.
I learnt a lot about genomics and how interconnected the different genes in our body are. It also helped me work on my project on Alzheimer's and the APOE 4 allele.
I was able to learn more about PTSD and identifying candidate genes for PTSD. I performed this project with DAVID and learned a lot about the Gene Ontology System.
The DAVID tools helped me cluster annotation terms based on keywords associated with neurological disorders. Also, the KEGG Pathway map allowed me to visualize genes and how they interact.
This was the first time that I worked with either of these tools and I think I benefited mostly because I got an idea of what working in the field of Biology/Neuroscience would be like.
I was able to learn more about the GO System, different databases such as the Allen Brain Atlas and StringDB, which I used in conjunction with DAVID and learned about how DAVID analyzes the data.
I found that working with the DAVID tools and Gene Ontology System I was exposed to a real world experience in science that gave me a better understanding of where we are in research now, and what is still to be done. The benefits I found in myself were a capital in scientific nomenclature, new skills in analysis of data, and a wholesome exposure into the field of genomics research.
I learned a lot about gene databases and how to look at specific sets of data while ignoring information that may be necessary.
I felt like working with the DAVID tools exposed me to an area of research that I wasn't previously familiar with in an engaging and fascinating way. Learning how to find specific information about gene interactions/gene functions through an online database is a valuable tool that I believe will benefit me in my future research.
questions was used to organize the results from the text analysis (Workbooks 1 and 2).
Response category words chosen represent replies to the question asked. Words containing the same root such as learn, learned, learnt, learning were grouped. The response categories selected for question 3a that may provide insight into why students believe they have benefited intellectually are: expose (5), analyze (4) understand (3), help (3), experience (2), and gained (2) (Workbook 1).
For question 4a, the response categories chosen which may provide insight into why students enjoyed the experience are: interesting (6), gained (2), new (2) and explore (2). Other adjectives used in single responses were: interactive, rewarding, and refreshing (Workbook 2). Working with DAVID and the other databases gave me a chance to learn about the field of bioinformatics and genetics, beyond our school curriculum. It was extremely interesting to learn about the hundreds of genes in the human genome, their various functions both at the molecular and biological level and how they affect the neurological characteristics of human beings. The databases were also very interactive and allowed me to explore the other parts of the database myself.
Although it was initially difficult to navigate the DAVID tools, I found the experience rewarding in the end. The process became easier as I persisted in using the database, and I enjoyed being able to explore the realm of gene profiling.
I enjoyed working with the DAVID tools and learning about gene profiling because from a biological point of view, DAVID is very good at finding relevant information, like other correlates, to the keyword I'm looking for.
I felt like what I was learning about through DAVID and the Gene Ontology system was applicable to my project and I was able to utilize and apply my knowledge of these tools effectively, making me more excited to use it.
I gained insight into gene analysis and gained knowledge that I can use in other situations.
I got practical hands-on experience working with a scientific database which was quite different from the textbook learning taught in schools. I found this quite refreshing and enjoyable.
Although it was pretty confusing for me at the time, I think what made it enjoyable was that I found using these tools interesting especially since this was all new to me! Also it was not too difficult or overwhelming since the provided instructions for the internship walked me through each step. I'm actually now studying Neuroscience at BU and I'm actually hoping to get back to relearning how to use these tools again now that I have a better understanding.
I enjoyed working with the DAVID tools and learning about gene profiling with the GO System because it was very interesting to see how different genes were connected to each other and how far reaching the effects of certain genes are.
Because the breadth and depth of information felt like a million different rabbit holes that I could fall into and learn something new from. However, these tools required navigational help and direction from the supervisor and fellow interns for me to truly reach this point of knowing how to immerse myself in it out of mere curiosity, because of how complex felt at first, and I still have so much more to learn, but overall enjoyed working with these tools once I was comfortable with them.
It was a really interesting and valuable experience to have, and I feel like I learned a lot about how different genes may be connected to each other and what is important to consider and look for in gene profiling.
I thought it was interesting to be able to visualize some of the molecular pathways through the diagrams provided. In completing my research, it was helpful to have all of the biological processes and molecular functions of certain genes all in one place I really enjoyed working on a topic that was interesting to me. I could learn about PTSD while learning more about biology and gene profiling.

Conclusions
We describe a procedure for students to become acquainted with the Gene Ontology classification system which is widely used in genomics and systems biology research to characterize gene function. Grouping genes with GO Terms and the DAVID database is based on a protocol that we use with our summer interns to profile gene expression data related to behavioral neuroscience studies (Crusio et al., 2017). Grouping genes in this way identifies genes that function in like processes and also provides information about the overall distribution of a set of genes associated with a particular tissue or process. This tutorial will familiarize early stage students with a biological database and teach them how to mine it and extract useful information from a sample list of genes.
Survey response data from twelve students indicate that they believe they have benefited intellectually from this work and that they enjoy this type of learning experience. Based on coding of the open ended survey responses, the underlying reasons are because they have learned something beneficial and that they find it interesting. The majority of the students (92%) also state that as a result of this experience, they are better prepared for future database use. Entry and exit ticket questions designed to assess student prior and post knowledge as well as stimulate ideas on how these databases and tools may be used in a research project, are included as a formative assessment strategy.

Grant information
The author(s) declared that no grants were involved in supporting this work.

References
How are the Gene Ontology, and ontologies in general, introduced to the students? Clearly what is presented in the manuscript is not sufficient. Can the authors provide easily accessible introductory tutorials?
I am surprised by how much high school students are expected to know about genes and their function, for example when the activity of RAB5A is described. Can the authors provide recommended pre-requisites for this tutorial?
Likewise, the authors intend to demonstrate how to do a gene profiling, but don't describe what is the purpose of this test.
Nowhere do authors mention 'annotation' or 'curation'. It is important to explain this concept -how genes come about to be linked to GO terms. There are many sentences in the text that are unclear or imprecise: The authors state ""Ontology" is not a commonly encountered term and there are several definitions that are related to philosophical concepts.", and then go on to explain only one of these concepts: "In the context of information science, as described here, "ontology" is concerned with the representation, formal naming and classification system with the purpose of describing the relationship categories and properties of the data." In this context, I am not clear what the authors mean by a "classification system".
The importance of the GO term system becomes apparent when analyzing the organization of genomes (...)": As formulated, this information is incorrect.
Section "Making sense of a gene": that title isn't clear; change title to something along the lines of "Overview of gene function with GO".
This section doesn't explain how to get this information from the DAVID site. This is essential if this method is meant to be used by other groups.
Section "Gene profiling in DAVID": No tools on the DAVID website are named "Gene Profiling"; on the DAVID website I get 0 results when I search on "profiling". The authors should either rename the section with the name of the DAVID tool being described, or at the very least explain what they mean by gene profiling.
The first sentence of that section is "DAVID is primarily a clustering program that groups genes based on different criteria related to GO terms." However, the protocol, demonstrated in Video 1, only shows how to obtain all the functional annotations of all genes from the list inputted, there is no clustering analysis here. The graph that provides clustering is the "Functional Annotation Clustering", but I couldn't get it to work; I get an error message stating that there are too many genes on my list (3000), which by the way was incorrect.
The authors state that "DAVID links to other databases that contain complementary information like The Gene Ontology". This is incorrect: the Gene Ontology is the source of the data (annotations) presented in DAVID. The Gene Ontology therefore does not contain complementary information; it contains the source data. (Note also that DAVID is much out of date with respect to the GO data; the last update was in 2018, and the previous one in 2009. This should not impact the ability to use this tool for demonstration purposes, but for research project, as those mentioned by the students in their comments, the gene enrichment analysis tool from the GO website should be favored, as its data is updated monthly).
its data is updated monthly).
Screenshot 3 describing Figure 4: The authors state " You will see a notice: "Multiple Species, have been Detected""; I don't see this text in the page. I do see "Select to limit annotations by one or more species"; maybe this is what the authors are referring to? In that case, for the DEMOLIST sets, there are not multiple species, there is 'human' and 'unknown'. It looks like the 'unknowns' are genes that DAVID cannot map (if you follow the link to Ensembl you will notice it is not there).
Students should understand this limitation if they are faced with it: identifiers are unstable, some identifiers may disappear. It's unfortunate that DAVID provides examples containing these unknown identifiers, but if one does the analysis with the 'human list', you are excluding the genes in the 'unknown' list.
Screenshot 6 describing Figure 7: Keyword search, only demonstrates how to use the built-in functionality of a browser to search text in a web page. While this is good knowledge to have to successfully navigate web-based resources, this is not relevant here.
There is a mistake in Video 2: When the list is selected from the Random Gene List generator tool, the button 'Copy' should be used, not 'Paste'. In this case, it worked because probably the list was already in the clipboard. Again, this is pretty basic web browsing or even text management knowledge, I don't think it needs to be part of this tutorial.
There are no answers to Entry ticket questions 5, 6, and 7. Typos and errors: Figure 2 'David' should be DAVID.
Lists are inconsistent: sometimes lines end with a period, sometimes not. I strongly recommend that these points are addressed before the manuscript is indexed.

Are sufficient details provided to allow replication of the method development and its use by others? No
If any results are presented, are all the source data underlying the results available to ensure full reproducibility? Partly Are the conclusions about the method and its performance adequately supported by the findings presented in the article? Yes No competing interests were disclosed.

Competing Interests:
No competing interests were disclosed. Competing Interests: Reviewer Expertise: Biocuration, gene ontology.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Author Response 31 Jul 2019 , BioScience Project, USA Anna Delprato

Reviewer 4
The method presented in this manuscript is of interest to the education community, in that it describes concisely how to use DAVID, a widely-used tool for analysis using GO. However, some points should be addressed to make the method presented here more realistically usable by other schools wishing to introduce this tutorial: How are the Gene Ontology, and ontologies in general, introduced to the students? Clearly what is presented in the manuscript is not sufficient. Can the authors provide easily accessible introductory tutorials?
We use video conferencing, screen sharing, screenshots, and instructional videos to teach our student interns about the Gene Ontology system. The screenshots and videos that we use for teaching are included in this manuscript and the extended data. We would be willing to provide a tutorial geared to instructors via videoconferencing and screen sharing either one on one or in small groups.
I am surprised by how much high school students are expected to know about genes and their function, for example when the activity of RAB5A s described. Can the authors provide recommended pre-requisites for this tutorial? Students are not expected to have prior knowledge about genes and their function. This is what they learn about in the internship through direct discussion with the instructor and while working on their projects. We do not use pre-requisites for this tutorial.
Likewise, the authors intend to demonstrate how to do a gene profiling, but don't describe what is the purpose of this test. By gene profiling, we simply mean analysis to understand function. We have changed the word "profiling" to "analyses" for clarification.
Nowhere do authors mention 'annotation' or 'curation'. It is important to explain this concept -how genes come about to be linked to GO terms.
We agree and as stated above, we have now discussed this point in the introduction. There are many sentences in the text that are unclear or imprecise: The authors state ""Ontology" is not a commonly encountered term and there are several definitions that are related to philosophical concepts.", and then go on to explain only one of these concepts: "In the context of information science, as described here, "ontology" is concerned with the representation, formal naming and classification system with the purpose of describing the relationship categories and properties of the data." In this context, I am not clear what the authors mean by a "classification system".
By classification system we mean the organization and retrieval of information. We have attempted By classification system we mean the organization and retrieval of information. We have attempted to clarify this point by using an analogy with Wikipedia which also uses a structured and controlled vocabulary like GO. We have added this to the introduction.
The importance of the GO term system becomes apparent when analyzing the organization of genomes (...)": As formulated, this information is incorrect. .

We apologize, but we fail to see what is incorrect about this phrase
Section "Making sense of a gene": that title isn't clear; change title to something along the lines of "Overview of gene function with GO".

The section title has been changed.
This section doesn't explain how to get this information from the DAVID site. This is essential if this method is meant to be used by other groups. Section "Gene profiling in DAVID": No tools on the DAVID website are named "Gene Profiling"; on the DAVID website I get 0 results when I search on "profiling". The authors should either rename the section with the name of the DAVID tool being described, or at the very least explain what they mean by gene profiling.
We apologize for the confusion. We have changed the word "profiling" to "analyses" for clarification.
The first sentence of that section is "DAVID is primarily a clustering program that groups genes based on different criteria related to GO terms." However, the protocol, demonstrated in Video 1, only shows how to obtain all the functional annotations of all genes from the list inputted, there is no clustering analysis here. The graph that provides clustering is the "Functional Annotation Clustering", but I couldn't get it to work; I get an error message stating that there are too many genes on my list (3000), which by the way was incorrect.
We apologize for the confusion. We have omitted the word "clustering" from the sentence.
The authors state that "DAVID links to other databases that contain complementary information like The Gene Ontology". This is incorrect: the Gene Ontology is the source of the data (annotations) presented in DAVID. The Gene Ontology therefore does not contain complementary information; it contains the source data.
We have removed the word "complimentary" from the sentence.
(Note also that DAVID is much out of date with with respect to the GO data; the last update was in 2018, and the previous one in 2009. This should not impact the ability to use this tool for demonstration purposes, but for research project, as those mentioned by the students in their comments, the gene enrichment analysis tool from the GO website should be favored, as its data is updated monthly).

Students do in fact use the Gene Ontology site in addition to DAVID for their research projects.
Based on a suggestion made by Reviewer 3, we have now added an optional gene enrichment tools. exercise using the Gene Ontology Screenshot 3 describing Figure 4: The authors state " You will see a notice: "Multiple Species, Screenshot 3 describing Figure 4: The authors state " You will see a notice: "Multiple Species, have been Detected""; I don't see this text in the page. I do see "Select to limit annotations by one or more species"; maybe this is what the authors are referring to? In that case, for the DEMOLIST sets, there are not multiple species, there is 'human' and 'unknown'. It looks like the 'unknowns' are genes that DAVID cannot map (if you follow the link to Ensembl you will notice it is not there). Students should understand this limitation if they are faced with it: identifiers are unstable, some identifiers may disappear. It's unfortunate that DAVID provides examples containing these unknown identifiers, but if one does the analysis with the 'human list', you are excluding the genes in the 'unknown' list.
We have attempted to clarify this point in the figure legend.
There are no answers to Entry ticket questions 5, 6, and 7.
We have now provided answers to Entry ticket questions 5 and 6. Question 7, is subjective (Have you ever worked with a biological database?) and we have not included an answer but have indicated that it is a subjective question.
Typos and errors:

Ruth Lovering
Functional Gene Annotation, Institute of Cardiovascular Science, University College London, London, UK This article describes the resources used to teach students about the Gene Ontology and one of the many functional analysis tools that are available. As someone who provides Gene Ontology (GO) annotations and teaches MSc students about bioinformatics resources, I found this article very interesting. I am surprised that high school students were engaged and understood sufficient biology to appreciate the application of GO and DAVID. An aspect not apparently covered by this course is the source of the annotations included in these resources. Biocuration is a poorly appreciated aspect of annotation resources, the majority of scientists have no idea how the data from scientific papers is being summarised and incorporated into annotation files. This is an ongoing process and it would be useful to discuss this as well, for example the similarity of the GO resource with Wikipedia (which also includes GO annotations). Also to point out that the annotation files are not static but updated on a monthly basis. My final question is whether the authors considered using any other tool? DAVID does not regularly update the annotations used in the analysis (last updated in 2016) and it is difficult to see how the same genes are enriched with similar terms. DAVID also has a complicated system of GO FAT etc, which requires the user to understand the ontology structure. But this does help to limit view of the GO terms retrieved. However, the figures only seem to show the list of GO (and other) terms associated with each gene. A figure showing the enriched terms for the full list would be useful.

Reviewer 3
This article describes the resources used to teach students about the Gene Ontology and one of the many functional analysis tools that are available. As someone who provides Gene Ontology (GO) annotations and teaches MSc students about bioinformatics resources, I found this article very interesting. I am surprised that high school students were engaged and understood sufficient biology to appreciate the application of GO and DAVID.
An aspect not apparently covered by this course is the source of the annotations included in these resources. Biocuration is a poorly appreciated aspect of annotation resources, the majority of scientists have no idea how the data from scientific papers is being summarised and incorporated into annotation files. This is an ongoing process and it would be useful to discuss this as well, for example the similarity of the GO resource with Wikipedia (which also includes GO annotations). Also to point out that the annotation files are not static but updated on a monthly basis.
We have now discussed these important points in the introduction of the revised article -version 4.
My final question is whether the authors considered using any other tool? DAVID does not regularly update the annotations used in the analysis (last updated in 2016) and it is difficult to see how the same genes are enriched with similar terms. DAVID also has a complicated system of GO FAT etc, which requires the user to understand the ontology structure. But this does help to limit view of the GO terms retrieved. However, the figures only seem to show the list of GO (and other) terms associated with each gene. A figure showing the enriched terms for the full list would be useful.
We use DAVID because the site and its output is well organized and highly informative for students. In addition to the Gene Ontology classification system and its structure as part of the tutorial. In addition to DAVID we use the Gene Ontology site as well.
We have included an optional gene enrichment exercise along with a step by step video and an additional figure (Figure 8) showing the results of the gene enrichment.
No competing interests to declare. Competing Interests: This is an open access peer review report distributed under the terms of the Creative Commons , which permits unrestricted use, distribution, and reproduction in any medium, provided the original Attribution Licence work is properly cited.

© 2019 Grisham W. This is an open access peer review report distributed under the terms of the Creative Commons
, which permits unrestricted use, distribution, and reproduction in any medium, provided the original Attribution Licence work is properly cited.

William Grisham
University of California, Los Angeles (UCLA), Department of Psychology, Los Angeles, CA, USA This article is aimed at describing an easily searchable database that can be used by students to search by gene ontologies. The point of the article is to provide a guide to use this tool in education at the high school level, but the article could provide a good starting point for college level instructors and even industry trainers. The article describes the underlying question well and chooses a gene, RAB5A, whose function should be readily understood by most biologists.
The submitted article is timely because the DAVID tool described is not very intuitive. Thus, an easy to follow guide such as the article provides is most welcome. Notably, database tools are becoming more elaborate, and their complexity has created a barrier for effectively use in teaching, particularly at lower levels. The article does a good job of describing this tool, DAVID, in a fashion that will be intelligible to high school and college level students. Lavish illustrations are provided as are links to videos to aid in instruction.
The use of Entry Ticket and Exit Ticket questions is an excellent pedagogical idea. The questions, however, seem a bit simplistic.
A bigger problem with the article is that there are no data showing how using the DAVID tool in teaching has produced changes in measurable ways. Do students do better on tests of content knowledge? Are students more sophisticated in their thinking as a function of using the DAVID tool? Are they better able to navigate other genomic tools? Do they simply enjoy the lessons more than with traditional instruction? Many of these questions can be answered with a simple pre-and posttest quiz or even comparing current scores on tests/evaluations to historical scores. This addition would make this article of much more value to educators who could then see if incorporating this tool into their pedagogy is worthwhile.

Are sufficient details provided to allow replication of the method development and its use by others? Yes
If any results are presented, are all the source data underlying the results available to ensure full reproducibility?

No source data required
Are the conclusions about the method and its performance adequately supported by the findings presented in the article?
Partly school students to enhance biological education. While the proposal describes the method in detail, there is no data presented to show that the method actually works. While this is an interesting protocol, it is not a fully fleshed out methods article. The authors will need to present data to show the viability of the method, or resubmit potentially as a protocol.

Response: This point has been addressed in our response to Reviewer 1.
A few more specific comments about the article are listed below: 1) The abstract is worded as more of a proposal rather than a paper.
Response: The abstract has been revised to make it sound less like a proposal and we have incorporated . the information obtained from the survey responses 2) The protocol seems very much like following a set of directions (not very thought-provoking or inquiry-based), without relating it to the neuroscience. I think the paper would be much stronger if the authors were to relate it back to the behavioral neuroscience projects that are mentioned in the abstract.
Response: The method presented in this article is intended to introduce students to an unfamiliar and sophisticated topic. This instruction may serve as a stepping stone for inquiry-based projects.
We expanded the set of Exit Ticket questions to stimulate ideas on how these databases and tools may be used in a research project.
A paragraph to relate the tutorial back to the behavioral neuroscience has been added. We also provide a suggestion for expanding the scope of the tutorial into an inquiry-based project.
No competing interests were disclosed.

Competing Interests:
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias You can publish traditional articles, null/negative results, case reports, data notes and more The peer review process is transparent and collaborative Your article is indexed in PubMed after passing peer review Dedicated customer support at every stage For pre-submission enquiries, contact research@f1000.com