<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="other" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.11317.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Software Tool Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                    <subj-group>
                        <subject>Bioinformatics</subject>
                    </subj-group>
                    <subj-group>
                        <subject>Diabetes &amp; Obesity</subject>
                    </subj-group>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>ogttMetrics: Data structures and algorithms for oral glucose tolerance tests</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 1 approved with reservations]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Stubbs</surname>
                        <given-names>Benjamin J.</given-names>
                    </name>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Frankston</surname>
                        <given-names>Keith</given-names>
                    </name>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Ramos</surname>
                        <given-names>Marcel</given-names>
                    </name>
                    <xref ref-type="aff" rid="a3">3</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Laranjo</surname>
                        <given-names>Nancy</given-names>
                    </name>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Sacks</surname>
                        <given-names>Frank M.</given-names>
                    </name>
                    <xref ref-type="aff" rid="a4">4</xref>
                </contrib>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Carey</surname>
                        <given-names>Vincent J.</given-names>
                    </name>
                    <uri content-type="orcid">https://orcid.org/0000-0003-4046-0063</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Channing Division of Network Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, 02115, USA</aff>
                <aff id="a2">
                    <label>2</label>Department of Mathematics, Rutgers University, Piscataway, NJ, 08854, USA</aff>
                <aff id="a3">
                    <label>3</label>Department of Biostatistics, CUNY Graduate School of Public Health and Health Policy, New York, NY, 10027, USA</aff>
                <aff id="a4">
                    <label>4</label>Department of Nutrition, Harvard T.H. Chan School of Public Health, Boston, MA, 02115, USA</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:stvjc@channing.harvard.edu">stvjc@channing.harvard.edu</email>
                </corresp>
                <fn fn-type="con">
                    <p>Benjamin Stubbs and Keith Frankston developed software and visualizations, analyzed the data, and participated in manuscript development. Marcel Ramos developed the MultiAssayExperiment package of Bioconductor. Frank M. Sacks and Nancy Laranjo conceived and executed the OMNICarb study created the database from which 
                        <italic toggle="yes">ogttMetrics</italic> data are derived, and participated in manuscript development. Vincent Carey acquired funding for software development, developed software and visualizations, and wrote the manuscript.</p>
                </fn>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>16</day>
                <month>5</month>
                <year>2017</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2017</year>
            </pub-date>
            <volume>6</volume>
            <elocation-id>684</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>8</day>
                    <month>5</month>
                    <year>2017</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2017 Stubbs BJ et al.</copyright-statement>
                <copyright-year>2017</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/6-684/pdf"/>
            <abstract>
                <p>We describe an open source software package, 
                    <italic toggle="yes">ogttMetrics</italic>, to compute diverse measures of glucose metabolism derived from oral glucose tolerance tests (OGTTs). Tools are provided to organize, visualize and compare OGTT data from large cohorts. Numerical difficulties in estimation of parameters of the Bergman minimal model are described, and in one large clinical trial, the simpler closed form index of Matsuda is observed to lead to similar rankings of individuals with respect to insulin sensitivity, and similar inferences concerning effects of modifications to carbohydrate content and glycemic index of experimental diets. </p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>diabetes</kwd>
                <kwd>carbohydrate metabolism</kwd>
                <kwd>clinical trials</kwd>
                <kwd>nonlinear models</kwd>
                <kwd>multivariate analysis</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1">
                    <funding-source>National Cancer Institute</funding-source>
                    <award-id>5U24CA180996-04</award-id>
                </award-group>
                <award-group id="fund-2">
                    <funding-source>National Institute of Diabetes and Digestive and Kidney Diseases</funding-source>
                    <award-id>5R21DK098720-02</award-id>
                </award-group>
                <funding-statement>This work was supported by US National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases (5R21DK098720-02; V. Carey, PI), and National Cancer Institute (5U24 CA180996-04; M. Morgan, PI.)</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec>
            <title>Introduction</title>
            <p>Disorders of carbohydrate metabolism contribute substantially to overall disease burden throughout the world. According to the International Diabetes Foundation (
                <xref ref-type="bibr" rid="ref-9">International Diabetes Federation: IDF Diabetes Atlas, 2015</xref>), over 400 million individuals are diabetic, and numbers afflicted continue to rise.</p>
            <p>Various tests are used to diagnose diabetes or assess risk of diabetes. In the oral glucose tolerance test (OGTT), a specified quantity of glucose is ingested orally. Plasma concentrations of glucose and insulin are measured at specific times after ingestion. Panels (a) and (b) of 
                <xref ref-type="fig" rid="f1">Figure 1</xref> illustrate trajectories of glucose and insulin concentrations in a single patient performing the 120 minute protocol.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>Output of plot_OGTT_fit for a single OGTT series, from the baseline contribution of an OMNICarb participant.</title>
                    <p>(
                        <bold>a</bold>) The observed glucose concentrations (dots) and predictions (line). (
                        <bold>b</bold>) Insulin concentrations (dots) and linear interpolation. (
                        <bold>c</bold>) Predicted glucose vs. insulin action 
                        <italic toggle="yes">X</italic>(
                        <italic toggle="yes">t)</italic>. (
                        <bold>d</bold>) Rate of appearance of glucose, 
                        <italic toggle="yes">Ra</italic>(
                        <italic toggle="yes">t</italic>).</p>
                </caption>
                <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/12213/e2792e7f-5ae1-4b13-a099-164f538b3592_figure1.gif"/>
            </fig>
            <p>Methods for administering, analyzing, and clinically interpreting OGTT results are subjects of active research. Concerns with the use of individualized compartmental models for OGTT analysis are discussed in the work of 
                <xref ref-type="bibr" rid="ref-14">Theodorakis 
                    <italic toggle="yes">et al.</italic> (2017)</xref>. These authors propose population level nonlinear modeling for estimation of insulin sensitivity, and demonstrate that empirical Bayes procedures have desirable properties for computation and interpretation.</p>
            <p>In this report, we describe an open-source software, 
                <italic toggle="yes">ogttMetrics</italic>, for the management and analysis of OGTT series collected in large cohorts, and illustrate the application of models and metrics to a cross-over clinical trial (
                <xref ref-type="bibr" rid="ref-12">Sacks 
                    <italic toggle="yes">et al.</italic> (2014)</xref>) of effects of varying glycemic index and carbohydrate content of controlled diets. A widely cited proprietary software tool for fitting compartmental models to OGTT data is SAAM-II (
                <xref ref-type="bibr" rid="ref-1">Barrett 
                    <italic toggle="yes">et al.</italic> (1998)</xref>). We developed 
                <italic toggle="yes">ogttMetrics</italic> to allow open investigation into properties of OGTT series, for which the SAAM-II models yield untenable estimates or do not converge. In addition, we saw an opportunity to develop a formal structure for collections of large numbers of OGTT series. We adopted the Bioconductor 
                <xref ref-type="bibr" rid="ref-10">MultiAssayExperiment</xref> structure for this purpose, and introduced methods for interactive visualization and quality assessment of OGTT series, exploiting structures and functions of this package to simplify the coding.</p>
        </sec>
        <sec sec-type="methods">
            <title>Methods</title>
            <sec>
                <title>Informal derivation of insulin sensitivity via the minimal model</title>
                <p>Following 
                    <xref ref-type="bibr" rid="ref-2">Bergman 
                        <italic toggle="yes">et al.</italic> (1979)</xref>, let 
                    <italic toggle="yes">G</italic>(
                    <italic toggle="yes">t</italic>) and 
                    <italic toggle="yes">I</italic>(
                    <italic toggle="yes">t</italic>) denote time-dependent plasma concentrations of glucose and insulin respectively. Various time-dependent factors affect the trajectories of these concentration functions, and we will assume that derivatives of these function with respect to time, and partial derivatives of these functions with respect to relevant time-dependent variables, can be defined. Let 
                    <italic toggle="yes">&#x0120;</italic> denote the rate of change of glucose concentration in plasma over time. Glucose effectiveness is defined as 
                    <italic toggle="yes">E</italic> = &#x2013;
                    <italic toggle="yes">&#x2202;&#x0120;/&#x2202;G</italic>. This is described as &#x201c;the quantitative enhancement of glucose disappearance due to an increase in the plasma glucose concentration&#x201d; (
                    <xref ref-type="bibr" rid="ref-2">Bergman 
                        <italic toggle="yes">et al.</italic>, 1979</xref>, p. E673). At steady state, insulin sensitivity is 
                    <italic toggle="yes">S
                        <sub>I</sub>
                    </italic> = 
                    <italic toggle="yes">&#x2202;E/&#x2202;I</italic>. A four compartment model (model VI of 
                    <xref ref-type="bibr" rid="ref-2">Bergman 
                        <italic toggle="yes">et al.</italic> (1979)</xref>) leads to differential equations</p>
                <p>&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;
                    <italic toggle="yes">G</italic>&#x2032;(
                    <italic toggle="yes">t</italic>) = (
                    <italic toggle="yes">p</italic>
                    <sub>1</sub> &#x2013; 
                    <italic toggle="yes">X</italic>(
                    <italic toggle="yes">t</italic>))
                    <italic toggle="yes">G</italic>(
                    <italic toggle="yes">t</italic>) + 
                    <italic toggle="yes">B</italic>
                    <sub>0</sub>
                </p>
                <p>and</p>
                <p>&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;&#x00a0;
                    <italic toggle="yes">X</italic>&#x2032;(
                    <italic toggle="yes">t</italic>) = 
                    <italic toggle="yes">p</italic>
                    <sub>2</sub>
                    <italic toggle="yes">X</italic>(
                    <italic toggle="yes">t</italic>) + 
                    <italic toggle="yes">p</italic>
                    <sub>3</sub>
                    <italic toggle="yes">I</italic>(
                    <italic toggle="yes">t</italic>)</p>
                <p>where 
                    <italic toggle="yes">X</italic>(
                    <italic toggle="yes">t</italic>) is an abstract time-dependent function representing insulin action, 
                    <italic toggle="yes">G</italic>(
                    <italic toggle="yes">t</italic>) is the time-dependent function representing glucose concentration, 
                    <italic toggle="yes">I</italic>(
                    <italic toggle="yes">t</italic>) represents time-dependent insulin concentration, and 
                    <italic toggle="yes">B</italic>
                    <sub>0</sub> represents &#x201c;glucose balance&#x201d; (difference between rates of hepatic release to circulation and uptake in peripheral tissue) extrapolated to zero glucose concentration. By the definition of glucose effectiveness, the first differential equation implies 
                    <italic toggle="yes">E</italic>(
                    <italic toggle="yes">t</italic>) = 
                    <italic toggle="yes">X</italic>(
                    <italic toggle="yes">t</italic>) &#x2013; 
                    <italic toggle="yes">p</italic>
                    <sub>1</sub>, and, at steady state, 
                    <italic toggle="yes">X
                        <sub>SS</sub>
                    </italic> = &#x2013;
                    <italic toggle="yes">I
                        <sub>SS</sub>p
                        <sub>3</sub>
                    </italic>/
                    <italic toggle="yes">p
                        <sub>2</sub>
                    </italic>. This final expression is substituted into the expression for 
                    <italic toggle="yes">E</italic> just obtained, and after formal partial differentiation by 
                    <italic toggle="yes">I</italic>, we obtain 
                    <italic toggle="yes">S
                        <sub>I</sub> = &#x2013;
                        <italic toggle="yes">p</italic>
                        <sub>3</sub>
                    </italic>/
                    <italic toggle="yes">p</italic>
                    <sub>2</sub>.</p>
            </sec>
            <sec>
                <title>A formal specification</title>
                <p>In the following,</p>
                <list list-type="bullet">
                    <list-item>
                        <p>
                            <italic toggle="yes">G</italic>(
                            <italic toggle="yes">t</italic>) is the plasma glucose concentration (mg/dl),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">I</italic>(
                            <italic toggle="yes">t</italic>) is the plasma insulin concentration (
                            <italic toggle="yes">&#x03bc;</italic>U/ml),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">G
                                <sub>b</sub>
                            </italic> and 
                            <italic toggle="yes">I
                                <sub>b</sub>
                            </italic> are the baseline values of glucose and insulin,</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">X</italic>(
                            <italic toggle="yes">t</italic>) represents insulin action on glucose production and disposal (min
                            <sup>&#x2013;1</sup>),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">S
                                <sub>I</sub>
                            </italic> is insulin sensitivity (min
                            <sup>&#x2013;1</sup>/
                            <italic toggle="yes">&#x03bc;</italic>
                            <italic toggle="yes">U</italic> &#x00b7; 
                            <italic toggle="yes">ml</italic>
                            <sup>&#x2013;1</sup>),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">p</italic>
                            <sub>2</sub> is a rate constant for dynamics of insulin action (min
                            <sup>&#x2013;1</sup>),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">Ra</italic>(
                            <italic toggle="yes">&#x03b1;</italic>, 
                            <italic toggle="yes">t</italic>) denotes a time-dependent function representing appearance of glucose in plasma, with parameters 
                            <italic toggle="yes">&#x03b1;</italic> (mg &#x00b7; min
                            <sup>&#x2013;1</sup>/kg),</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">V</italic> is volume of distribution (dl/kg), and</p>
                    </list-item>
                    <list-item>
                        <p>
                            <italic toggle="yes">S
                                <sub>G</sub>
                            </italic> is glucose effectiveness per unit volume (min
                            <sup>-1</sup>).</p>
                    </list-item>
                </list>
                <p>We consider the specific formalism for the dalla Man 
                    <italic toggle="yes">et al.</italic> minimal model given by 
                    <xref ref-type="bibr" rid="ref-3">Burattini 
                        <italic toggle="yes">et al.</italic> (2006)</xref>:</p>
                <p>
                    <disp-formula>
                        <mml:math display="block" id="math1">
                            <mml:mrow>
                                <mml:mfrac>
                                    <mml:mrow>
                                        <mml:mi>d</mml:mi>
                                        <mml:mi>G</mml:mi>
                                        <mml:mo stretchy="false">(</mml:mo>
                                        <mml:mi>t</mml:mi>
                                        <mml:mo stretchy="false">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mrow>
                                        <mml:mi>d</mml:mi>
                                        <mml:mi>t</mml:mi>
                                    </mml:mrow>
                                </mml:mfrac>
                                <mml:mo>=</mml:mo>
                                <mml:mo>&#x2212;</mml:mo>
                                <mml:mo stretchy="false">[</mml:mo>
                                <mml:msub>
                                    <mml:mi>S</mml:mi>
                                    <mml:mi>G</mml:mi>
                                </mml:msub>
                                <mml:mo>+</mml:mo>
                                <mml:mi>X</mml:mi>
                                <mml:mo stretchy="false">(</mml:mo>
                                <mml:mi>t</mml:mi>
                                <mml:mo stretchy="false">)</mml:mo>
                                <mml:mo stretchy="false">]</mml:mo>
                                <mml:mo> . </mml:mo>
                                <mml:mi>G</mml:mi>
                                <mml:mo stretchy="false">(</mml:mo>
                                <mml:mi>t</mml:mi>
                                <mml:mo stretchy="false">)</mml:mo>
                                <mml:mo>+</mml:mo>
                                <mml:msub>
                                    <mml:mi>S</mml:mi>
                                    <mml:mi>G</mml:mi>
                                </mml:msub>
                                <mml:mo> &#x00b7; </mml:mo>
                                <mml:msub>
                                    <mml:mi>G</mml:mi>
                                    <mml:mi>b</mml:mi>
                                </mml:msub>
                                <mml:mo>+</mml:mo>
                                <mml:mfrac>
                                    <mml:mrow>
                                        <mml:mi>R</mml:mi>
                                        <mml:mi>a</mml:mi>
                                        <mml:mo stretchy="false">(</mml:mo>
                                        <mml:mi>&#x03b1;</mml:mi>
                                        <mml:mo>,</mml:mo>
                                        <mml:mi>t</mml:mi>
                                        <mml:mo stretchy="false">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mi>V</mml:mi>
                                </mml:mfrac>
                            </mml:mrow>
                        </mml:math>
                    </disp-formula>
                </p>
                <p>
                    <disp-formula>
                        <mml:math display="block" id="math2">
                            <mml:mrow>
                                <mml:mfrac>
                                    <mml:mrow>
                                        <mml:mi>d</mml:mi>
                                        <mml:mi>X</mml:mi>
                                        <mml:mo stretchy="false">(</mml:mo>
                                        <mml:mi>t</mml:mi>
                                        <mml:mo stretchy="false">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mrow>
                                        <mml:mi>d</mml:mi>
                                        <mml:mi>t</mml:mi>
                                    </mml:mrow>
                                </mml:mfrac>
                                <mml:mo>=</mml:mo>
                                <mml:mo>&#x2212;</mml:mo>
                                <mml:msub>
                                    <mml:mi>p</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msub>
                                <mml:mo> &#x00b7; </mml:mo>
                                <mml:mi>X</mml:mi>
                                <mml:mo stretchy="false">(</mml:mo>
                                <mml:mi>t</mml:mi>
                                <mml:mo stretchy="false">)</mml:mo>
                                <mml:mo>+</mml:mo>
                                <mml:msub>
                                    <mml:mi>p</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msub>
                                <mml:mo> &#x00b7; </mml:mo>
                                <mml:msub>
                                    <mml:mi>S</mml:mi>
                                    <mml:mi>I</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="false">[</mml:mo>
                                <mml:mi>I</mml:mi>
                                <mml:mo stretchy="false">(</mml:mo>
                                <mml:mi>t</mml:mi>
                                <mml:mo stretchy="false">)</mml:mo>
                                <mml:mo>&#x2212;</mml:mo>
                                <mml:msub>
                                    <mml:mi>I</mml:mi>
                                    <mml:mi>b</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="false">]</mml:mo>
                                <mml:mo>,</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </disp-formula>
                </p>
                <p>with initial conditions 
                    <italic toggle="yes">G</italic>(0) = 
                    <italic toggle="yes">G
                        <sub>b</sub>
                    </italic> and 
                    <italic toggle="yes">X</italic>(0) = 0.</p>
            </sec>
            <sec>
                <title>Estimation</title>
                <p>The procedure of 
                    <xref ref-type="bibr" rid="ref-6">Dalla Man 
                        <italic toggle="yes">et al.</italic> (2002)</xref> involves two phases. In the first phase, the system of ordinary differential equations (ODE) above is solved on the basis of provisional settings of unknown parameters. The solution yields pointwise predictions of glucose concentrations 
                    <italic toggle="yes">&#x011c;
                        <sub>t</sub>
                    </italic> with t ranging over the sampling time course of the OGTT. In the second phase, parameters of the ODE system are updated using non-linear least squares. The phases are iterated until the sum of squared discrepancies &#x2211;
                    <italic toggle="yes">
                        <sub>t</sub>
                    </italic>(
                    <italic toggle="yes">G
                        <sub>t</sub>
                    </italic> &#x2013; 
                    <italic toggle="yes">&#x011c;
                        <sub>t</sub>
                    </italic>)
                    <sup>2</sup> converges to a minimum. Inputs to the algorithm are measured time series of glucose and insulin concentrations, and individual body weight; other quantities, such as glucose effectiveness (
                    <italic toggle="yes">S
                        <sub>G</sub>
                    </italic>) fraction of ingested dose absorbed (
                    <italic toggle="yes">FA</italic>), and volume of distribution (
                    <italic toggle="yes">V</italic>) are taken as fixed constants, with values derived from results of other experiments.</p>
            </sec>
            <sec>
                <title>Programming considerations</title>
                <p>The kernel of 
                    <monospace>fitOneMinMod</monospace> in the 
                    <italic toggle="yes">ogttMetrics</italic> package is</p>
                <p>
                    <preformat orientation="portrait" position="float" preformat-type="computer code" xml:space="preserve">    
                        <styled-content style="font-size:15px;">model &lt;- function(t, Y, parameters) {
        with(as.list(parameters), {
            dy1 = -(Sg + Y[2]) * Y[1] + Sg * Gb + (ra(a1, a2,
                a3, t, BW, D, FA, DC)/V)
            dy2 = -p2 * Y[2] + p2 * SI * (Insulin(t) - Ib)
            list(c(dy1, dy2))
        })
    }
</styled-content>
                    </preformat>
                </p>
                <p>Here the interface to 
                    <monospace>lsoda</monospace> in the 
                    <italic toggle="yes">deSolve</italic> package is employed (
                    <xref ref-type="bibr" rid="ref-8">Hindmarsh, 1983</xref>; 
                    <xref ref-type="bibr" rid="ref-11">Petzold, 1983</xref>). The formal variables 
                    <monospace>Y[1]</monospace> and 
                    <monospace>Y[2]</monospace> represent 
                    <italic toggle="yes">G</italic>(
                    <italic toggle="yes">t</italic>) and 
                    <italic toggle="yes">X</italic>(
                    <italic toggle="yes">t</italic>) respectively; 
                    <monospace>ra()</monospace> and 
                    <monospace>Insulin()</monospace> are specially defined functions that return, for any given time in the course of the OGTT, the rate of glucose appearance, and insulin concentration, respectively. The 
                    <monospace>lsoda</monospace> solver is invoked in the function 
                    <monospace>mmsolfn</monospace>, whose inputs 
                    <monospace>a1</monospace>, 
                    <monospace>a2</monospace>, 
                    <monospace>a3</monospace> are free parameters of a piecewise linear model for 
                    <italic toggle="yes">Ra</italic>(
                    <italic toggle="yes">t</italic>), the rate of appearance of glucose; input 
                    <monospace>SI</monospace> is the target quantity of interest, the measure of insulin sensitivity. The values of free parameters are obtained by minimizing the sum of squared differences between observed glucose 
                    <monospace>g</monospace> and values predicted by the ODE system for current values of the unknown parameters:</p>
                <p>
                    <preformat orientation="portrait" position="float" preformat-type="computer code" xml:space="preserve">    
                        <styled-content style="font-size:15px;">mmsolfn = function(a1, a2, a3, SI) lsoda(c(Gb, 0), t, model,
        c(a1 = a1, a2 = a2, a3 = a3, SI = SI))
    fit = nls(g 
                            <sup>~</sup> mmsolfn(a1, a2, a3, SI)[, 2], start = nlsinit,
        trace = nlstrace, control = fullNLScontrol)</styled-content>
                    </preformat>
                </p>
                <p>Additional quantities 
                    <monospace>BW</monospace>, 
                    <monospace>D</monospace>, 
                    <monospace>FA</monospace>, 
                    <monospace>DC</monospace> are used to implement the constraint of 
                    <xref ref-type="bibr" rid="ref-6">Dalla Man 
                        <italic toggle="yes">et al.</italic> (2002)</xref>
                </p>
                <p>
                    <disp-formula>
                        <mml:math display="block" id="math3">
                            <mml:mrow>
                                <mml:mstyle displaystyle="true">
                                    <mml:mrow>
                                        <mml:munderover>
                                            <mml:mo>&#x222b;</mml:mo>
                                            <mml:mn>0</mml:mn>
                                            <mml:mrow>
                                                <mml:mn>420</mml:mn>
                                            </mml:mrow>
                                        </mml:munderover>
                                        <mml:mrow>
                                            <mml:mi>R</mml:mi>
                                            <mml:mi>a</mml:mi>
                                            <mml:mo stretchy="false">(</mml:mo>
                                            <mml:mi>t</mml:mi>
                                            <mml:mo stretchy="false">)</mml:mo>
                                            <mml:mi>d</mml:mi>
                                            <mml:mi>t</mml:mi>
                                            <mml:mo>=</mml:mo>
                                            <mml:mfrac>
                                                <mml:mrow>
                                                    <mml:mi>D</mml:mi>
                                                    <mml:mo> &#x00b7; </mml:mo>
                                                    <mml:mi>F</mml:mi>
                                                    <mml:mi>A</mml:mi>
                                                </mml:mrow>
                                                <mml:mrow>
                                                    <mml:mi>B</mml:mi>
                                                    <mml:mi>W</mml:mi>
                                                </mml:mrow>
                                            </mml:mfrac>
                                        </mml:mrow>
                                    </mml:mrow>
                                </mml:mstyle>
                            </mml:mrow>
                        </mml:math>
                    </disp-formula>
                </p>
                <p>in which 
                    <italic toggle="yes">BW</italic> is participant body weight, 
                    <italic toggle="yes">D</italic> is the dose of glucose ingested, and 
                    <italic toggle="yes">FA</italic> is the fraction of ingested glucose that is actually absorbed; 
                    <italic toggle="yes">DC</italic> is constant that determines the rate of exponential decay of glucose concentration in plasma past minute 120.</p>
                <p>For concreteness, 
                    <xref ref-type="fig" rid="f1">Figure 1</xref> displays all components of a minimal model fitted to a single 120 minute OGTT.</p>
            </sec>
            <sec>
                <title>Data management and reporting</title>
                <p>In practice, OGTT series can be collected according to different protocols and may include additional biomarkers such as c-peptide concentrations. For flexible data management and analysis, we adopted the data structure of the 
                    <italic toggle="yes">MultiAssayExperiment</italic> package of Bioconductor. We extended this structure in a class called 
                    <monospace>ogttCohort</monospace>, which includes metadata about timing of concentration measures. Each biomarker series for each individual is stored as a column of an R matrix, with rows and columns coordinated across assays. Arbitrary additional sample-level information can be linked to assay data. High level functions 
                    <monospace>getMinmodSIs</monospace> and 
                    <monospace>addMatsuda120</monospace> fit the minimal model or compute the Matsuda index for each series, and append results to the data container. Because the minimal model may be time-consuming to fit, support is provided for parallel computation of multiple models. Use of a compact formal representation of all the OGTT data collected on a cohort simplifies creation of generic reports and visualizations. 
                    <xref ref-type="fig" rid="f2">Figure 2</xref> is based on the 
                    <monospace>QCplots</monospace> function, that can be applied to any 
                    <monospace>ogttCohort</monospace> instance. The top two panels display aspects of marginal (time-specific) distributions using boxplots. The bottom two panels are views of joint distributions of features and samples using the biplot methodology of 
                    <xref ref-type="bibr" rid="ref-7">Gabriel (1971)</xref>. Calibrated outlier detection, proceeding under the assumption that the OGTT series are multivariate normal with a common mean vector and unspecified covariance matrix, can be conducted for glucose and insulin series separately, using 
                    <monospace>mvOutliers</monospace>. The procedure of 
                    <xref ref-type="bibr" rid="ref-5">Caroni &amp; Prescott (1992)</xref> is used.</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>Figure 2. </label>
                    <caption>
                        <title>Output of 
                            <monospace>QCplots</monospace> for a sample of 50 observations from the OMNICarb study in the 
                            <monospace>obaSamp</monospace> object distributed with ogttMetrics.</title>
                        <p>Top two panels are time-specific boxplots, bottom two are biplots based on principal components analysis of the 50
                            <italic toggle="yes">x</italic>2 7-dimensional vectors of glucose and insulin concentrations in the dataset.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/12213/e2792e7f-5ae1-4b13-a099-164f538b3592_figure2.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Numerical considerations</title>
                <p>
                    <xref ref-type="bibr" rid="ref-14">Theodorakis 
                        <italic toggle="yes">et al.</italic> (2017)</xref> mentions that the standard (proprietary, closed source) software tool SAAM-II (
                    <xref ref-type="bibr" rid="ref-1">Barrett 
                        <italic toggle="yes">et al.</italic> (1998)</xref>) failed to produce accceptable estimates of insulin sensitivity in over one-third of 106 samples. Similar difficulties were encountered in the OMNICarb study. These challenges motivated us to create an open source solution that would foster investigation of aspects of glucose and insulin series for which the minimal model fails to converge, and allow comparison of alternative metrics of carbohydrate metabolism on large datasets. 
                    <xref ref-type="fig" rid="f3">Figure 3</xref> displays the 
                    <monospace>SIexplorer</monospace> interactive interface. Given a collection of OGTT results in an 
                    <monospace>ogttCohort</monospace> structure, the 
                    <monospace>SI vs Matsuda</monospace> panel shows the association between estimated SI, Matsuda&#x2019;s index, and convergence status of the Burattini 
                    <italic toggle="yes">et al.</italic> formulation of the minimal model. The display is made with transformed axes (log10 for SI, square root for Matsuda&#x2019;s index). Negative estimates of SI are Winsorized to the smallest positive estimate observed in the data. Positive correlation between the indices is apparent, and the general trend appears to be obeyed for the majority of estimates of SI for which the dalla Man 
                    <italic toggle="yes">et al.</italic>, algorithm does not converge.</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>Figure 3. </label>
                    <caption>
                        <title>SIexplorer interactive display of association of insulin sensitivity, Matsuda index, and minimal model convergence with default settings.</title>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/12213/e2792e7f-5ae1-4b13-a099-164f538b3592_figure3.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Application to a cross-over trial</title>
                <p>We created the 
                    <italic toggle="yes">ogttMetrics</italic> package to analyze data from the OMNICarb study (
                    <xref ref-type="bibr" rid="ref-12">Sacks 
                        <italic toggle="yes">et al.</italic> (2014)</xref>). This study involved over 150 overweight individuals (BMI &gt; 25
                    <italic toggle="yes">kg=m</italic>
                    <sup>2</sup>) whose systolic blood pressure was in the interval 120&#x2013;159 mmHg, or diastolic blood pressure in the interval 70&#x2013;90 mmHg. Individuals with diagnoses of diabetes, cardiovascular disease, or chronic kidney disease were excluded. Four experimental diets were designed to provide contrasting values of overall carbohydrate content and glycemic index of foods consumed. Carbohydrate and glycemic index each had two levels denoted C and c (G and g) respectively, leading to the set (CG, Cg, cG, cg) of experimental diets. Each patient received a randomly ordered sequence of diets from this set, consuming each assigned diet for five weeks, with a pause of two weeks between diets. At the end of each feeding period a 120-minute OGTT protocol was administered. As noted previously, attempts to fit the Bergman minimal model with SAAM-II frequently failed to produce acceptable values, and so the study report of effects on insulin sensitivity used Matsuda&#x2019;s index. We have used the ogttMetrics package to structure the data and compute both Matsuda&#x2019;s index and the minimal model SI. 
                    <xref ref-type="fig" rid="f4">Figure 4</xref> shows how the diet effects are estimated using these two indices. Confidence intervals are presented for five different contrasts. The left panel of 
                    <xref ref-type="fig" rid="f4">Figure 4</xref> is identical in content to the Insulin sensitivity panel of 
                    <xref ref-type="fig" rid="f3">Figure 3</xref> of 
                    <xref ref-type="bibr" rid="ref-12">Sacks 
                        <italic toggle="yes">et al.</italic> (2014)</xref>. The right panel shows results based on SI that are qualitatively similar to those found with Matsuda&#x2019;s index, with the exception of the estimated effect of lowering glycemic index in the context of high overall carbohydrate content. With Matsuda&#x2019;s index, the 95% confidence interval excludes zero, but this is not observed when SI is used. Further work on optimizing estimation of insulin sensitivity from the 120 minute OGTT protocol is warranted; the empirical Bayes approach of 
                    <xref ref-type="bibr" rid="ref-14">Theodorakis 
                        <italic toggle="yes">et al.</italic> (2017)</xref> is of particular interest as individual-level estimation in that procedure borrows strength from information assembled for the cohort as a whole.</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Left: Two-sided 95 percent confidence intervals for within-person diet contrasts based on Matsuda&#x2019;s index.</title>
                        <p>Right: Analogous confidence intervals for within-person diet contrasts based on SI estimated using ogttMetrics.</p>
                    </caption>
                    <graphic orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/12213/e2792e7f-5ae1-4b13-a099-164f538b3592_figure4.gif"/>
                </fig>
            </sec>
            <sec>
                <title>Installation and operation</title>
                <p>Installation of ogttMetrics can be accomplished using R 3.4 via 
                    <monospace>devtools::install_github("vjcitn/ogttMetrics", dependencies=c("Depends", "Imports", &#x201c;Suggests"))</monospace>. The key infrastructure components required for 
                    <italic toggle="yes">ogttMetrics</italic> are CRAN package 
                    <italic toggle="yes">deSolve</italic> for minimal model estimation, and Bioconductor package 
                    <italic toggle="yes">MultiAssayExperiment</italic> for data management. The 
                    <monospace>SIexplorer</monospace> utility employs the 
                    <italic toggle="yes">shiny</italic> package. These key components have extensive dependencies among other CRAN packages, but these dependencies are automatically resolved by the 
                    <monospace>install_github()</monospace> command given above. All example data analyzed or visualized in this paper are accessible using the 
                    <monospace>data()</monospace> function. For example, to reproduce 
                    <xref ref-type="fig" rid="f1">Figure 1</xref>, use the commands 
                    <monospace>library(ogttMetrics); data(obaSamp); m1 = minmodByID(obaSamp, "1"); plot_OGTT_fit(m1)</monospace>. For 
                    <xref ref-type="fig" rid="f2">Figure 2</xref>, in the same session, use 
                    <monospace>QCplots(obasamp)</monospace>.</p>
                <p>Users may import data managed in spreadsheets (CSV format) for use with this software. An executable example is available with 
                    <monospace>example(csvImport)</monospace>.</p>
            </sec>
        </sec>
        <sec sec-type="conclusions">
            <title>Conclusions</title>
            <p>The reference assay for glucose metabolism is the hyperinsulinemic-euglycemic clamp (
                <xref ref-type="bibr" rid="ref-13">Soonthornpun 
                    <italic toggle="yes">et al.</italic> (2003)</xref>). Because it is less expensive and much less invasive, the OGTT is an attractive assay for assessing insulin sensitivity, particularly in large studies. We have presented, and made freely available (at 
                <ext-link ext-link-type="uri" xlink:href="http://github.com/vjcitn/ogttMetrics">http://github.com/vjcitn/ogttMetrics</ext-link>), a collection of data structures and functions in the R programming language that help manage and interpret OGTT series collected in cohort studies and clinical trials.</p>
            <p>We and others have found that the minimal model frequently fails to generate reasonable values for SI in OGTT series encountered in practice. In part this is manifested in non-convergence of the basic nonlinear model for the glucose trajectory. However, we have not observed a striking disparity between rankings of participants using the estimate of SI based on an unsatisfactory minimal model fit, and rankings obtained when the closed form Matsuda index is computed on the same OGTT data (
                <xref ref-type="fig" rid="f3">Figure 3</xref>, Spearman correlation between Matsuda and estimated SI = 0.5782, 
                <italic toggle="yes">p</italic> &lt; .0001). The estimated SI may be good enough for practical use, but further investigation of features of OGTT data associated with non-convergence of the minimal model, and biologically motivated elaborations of the model that yield successful fits more generally, should be undertaken.</p>
            <p>The tools for multivariate analysis and interactive model visualization in the SIexplorer component of ogttMetrics will be useful for gaining additional insight into subtyping of patients according to features of glucose and insulin trajectories.</p>
        </sec>
        <sec>
            <title>Software and data availability</title>
            <p>Software and all data analyzed in this paper are available from: 
                <ext-link ext-link-type="uri" xlink:href="http://github.com/vjcitn/ogttMetrics">http://github.com/vjcitn/ogttMetrics</ext-link>
            </p>
            <p>Archived source code as at time of publication: DOI, 
                <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.5281/zenodo.570174">10.5281/zenodo.570174</ext-link> (
                <xref ref-type="bibr" rid="ref-4">Carey, 2017</xref>)</p>
            <p>License: GPL-3</p>
        </sec>
    </body>
    <back>
        <ref-list>
            <ref id="ref-1">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Barrett</surname>
                            <given-names>PH</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bell</surname>
                            <given-names>BM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Cobelli</surname>
                            <given-names>C</given-names>
                        </name>
</person-group>:
                    <article-title>SAAM II: Simulation, Analysis, and Modeling Software for tracer and pharmacokinetic studies.</article-title>
                    <source>

                        <italic toggle="yes">Metabolism.</italic>
</source>
                    <year>1998</year>;<volume>47</volume>(<issue>4</issue>):<fpage>484</fpage>&#x2013;<lpage>92</lpage>.
                    <pub-id pub-id-type="pmid">9550550</pub-id>
                    <pub-id pub-id-type="doi">10.1016/S0026-0495(98)90064-6</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-2">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Bergman</surname>
                            <given-names>RN</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ider</surname>
                            <given-names>YZ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bowden</surname>
                            <given-names>CR</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Quantitative estimation of insulin sensitivity.</article-title>
                    <source>

                        <italic toggle="yes">Am J Physiol.</italic>
</source>
                    <year>1979</year>;<volume>236</volume>(<issue>6</issue>):<fpage>E667</fpage>&#x2013;<lpage>77</lpage>.
                    <pub-id pub-id-type="pmid">443421</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-3">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Burattini</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Casagrande</surname>
                            <given-names>F</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Di Nardo</surname>
                            <given-names>F</given-names>
                        </name>
</person-group>:
                    <article-title>Insulin sensitivity and plasma glucose appearance profile by oral minimal model in normotensive and normoglycemic humans.</article-title>
                    <source>

                        <italic toggle="yes">Lecture Notes in Computer Science, Biological and Medical Data Analysis.</italic>
</source>
                    <year>2006</year>;<volume>4345</volume>:<fpage>128</fpage>&#x2013;<lpage>36</lpage>.
                    <pub-id pub-id-type="doi">10.1007/11946465_12</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-4">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Carey</surname>
                            <given-names>V</given-names>
                        </name>
</person-group>:
                    <article-title>vjcitn/ogttMetrics: Runs on R 3.4 [Data set].</article-title>
                    <source>

                        <italic toggle="yes">Zenodo.</italic>
</source>
                    <year>2017</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://dx.doi.org/10.5281/zenodo.570174">Data Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-5">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Caroni</surname>
                            <given-names>C</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Prescott</surname>
                            <given-names>P</given-names>
                        </name>
</person-group>:
                    <article-title>Sequential application of Wilks&#x2019;s multivariate outlier test.</article-title>
                    <source>

                        <italic toggle="yes">J R Stat Soc Ser C Appl Stat.</italic>
</source>
                    <year>1992</year>;<volume>41</volume>(<issue>2</issue>):<fpage>355</fpage>&#x2013;<lpage>64</lpage>.
                    <pub-id pub-id-type="doi">10.2307/2347567</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-6">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Dalla Man</surname>
                            <given-names>C</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Caumo</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Cobelli</surname>
                            <given-names>C</given-names>
                        </name>
</person-group>:
                    <article-title>The oral glucose minimal model: estimation of insulin sensitivity from a meal test.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans Biomed Eng.</italic>
</source>
                    <year>2002</year>;<volume>49</volume>(<issue>5</issue>):<fpage>419</fpage>&#x2013;<lpage>29</lpage>.
                    <pub-id pub-id-type="pmid">12002173</pub-id>
                    <pub-id pub-id-type="doi">10.1109/10.995680</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-7">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Gabriel</surname>
                            <given-names>KR</given-names>
                        </name>
</person-group>:
                    <article-title>The biplot graphic display of matrices with application to principal component analysis.</article-title>
                    <source>

                        <italic toggle="yes">Biometrika.</italic>
</source>
                    <year>1971</year>;<volume>58</volume>(<issue>3</issue>):<fpage>453</fpage>&#x2013;<lpage>67</lpage>.
                    <pub-id pub-id-type="doi">10.1093/biomet/58.3.453</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-8">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Hindmarsh</surname>
                            <given-names>AC</given-names>
                        </name>
</person-group>:
                    <article-title>ODEPACK, a Systematized Collection of ODE Solvers.</article-title>
                    <source>

                        <italic toggle="yes">IMACS Transactions on Scientific Computation.</italic>
</source>
                    <year>1983</year>;<volume>1</volume>:<fpage>55</fpage>&#x2013;<lpage>64</lpage>.</mixed-citation>
            </ref>
            <ref id="ref-9">
                <mixed-citation publication-type="journal">
                    <collab>International Diabetes Federation: </collab>
                    <article-title>IDF Diabetes Atlas</article-title>.<year>2015</year>.
                    <ext-link ext-link-type="uri" xlink:href="http://www.diabetesatlas.org">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-10">
                <mixed-citation publication-type="journal">
                    <article-title>MultiAssayExperiment: Software for the integration of multi-omics experiments in Bioconductor</article-title>. R package version 1.2.0.
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/waldronlab/MultiAssayExperiment/wiki/MultiAssayExperiment-API">Reference Source</ext-link>
                </mixed-citation>
            </ref>
            <ref id="ref-11">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Petzold</surname>
                            <given-names>L</given-names>
                        </name>
</person-group>:
                    <article-title>Automatic Selection of Methods for Solving Stiff and Nonstiff Systems of Ordinary Differential Equations.</article-title>
                    <source>

                        <italic toggle="yes">SIAM J Sci and Stat Comput.</italic>
</source>
                    <year>1983</year>;<volume>4</volume>(<issue>1</issue>):<fpage>136</fpage>&#x2013;<lpage>48</lpage>.
                    <pub-id pub-id-type="doi">10.1137/0904010</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-12">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Sacks</surname>
                            <given-names>FM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Carey</surname>
                            <given-names>VJ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Anderson</surname>
                            <given-names>CA</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Effects of high vs low glycemic index of dietary carbohydrate on cardiovascular disease Risk factors and insulin sensitivity: the OmniCarb randomized clinical trial.</article-title>
                    <source>

                        <italic toggle="yes">JAMA.</italic>
</source>
                    <year>2014</year>;<volume>312</volume>(<issue>23</issue>):<fpage>2531</fpage>&#x2013;<lpage>41</lpage>.
                    <pub-id pub-id-type="pmid">25514303</pub-id>
                    <pub-id pub-id-type="doi">10.1001/jama.2014.16658</pub-id>
                    <pub-id pub-id-type="pmcid">4370345</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-13">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Soonthornpun</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Setasuban</surname>
                            <given-names>W</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Thamprasit</surname>
                            <given-names>A</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Novel insulin sensitivity index derived from oral glucose tolerance test.</article-title>
                    <source>

                        <italic toggle="yes">J Clin Endocrinol Metab.</italic>
</source>
                    <year>2003</year>;<volume>88</volume>(<issue>3</issue>):<fpage>1019</fpage>&#x2013;<lpage>23</lpage>.
                    <pub-id pub-id-type="pmid">12629079</pub-id>
                    <pub-id pub-id-type="doi">10.1210/jc.2002-021127</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref-14">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Theodorakis</surname>
                            <given-names>MJ</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Katsiki</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Arampatzi</surname>
                            <given-names>K</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Modeling the oral glucose tolerance test in normal and impaired glucose tolerant states: a population approach.</article-title>
                    <source>

                        <italic toggle="yes">Curr Med Res Opin.</italic>
</source>
                    <year>2017</year>;<volume>33</volume>(<issue>2</issue>):<fpage>305</fpage>&#x2013;<lpage>13</lpage>.
                    <pub-id pub-id-type="pmid">27794624</pub-id>
                    <pub-id pub-id-type="doi">10.1080/03007995.2016.1254607</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report22802">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.12213.r22802</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Honkela</surname>
                        <given-names>Antti</given-names>
                    </name>
                    <xref ref-type="aff" rid="r22802a1">1</xref>
                    <xref ref-type="aff" rid="r22802a2">2</xref>
                    <xref ref-type="aff" rid="r22802a3">3</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-9193-8093</uri>
                </contrib>
                <aff id="r22802a1">
                    <label>1</label>Helsinki Institute for Information Technology HIIT, Department of Computer Science, University of Helsinki, Helsinki, Finland</aff>
                <aff id="r22802a2">
                    <label>2</label>Department of Public Health, University of Helsinki, Helsinki, Finland</aff>
                <aff id="r22802a3">
                    <label>3</label>Department of Mathematics and Statistics, University of Helsinki, Helsinki, Finland</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>1</day>
                <month>6</month>
                <year>2017</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2017 Honkela A</copyright-statement>
                <copyright-year>2017</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport22802" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.11317.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The submission describes an open source R software package for managing, visualising and analysing data from oral glucose tolerance tests (OGTTs).</p>
            <p> </p>
            <p> The aim of the work in providing open source tools for the analysis of OGTT data is highly commendable. From a technical standpoint, the package clearly follows good software development practices by using existing data management infrastructure and extensive automated testing. The authors make a strong effort to make the results reported in the paper reproducible by providing code for reproducing half of the figures. The package includes a vignette that provides some example workflows that seem potentially very useful.</p>
            <p> </p>
            <p> While these basics are well covered, the package still has quite a few rough edges that may make it more difficult to adopt for potential end users. I believe these should be addressed to make the submission scientifically sound.</p>
            <p> </p>
            <p> 1. Installation of the package on a fresh R 3.4 according to instructions fails, presumably due to inability of devtools::install_github() to install required dependencies:</p>
            <p> ERROR: dependencies &#x2018;S4Vectors&#x2019;, &#x2018;MultiAssayExperiment&#x2019;, &#x2018;Biobase&#x2019;, &#x2018;SummarizedExperiment&#x2019;, &#x2018;parody&#x2019;, &#x2018;ggbiplot&#x2019; are not available for package &#x2018;ogttMetrics&#x2019;</p>
            <p> </p>
            <p> 2. After manual install of the required Bioconductor packages, installation still fails because ggbiplot is not available in any standard repositories but only on GitHub.</p>
            <p> </p>
            <p> 3. Installing the package using suggested approach after a manual install of all&#x00a0;missing dependencies seems to fail to install the vignette. (Not visible in the listing provided by vignette().)</p>
            <p> </p>
            <p> 4. Running the examples provided in the paper produces some errors:</p>
            <p> &gt; QCplots(obasamp)</p>
            <p> Error in experiments(oc) : object 'obasamp' not found</p>
            <p> </p>
            <p> After fixing the command the first run gives:</p>
            <p> &gt; QCplots(obaSamp)</p>
            <p> ...</p>
            <p> Error in UseMethod("depth") :&#x00a0;</p>
            <p> &#x00a0; no applicable method for 'depth' applied to an object of class "NULL"</p>
            <p> &gt;</p>
            <p> Oddly enough this works when used later.</p>
            <p> </p>
            <p> 5. Additionally, when reading the example from the PDF, the command plot_OGTT_fit contains 'fi' ligature which breaks copy-paste of the command from the PDF.</p>
            <p> </p>
            <p> 6. Running "R CMD check" produces notes and a warning, which probably would not be acceptable at the major repositories:</p>
            <p> * checking for missing documentation entries ... WARNING</p>
            <p> Undocumented data sets:</p>
            <p> &#x00a0; &#x2018;omnicCG_samp&#x2019; &#x2018;omniccG_samp&#x2019; &#x2018;omniccg_samp&#x2019;</p>
            <p> All user-level objects in a package should have documentation entries.</p>
            <p> See chapter &#x2018;Writing R documentation files&#x2019; in the &#x2018;Writing R</p>
            <p> Extensions&#x2019; manual.</p>
            <p> </p>
            <p> </p>
            <p> 7. These fairly trivial technical issues aside, I am unsure what is the intended audience of the package and how useful it would be for that audience. The authors present a smooth workflow for analysing pre-packaged data from existing large studies, but instructions for importing new data are limited to one sparsely documented example and it is not immediately obvious how to e.g. compute the minimal models for this example. The vignette contains some code snippets that are likely relevant, but more comments and explanation would be needed. I tried a little but could not get this working easily. In general the vignette would need to be clearer to be useful to new users.</p>
            <p> </p>
            <p> 8. Related to the above note, the csvImport format should be documented better. The vignette could contain an example with different time points. A hard-coded default of time points seems difficult for something where there probably is no generally applicable default.</p>
            <p> </p>
            <p> 9. The minimal model code contains a number of magic constants with some assumed default values. It would be very good to document with proper references where these come from. It is especially unclear where the constant 420 in the integral in Programming considerations comes from and if that can be safely used for data with a different sampling period.</p>
            <p> </p>
            <p> 10. The implications of the piece-wise linear model using a different model before and after 120 min for data with different (either shorter or longer) sampling period and times should be discussed. Can the model be safely applied in these cases? Are there other hidden assumptions that could impact the end users?</p>
            <p> </p>
            <p> 11. The unit for BMI in "Application to a cross-over trial" is reported incorrectly (25kg=m^2, units incorrectly in italics).</p>
            <p> </p>
            <p> Further suggestions:</p>
            <p> </p>
            <p> 12. It is good that the code contains many stopifnot() sanity checks, but more informative error messages suggesting how to fix things would be useful for the end users.</p>
            <p> </p>
            <p> 13. The specification of the model might benefit from more consistent notation for derivatives. (Now sometimes d/dt, sometimes G' and X'.)</p>
            <p> </p>
            <p> 14. It would be good to include a copyright notice with author and license information to each source file. See https://www.gnu.org/licenses/gpl-howto.html</p>
            <p>Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?</p>
            <p>Yes</p>
            <p>Is the rationale for developing the new software tool clearly explained?</p>
            <p>Yes</p>
            <p>Is the description of the software tool technically sound?</p>
            <p>Partly</p>
            <p>Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?</p>
            <p>Partly</p>
            <p>Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>NA</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
    </sub-article>
</article>
