<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.145100.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Research Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Intestelligence: A pharmacological neural network using intestine data</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 2 approved with reservations]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Watanabe</surname>
                        <given-names>Yusuke</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Formal Analysis</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Software</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Visualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0001-9541-6073</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                    <xref ref-type="aff" rid="a2">2</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Ban</surname>
                        <given-names>Hiroshi</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a3">3</xref>
                </contrib>
                <contrib contrib-type="author" corresp="no">
                    <name>
                        <surname>Hagura</surname>
                        <given-names>Nobuhiro</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="aff" rid="a3">3</xref>
                </contrib>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Ikegaya</surname>
                        <given-names>Yuji</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Funding Acquisition</role>
                    <role content-type="http://credit.niso.org/">Investigation</role>
                    <role content-type="http://credit.niso.org/">Project Administration</role>
                    <role content-type="http://credit.niso.org/">Resources</role>
                    <role content-type="http://credit.niso.org/">Supervision</role>
                    <role content-type="http://credit.niso.org/">Validation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <xref ref-type="corresp" rid="c2">b</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                    <xref ref-type="aff" rid="a3">3</xref>
                    <xref ref-type="aff" rid="a4">4</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Graduate School of Pharmaceutical Sciences, The University of Tokyo, Hongo, Bunkyo-ku, Tokyo, 1130033, Japan</aff>
                <aff id="a2">
                    <label>2</label>Department of Biomedical Engineering, The University of Melbourne, Melbourne, Victoria, 3010, Australia</aff>
                <aff id="a3">
                    <label>3</label>Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita, Osaka, 5650871, Japan</aff>
                <aff id="a4">
                    <label>4</label>Institute for AI and Beyond, The University of Tokyo, Hongo, Bunkyo-ku, Tokyo, 1130033, Japan</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:ywatanabe@alumni.u-tokyo.ac.jp">ywatanabe@alumni.u-tokyo.ac.jp</email>
                </corresp>
                <corresp id="c2">
                    <label>b</label>
                    <email xlink:href="mailto:yuji@ikegaya.jp">yuji@ikegaya.jp</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>28</day>
                <month>6</month>
                <year>2024</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2024</year>
            </pub-date>
            <volume>13</volume>
            <elocation-id>714</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>20</day>
                    <month>5</month>
                    <year>2024</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2024 Watanabe Y et al.</copyright-statement>
                <copyright-year>2024</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/13-714/pdf"/>
            <abstract>
                <sec>
                    <title>Background</title>
                    <p>A neural network is a machine learning algorithm that can learn and make predictions by adjusting the strength of the connections between nodes. The sigmoid function is commonly used as an activation function in these nodes. This study explores the potential applicability of biological materials in the development of alternative activation functions.</p>
                </sec>
                <sec>
                    <title>Methods</title>
                    <p>Inspired by the fact that acetylcholine induces intestinal contractions that follow a sigmoid function, we used pharmacological data obtained from guinea pig ilea in a layered neural network for image classification tasks.</p>
                </sec>
                <sec>
                    <title>Results and Conclusions</title>
                    <p>We found that the intestinal data-based neural network with the same structure as a conventional three-layer perceptron achieved an impressive classification accuracy of 85.7% &#x00b1; 0.6% based on the MNIST handwritten digit dataset (chance = 10%). Additionally, the neural network was trained to determine whether objects in photographs collected from the internet were digestible, achieving an accuracy of 88.5% &#x00b1; 0.9% (chance = 50%). Our approach highlights the potential applicability of intestine data in neural computations based on pharmacological mechanisms.</p>
                </sec>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>intestine</kwd>
                <kwd>neural network</kwd>
                <kwd>activation function</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1" xlink:href="http://dx.doi.org/10.13039/501100002241">
                    <funding-source>The Exploratory Research for Advanced Technology by Japan Science and Technology Agency (JST ERATO)</funding-source>
                    <award-id>JPMJER1801</award-id>
                </award-group>
                <funding-statement>The Exploratory Research for Advanced Technology by Japan Science and Technology Agency (JST ERATO; Grant Number: JPMJER1801). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec id="sec4" sec-type="intro">
            <title>Introduction</title>
            <p>Neural networks (NNs), which were first introduced by 
                <xref ref-type="bibr" rid="ref1">Amari (1967)</xref> and later improved by 
                <xref ref-type="bibr" rid="ref20">Rumelhart et al. (1986)</xref>, are mathematical models that are inspired by the structure and function of the central nervous system (CNS). A critical aspect of NNs is their nonlinearity, which is provided by activation functions that control the nodes&#x2019; outputs, similar to biological neurons.</p>
            <p>Despite their important impact on neural network behavior, many activation functions have been proposed without theoretical justification (
                <xref ref-type="bibr" rid="ref7">Glorot et al., 2011</xref>; 
                <xref ref-type="bibr" rid="ref8">Hahnloser et al., 2000</xref>; 
                <xref ref-type="bibr" rid="ref9">Jarrett et al., 2009</xref>; 
                <xref ref-type="bibr" rid="ref16">Nair &amp; Hinton, 2010</xref>; 
                <xref ref-type="bibr" rid="ref18">Ramachandran et al., 2017</xref>; 
                <xref ref-type="bibr" rid="ref15">Misra, 2020</xref>). Therefore, the use of biologically verified activation functions, such as those derived from the enteric nervous system, may improve the performance of existing activation functions. The enteric nervous system shares many features with the central nervous system (
                <xref ref-type="bibr" rid="ref24">Goyal &amp; Hirano, 1996</xref>; 
                <xref ref-type="bibr" rid="ref19">Rao &amp; Gershon, 2016</xref>) and may show promise for the development of unconventional neural network models.</p>
            <p>The intestine, which is sometimes referred to as the &#x201c;second brain&#x201d; (
                <xref ref-type="bibr" rid="ref14">Mayer, 2011</xref>; 
                <xref ref-type="bibr" rid="ref17">Obata &amp; Pachnis, 2016</xref>; 
                <xref ref-type="bibr" rid="ref19">Rao &amp; Gershon, 2016</xref>; 
                <xref ref-type="bibr" rid="ref13">Maniscalco &amp; Rinaman, 2018</xref>), responds to external stimuli. Its smooth muscle undergoes reversible contractions in response to acetylcholine (ACh) in a dose-dependent manner (
                <xref ref-type="bibr" rid="ref4">Burgen &amp; Spero, 1997</xref>; 
                <xref ref-type="bibr" rid="ref19">Rao &amp; Gershon, 2016</xref>). These contractions conform to a dose&#x2013;response curve that reflects the activation of ACh receptors and subsequent signal pathways; thus, contraction responses are macroscopic expressions of biochemical reactions. Therefore, intestinal contraction data may be a valuable resource for developing novel biologically inspired activation functions.</p>
            <p>In this study, we developed an intestinal data-based neural network (INN) by incorporating pharmacological data based on intestinal contractions into a neural network model. We hypothesized that the INN model could perform classification tasks as effectively as neural networks based on the central nervous system.</p>
        </sec>
        <sec id="sec5" sec-type="methods">
            <title>Methods</title>
            <sec id="sec6">
                <title>Animal ethics</title>
                <p>The intestinal tissues used in this study were surplus samples in a university student practical course that would otherwise have been discarded. Specifically, retired male breeder guinea pigs weighing 300&#x2013;400 g (
                    <italic toggle="yes">n</italic> = 14; SLC, Shizuoka, Japan) were used in our experiments. Consequently, the sample size was dictated by the enrollment of students in the course. The animals were housed in a temperature-controlled room (22 &#x00b1; 1&#x00b0;C) with a 12-hour light-dark cycle (light from 07:00 to 19:00) and given 
                    <italic toggle="yes">ad libitum</italic> access to food and water. All animal experiments were performed with the approval of the Animal Experiment Ethics Committee of the University of Tokyo (approval number: P24-1) and in accordance with the University of Tokyo guidelines for the care and use of laboratory animals. Our experimental protocols adhered to the Fundamental Guidelines for Proper Conduct of Animal Experiments and Related Activities in Academic Research Institutions (Ministry of Education, Culture, Sports, Science and Technology, Notice No. 71 of 2006), the Standards for Breeding and Housing of and Pain Alleviation for Experimental Animals (Ministry of the Environment, Notice No. 88 of 2006), and the Guidelines on the Method of Animal Disposal (Prime Minister&#x2019;s Office, Notice No. 40 of 1995). The primary outcome measure was the variation in ileum tissue length when subjected to different concentrations of acetylcholine (ACh) in a controlled bath setting. Animal identifiers were not identifiable to data analysts. All efforts were made to ameliorate the suffering of animals. Euthanasia methods involved intraperitoneal injection of sodium pentobarbital at a lethal dose of 100 mg/kg body weight to ensure a humane termination of life. Additionally, pre-established criteria for euthanizing animals displaying severe distress or declining health included indicators such as unexplained weight loss of over 20%, labored breathing, and other signs of suffering; however, none of the animals met these criteria. At the conclusion of the educational pharmacology course, all surviving animals were humanely euthanized, adhering to the same ethical standards, and were not used for subsequent experiments or educational activities.</p>
            </sec>
            <sec id="sec7">
                <title>Drugs</title>
                <p>Acetylcholine chloride (C
                    <sub>7</sub>H
                    <sub>16</sub>ClNO
                    <sub>2</sub>; Wako, Japan) was dissolved in double distilled water at concentrations of 2&#x00d7;10
                    <sup>&#x2212;7</sup>, 2&#x00d7;10
                    <sup>&#x2212;6</sup>, 2&#x00d7;10
                    <sup>&#x2212;5</sup>, 2&#x00d7;10
                    <sup>&#x2212;4</sup>, 2&#x00d7;10
                    <sup>&#x2212;3</sup>, 2&#x00d7;10
                    <sup>&#x2212;2</sup>, and 2&#x00d7;10
                    <sup>&#x2212;2</sup> M.</p>
            </sec>
            <sec id="sec8">
                <title>Intestinal contraction data acquisition</title>
                <p>To isolate ileum tissue, guinea pigs were deeply anesthetized with 100 mg/kg pentobarbital (i.p.). The chest was opened, and the ileum was removed and washed with nutrient solution (in mM: NaCl 137, KCl 2.7, NaHCO
                    <sub>3</sub> 11.9, CaCl
                    <sub>2</sub> 1.8, MgCl
                    <sub>2</sub> 1.0, D-glucose 5.5) that was bubbled with 5% CO
                    <sub>2</sub>/95% O
                    <sub>2</sub>. The tissue was cut to a length of approximately 3 cm, and mesenteric arteries were removed. Two hooks were used to mount the tissue; one hook was inserted at one end of the tissue and fixed to the tip of an aeration tube, while the other was connected to an isotonic displacement transducer (45347, NEC San-ei Instrument, Tokyo, Japan). The transducer was connected to an amplifier (AS1202, NEC San-ei Instruments) and a flatbed recorder (FBR-251A, TOA Electronics, Tokyo, Japan), which was used to record the ileum tissue responses (
                    <xref ref-type="fig" rid="f1">Figure 1A</xref>). Before starting the experiments, the ileum tissue strips were equilibrated, and a stable baseline recording was obtained at a resting tension of 0.5 g. Acetylcholine (ACh) solutions were then sequentially bath-applied to the organ bath at the following concentrations (in M): 1&#x00d7;10
                    <sup>&#x2212;9</sup>, 3&#x00d7;10
                    <sup>&#x2212;9</sup>, 1&#x00d7;10
                    <sup>&#x2212;8</sup>, 3&#x00d7;10
                    <sup>&#x2212;8</sup>, 1&#x00d7;10
                    <sup>&#x2212;7</sup>, 3&#x00d7;10
                    <sup>&#x2212;7</sup>, 1&#x00d7;10
                    <sup>&#x2212;6</sup>, 3&#x00d7;10
                    <sup>&#x2212;6</sup>, 1&#x00d7;10
                    <sup>&#x2212;5</sup>, 3&#x00d7;10
                    <sup>&#x2212;5</sup>, 1&#x00d7;10
                    <sup>&#x2212;4</sup>, 3&#x00d7;10
                    <sup>&#x2212;4</sup>, 1&#x00d7;10
                    <sup>&#x2212;3</sup>, and 3&#x00d7;10
                    <sup>&#x2212;3</sup>. Bath application was performed when a local maximum of the contraction reaction at the previous ACh concentration was confirmed. The ACh concentration [M] and intestine length [cm] were recorded, and these steps were repeated 2&#x2013;3 times. The intestinal contraction rate [%] was calculated by averaging the contraction lengths over different trials and dividing the mean values by the maximum value. We collected the data for the dose&#x2013;response curves from 512 ileum samples.</p>
            </sec>
            <sec id="sec9">
                <title>Sigmoid-curve fitting for the intestine contraction data at different acetylcholine concentrations</title>
                <p>For each of the 512 data pairs, the ACh concentration was natural-log-transformed:
                    <disp-formula id="e1">
                        <mml:math display="block">
                            <mml:msup>
                                <mml:mi mathvariant="normal">C</mml:mi>
                                <mml:mo>&#x2032;</mml:mo>
                            </mml:msup>
                            <mml:mo>=</mml:mo>
                            <mml:mo>ln</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="normal">C</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>+</mml:mo>
                            <mml:mn>4</mml:mn>
                        </mml:math>
                        <label>(1)</label>
                    </disp-formula>where C is the ACh concentration in the organ bath. The dose&#x2013;response curves were plotted as
                    <disp-formula id="e2">
                        <mml:math display="block">
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">x</mml:mi>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">y</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                            <mml:mo>=</mml:mo>
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:msup>
                                        <mml:mi mathvariant="normal">C</mml:mi>
                                        <mml:mo>&#x2032;</mml:mo>
                                    </mml:msup>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">R</mml:mi>
                                        <mml:mi mathvariant="normal">i</mml:mi>
                                    </mml:msub>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                        <label>(2)</label>
                    </disp-formula>where 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">R</mml:mi>
                                <mml:mi mathvariant="normal">i</mml:mi>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> is the i
                    <sup>th</sup> intestine contraction rate [%] (i = 1, 2, &#x2026;, 512; 
                    <xref ref-type="fig" rid="f1">Figure 1B</xref>). These curves were fitted by sigmoid functions:
                    <disp-formula id="e3">
                        <mml:math display="block">
                            <mml:mi mathvariant="normal">&#x03c2;</mml:mi>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="normal">x</mml:mi>
                                <mml:mo>,</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                    <mml:msub>
                                        <mml:mn>0</mml:mn>
                                        <mml:mi mathvariant="normal">i</mml:mi>
                                    </mml:msub>
                                </mml:msub>
                                <mml:mo>,</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                    <mml:msub>
                                        <mml:mn>1</mml:mn>
                                        <mml:mi mathvariant="normal">i</mml:mi>
                                    </mml:msub>
                                </mml:msub>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mn>1</mml:mn>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>+</mml:mo>
                                    <mml:mtext>exp</mml:mtext>
                                    <mml:mspace width="0.25em"/>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mo>&#x2212;</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                            <mml:msub>
                                                <mml:mn>0</mml:mn>
                                                <mml:mi mathvariant="normal">i</mml:mi>
                                            </mml:msub>
                                        </mml:msub>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:mo>+</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                            <mml:msub>
                                                <mml:mn>1</mml:mn>
                                                <mml:mi mathvariant="normal">i</mml:mi>
                                            </mml:msub>
                                        </mml:msub>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mspace width="0.25em"/>
                                </mml:mrow>
                            </mml:mfrac>
                        </mml:math>
                        <label>(3)</label>
                    </disp-formula>where 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>1</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> are parameters to fit the sigmoid function based on the i
                    <sup>th</sup> intestine data. The parameters 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>1</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> were optimized simultaneously using the MATLAB function nlinfit (MathWorks, Natick, MA, USA; 
                    <ext-link ext-link-type="uri" xlink:href="https://www.mathworks.com/help/stats/nlinfit.html">https://www.mathworks.com/help/stats/nlinfit.html</ext-link>). The determined sigmoid functions were shifted by 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">x</mml:mi>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">y</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                    </inline-formula> = 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mo>&#x2212;</mml:mo>
                                    <mml:mfrac>
                                        <mml:msub>
                                            <mml:mover accent="true">
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:mo stretchy="true">&#x00af;</mml:mo>
                                            </mml:mover>
                                            <mml:mn>1</mml:mn>
                                        </mml:msub>
                                        <mml:msub>
                                            <mml:mover accent="true">
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:mo stretchy="true">&#x00af;</mml:mo>
                                            </mml:mover>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                    </mml:mfrac>
                                    <mml:mo>,</mml:mo>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                    </inline-formula>as follows:
                    <disp-formula id="e4">
                        <mml:math display="block">
                            <mml:mtable columnalign="left" displaystyle="true">
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03c2;</mml:mi>
                                            <mml:mtext>shifted</mml:mtext>
                                        </mml:msub>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">x</mml:mi>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>0</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>1</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                        <mml:mo>=</mml:mo>
                                        <mml:mi mathvariant="normal">&#x03c2;</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">x</mml:mi>
                                            <mml:mo>+</mml:mo>
                                            <mml:mfrac>
                                                <mml:msub>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                        <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mn>1</mml:mn>
                                                </mml:msub>
                                                <mml:msub>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                        <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mn>0</mml:mn>
                                                </mml:msub>
                                            </mml:mfrac>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>0</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>1</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="7.6em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mn>1</mml:mn>
                                            <mml:mrow>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>+</mml:mo>
                                                <mml:mo mathvariant="italic">exp</mml:mo>
                                                <mml:mspace width="0.25em"/>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">{</mml:mo>
                                                    <mml:mo>&#x2212;</mml:mo>
                                                    <mml:msub>
                                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                        <mml:msub>
                                                            <mml:mn>0</mml:mn>
                                                            <mml:mi mathvariant="normal">i</mml:mi>
                                                        </mml:msub>
                                                    </mml:msub>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">x</mml:mi>
                                                        <mml:mo>+</mml:mo>
                                                        <mml:mfrac>
                                                            <mml:msub>
                                                                <mml:mover accent="true">
                                                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                                </mml:mover>
                                                                <mml:mn>1</mml:mn>
                                                            </mml:msub>
                                                            <mml:msub>
                                                                <mml:mover accent="true">
                                                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                                </mml:mover>
                                                                <mml:mn>0</mml:mn>
                                                            </mml:msub>
                                                        </mml:mfrac>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                    <mml:mo>+</mml:mo>
                                                    <mml:msub>
                                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                        <mml:msub>
                                                            <mml:mn>1</mml:mn>
                                                            <mml:mi mathvariant="normal">i</mml:mi>
                                                        </mml:msub>
                                                    </mml:msub>
                                                    <mml:mo stretchy="true">}</mml:mo>
                                                </mml:mrow>
                                                <mml:mspace width="0.25em"/>
                                            </mml:mrow>
                                        </mml:mfrac>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="7.6em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mn>1</mml:mn>
                                            <mml:mrow>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>+</mml:mo>
                                                <mml:mo mathvariant="italic">exp</mml:mo>
                                                <mml:mspace width="0.25em"/>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">{</mml:mo>
                                                    <mml:mo>&#x2212;</mml:mo>
                                                    <mml:msub>
                                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                        <mml:msub>
                                                            <mml:mn>0</mml:mn>
                                                            <mml:mi mathvariant="normal">i</mml:mi>
                                                        </mml:msub>
                                                    </mml:msub>
                                                    <mml:mi mathvariant="normal">x</mml:mi>
                                                    <mml:mo>+</mml:mo>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mo>&#x2212;</mml:mo>
                                                        <mml:mfrac>
                                                            <mml:msub>
                                                                <mml:mover accent="true">
                                                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                                </mml:mover>
                                                                <mml:mn>1</mml:mn>
                                                            </mml:msub>
                                                            <mml:msub>
                                                                <mml:mover accent="true">
                                                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                                </mml:mover>
                                                                <mml:mn>0</mml:mn>
                                                            </mml:msub>
                                                        </mml:mfrac>
                                                        <mml:msub>
                                                            <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                            <mml:msub>
                                                                <mml:mn>0</mml:mn>
                                                                <mml:mi mathvariant="normal">i</mml:mi>
                                                            </mml:msub>
                                                        </mml:msub>
                                                        <mml:mo>+</mml:mo>
                                                        <mml:msub>
                                                            <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                                            <mml:msub>
                                                                <mml:mn>1</mml:mn>
                                                                <mml:mi mathvariant="normal">i</mml:mi>
                                                            </mml:msub>
                                                        </mml:msub>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                    <mml:mo stretchy="true">}</mml:mo>
                                                </mml:mrow>
                                                <mml:mspace width="0.25em"/>
                                            </mml:mrow>
                                        </mml:mfrac>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="7.6em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mn>1</mml:mn>
                                            <mml:mrow>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>+</mml:mo>
                                                <mml:mo mathvariant="italic">exp</mml:mo>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:mo>&#x2212;</mml:mo>
                                                    <mml:msub>
                                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                        <mml:msub>
                                                            <mml:mn>0</mml:mn>
                                                            <mml:mi mathvariant="normal">i</mml:mi>
                                                        </mml:msub>
                                                    </mml:msub>
                                                    <mml:mi mathvariant="normal">x</mml:mi>
                                                    <mml:mo>+</mml:mo>
                                                    <mml:msub>
                                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                        <mml:msub>
                                                            <mml:mn>1</mml:mn>
                                                            <mml:mi mathvariant="normal">i</mml:mi>
                                                        </mml:msub>
                                                    </mml:msub>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                            </mml:mrow>
                                        </mml:mfrac>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="7.6em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03c2;</mml:mi>
                                            <mml:mtext>shifted</mml:mtext>
                                        </mml:msub>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">x</mml:mi>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>0</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo>,</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                <mml:msub>
                                                    <mml:mn>1</mml:mn>
                                                    <mml:mi mathvariant="normal">i</mml:mi>
                                                </mml:msub>
                                            </mml:msub>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                            </mml:mtable>
                        </mml:math>
                        <label>(4)</label>
                    </disp-formula>where
                    <disp-formula id="e5">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                        <label>(5)</label>
                    </disp-formula>
                    <disp-formula id="e6">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:msub>
                                    <mml:mn>1</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:mo>&#x2212;</mml:mo>
                            <mml:mfrac>
                                <mml:msub>
                                    <mml:mover accent="true">
                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                        <mml:mo stretchy="true">&#x00af;</mml:mo>
                                    </mml:mover>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                                <mml:msub>
                                    <mml:mover accent="true">
                                        <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                        <mml:mo stretchy="true">&#x00af;</mml:mo>
                                    </mml:mover>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:mfrac>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                            <mml:mo>+</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                <mml:msub>
                                    <mml:mn>1</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                        <label>(6)</label>
                    </disp-formula>
                </p>
                <p>Thus, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mover accent="true">
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                </mml:mover>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mover accent="true">
                                    <mml:mi mathvariant="normal">&#x03b1;</mml:mi>
                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                </mml:mover>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mover accent="true">
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                </mml:mover>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 0 (
                    <xref ref-type="fig" rid="f1">Figure 1C</xref>).</p>
                <p>The mean square error (MSE) between the fitted sigmoid curve and the corresponding sample was calculated. Samples that fell outside the range of the mean &#x00b1; 3 standard deviations (SDs) in the |
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:msub>
                                    <mml:mn>0</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>|, |
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:msub>
                                    <mml:mn>1</mml:mn>
                                    <mml:mi mathvariant="normal">i</mml:mi>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>|, or MSE value for the i
                    <sup>th</sup> intestine sample were excluded from further analyses, leaving 503 intestine samples. It is important to emphasize that these criteria were not predetermined.</p>
            </sec>
            <sec id="sec10">
                <title>Forward passes of neural networks</title>
                <p>The three-layer perceptron (control) and INN model had the same structure except for their activation functions. The common structure is written as follows:
                    <disp-formula id="e7">
                        <mml:math display="block">
                            <mml:mi mathvariant="bold">x</mml:mi>
                            <mml:mo mathvariant="bold">&#x2254;</mml:mo>
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:mn>2</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:mo>&#x2026;</mml:mo>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">D</mml:mi>
                                            <mml:mi mathvariant="normal">x</mml:mi>
                                        </mml:msub>
                                    </mml:msub>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                        <label>(7)</label>
                    </disp-formula>
                    <disp-formula id="e8">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">a</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:munderover>
                                <mml:mo>&#x2211;</mml:mo>
                                <mml:mi mathvariant="normal">i</mml:mi>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">D</mml:mi>
                                    <mml:mi mathvariant="normal">x</mml:mi>
                                </mml:msub>
                            </mml:munderover>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">w</mml:mi>
                                <mml:mi>ji</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:msub>
                                <mml:mi mathvariant="normal">x</mml:mi>
                                <mml:mi mathvariant="normal">i</mml:mi>
                            </mml:msub>
                            <mml:mo>+</mml:mo>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">b</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                        <label>(8)</label>
                    </disp-formula>
                    <disp-formula id="e9">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">h</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">a</mml:mi>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(9)</label>
                    </disp-formula>
                    <disp-formula id="e10">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">a</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:munderover>
                                <mml:mo>&#x2211;</mml:mo>
                                <mml:mi mathvariant="normal">j</mml:mi>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">D</mml:mi>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                </mml:msub>
                            </mml:munderover>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">w</mml:mi>
                                <mml:mi>kj</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:msub>
                                <mml:mi mathvariant="normal">h</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>+</mml:mo>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">b</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                        <label>(10)</label>
                    </disp-formula>
                    <disp-formula id="e11">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mover accent="true">
                                    <mml:mi mathvariant="normal">y</mml:mi>
                                    <mml:mo stretchy="true">&#x0302;</mml:mo>
                                </mml:mover>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">a</mml:mi>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(11)</label>
                    </disp-formula>
                    <disp-formula id="e12">
                        <mml:math display="block">
                            <mml:mover accent="true">
                                <mml:mi mathvariant="bold">y</mml:mi>
                                <mml:mo stretchy="true">&#x0302;</mml:mo>
                            </mml:mover>
                            <mml:mo mathvariant="bold">&#x2254;</mml:mo>
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:msub>
                                        <mml:mover accent="true">
                                            <mml:mi mathvariant="normal">y</mml:mi>
                                            <mml:mo stretchy="true">&#x0302;</mml:mo>
                                        </mml:mover>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mover accent="true">
                                            <mml:mi mathvariant="normal">y</mml:mi>
                                            <mml:mo stretchy="true">&#x0302;</mml:mo>
                                        </mml:mover>
                                        <mml:mn>2</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:mo>&#x2026;</mml:mo>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mover accent="true">
                                            <mml:mi mathvariant="normal">y</mml:mi>
                                            <mml:mo stretchy="true">&#x0302;</mml:mo>
                                        </mml:mover>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">D</mml:mi>
                                            <mml:mi mathvariant="normal">y</mml:mi>
                                        </mml:msub>
                                    </mml:msub>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                        <label>(12)</label>
                    </disp-formula>where</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="bold">x</mml:mi>
                            <mml:mo>&#x2208;</mml:mo>
                            <mml:msup>
                                <mml:mi>&#x211d;</mml:mi>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">D</mml:mi>
                                    <mml:mi mathvariant="normal">x</mml:mi>
                                </mml:msub>
                            </mml:msup>
                        </mml:math>
                    </inline-formula>: input vector (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">D</mml:mi>
                                <mml:mi mathvariant="normal">x</mml:mi>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>: dimension of the input vector)</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">w</mml:mi>
                                <mml:mi>ji</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                    </inline-formula>: the weight between the i
                    <sup>th</sup> unit in the input layer and the j
                    <sup>th</sup> unit in the hidden layer (j = 1, 2, &#x2026;, 100)</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">b</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                    </inline-formula>: 1, the bias of the j
                    <sup>th</sup> unit in the hidden layer</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mo>&#x00b7;</mml:mo>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula>: the activation function of the j
                    <sup>th</sup> unit in the hidden layer</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">w</mml:mi>
                                <mml:mi>kj</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                    </inline-formula>: the weight between the j
                    <sup>th</sup> unit in the hidden layer and the k
                    <sup>th</sup> unit in the output layer (k = 1, 2, &#x2026;, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">D</mml:mi>
                                <mml:mi mathvariant="normal">y</mml:mi>
                            </mml:msub>
                            <mml:mo>;</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">D</mml:mi>
                                <mml:mi mathvariant="normal">y</mml:mi>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> is dimension of the output vector)</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">b</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                        </mml:math>
                    </inline-formula>: 1, the bias of the k
                    <sup>th</sup> unit in the hidden layer</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mo>&#x00b7;</mml:mo>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula>: the activation function of the k
                    <sup>th</sup> unit in the output layer</p>
                <p>
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mover accent="true">
                                <mml:mi mathvariant="bold">y</mml:mi>
                                <mml:mo stretchy="true">&#x0302;</mml:mo>
                            </mml:mover>
                            <mml:mo>&#x2208;</mml:mo>
                            <mml:msup>
                                <mml:mi>&#x211d;</mml:mi>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">D</mml:mi>
                                    <mml:mi mathvariant="normal">y</mml:mi>
                                </mml:msub>
                            </mml:msup>
                        </mml:math>
                    </inline-formula>: output vector.
                    <disp-formula id="e13">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mtext>Control</mml:mtext>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="normal">x</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mn>1</mml:mn>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>+</mml:mo>
                                    <mml:mtext>exp</mml:mtext>
                                    <mml:mspace width="0.25em"/>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mo>&#x2212;</mml:mo>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mspace width="0.25em"/>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(13)</label>
                    </disp-formula>
                </p>
                <p>On the other hand, the activation functions of the INN model, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">a</mml:mi>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mspace width="0.25em"/>
                        </mml:math>
                    </inline-formula>and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">a</mml:mi>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                </mml:msub>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula>, included two random variables 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>as parameters:
                    <disp-formula id="e14">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mi>INN</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="normal">x</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mn>1</mml:mn>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>+</mml:mo>
                                    <mml:mtext>exp</mml:mtext>
                                    <mml:mspace width="0.25em"/>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mo>&#x2212;</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">B</mml:mi>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                        <mml:mi mathvariant="normal">x</mml:mi>
                                        <mml:mo>+</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">B</mml:mi>
                                            <mml:mn>1</mml:mn>
                                        </mml:msub>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mspace width="0.25em"/>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(14)</label>
                    </disp-formula>
                </p>
                <p>Note that 
                    <xref ref-type="disp-formula" rid="e13">Eq. 13</xref> and 
                    <xref ref-type="disp-formula" rid="e14">Eq. 14</xref> are identical when (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>) = (1, 0). For each INN activation function, namely, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                    <mml:mo>_</mml:mo>
                                    <mml:mi>INN</mml:mi>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">h</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mo>&#x00b7;</mml:mo>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula> (j = 1, 2, &#x2026;, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mn>100</mml:mn>
                        </mml:math>
                    </inline-formula>) and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mrow>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                    <mml:mo>_</mml:mo>
                                    <mml:mi>INN</mml:mi>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mo>&#x00b7;</mml:mo>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula> (k = 1, 2, &#x2026;, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">D</mml:mi>
                                <mml:mi mathvariant="normal">y</mml:mi>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>), the random variables 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo stretchy="true">(</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">j</mml:mi>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>) and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo stretchy="true">(</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">B</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>,</mml:mo>
                                    <mml:mi mathvariant="normal">k</mml:mi>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>) were sampled from 503 sets of experimentally obtained parameters {(
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>1</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                            <mml:mo>,</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>1</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>), (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>2</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                            <mml:mo>,</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>2</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>), &#x2026;, (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>503</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                            <mml:mo>,</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mn>503</mml:mn>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>)} (
                    <xref ref-type="disp-formula" rid="e5">Eqs. 1</xref>&#x2013;
                    <xref ref-type="disp-formula" rid="e10">6</xref>). The source code for the three-layer perceptron and INN is available on our GitHub repository: 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/ywatanabe1989/intestelligence">https://github.com/ywatanabe1989/intestelligence</ext-link>.</p>
            </sec>
            <sec id="sec11">
                <title>Cost function</title>
                <p>The cross-entropy loss was adopted as the cost function, which was calculated as follows:
                    <disp-formula id="e15">
                        <mml:math display="block">
                            <mml:mtable columnalign="left" displaystyle="true">
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mn>1</mml:mn>
                                            <mml:mi mathvariant="normal">M</mml:mi>
                                        </mml:mfrac>
                                        <mml:mspace width="0.25em"/>
                                        <mml:munderover>
                                            <mml:mo>&#x2211;</mml:mo>
                                            <mml:mrow>
                                                <mml:mi mathvariant="normal">m</mml:mi>
                                                <mml:mo>=</mml:mo>
                                                <mml:mn>1</mml:mn>
                                            </mml:mrow>
                                            <mml:mi mathvariant="normal">M</mml:mi>
                                        </mml:munderover>
                                        <mml:mspace width="0.25em"/>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">J</mml:mi>
                                            <mml:mi mathvariant="normal">m</mml:mi>
                                        </mml:msub>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="1.7em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mn>1</mml:mn>
                                            <mml:mi mathvariant="normal">M</mml:mi>
                                        </mml:mfrac>
                                        <mml:munderover>
                                            <mml:mo>&#x2211;</mml:mo>
                                            <mml:mrow>
                                                <mml:mi mathvariant="normal">m</mml:mi>
                                                <mml:mo>=</mml:mo>
                                                <mml:mn>1</mml:mn>
                                            </mml:mrow>
                                            <mml:mi mathvariant="normal">M</mml:mi>
                                        </mml:munderover>
                                        <mml:munderover>
                                            <mml:mo>&#x2211;</mml:mo>
                                            <mml:mrow>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mo>=</mml:mo>
                                                <mml:mn>1</mml:mn>
                                            </mml:mrow>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">D</mml:mi>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                            </mml:msub>
                                        </mml:munderover>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">[</mml:mo>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:msubsup>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:mi mathvariant="normal">m</mml:mi>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                            </mml:msubsup>
                                            <mml:mo>log</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">y</mml:mi>
                                                        <mml:mo stretchy="true">&#x0302;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">y</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>log</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">y</mml:mi>
                                                        <mml:mo stretchy="true">&#x0302;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo stretchy="true">]</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                            </mml:mtable>
                        </mml:math>
                        <label>(15)</label>
                    </disp-formula>where 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                        </mml:math>
                    </inline-formula> represents all the learnable parameters
                    <italic toggle="yes">,</italic> M denotes the mini-batch size (the amount of training data used in each training loop), 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mspace width="0.25em"/>
                            <mml:msub>
                                <mml:mi mathvariant="normal">J</mml:mi>
                                <mml:mi mathvariant="normal">m</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula> represents the cost function derived from the m
                    <sup>th</sup> sample in the mini-batch, and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">y</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> denotes the k
                    <sup>th</sup> element of 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="normal">y</mml:mi>
                        </mml:math>
                    </inline-formula>, which is a one-hot encoded vector of the ground-truth label:
                    <disp-formula id="e16">
                        <mml:math display="block">
                            <mml:mi mathvariant="bold">y</mml:mi>
                            <mml:mo mathvariant="bold">&#x2254;</mml:mo>
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">y</mml:mi>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">y</mml:mi>
                                        <mml:mn>2</mml:mn>
                                    </mml:msub>
                                    <mml:mo>,</mml:mo>
                                    <mml:mo>&#x2026;</mml:mo>
                                    <mml:mo>,</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">y</mml:mi>
                                        <mml:msub>
                                            <mml:mi>D</mml:mi>
                                            <mml:mi>y</mml:mi>
                                        </mml:msub>
                                    </mml:msub>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                        </mml:math>
                    </disp-formula>
                    <disp-formula id="e17">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">y</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">{</mml:mo>
                                <mml:mtable>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:mn>1</mml:mn>
                                        </mml:mtd>
                                        <mml:mtd>
                                            <mml:mtext mathvariant="italic">when</mml:mtext>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mi mathvariant="bold">x</mml:mi>
                                            <mml:mo>&#x2208;</mml:mo>
                                            <mml:mtext mathvariant="italic">class</mml:mtext>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mi mathvariant="normal">K</mml:mi>
                                        </mml:mtd>
                                    </mml:mtr>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:mn>0</mml:mn>
                                        </mml:mtd>
                                        <mml:mtd>
                                            <mml:mtext mathvariant="italic">otherwise</mml:mtext>
                                            <mml:mspace width="0.25em"/>
                                        </mml:mtd>
                                    </mml:mtr>
                                </mml:mtable>
                            </mml:mrow>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                    </disp-formula>
                </p>
            </sec>
            <sec id="sec12">
                <title>Backpropagation</title>
                <p>The backpropagation method (
                    <xref ref-type="bibr" rid="ref20">Rumelhart et al., 1986</xref>) was adopted to efficiently calculate the gradient of the cost function, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo>&#x2207;</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">J</mml:mi>
                                <mml:mi mathvariant="normal">m</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula>.
                    <disp-formula id="e18">
                        <mml:math display="block">
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msubsup>
                                        <mml:mi mathvariant="normal">w</mml:mi>
                                        <mml:mi>kj</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">o</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:msubsup>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msubsup>
                                        <mml:mi mathvariant="normal">w</mml:mi>
                                        <mml:mi>kj</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">o</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:msubsup>
                                </mml:mrow>
                            </mml:mfrac>
                        </mml:math>
                        <label>(16)</label>
                    </disp-formula>
                </p>
                <p>The following notation was introduced:
                    <disp-formula id="e19">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:mo>&#x2254;</mml:mo>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(17)</label>
                    </disp-formula>which is analytically defined as:
                    <disp-formula id="e20">
                        <mml:math display="block">
                            <mml:mtable columnalign="left" displaystyle="true">
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:msubsup>
                                            <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                            <mml:mi mathvariant="normal">k</mml:mi>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mi mathvariant="normal">m</mml:mi>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                        </mml:msubsup>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mi>&#x2202;</mml:mi>
                                            <mml:mrow>
                                                <mml:mi>&#x2202;</mml:mi>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">a</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                            </mml:mrow>
                                        </mml:mfrac>
                                        <mml:msubsup>
                                            <mml:mo>&#x2211;</mml:mo>
                                            <mml:mrow>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mo>=</mml:mo>
                                                <mml:mn>1</mml:mn>
                                            </mml:mrow>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">D</mml:mi>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                            </mml:msub>
                                        </mml:msubsup>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">[</mml:mo>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:msubsup>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:mi mathvariant="normal">m</mml:mi>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                            </mml:msubsup>
                                            <mml:mo>log</mml:mo>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">y</mml:mi>
                                                        <mml:mo stretchy="true">&#x0302;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">y</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>log</mml:mo>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mover accent="true">
                                                        <mml:mi mathvariant="normal">y</mml:mi>
                                                        <mml:mo stretchy="true">&#x0302;</mml:mo>
                                                    </mml:mover>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mo stretchy="true">]</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                                <mml:mtr>
                                    <mml:mtd>
                                        <mml:mspace width="2em"/>
                                        <mml:mo>=</mml:mo>
                                        <mml:mfrac>
                                            <mml:mi>&#x2202;</mml:mi>
                                            <mml:mrow>
                                                <mml:mi>&#x2202;</mml:mi>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">a</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                            </mml:mrow>
                                        </mml:mfrac>
                                        <mml:msubsup>
                                            <mml:mo>&#x2211;</mml:mo>
                                            <mml:mrow>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mo>=</mml:mo>
                                                <mml:mn>1</mml:mn>
                                            </mml:mrow>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">D</mml:mi>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                            </mml:msub>
                                        </mml:msubsup>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">[</mml:mo>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:msubsup>
                                                <mml:mi mathvariant="normal">y</mml:mi>
                                                <mml:mi mathvariant="normal">k</mml:mi>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:mi mathvariant="normal">m</mml:mi>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                            </mml:msubsup>
                                            <mml:mo>log</mml:mo>
                                            <mml:mspace width="0.25em"/>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">{</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">o</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:msubsup>
                                                        <mml:mi mathvariant="normal">a</mml:mi>
                                                        <mml:mi mathvariant="normal">k</mml:mi>
                                                        <mml:mrow>
                                                            <mml:mo stretchy="true">(</mml:mo>
                                                            <mml:mi mathvariant="normal">m</mml:mi>
                                                            <mml:mo stretchy="true">)</mml:mo>
                                                        </mml:mrow>
                                                    </mml:msubsup>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                                <mml:mo stretchy="true">}</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">(</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">y</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">m</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mo stretchy="true">)</mml:mo>
                                            </mml:mrow>
                                            <mml:mo>log</mml:mo>
                                            <mml:mrow>
                                                <mml:mo stretchy="true">{</mml:mo>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>&#x2212;</mml:mo>
                                                <mml:msubsup>
                                                    <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                                    <mml:mi mathvariant="normal">k</mml:mi>
                                                    <mml:mrow>
                                                        <mml:mo stretchy="true">(</mml:mo>
                                                        <mml:mi mathvariant="normal">o</mml:mi>
                                                        <mml:mo stretchy="true">)</mml:mo>
                                                    </mml:mrow>
                                                </mml:msubsup>
                                                <mml:mrow>
                                                    <mml:mo stretchy="true">(</mml:mo>
                                                    <mml:msubsup>
                                                        <mml:mi mathvariant="normal">a</mml:mi>
                                                        <mml:mi mathvariant="normal">k</mml:mi>
                                                        <mml:mrow>
                                                            <mml:mo stretchy="true">(</mml:mo>
                                                            <mml:mi mathvariant="normal">m</mml:mi>
                                                            <mml:mo stretchy="true">)</mml:mo>
                                                        </mml:mrow>
                                                    </mml:msubsup>
                                                    <mml:mo stretchy="true">)</mml:mo>
                                                </mml:mrow>
                                                <mml:mo stretchy="true">}</mml:mo>
                                            </mml:mrow>
                                            <mml:mo stretchy="true">]</mml:mo>
                                        </mml:mrow>
                                    </mml:mtd>
                                </mml:mtr>
                            </mml:mtable>
                        </mml:math>
                    </disp-formula>where:
                    <disp-formula id="e21">
                        <mml:math display="block">
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                        <mml:mtext>Control</mml:mtext>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mi>&#x2202;a</mml:mi>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                <mml:mtext>Control</mml:mtext>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="normal">x</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mrow>
                                <mml:mo stretchy="true">{</mml:mo>
                                <mml:mn>1</mml:mn>
                                <mml:mo>&#x2212;</mml:mo>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                    <mml:mtext>Control</mml:mtext>
                                </mml:msub>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">a</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mo stretchy="true">}</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(18)</label>
                    </disp-formula>
                    <disp-formula id="e22">
                        <mml:math display="block">
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                        <mml:mi>INN</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mi>&#x2202;a</mml:mi>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mo>&#x2212;</mml:mo>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">B</mml:mi>
                                        <mml:mn>0</mml:mn>
                                    </mml:msub>
                                    <mml:mspace width="0.25em"/>
                                    <mml:mtext>exp</mml:mtext>
                                    <mml:mspace width="0.25em"/>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mo>&#x2212;</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">B</mml:mi>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mo>+</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">B</mml:mi>
                                            <mml:mn>1</mml:mn>
                                        </mml:msub>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                    <mml:mspace width="0.25em"/>
                                </mml:mrow>
                                <mml:msup>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">{</mml:mo>
                                        <mml:mn>1</mml:mn>
                                        <mml:mo>+</mml:mo>
                                        <mml:mtext>exp</mml:mtext>
                                        <mml:mspace width="0.25em"/>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mo>&#x2212;</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">B</mml:mi>
                                                <mml:mn>0</mml:mn>
                                            </mml:msub>
                                            <mml:mi mathvariant="normal">a</mml:mi>
                                            <mml:mo>+</mml:mo>
                                            <mml:msub>
                                                <mml:mi mathvariant="normal">B</mml:mi>
                                                <mml:mn>1</mml:mn>
                                            </mml:msub>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                        <mml:mo stretchy="true">}</mml:mo>
                                    </mml:mrow>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                            </mml:mfrac>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(19)</label>
                    </disp-formula>
                </p>
                <p>According to 
                    <xref ref-type="disp-formula" rid="e10">Eq. 10</xref>, we have
                    <disp-formula id="e23">
                        <mml:math display="block">
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msubsup>
                                        <mml:mi mathvariant="normal">w</mml:mi>
                                        <mml:mi>kj</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">o</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:msubsup>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">h</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(20)</label>
                    </disp-formula>
                </p>
                <p>Substituting 
                    <xref ref-type="disp-formula" rid="e19">Eqs. 17</xref> and 
                    <xref ref-type="disp-formula" rid="e23">20</xref> into 
                    <xref ref-type="disp-formula" rid="e18">Eq. 16</xref>, we obtain
                    <disp-formula id="e24">
                        <mml:math display="block">
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msubsup>
                                        <mml:mi mathvariant="normal">w</mml:mi>
                                        <mml:mi>kj</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">o</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:msubsup>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:msub>
                                <mml:mi mathvariant="normal">h</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(21)</label>
                    </disp-formula>
                </p>
                <p>For the 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                        </mml:math>
                    </inline-formula>s of the hidden units, the chain rule for partial derivatives is applied to obtain:
                    <disp-formula id="e25">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>&#x2254;</mml:mo>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">j</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>=</mml:mo>
                            <mml:munder>
                                <mml:mo>&#x2211;</mml:mo>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:munder>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:mspace width="0.25em"/>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">J</mml:mi>
                                        <mml:mi mathvariant="normal">m</mml:mi>
                                    </mml:msub>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">k</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">j</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(22)</label>
                    </disp-formula>
                </p>
                <p>By substituting 
                    <xref ref-type="disp-formula" rid="e19">Eq. 17</xref> into 
                    <xref ref-type="disp-formula" rid="e25">Eq. 22</xref> and using 
                    <xref ref-type="disp-formula" rid="e9">Eqs. 9</xref> and 
                    <xref ref-type="disp-formula" rid="e10">10</xref>, the following backpropagation formula is obtained:
                    <disp-formula id="e26">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                <mml:mi mathvariant="normal">j</mml:mi>
                            </mml:msub>
                            <mml:mo>=</mml:mo>
                            <mml:mfrac>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msubsup>
                                        <mml:mi mathvariant="normal">&#x03c6;</mml:mi>
                                        <mml:mi mathvariant="normal">j</mml:mi>
                                        <mml:mrow>
                                            <mml:mo stretchy="true">(</mml:mo>
                                            <mml:mi mathvariant="normal">h</mml:mi>
                                            <mml:mo stretchy="true">)</mml:mo>
                                        </mml:mrow>
                                    </mml:msubsup>
                                    <mml:mrow>
                                        <mml:mo stretchy="true">(</mml:mo>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">a</mml:mi>
                                            <mml:mi mathvariant="normal">j</mml:mi>
                                        </mml:msub>
                                        <mml:mo stretchy="true">)</mml:mo>
                                    </mml:mrow>
                                </mml:mrow>
                                <mml:mrow>
                                    <mml:mi>&#x2202;</mml:mi>
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">a</mml:mi>
                                        <mml:mi mathvariant="normal">j</mml:mi>
                                    </mml:msub>
                                </mml:mrow>
                            </mml:mfrac>
                            <mml:munder>
                                <mml:mo>&#x2211;</mml:mo>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:munder>
                            <mml:msubsup>
                                <mml:mi mathvariant="normal">w</mml:mi>
                                <mml:mi>kj</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">o</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msubsup>
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b4;</mml:mi>
                                <mml:mi mathvariant="normal">k</mml:mi>
                            </mml:msub>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(23)</label>
                    </disp-formula>
                </p>
                <p>Thus, the gradient of the cost function 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo>&#x2207;</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">J</mml:mi>
                                <mml:mi mathvariant="normal">m</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula> can be calculated using the backpropagation method.</p>
            </sec>
            <sec id="sec13">
                <title>Weight update</title>
                <p>The model was trained using the stochastic gradient descent (SGD) algorithm.</p>
                <p>If the error function consists of a sum over data points, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="normal">J</mml:mi>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>=</mml:mo>
                            <mml:msub>
                                <mml:mo>&#x2211;</mml:mo>
                                <mml:mi mathvariant="normal">n</mml:mi>
                            </mml:msub>
                            <mml:msub>
                                <mml:mi mathvariant="normal">J</mml:mi>
                                <mml:mi mathvariant="normal">n</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </inline-formula>, after the presentation of pattern n, the SGD algorithm updates the parameter vector 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                        </mml:math>
                    </inline-formula> using the following formula:
                    <disp-formula id="e27">
                        <mml:math display="block">
                            <mml:msup>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">&#x03c4;</mml:mi>
                                    <mml:mo>+</mml:mo>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msup>
                            <mml:mo>=</mml:mo>
                            <mml:msup>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mi mathvariant="normal">&#x03c4;</mml:mi>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                            </mml:msup>
                            <mml:mo>&#x2212;</mml:mo>
                            <mml:mo>&#x03b7;</mml:mo>
                            <mml:mo>&#x2207;</mml:mo>
                            <mml:msub>
                                <mml:mi mathvariant="normal">J</mml:mi>
                                <mml:mi mathvariant="normal">n</mml:mi>
                            </mml:msub>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03b8;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                        <label>(24)</label>
                    </disp-formula>
                </p>
                <p>Here, &#x03c4; denotes the iteration number, and &#x03b7; is the learning rate parameter.</p>
            </sec>
            <sec id="sec14">
                <title>Classification of handwritten digits</title>
                <p>The MNIST database of handwritten digits (
                    <xref ref-type="bibr" rid="ref12">LeCun et al., 1998</xref>) was used for the experiments in this study. The control and INN models were trained based on 60,000 training data points and evaluated based on 10,000 test data points. Each image was 784 pixels (= 28 columns&#x00d7;28 rows), and the images were fed into models after being flattened into a vector. The mini-batch size was set to 100, and the learning rate was set to 0.05.</p>
            </sec>
            <sec id="sec15">
                <title>Simulations of INNs</title>
                <p>The INN models were simulated by sampling the fitting parameters 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>. Dummy fitting parameters 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mtext>dummy</mml:mtext>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mtext>dummy</mml:mtext>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>1</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mtext>dummy</mml:mtext>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>)
                    <sup>T</sup> were sampled from a bimodal Gaussian distribution, as described below.
                    <disp-formula id="e28">
                        <mml:math display="block">
                            <mml:msub>
                                <mml:mi mathvariant="bold">&#x03b2;</mml:mi>
                                <mml:mtext mathvariant="bold">dummy</mml:mtext>
                            </mml:msub>
                            <mml:mo>&#x223c;</mml:mo>
                            <mml:mi mathvariant="normal">N</mml:mi>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mi mathvariant="bold">&#x03bc;</mml:mi>
                                <mml:mo>,</mml:mo>
                                <mml:mi mathvariant="bold">&#x03a3;</mml:mi>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(25)</label>
                    </disp-formula>
                    <disp-formula id="e29">
                        <mml:math display="block">
                            <mml:mi mathvariant="bold">&#x03bc;</mml:mi>
                            <mml:mo>=</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mtable>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:msub>
                                                <mml:mover accent="true">
                                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                </mml:mover>
                                                <mml:mn>0</mml:mn>
                                            </mml:msub>
                                        </mml:mtd>
                                    </mml:mtr>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:msub>
                                                <mml:mover accent="true">
                                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                    <mml:mo stretchy="true">&#x00af;</mml:mo>
                                                </mml:mover>
                                                <mml:mn>1</mml:mn>
                                            </mml:msub>
                                        </mml:mtd>
                                    </mml:mtr>
                                </mml:mtable>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                            <mml:mo>=</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mtable>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:mn>3.06</mml:mn>
                                        </mml:mtd>
                                    </mml:mtr>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:mn>0.00</mml:mn>
                                        </mml:mtd>
                                    </mml:mtr>
                                </mml:mtable>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(26)</label>
                    </disp-formula>
                    <disp-formula id="e30">
                        <mml:math display="block">
                            <mml:mi mathvariant="bold">&#x03a3;</mml:mi>
                            <mml:mo>=</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">(</mml:mo>
                                <mml:mtable>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:msub>
                                                <mml:msup>
                                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                                    <mml:mn>2</mml:mn>
                                                </mml:msup>
                                                <mml:msub>
                                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                    <mml:mn>0</mml:mn>
                                                </mml:msub>
                                            </mml:msub>
                                        </mml:mtd>
                                        <mml:mtd>
                                            <mml:mn>0</mml:mn>
                                        </mml:mtd>
                                    </mml:mtr>
                                    <mml:mtr>
                                        <mml:mtd>
                                            <mml:mn>0</mml:mn>
                                        </mml:mtd>
                                        <mml:mtd>
                                            <mml:msub>
                                                <mml:msup>
                                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                                    <mml:mn>2</mml:mn>
                                                </mml:msup>
                                                <mml:msub>
                                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                                    <mml:mn>1</mml:mn>
                                                </mml:msub>
                                            </mml:msub>
                                        </mml:mtd>
                                    </mml:mtr>
                                </mml:mtable>
                                <mml:mo stretchy="true">)</mml:mo>
                            </mml:mrow>
                        </mml:math>
                        <label>(27)</label>
                    </disp-formula>
                    <disp-formula id="e31">
                        <mml:math display="block">
                            <mml:mo>&#x2200;</mml:mo>
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                            <mml:mo>&#x2208;</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">{</mml:mo>
                                <mml:mn>0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>0.1</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>0.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>1.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>1.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>2.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>2.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>3.0</mml:mn>
                                <mml:mo stretchy="true">}</mml:mo>
                            </mml:mrow>
                        </mml:math>
                    </disp-formula>
                    <disp-formula id="e32">
                        <mml:math display="block">
                            <mml:mo>&#x2200;</mml:mo>
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                            </mml:msub>
                            <mml:mo>&#x2208;</mml:mo>
                            <mml:mrow>
                                <mml:mo stretchy="true">{</mml:mo>
                                <mml:mn>0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>0.1</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>0.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>1</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>1.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>2.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>2.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>3.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>3.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>4.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>4.5</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>5.0</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>10</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>30</mml:mn>
                                <mml:mo>,</mml:mo>
                                <mml:mn>50</mml:mn>
                                <mml:mo stretchy="true">}</mml:mo>
                            </mml:mrow>
                            <mml:mo>.</mml:mo>
                        </mml:math>
                    </disp-formula>
                </p>
                <p>When the slope at the inflection point 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo stretchy="true">(</mml:mo>
                            <mml:msup>
                                <mml:mrow>
                                    <mml:mo stretchy="true">(</mml:mo>
                                    <mml:mfrac>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mrow>
                                                <mml:mn>1</mml:mn>
                                                <mml:mo>_</mml:mo>
                                                <mml:mtext>dummy</mml:mtext>
                                            </mml:mrow>
                                        </mml:msub>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mrow>
                                                <mml:mn>0</mml:mn>
                                                <mml:mo>_</mml:mo>
                                                <mml:mtext>dummy</mml:mtext>
                                            </mml:mrow>
                                        </mml:msub>
                                    </mml:mfrac>
                                    <mml:mo>,</mml:mo>
                                    <mml:mfrac>
                                        <mml:mn>1</mml:mn>
                                        <mml:mn>2</mml:mn>
                                    </mml:mfrac>
                                    <mml:mo stretchy="true">)</mml:mo>
                                </mml:mrow>
                                <mml:mi mathvariant="normal">T</mml:mi>
                            </mml:msup>
                            <mml:mo stretchy="true">)</mml:mo>
                        </mml:math>
                    </inline-formula> of the simulated sigmoid function 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mo stretchy="true">(</mml:mo>
                            <mml:mtext>slope:</mml:mtext>
                            <mml:mo>&#x2212;</mml:mo>
                            <mml:mfrac>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mrow>
                                        <mml:mn>0</mml:mn>
                                        <mml:mo>_</mml:mo>
                                        <mml:mtext>dummy</mml:mtext>
                                    </mml:mrow>
                                </mml:msub>
                                <mml:mn>4</mml:mn>
                            </mml:mfrac>
                            <mml:mo stretchy="true">)</mml:mo>
                        </mml:math>
                    </inline-formula> was negative (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mrow>
                                    <mml:mn>0</mml:mn>
                                    <mml:mo>_</mml:mo>
                                    <mml:mtext>dummy</mml:mtext>
                                </mml:mrow>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> &gt; 0), 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="bold">&#x03b2;</mml:mi>
                                <mml:mtext mathvariant="bold">dummy</mml:mtext>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> was resampled. A total of 140 (= 8 &#x00d7; 15) experiments were performed.</p>
            </sec>
            <sec id="sec16">
                <title>Classification of edible/inedible images</title>
                <p>A total of 1,200 images of edible objects and 1,200 images of inedible objects were randomly sampled from 
                    <ext-link ext-link-type="uri" xlink:href="https://www.sozaijiten.com">https://www.sozaijiten.com</ext-link>. Then, 1,000 images from each class (edible/inedible) were used as training data, and the remaining 200 images from each class were used as test data.</p>
                <p>The edible images included images of cooked meals and raw foods such as fruits, vegetables, and fish. The inedible images included images of tools, parts, coins, and other objects.</p>
                <p>The images were converted into feature vectors using a pretrained convolutional neural network model (AlexNet; 
                    <xref ref-type="bibr" rid="ref11">Krizhevsky et al., 2012</xref>) based on the Caffe deep learning framework (
                    <xref ref-type="bibr" rid="ref10">Jia et al., 2014</xref>). Specifically, for every input image, the output of the activation of the fc6 layer (4,096-dimension vector) was extracted.</p>
                <p>These feature vectors were fed into the INN model instead of the original images to perform the 2-class classification task. The model consists of three layers: an input layer, a hidden layer, and an output layer. The input layer contains 4,097 nodes, including a bias unit. The hidden layer has 101 nodes, also inclusive of a bias unit. The output layer consists of 2 nodes corresponding to two classes: edible or inedible. The learning rate was set to 0.6.</p>
            </sec>
            <sec id="sec17">
                <title>Statistical methods</title>
                <p>To compare classification accuracies of INN and Control, we employed the Mann-Whitney U test due to the absence of any specific parametric hypotheses. The test was conducted using the Python package SciPy (
                    <ext-link ext-link-type="uri" xlink:href="https://scipy.org/">https://scipy.org/</ext-link>). The rank-biserial correlation was used as the effect size measure.</p>
            </sec>
        </sec>
        <sec id="sec18" sec-type="results">
            <title>Results</title>
            <sec id="sec19">
                <title>Ileal sample contraction during acetylcholine bath application</title>
                <p>After different concentrations of Ach were sequentially applied, the lengths of the isolated ileum tissues decreased. We used an experimental device to measure the isotonic contraction of the intestinal tract (
                    <xref ref-type="fig" rid="f1">Figure 1A</xref>) and recorded the dose&#x2013;response curves of the ilea during bath application of ACh at concentrations ranging from 10
                    <sup>&#x2212;9</sup> to 10
                    <sup>&#x2212;3</sup> M (
                    <xref ref-type="fig" rid="f1">Figure 1B</xref>, 
                    <italic toggle="yes">n</italic> = 512 tissues from 14 guinea pigs). Each dose&#x2013;response curve was obtained from a single tissue and was normalized and fitted to a sigmoid function (
                    <xref ref-type="fig" rid="f1">Figure 1C</xref>) using two fitting parameters, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> (
                    <xref ref-type="disp-formula" rid="e5">Eqs. 1</xref>&#x2013;
                    <xref ref-type="disp-formula" rid="e10">6</xref>). The MSE between the experimentally recorded data and the fitted sigmoid curve was calculated for all 512 samples. Data samples for which the |
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>|, |
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>| or MSE values exceeded the mean &#x00b1; 3SD were omitted from the following analysis, and the remaining 503 data points were used to construct the INN model. The distributions of 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> are plotted in 
                    <xref ref-type="fig" rid="f1">Figure 1D</xref> and 
                    <xref ref-type="fig" rid="f1">1E</xref>. For 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, the sample mean 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mover accent="true">
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                                <mml:mo stretchy="true">&#x00af;</mml:mo>
                            </mml:mover>
                        </mml:math>
                    </inline-formula> was 3.06, and the sample variance 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">s</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> was 1.38; for 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, the sample mean 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:mover accent="true">
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                                <mml:mo stretchy="true">&#x00af;</mml:mo>
                            </mml:mover>
                        </mml:math>
                    </inline-formula> was 0.00 (due to the adjustment), and the sample variance 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">s</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> was 3.23).</p>
                <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                    <label>Figure 1. </label>
                    <caption>
                        <title>Dose&#x2013;response curves of ACh-induced contractions of the ileum.</title>
                        <p>(A) Illustration of our experimental devices for measuring ileum contraction.</p>
                        <p>(B) The dose&#x2013;response curves of 512 data points for contractions in response to bath application of ACh. Each 
                            <italic toggle="yes">gray</italic> line indicates a single data point, whereas the 
                            <italic toggle="yes">black</italic> line indicates the mean.</p>
                        <p>(C) Five examples of sigmoid fitting of the dose&#x2013;response curves of ACh-induced ileum contraction. Note that 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>0</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> and 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> are fitting parameters.</p>
                        <p>(D, E) Distributions of the fitting parameters 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>0</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> (D) and 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> (E).</p>
                    </caption>
                    <graphic id="gr1" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/158989/51553c93-6e33-4599-8a5e-9c7537c7a123_figure1.gif"/>
                </fig>
            </sec>
            <sec id="sec20">
                <title>Classification of handwritten digits by the INN model</title>
                <p>We constructed a three-layer perceptron (Control) and an INN, both with the same structure as illustrated in 
                    <xref ref-type="fig" rid="f2">Figure 2A</xref>. These models were used for a 10-class classification task using handwritten digits from the MNIST database (
                    <xref ref-type="bibr" rid="ref12">LeCun et al., 1998</xref>). During 6,000 rounds of training, the cost of the Control model was lower than that of the INN model (
                    <xref ref-type="fig" rid="f2">Figure 2B</xref>; at Loop #6,000, Control: 0.558 &#x00b1; 0.097; INN: 0.923 &#x00b1; 0.188; 
                    <italic toggle="yes">p</italic> &lt; 0.001; 
                    <italic toggle="yes">n</italic>s = 40; 
                    <italic toggle="yes">U</italic> = 1,556; Mann&#x2013;Whitney U test; the rank-biserial correlation = - 0.945). Both models converged, but the sum of the absolute weight update of the Control model was lower than that of the INN model (
                    <xref ref-type="fig" rid="f2">Figure 2C</xref>; at Loop #6,000, Control: 1.83 &#x00b1; 0.86; INN: 2.58 &#x00b1; 0.94; 
                    <italic toggle="yes">p</italic> &lt; 0.001; 
                    <italic toggle="yes">n</italic>s = 40; 
                    <italic toggle="yes">U</italic> = 1,182; Mann&#x2013;Whitney U test; the rank-biserial correlation = - 0.478). The accuracy of the Control model was higher than that of the INN model (
                    <xref ref-type="fig" rid="f2">Figure 2D</xref>; at Loop #6,000, Control: 92.3 &#x00b1; 0.4; INN: 85.7 &#x00b1; 0.6; Chance: 10%; 
                    <italic toggle="yes">p</italic> &lt; 0.001; 
                    <italic toggle="yes">n</italic>s = 40; 
                    <italic toggle="yes">U</italic> = 0; Mann&#x2013;Whitney U test; the rank-biserial correlation = 1.000).</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>Figure 2. </label>
                    <caption>
                        <title>Handwritten number identification using the INN model.</title>
                        <p>(A) A schematic diagram of the three-layer INN model. A conventional perceptron with the same structure was used as the control.</p>
                        <p>(B&#x2013;D) Output values of test data during training. B, cost function; C, sum of the absolute weight updates; D, classification accuracy. The 
                            <italic toggle="yes">blue</italic> and 
                            <italic toggle="yes">black</italic> lines represent the INN and control model results, respectively. Each line indicates a single data point (
                            <italic toggle="yes">n</italic> = 32 tuning trials), whereas the thick line indicates the mean.</p>
                    </caption>
                    <graphic id="gr2" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/158989/51553c93-6e33-4599-8a5e-9c7537c7a123_figure2.gif"/>
                </fig>
                <p>We also observed the sample variance in the fitting parameters 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> (
                    <xref ref-type="fig" rid="f1">Figure 1D</xref> and 
                    <xref ref-type="fig" rid="f1">1E</xref>). These fluctuations might originate from (i) biological noise and (ii) differences in the experimental conditions. We hypothesized that the variations in 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>0</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                <mml:mn>1</mml:mn>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> (
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>, 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>) affect the classification accuracy of the model based on the MNIST dataset. To test this hypothesis, we simulated INNs using various values of 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula>. 
                    <xref ref-type="fig" rid="f3">Figure 3</xref> shows the accuracies of the virtually generated INNs in a pseudocolor map. The highest mean accuracy was 92.3% &#x00b1; 0.3% for 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 0.0 and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 0.0, and the lowest mean accuracy was 57.3% &#x00b1; 5.5% for 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>0</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 3.0 and 
                    <inline-formula>
                        <mml:math display="inline">
                            <mml:msub>
                                <mml:msup>
                                    <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                    <mml:mn>2</mml:mn>
                                </mml:msup>
                                <mml:msub>
                                    <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                    <mml:mn>1</mml:mn>
                                </mml:msub>
                            </mml:msub>
                        </mml:math>
                    </inline-formula> = 50.</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>Figure 3. </label>
                    <caption>
                        <title>Relationship between INN performance in number identification tasks and parameter variation.</title>
                        <p>Accuracies of INNs as a function of the variance in the hyperparameters 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>0</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> and 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                        <mml:mn>1</mml:mn>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> (
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:msup>
                                            <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                            <mml:mn>2</mml:mn>
                                        </mml:msup>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> and 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:msup>
                                            <mml:mi mathvariant="normal">&#x03c3;</mml:mi>
                                            <mml:mn>2</mml:mn>
                                        </mml:msup>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula>). The experimentally obtained sample variance (
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:msup>
                                            <mml:mi mathvariant="normal">s</mml:mi>
                                            <mml:mn>2</mml:mn>
                                        </mml:msup>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mn>0</mml:mn>
                                        </mml:msub>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> = 1.38; 
                            <inline-formula>
                                <mml:math display="inline">
                                    <mml:msub>
                                        <mml:msup>
                                            <mml:mi mathvariant="normal">s</mml:mi>
                                            <mml:mn>2</mml:mn>
                                        </mml:msup>
                                        <mml:msub>
                                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                                            <mml:mn>1</mml:mn>
                                        </mml:msub>
                                    </mml:msub>
                                </mml:math>
                            </inline-formula> = 3.23) is indicated by the 
                            <italic toggle="yes">black</italic> dot.</p>
                    </caption>
                    <graphic id="gr3" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/158989/51553c93-6e33-4599-8a5e-9c7537c7a123_figure3.gif"/>
                </fig>
            </sec>
            <sec id="sec21">
                <title>Edible/inedible image classification by the INN model</title>
                <p>A total of 1,200 images of edible and inedible objects were randomly chosen from the dataset and split into training/test datasets at a ratio of 5:1 (
                    <xref ref-type="fig" rid="f4">Figure 4A</xref>). The accuracies of the model based on the test data were evaluated during training. At loop #6,000, the accuracy was 88.5% &#x00b1; 0.9% (
                    <xref ref-type="fig" rid="f4">Figure 4</xref>; 
                    <italic toggle="yes">n</italic> = 32 cross-validations).</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Image classification using the INN model.</title>
                        <p>(A) Examples of edible (
                            <italic toggle="yes">left</italic>) and inedible (
                            <italic toggle="yes">right</italic>) images in the dataset. The top images were used for training, and the bottom images were used for testing. (B) The learning curves of the INN prediction results for the test data. Each 
                            <italic toggle="yes">gray</italic> line indicates a single data point (
                            <italic toggle="yes">n</italic> = 32 cross-validation), whereas the 
                            <italic toggle="yes">black</italic> line indicates the mean.</p>
                    </caption>
                    <graphic id="gr4" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/158989/51553c93-6e33-4599-8a5e-9c7537c7a123_figure4.gif"/>
                </fig>
            </sec>
        </sec>
        <sec id="sec22" sec-type="discussion">
            <title>Discussion</title>
            <p>In this study, we confirmed that different intestine samples exhibited distinct contraction responses to ACh (
                <xref ref-type="fig" rid="f1">Figure 1B</xref> and 
                <xref ref-type="fig" rid="f1">1C</xref>). Using the response curves, we built an INN model and demonstrated that the INN model could be used to precisely classify handwritten digits. Furthermore, we artificially generated various INN models and found that the INN model with the average activation function, or the three-layer perceptron, achieved the highest accuracy (92.3% &#x00b1; 0.3%). Finally, the INN model could be used to classify edible/inedible images, achieving a classification accuracy of 88.5% &#x00b1; 0.9%.</p>
            <p>This study presents a novel approach for building neural networks using biological data and demonstrates the potential of the enteric nervous system for developing activation functions for machine learning models. Existing activation functions can be broadly classified into two groups: (i) the sigmoid group, including the logistic sigmoid function and the hyperbolic tangent function, and (ii) the V-shaped group, including the ReLU (Hasnloser et al., 2000; 
                <xref ref-type="bibr" rid="ref16">Nair &amp; Hinton, 2010</xref>), Leaky ReLU (Maas et al., 2013), PReLU (He et al., 2015), Swish (
                <xref ref-type="bibr" rid="ref18">Ramachandran et al., 2017</xref>), and Mish (Misra, 2019) activation functions. However, the use of sigmoid functions has recently decreased because sigmoid functions lead to slow weight updates and vanishing gradients (
                <xref ref-type="bibr" rid="ref3">Bengio et al., 1994</xref>) in the backpropagation framework (
                <xref ref-type="bibr" rid="ref20">Rumelhart et al., 1986</xref>). Despite this disadvantage, sigmoid functions remain important for biologically inspired network models (
                <xref ref-type="bibr" rid="ref21">Shinozaki, 2021</xref>; 
                <xref ref-type="bibr" rid="ref22">Zhan et al., 2023</xref>), as backpropagation is unlikely to occur in a real brain (
                <xref ref-type="bibr" rid="ref2">Bartunov et al., 2018</xref>). Sigmoid curves are known to emerge as cumulative functions for bell-shaped curves, such as normal distributions. The sigmoid curves observed in our pharmacological experiments are regarded as macroscopic expressions of decision-making processes at the biomolecular level in the enteric nervous system because the ACh signaling pathways via muscarinic receptors trigger contractions in smooth muscles (
                <xref ref-type="bibr" rid="ref6">Eglen et al., 1996</xref>).</p>
            <p>One limitation of our study is the granularity of the model. Our intestine samples were modeled as nodes in an INN, rather than as the network itself. Thus, the developed INN can be viewed as an ensemble model based on intestine samples. This modeling approach was necessitated by the difficulty in recording a large number of individual neurons and receptors. Our model may not necessarily be inaccurate because the central and enteric nervous system exhibit plasticity (
                <italic toggle="yes">e.g.</italic>, updated functional connectivity at synapses; 
                <xref ref-type="bibr" rid="ref5">Citri &amp; Malenka, 2008</xref>) and can be viewed as an ensemble model of different network states at various timepoints. On the other hand, we observed different ACh responsiveness among individual intestine samples (approximately 10
                <sup>2</sup> scale). This variation may be caused by biological noise, as indicated by the observed parameters 
                <inline-formula>
                    <mml:math display="inline">
                        <mml:msub>
                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                            <mml:mn>0</mml:mn>
                        </mml:msub>
                    </mml:math>
                </inline-formula> and 
                <inline-formula>
                    <mml:math display="inline">
                        <mml:msub>
                            <mml:mi mathvariant="normal">&#x03b2;</mml:mi>
                            <mml:mn>1</mml:mn>
                        </mml:msub>
                    </mml:math>
                </inline-formula>, which were located in local maxima. Due to these variations, the INN model may have shown different sensitivity to various input data (
                <xref ref-type="fig" rid="f2">Figure 2B</xref>) and plasticity (
                <xref ref-type="fig" rid="f2">Figure 2C</xref>), which are desirable characteristics in real-world settings, in which the data are often unpredictable and unfixed and balancing the explore-exploit tradeoff is important. To verify this hypothesis, novel datasets and evaluation methods that closely mimic real-world scenarios should be developed to make fair comparisons between 
                <italic toggle="yes">in silico</italic> and biological models.</p>
        </sec>
        <sec id="sec23" sec-type="conclusion">
            <title>Conclusion</title>
            <p>This study has pioneered the development of an INN model based on biological intestine data, offering a novel approach to neural network construction. While the model&#x2019;s performance in image classification tasks may currently be inferior to traditional three-layer perceptrons, this work focuses on the innovative methodology and its potential for future improvements. The model demonstrates the feasibility and uniqueness of using biological data as a basis for neural network development. Future work could explore the effects of biological diversity among ileum samples in their responsiveness to acetylcholine on the generalization ability of such networks. This could advance our understanding and the practical utility of biologically inspired models.</p>
        </sec>
        <sec id="sec26">
            <title>Ethics approval</title>
            <p>All animal experiments were performed with the approval of the Animal Experiment Ethics Committee of the University of Tokyo (approval number: P24-1) and in accordance with the University of Tokyo guidelines for the care and use of laboratory animals</p>
        </sec>
        <sec id="sec27">
            <title>Author contributions</title>
            <p>Y.I., H.B., and N.H. conceptualized the study; Y.W. performed the experiments and data analysis; Y.W. and Y.I. wrote the original draft; and all authors reviewed and edited the final manuscript.</p>
        </sec>
    </body>
    <back>
        <sec id="sec30" sec-type="data-availability">
            <title>Data availability statement</title>
            <sec id="sec31">
                <title>Underlying data</title>
                <p>Zenodo: Intestelligence: A pharmacological neural network using intestine data, 
                    <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/records/11213157">https://zenodo.org/records/11213157</ext-link> (
                    <xref ref-type="bibr" rid="ref23">Watanabe et al., 2024</xref>).</p>
                <p>Data is available under the terms of Creative Commons Zero v1.0 Universal</p>
            </sec>
            <sec id="sec24">
                <title>Reporting guidelines</title>
                <p>Zenode: Checklist for &#x201c;Intestelligence: A pharmacological neural network using intestine data&#x201d;, 
                    <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/records/11213157">https://zenodo.org/records/11213157</ext-link>
                </p>
            </sec>
        </sec>
        <sec id="sec25">
            <title>Software availability statement</title>
            <p>

                <list list-type="bullet">
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Source code available from: 
                            <ext-link ext-link-type="uri" xlink:href="https://github.com/ywatanabe1989/intestelligence">https://github.com/ywatanabe1989/intestelligence</ext-link>
                        </p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>Archived software available from: 
                            <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/records/11213157">https://zenodo.org/records/11213157</ext-link>
                        </p>
                    </list-item>
                    <list-item>
                        <label>&#x2022;</label>
                        <p>License: Creative Commons Zero v1.0 Universal</p>
                    </list-item>
                </list>
            </p>
            <p>The source code for this study is available at Zenode (
                <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/records/11213157">https://zenodo.org/records/11213157</ext-link>).</p>
        </sec>
        <ack>
            <title>Acknowledgments</title>
            <p>We thank Dr. Hideyoshi Igata for his technical support.</p>
        </ack>
        <ref-list>
            <title>References</title>
            <ref id="ref1">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Amari</surname>
                            <given-names>S</given-names>
                        </name>
</person-group>:
                    <article-title>A Theory of Adaptive Pattern Classifiers.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans. Electron. Comput.</italic>
</source>
                    <year>1967</year>;<volume>EC-16</volume>:<fpage>299</fpage>&#x2013;<lpage>307</lpage>.
                    <pub-id pub-id-type="doi">10.1109/PGEC.1967.264666</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref2">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Bartunov</surname>
                            <given-names>S</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Assessing the scalability of biologically-motivated deep learning algorithms and architectures.</chapter-title>
                    <source>

                        <italic toggle="yes">Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS&#x2019;18.</italic>
</source>
                    <publisher-name>Curran Associates Inc.</publisher-name>;<year>2018</year>; pp.<fpage>9390</fpage>&#x2013;<lpage>9400</lpage>.</mixed-citation>
            </ref>
            <ref id="ref3">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Bengio</surname>
                            <given-names>Y</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Simard</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Frasconi</surname>
                            <given-names>P</given-names>
                        </name>
</person-group>:
                    <article-title>Learning long-term dependencies with gradient descent is difficult.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans. Neural Netw.</italic>
</source>
                    <year>1994</year>;<volume>5</volume>:<fpage>157</fpage>&#x2013;<lpage>166</lpage>.
                    <pub-id pub-id-type="pmid">18267787</pub-id>
                    <pub-id pub-id-type="doi">10.1109/72.279181</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref4">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Burgen</surname>
                            <given-names>AS</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Spero</surname>
                            <given-names>L</given-names>
                        </name>
</person-group>:
                    <article-title>The action of acetylcholine and other drugs on the efflux of potassium and rubidium from smooth muscle of the guinea-pig intestine.</article-title>
                    <source>

                        <italic toggle="yes">Br. J. Pharmacol.</italic>
</source>
                    <year>1997</year>;<volume>120</volume>:<fpage>317</fpage>&#x2013;<lpage>333</lpage>. discussion 315-316. 1968.
                    <pub-id pub-id-type="pmid">9142413</pub-id>
                    <pub-id pub-id-type="doi">10.1111/j.1476-5381.1997.tb06813.x</pub-id>
                    <pub-id pub-id-type="pmcid">PMC3224308</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref5">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Citri</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Malenka</surname>
                            <given-names>RC</given-names>
                        </name>
</person-group>:
                    <article-title>Synaptic Plasticity: Multiple Forms, Functions, and Mechanisms.</article-title>
                    <source>

                        <italic toggle="yes">Neuropsychopharmacology.</italic>
</source>
                    <year>2008</year>;<volume>33</volume>:<fpage>18</fpage>&#x2013;<lpage>41</lpage>.
                    <pub-id pub-id-type="doi">10.1038/sj.npp.1301559</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref6">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Eglen</surname>
                            <given-names>RM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hegde</surname>
                            <given-names>SS</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Watson</surname>
                            <given-names>N</given-names>
                        </name>
</person-group>:
                    <article-title>Muscarinic receptor subtypes and smooth muscle function.</article-title>
                    <source>

                        <italic toggle="yes">Pharmacol. Rev.</italic>
</source>
                    <year>1996</year>;<volume>48</volume>:<fpage>531</fpage>&#x2013;<lpage>565</lpage>.
                    <pub-id pub-id-type="pmid">8981565</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref7">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Glorot</surname>
                            <given-names>X</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bordes</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bengio</surname>
                            <given-names>Y</given-names>
                        </name>
</person-group>:
                    <article-title>Deep Sparse Rectifier Neural Networks.</article-title>
                    <source>

                        <italic toggle="yes">Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, (JMLR Workshop and Conference Proceedings.</italic>
</source>
                    <year>2011</year>; pp.<fpage>315</fpage>&#x2013;<lpage>323</lpage>.</mixed-citation>
            </ref>
            <ref id="ref24">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Goyal</surname>
                            <given-names>RK</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hirano</surname>
                            <given-names>I</given-names>
                        </name>
</person-group>:
                    <article-title>The enteric nervous system.</article-title>
                    <source>

                        <italic toggle="yes">N. Engl. J. Med.</italic>
</source>
                    <year>1996</year>;<volume>334</volume>(<issue>17</issue>):<fpage>1106</fpage>&#x2013;<lpage>1115</lpage>. Erratum in: N Engl J Med 1996 Jul 18;335(3):215.
                    <pub-id pub-id-type="pmid">8598871</pub-id>
                    <pub-id pub-id-type="doi">10.1056/NEJM199604253341707</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref8">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Hahnloser</surname>
                            <given-names>RHR</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sarpeshkar</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Mahowald</surname>
                            <given-names>MA</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit.</article-title>
                    <source>

                        <italic toggle="yes">Nature.</italic>
</source>
                    <year>2000</year>;<volume>405</volume>:<fpage>947</fpage>&#x2013;<lpage>951</lpage>.
                    <pub-id pub-id-type="pmid">10879535</pub-id>
                    <pub-id pub-id-type="doi">10.1038/35016072</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref9">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Jarrett</surname>
                            <given-names>K</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kavukcuoglu</surname>
                            <given-names>K</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ranzato</surname>
                            <given-names>M</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>What is the best multi-stage architecture for object recognition?</article-title>
                    <source>

                        <italic toggle="yes">2009 IEEE 12th International Conference on Computer Vision.</italic>
</source>
                    <year>2009</year>; pp.<fpage>2146</fpage>&#x2013;<lpage>2153</lpage>.</mixed-citation>
            </ref>
            <ref id="ref10">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Jia</surname>
                            <given-names>Y</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Caffe: Convolutional Architecture for Fast Feature Embedding.</article-title>
                    <source>

                        <italic toggle="yes">Proceedings of the 22nd ACM International Conference on Multimedia, MM&#x2019;14. (Association for Computing Machinery).</italic>
</source>
                    <year>2014</year>; pp.<fpage>675</fpage>&#x2013;<lpage>678</lpage>.</mixed-citation>
            </ref>
            <ref id="ref11">
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Krizhevsky</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sutskever</surname>
                            <given-names>I</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hinton</surname>
                            <given-names>GE</given-names>
                        </name>
</person-group>:
                    <chapter-title>ImageNet Classification with Deep Convolutional Neural Networks.</chapter-title>
                    <source>

                        <italic toggle="yes">Advances in Neural Information Processing Systems.</italic>
</source>
                    <publisher-name>Curran Associates, Inc.</publisher-name>;<year>2012</year>.</mixed-citation>
            </ref>
            <ref id="ref12">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Lecun</surname>
                            <given-names>Y</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bottou</surname>
                            <given-names>L</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bengio</surname>
                            <given-names>Y</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Gradient-based learning applied to document recognition.</article-title>
                    <source>

                        <italic toggle="yes">Proc. IEEE.</italic>
</source>
                    <year>1998</year>;<volume>86</volume>:<fpage>2278</fpage>&#x2013;<lpage>2324</lpage>.
                    <pub-id pub-id-type="doi">10.1109/5.726791</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref13">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Maniscalco</surname>
                            <given-names>JW</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Rinaman</surname>
                            <given-names>L</given-names>
                        </name>
</person-group>:
                    <article-title>Vagal Interoceptive Modulation of Motivated Behavior.</article-title>
                    <source>

                        <italic toggle="yes">Physiology.</italic>
</source>
                    <year>2018</year>;<volume>33</volume>:<fpage>151</fpage>&#x2013;<lpage>167</lpage>.
                    <pub-id pub-id-type="pmid">29412062</pub-id>
                    <pub-id pub-id-type="doi">10.1152/physiol.00036.2017</pub-id>
                    <pub-id pub-id-type="pmcid">PMC5899236</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref14">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Mayer</surname>
                            <given-names>EA</given-names>
                        </name>
</person-group>:
                    <article-title>Gut feelings: the emerging biology of gut&#x2013;brain communication.</article-title>
                    <source>

                        <italic toggle="yes">Nat. Rev. Neurosci.</italic>
</source>
                    <year>2011</year>;<volume>12</volume>:<fpage>453</fpage>&#x2013;<lpage>466</lpage>.
                    <pub-id pub-id-type="pmid">21750565</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nrn3071</pub-id>
                    <pub-id pub-id-type="pmcid">PMC3845678</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref15">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Misra</surname>
                            <given-names>D</given-names>
                        </name>
</person-group>:
                    <article-title>Mish: A Self Regularized Non-Monotonic Activation Function.</article-title>
                    <year>2020</year>. (October 22, 2022).
                    <pub-id pub-id-type="doi">10.48550/arXiv.1908.08681</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref16">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Nair</surname>
                            <given-names>V</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hinton</surname>
                            <given-names>GE</given-names>
                        </name>
</person-group>:
                    <article-title>Rectified Linear Units Improve Restricted Boltzmann Machines.</article-title>
                    <year>2010</year>;<volume>8</volume>.</mixed-citation>
            </ref>
            <ref id="ref17">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Obata</surname>
                            <given-names>Y</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Pachnis</surname>
                            <given-names>V</given-names>
                        </name>
</person-group>:
                    <article-title>The Effect of Microbiota and the Immune System on the Development and Organization of the Enteric Nervous System.</article-title>
                    <source>

                        <italic toggle="yes">Gastroenterology.</italic>
</source>
                    <year>2016</year>;<volume>151</volume>:<fpage>836</fpage>&#x2013;<lpage>844</lpage>.
                    <pub-id pub-id-type="pmid">27521479</pub-id>
                    <pub-id pub-id-type="doi">10.1053/j.gastro.2016.07.044</pub-id>
                    <pub-id pub-id-type="pmcid">PMC5102499</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref18">
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Ramachandran</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zoph</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Le</surname>
                            <given-names>QV</given-names>
                        </name>
</person-group>:
                    <article-title>Searching for Activation Functions.</article-title>
                    <year>2017</year>. (October 22, 2022).
                    <pub-id pub-id-type="doi">10.48550/arXiv.1710.05941</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref19">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Rao</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Gershon</surname>
                            <given-names>MD</given-names>
                        </name>
</person-group>:
                    <article-title>The bowel and beyond: the enteric nervous system in neurological disorders.</article-title>
                    <source>

                        <italic toggle="yes">Nat. Rev. Gastroenterol. Hepatol.</italic>
</source>
                    <year>2016</year>;<volume>13</volume>:<fpage>517</fpage>&#x2013;<lpage>528</lpage>.
                    <pub-id pub-id-type="pmid">27435372</pub-id>
                    <pub-id pub-id-type="doi">10.1038/nrgastro.2016.107</pub-id>
                    <pub-id pub-id-type="pmcid">PMC5005185</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref20">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Rumelhart</surname>
                            <given-names>DE</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hinton</surname>
                            <given-names>GE</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Williams</surname>
                            <given-names>RJ</given-names>
                        </name>
</person-group>:
                    <article-title>Learning representations by back-propagating errors.</article-title>
                    <source>

                        <italic toggle="yes">Nature.</italic>
</source>
                    <year>1986</year>;<volume>323</volume>:<fpage>533</fpage>&#x2013;<lpage>536</lpage>.
                    <pub-id pub-id-type="doi">10.1038/323533a0</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref21">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Shinozaki</surname>
                            <given-names>T</given-names>
                        </name>
</person-group>:
                    <article-title>Biologically motivated learning method for deep neural networks using hierarchical competitive learning.</article-title>
                    <source>

                        <italic toggle="yes">Neural Netw.</italic>
</source>
                    <year>2021</year>;<volume>144</volume>:<fpage>271</fpage>&#x2013;<lpage>278</lpage>.
                    <pub-id pub-id-type="pmid">34520937</pub-id>
                    <pub-id pub-id-type="doi">10.1016/j.neunet.2021.08.027</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref23">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Watanabe</surname>
                            <given-names>Y</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Intestelligence: A pharmacological neural network using intestine data (v1.0).</article-title>
                    <source>

                        <italic toggle="yes">Zenodo.</italic>
</source>
                    <year>2024</year>.
                    <pub-id pub-id-type="doi">10.5281/zenodo.11213157</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref22">
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Zhan</surname>
                            <given-names>Q</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Liu</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Xie</surname>
                            <given-names>X</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Bio-inspired Active Learning method in spiking neural network.</article-title>
                    <source>

                        <italic toggle="yes">Knowl.-Based Syst.</italic>
</source>
                    <year>2023</year>;<volume>261</volume>:<fpage>110193</fpage>.
                    <pub-id pub-id-type="doi">10.1016/j.knosys.2022.110193</pub-id>
                </mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report422553">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.158989.r422553</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Siddiqui</surname>
                        <given-names>Samra</given-names>
                    </name>
                    <xref ref-type="aff" rid="r422553a1">1</xref>
                    <xref ref-type="aff" rid="r422553a2">2</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r422553a1">
                    <label>1</label>COMSATS University Islamabad - Wah Campus, Wah Cantt., Punjab, Pakistan</aff>
                <aff id="r422553a2">
                    <label>2</label>HITEC University, Taxila, Pakistan</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>14</day>
                <month>11</month>
                <year>2025</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Siddiqui S</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport422553" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.145100.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>1. The paper presents a novel concept of integrating neural architecture with biological data, but some points need to be considered explicitly.</p>
            <p> 2. While the conceptual innovation is clear, the paper would benefit from a more detailed quantitative comparison with standard architectures. Including accuracy metrics, loss curves, or ablation studies would help assess the model&#x2019;s performance more objectively. Some metrics, like accuracy, are stated but do not give promising results.</p>
            <p> 3.&#x00a0;The biological rationale for mapping intestinal data to network parameters could be further elaborated. Why just the ileum is selected as part of experimentation? Explaining how the chosen biological signals correspond to specific neural network behaviors would strengthen the study&#x2019;s interpretability.</p>
            <p> 4. More information on the preprocessing of biological data, model training procedures, and parameter settings is needed to ensure reproducibility and clarity.</p>
            <p> 5. The manuscript could discuss potential limitations, for example, how scalable the proposed INN framework is for larger datasets or different modalities beyond image classification.</p>
            <p> 
                <italic>6.&#x00a0;</italic>Some conceptual transitions (from biological modeling to computational implementation) could be made clearer for readers unfamiliar with both fields.</p>
            <p>Is the work clearly and accurately presented and does it cite the current literature?</p>
            <p>Yes</p>
            <p>If applicable, is the statistical analysis and its interpretation appropriate?</p>
            <p>Yes</p>
            <p>Are all the source data underlying the results available to ensure full reproducibility?</p>
            <p>Partly</p>
            <p>Is the study design appropriate and is the work technically sound?</p>
            <p>Partly</p>
            <p>Are the conclusions drawn adequately supported by the results?</p>
            <p>Partly</p>
            <p>Are sufficient details of methods and analysis provided to allow replication by others?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Machine learning, Digital Image processing</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
    </sub-article>
    <sub-article article-type="reviewer-report" id="report422554">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.158989.r422554</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Ziaul karim</surname>
                        <given-names>Dewan</given-names>
                    </name>
                    <xref ref-type="aff" rid="r422554a1">1</xref>
                    <role>Referee</role>
                </contrib>
                <aff id="r422554a1">
                    <label>1</label>Brac University, Dhaka, Bangladesh</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>7</day>
                <month>11</month>
                <year>2025</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2025 Ziaul karim D</copyright-statement>
                <copyright-year>2025</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport422554" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.145100.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The study is technically detailed, ethically transparent, and supported by a reproducible technique (including available GitHub code). However, some essential areas require clarity, quantitative rigor, and broader contextualization to fulfill the standards of a high-impact indexing.</p>
            <p> </p>
            <p> The research does not adequately explain why gut contraction behavior is a useful model for neuron activation. While the theory is fascinating and based on the concept of the &#x201c;second brain,&#x201d; there has to be a stronger scientific link between how smooth muscle contractions relate to how neurons function. The biological reasoning should be expanded to illustrate why this analogy makes sense in computational terms.</p>
            <p> </p>
            <p> The INN model achieved 85.7% accuracy on MNIST, which is lower than many basic neural networks. The authors state that the purpose is to highlight the method rather than performance, however it would still benefit to compare INN with alternative activation functions such as Swish, GELU, or other biologically inspired models.</p>
            <p> The research should also clarify why the accuracy is reduced - is it because of biological noise, the variability of the activation slopes, or limits of the sigmoid shape?</p>
            <p> </p>
            <p> The research does not properly explain what the parameter values (&#x03b2;&#x2080; and &#x03b2;&#x2081;) signify biologically or how they affect the network&#x2019;s learning. A simple analysis relating changes in slope or curve form to learning behavior or stability would make the results more accessible and useful.</p>
            <p> </p>
            <p> Although the ethical approval process is presented effectively, the study can clearly clarify why it was important to use real animal tissue. Since equivalent data might be created synthetically, the authors should define why live samples were required and how the advantages justified the ethical and methodological cost.</p>
            <p> </p>
            <p> The studies only employ two simple datasets - MNIST and edible/inedible image categorization. These are great for an initial demonstration, but adding one more test with a biologically related dataset (such as biomedical or physiological data) would make the work stronger and prove that the INN may be effective in real-world scientific situations.</p>
            <p>Is the work clearly and accurately presented and does it cite the current literature?</p>
            <p>Yes</p>
            <p>If applicable, is the statistical analysis and its interpretation appropriate?</p>
            <p>I cannot comment. A qualified statistician is required.</p>
            <p>Are all the source data underlying the results available to ensure full reproducibility?</p>
            <p>Partly</p>
            <p>Is the study design appropriate and is the work technically sound?</p>
            <p>Yes</p>
            <p>Are the conclusions drawn adequately supported by the results?</p>
            <p>Partly</p>
            <p>Are sufficient details of methods and analysis provided to allow replication by others?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>Neural Network, Medical Image processing.</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
    </sub-article>
</article>
