<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd"><article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="1.2" xml:lang="en">
    <front>
        <journal-meta>
            <journal-id journal-id-type="pmc">F1000Research</journal-id>
            <journal-title-group>
                <journal-title>F1000Research</journal-title>
            </journal-title-group>
            <issn pub-type="epub">2046-1402</issn>
            <publisher>
                <publisher-name>F1000 Research Limited</publisher-name>
                <publisher-loc>London, UK</publisher-loc>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.12688/f1000research.109148.1</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Research Article</subject>
                </subj-group>
                <subj-group>
                    <subject>Articles</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Study of bypassing Microsoft Windows Security using the MITRE CALDERA Framework</article-title>
                <fn-group content-type="pub-status">
                    <fn>
                        <p>[version 1; peer review: 1 approved with reservations]</p>
                    </fn>
                </fn-group>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author" corresp="yes">
                    <name>
                        <surname>Mohamed</surname>
                        <given-names>Nachaat</given-names>
                    </name>
                    <role content-type="http://credit.niso.org/">Conceptualization</role>
                    <role content-type="http://credit.niso.org/">Data Curation</role>
                    <role content-type="http://credit.niso.org/">Methodology</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Original Draft Preparation</role>
                    <role content-type="http://credit.niso.org/">Writing &#x2013; Review &amp; Editing</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-4202-1294</uri>
                    <xref ref-type="corresp" rid="c1">a</xref>
                    <xref ref-type="aff" rid="a1">1</xref>
                </contrib>
                <aff id="a1">
                    <label>1</label>Assistant Professor of Homeland Security, Rabdan Academy, Abu Dhabi, United Arab Emirates</aff>
            </contrib-group>
            <author-notes>
                <corresp id="c1">
                    <label>a</label>
                    <email xlink:href="mailto:eng.cne1@gmail.com">eng.cne1@gmail.com</email>
                </corresp>
                <fn fn-type="conflict">
                    <p>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>14</day>
                <month>4</month>
                <year>2022</year>
            </pub-date>
            <pub-date pub-type="collection">
                <year>2022</year>
            </pub-date>
            <volume>11</volume>
            <elocation-id>422</elocation-id>
            <history>
                <date date-type="accepted">
                    <day>18</day>
                    <month>3</month>
                    <year>2022</year>
                </date>
            </history>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2022 Mohamed N</copyright-statement>
                <copyright-year>2022</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <self-uri content-type="pdf" xlink:href="https://f1000research.com/articles/11-422/pdf"/>
            <abstract>
                <p>
                    <bold>Background:</bold> Microsoft Windows Security is a recently implemented safeguard for the Windows operating systems, including the latest versions of Windows10 and 11. However, there is a major shortcoming in this system to stop Advanced Persistent Threat (APT). These are government-financed groups that are funded to attack other government entities. Following the initial security breach, the hacked Windows device is used to access the rest of the network devices in order to transfer data to external storage (Exfiltration).</p>
                <p>
                    <bold>Methods:</bold> In this work, we have tested the Microsoft Windows Security system using MITRE CALDERA and ATT&amp;CK frameworks and explain how APT groups are able to bypass Windows Security.</p>
                <p>
                    <bold>Results:</bold> In this study we used "54ndc47" agent through GoLang feature in MITRE CALDERA platform to test and bypass Microsoft Windows Security systems (MS Windows 10). Through it, we were able to bypass the Windows Security system and display entire files in the victim&#x2019;s device.</p>
                <p>
                    <bold>Conclusions:</bold> In this paper, we have provided recommendations to Microsoft to improve their Windows Security tool through the use of Artificial intelligence (AI).</p>
            </abstract>
            <kwd-group kwd-group-type="author">
                <kwd>APT</kwd>
                <kwd>CPU</kwd>
                <kwd>Attack</kwd>
                <kwd>Exploit</kwd>
                <kwd>Detection</kwd>
                <kwd>Cyberattack.</kwd>
            </kwd-group>
            <funding-group>
                <award-group id="fund-1">
                    <funding-source>Rabdan Academy (Homeland Security) Abu Dhabi, United Arab Emirates.</funding-source>
                </award-group>
                <funding-statement>This work was fully funded by Rabdan Academy (Homeland Security) Abu Dhabi, United Arab Emirates.</funding-statement>
                <funding-statement>
                    <italic>The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</italic>
                </funding-statement>
            </funding-group>
        </article-meta>
    </front>
    <body>
        <sec id="sec1" sec-type="intro">
            <title>Introduction</title>
            <p>Windows 10 and 11 incorporate Windows Security, which provides users with the most recent antivirus assurance. Windows Security will begin operating and secure the system from the minute one begins Windows.
                <sup>
                    <xref ref-type="bibr" rid="ref1">1</xref>
                </sup> It ceaselessly looks for malware (pernicious programs), infections, and security dangers.
                <sup>
                    <xref ref-type="bibr" rid="ref1">1</xref>
                </sup> In addition to this real-time assurance, overhauls are downloaded automatically to assist in keeping the device secure from ongoing threats. Since this mode is streamlined for tighter security, the Infection &amp; Danger assurance zone has fewer alternatives.
                <sup>
                    <xref ref-type="bibr" rid="ref2">2</xref>
                </sup> Built-in security within this mode that naturally anticipates infections and other dangers running on user devices, and users then receive security overhauls as they continue using their device.</p>
            <p>As with each previously released version, Windows 10 was intended to be the most secure Windows operating system. As part of that release, Microsoft presented Windows Security as a beneficial addition.
                <sup>
                    <xref ref-type="bibr" rid="ref3">3</xref>
                </sup> This offered an improved approach to building, sending, and adjusting Windows data, and unused highlights are built persistently with each overhaul. Windows 10 also has more layers of assurance that assist in securing organizational information, as well as identifying unsafe behaviors and modern assaults. Windows 10 is therefore making a difference using superior secure data and with each subsequent release, Microsoft have built upon the existing security measures by including modern security highlights.
                <sup>
                    <xref ref-type="bibr" rid="ref3">3</xref>
                </sup> They have consciously tended to dangers through iterative design, ensuring that improved security is one of the operating system&#x2019;s greatest benefits.
                <sup>
                    <xref ref-type="bibr" rid="ref4">4</xref>
                </sup> However, there is a huge shortcoming in this product, and traditional hackers and Advanced Persistent Threat (APT) groups are utilizing varying methods to bypass this improved level of Windows security.</p>
            <p>In order to test the level of vulnerability, we simulated an APT attack to bypass the Windows security, using the CALDERA framework from 
                <ext-link ext-link-type="uri" xlink:href="https://github.com/mitre/caldera">MITRE</ext-link>. CALDERA is a tested framework to evaluate features of infrastructure security posture through penetration testing. It tests the entire suite of tactics and techniques used by APT.
                <sup>
                    <xref ref-type="bibr" rid="ref12">12</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref13">13</xref>
                </sup> The CALDERA framework is used by red teams to protect organizations against sophisticated attacks, and adversary emulation is resource intensive and can present challenges. To combat these challenges, the CALDERA framework offers an intelligent, automated system of red teamwork, which can reduce the resources needed by penetration testing and security teams for routine testing. This then leads to providing the right solution/recommendation to the blue teams/organizations.
                <sup>
                    <xref ref-type="bibr" rid="ref14">14</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref15">15</xref>
                </sup>
            </p>
            <p>CALDERA can also be utilized to test endpoint security arrangements and evaluate a network&#x2019;s security capability to withstand the common post-compromise, antagonistic strategies contained within the ATT&amp;CK approach.
                <sup>
                    <xref ref-type="bibr" rid="ref14">14</xref>
                </sup> CALDERA leverages the ATT&amp;CK approach to distinguish and imitate enemy behaviors as if a genuine interruption is happening. This empowers computerized evaluations of a network&#x2019;s defenselessness to enemy penetration, permitting organizations to see their systems through the eyes of a progressive, determined danger, on-demand and to confirm the strength of guards and security arrangements currently based upon known risk methods. It also employs an enemy representation dialect; the ATT&amp;CK profile. This is a motor choice to prepare assembled information and select ensuing activities, and a specialist conducting the operation. Utilizing CALDERA can therefore decrease assets required for appraisals and permit groups to focus on modern approaches to more difficult issues.
                <sup>
                    <xref ref-type="bibr" rid="ref16">16</xref>
                </sup> This can enable organizations to tune behavioral-based, interruption discovery frameworks more quickly as they are deployed.
                <sup>
                    <xref ref-type="bibr" rid="ref17">17</xref>
                </sup>
            </p>
            <p>CALDERA is also complementary to other types of security evaluation. The infrastructure security position is commonly surveyed based on program fix levels, security controls, and shield devices. Whereas numerous interruption location apparatuses depend on looking for known risk markers which alter as often as possible. Appraisals and enemy discovery are only typically based upon foe behavior.
                <sup>
                    <xref ref-type="bibr" rid="ref18">18</xref>
                </sup> This can change how shields respond to, identify, and react to dynamic dangers. CALDERA can also make a difference to shield approaches as it can move past discovery of pointers of compromise, through to location and reaction of foe behavior.
                <sup>
                    <xref ref-type="bibr" rid="ref17">17</xref>
                </sup> In addition to the expansion to the open-source adaptation of CALDERA, Miter maintains a closed-source form that highlights extra capabilities, creating superior adaptability to more endpoints. These are used to examine authorizing or collaboration exercises on closed-source CALDERA.
                <sup>
                    <xref ref-type="bibr" rid="ref18">18</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref19">19</xref>
                </sup>
            </p>
            <sec id="sec2">
                <title>Related work</title>
                <p>Microsoft&#x2019;s Protector is proficient at recognizing malware records, blocking misuses and network-based assaults, and hailing phishing destinations. It incorporates basic PC execution and wellbeing reports, as well as parental controls with substance sifting, utilization impediments, and area following. Antivirus software is fundamental if the user is utilizing a Mac or Windows device; both come with some level of infection assurance built in.
                    <sup>
                        <xref ref-type="bibr" rid="ref3">3</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref4">4</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref7">7</xref>
                    </sup> In order to build upon these protections and develop endpoint security, protection against malware and possibly undesirable programs, it is best to introduce a third-party antivirus application. Microsoft Guard was not present in the old versions of Windows operating systems, so users of Microsoft operating systems purchased or acquired an antivirus program for device protection. This has changed over time, and Microsoft has integrated an antivirus program to protect its operating systems and software running under them 10.
                    <sup>
                        <xref ref-type="bibr" rid="ref8">8</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref9">9</xref>
                    </sup>
                </p>
                <p>Microsoft now indirectly implies that when in the Microsoft system work environment, a user does not need the support of other companies for protection. Microsoft are now able to provide this protection to the end user utilizing Microsoft Protector. A user&#x2019;s device, files, and data are under its protection from direct or indirect tampering.
                    <sup>
                        <xref ref-type="bibr" rid="ref10">10</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref11">11</xref>
                    </sup> In addition, it is now true that Microsoft Shield is sometimes seen as a competitor with decent protection capabilities when compared to the large security systems in the free antivirus world.
                    <sup>
                        <xref ref-type="bibr" rid="ref10">10</xref>
                    </sup> This product has evolved significantly since being first developed due to Microsoft&#x2019;s relentless pursuit of this product. This could be because it would prevent the company&#x2019;s end users from buying protection products from other companies, which was clearly noted in the latest evaluations conducted by the main independent laboratories that conduct tests at fixed intervals to measure the readiness of antivirus applications.
                    <sup>
                        <xref ref-type="bibr" rid="ref7">7</xref>
                    </sup> A test of these available antivirus systems was conducted in July and October 2020 by AV-Comparatives, showing that Microsoft&#x2019;s performance improved considerably, and Microsoft was rated overall as &#x2018;good&#x2019; as it was able to stop 99.5% of the risks, and in this test it came in twelfth place among 17 competing anti-virus programs.
                    <sup>
                        <xref ref-type="bibr" rid="ref22">22</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref23">23</xref>
                    </sup> In another similar evaluation conducted by SE Labs, Microsoft&#x2019;s product scored 99%, making it the fifth out of 13 participants in the competition. In a further report on protection against intrusion, as of 2020, Microsoft ranked fourth; a result indicating its quality, especially considering the competition was with top providers in the antivirus industry.
                    <sup>
                        <xref ref-type="bibr" rid="ref24">24</xref>
                    </sup> The overall picture, therefore, is that Microsoft Guard is now more than robust in terms of protecting the user from hacking and malware. In short, according to Avira, we can say that Microsoft Protector may be sufficient for the user to satisfactorily protect their data, but it is also clear that it represents a reputable and reliable &#x2018;non-free&#x2019; antivirus option.
                    <sup>
                        <xref ref-type="bibr" rid="ref25">25</xref>
                    </sup>
                </p>
                <p>With regards to users who do not have high levels of technical experience, it is difficult for them to correctly judge whether this product is sufficient to protect them or not. It can therefore represent an advantageous product for some, and a limited one for others. This difference depends on the way users navigate their computer hardware, software and the Internet.
                    <sup>
                        <xref ref-type="bibr" rid="ref8">8</xref>
                    </sup> However, if the user can easily get a free protection program that meets their needs and does not put them at risk, then those can represent a suitable subscription for both experienced users and novice users.
                    <sup>
                        <xref ref-type="bibr" rid="ref7">7</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref8">8</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref10">10</xref>
                    </sup> Unfortunately, the area of antivirus software can also be used by malicious groups to distribute agents used to penetrate users&#x2019; systems. This has and will continue to occur, and therefore improved access to reputable antivirus programs for both ends of the market is a positive change.
                    <sup>
                        <xref ref-type="bibr" rid="ref11">11</xref>
                    </sup> However, it is also true that although Microsoft strives to maintain its reputation and increase the number of its users by providing products that meet user needs,
                    <sup>
                        <xref ref-type="bibr" rid="ref30">30</xref>
                    </sup> the findings from this paper prove with conclusive evidence that it is possible to bypass Microsoft Anti-Virus and take control of the device, just as the same device can then be used to hack the rest of the devices in the same infrastructure.</p>
            </sec>
        </sec>
        <sec id="sec3" sec-type="methods">
            <title>Methods</title>
            <p>The objective of robotized/simulated APT attacks imitating adversaries (APT groups) is to detect weaknesses in the current systems and provide system defenders with an apparatus able to execute a full-scale evaluation of their organization, working in a way that is comparable to a genuine adversary. Such an apparatus has noteworthy utility for guards, ultimately providing a standard for what their network looks like to an enemy, producing preparedness data, identifying shortcomings and/or misconfigurations, and testing in-place security measures and devices. This provides a valuable experimental proof for a cautious blue team to build on.
                <sup>
                    <xref ref-type="bibr" rid="ref2">2</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref31">31</xref>
                </sup> We differentiate this with a device that, for example, only distinguishes assault methods and approaches without actually executing them. This tool could provide an outline of what the organization looks like digitally, but typically will come up short to realize other utilization cases because it omits critical, hard-to-measure points of interest, and requires the authenticity of genuine execution. The objective of CALDERA therefore is to drive automated adversaries that do not just imitate, but also incorporate:
                <list list-type="order">
                    <list-item>
                        <label>1)</label>
                        <p>Selecting and chaining actions in ways comparable to how an attacker would.</p>
                    </list-item>
                    <list-item>
                        <label>2)</label>
                        <p>Allowing shields to be able to utilize the tool without requiring express arrangement of points of interest. These are both time-intensive to gather and are nearly incomprehensible for guards to completely track.</p>
                    </list-item>
                    <list-item>
                        <label>3)</label>
                        <p>The framework executes the same techniques that a genuine enemy would, and, like a genuine adversary, should begin at initial compromise, and progress to their intended end after achieving (or coming up short to realize) a particular set of goals.</p>
                    </list-item>
                    <list-item>
                        <label>4)</label>
                        <p>Clients of the framework ought to be able to run appraisals with methods of their choosing, as well as have the capacity to include modern strategies.
                            <sup>
                                <xref ref-type="bibr" rid="ref2">2</xref>
                            </sup>
                            <sup>,</sup>
                            <sup>
                                <xref ref-type="bibr" rid="ref30">30</xref>
                            </sup>
                            <sup>,</sup>
                            <sup>
                                <xref ref-type="bibr" rid="ref32">32</xref>
                            </sup>
                        </p>
                    </list-item>
                </list>
            </p>
            <p>CALDERA therefore offers and adds to the field of information security, as it automates the penetration-testing process and simulates the tactics and techniques used by state-funded hacking groups to attack other countries for the purposes of espionage or sabotage.
                <sup>
                    <xref ref-type="bibr" rid="ref16">16</xref>
                </sup> As a consequence, it is used as a tool by red teams to simulate a real attack, and blue teams can then use realistic data to explain protection plans and methods to protect public and private institutions.
                <sup>
                    <xref ref-type="bibr" rid="ref18">18</xref>
                </sup> It has also been designed to deal with aspects of the defensive and offensive systems of MITER ATT&amp;CK. This system consists of two components: 1) the server that contains all the operations with an application interface to control all the operations that take place on the victim&#x2019;s device.
                <sup>
                    <xref ref-type="bibr" rid="ref18">18</xref>
                </sup> 2) Additions. This is the main user interface through which it is possible to control the addition and deletion of components that serve the attack process, which must eventually lead to access and control of the largest possible amount of the target device&#x2019;s resources.
                <sup>
                    <xref ref-type="bibr" rid="ref19">19</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref20">20</xref>
                </sup> The end result therefore provides a product that will automatically test an organization&#x2019;s infrastructure against the tactics and techniques used by currently operating hackers.
                <sup>
                    <xref ref-type="bibr" rid="ref15">15</xref>
                </sup> It provides the red team a contemporary and informed ability to test the dangerous infrastructure in the shortest possible time and with as few people as possible, so one or two individuals can simulate an entire team of penetration testers. This is provided by CALDERA, produced and developed by MITER.
                <sup>
                    <xref ref-type="bibr" rid="ref17">17</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref32">32</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref37">37</xref>
                </sup> It can be directly used to design a client and test it on potential victims to test if there is a vulnerability that hackers can use against the targeted network and the rest of the network devices.
                <sup>
                    <xref ref-type="bibr" rid="ref16">16</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref33">33</xref>
                </sup> This paper presents an application of the CALDERA approach to Microsoft&#x2019;s security systems. It demonstrates that the protections were bypassed, new permissions were added, and all files on the victim&#x2019;s device were accessed.</p>
            <p>The client used in the attack is called Sandcat, a small command generation program identified as &#x201c;54ndc47&#x201d;, which can evade and attack the opponent, and reconnect with the CALDERA server. This &#x201c;54ndc47&#x201d; agent was created in GoLang to be compatible with most existing operating systems. The operation of &#x201c;54ndc47&#x201d; requires port 8888 to be opened to communicate with the server. To run &#x201c;54ndc47&#x201d;, one of the commands included in this framework that corresponds to the operating system or the so-called potential target is used, which allows the user to run remote commands.
                <sup>
                    <xref ref-type="bibr" rid="ref15">15</xref>
                </sup> These commands download the &#x201c;54ndc47&#x201d; executable compiled and provided by CALDERA and immediately run it on the victim or target machine. All commands and instructions can then be accessed through the Sandcat plugin. Once commands are executed on the target device, the attacker&#x2019;s device appears to have successfully communicated with the victim&#x2019;s device after sending it to CALDERA. With regards to the CALDERA server, every time a transfer command is run, the attack command forcefully reassembles itself and changes its source code to contact the attacker&#x2019;s machine, so that it gets a &#x201c;MD5&#x201d; miscellaneous registry hash. This certainly helps bypassing command-dependent signature detections in files.
                <sup>
                    <xref ref-type="bibr" rid="ref17">17</xref>
                </sup>
                <sup>,</sup>
                <sup>
                    <xref ref-type="bibr" rid="ref38">38</xref>
                </sup> When running &#x201c;54ndc47&#x201d;, important parameters can be used after the executable is running. The agent in this regard must be sent to at least one victim device within the target infrastructure.
                <sup>
                    <xref ref-type="bibr" rid="ref19">19</xref>
                </sup> In addition, virtual groups will be used during the attack process and the communication between the victim&#x2019;s device and the attacker. The attack process is organized and arranged to maintain the lightweight code, &#x201c;54NDC47&#x201d; or hack commands which are sometimes restrictive to avoid detection devices in general.</p>
            <p>The work used in the attack process has powerful additional features, referred to as GoCat extensions. It also includes an extension to the existing GoCat module tokens to provide many benefits, such as the use of a peer-to-peer broker, additional proxies, and additional C2 communication protocols.
                <sup>
                    <xref ref-type="bibr" rid="ref13">13</xref>
                </sup> To request other restricted plugins from GoCat, the client can perform a HTTP merge of all GoCat extensions when C2 is queried, which is called a custom assembler. The title should be a comma-separated list with technical considerations.
                <sup>
                    <xref ref-type="bibr" rid="ref34">34</xref>
                </sup> The server includes additional extensions and is not required if their conditions are met (e.g., if extension A requires a specific GoLang that is not available on the server, then extension A is almost certainly not included at this point). It is possible to set default values for these alternatives when Sandcat is pulled from an attacker&#x2019;s machine. Of course, this is highly valuable if hiding parameters from a method is required. This can be done by passing the values as headers instead of as parameters.</p>
            <p>For illustration, the following will download a windows executable that will utilize http://192.168.1.14:8888 as the server address rather than http://localhost:8888.</p>
            <fig fig-type="figure" id="f1" orientation="portrait" position="float">
                <label>Figure 1. </label>
                <caption>
                    <title>SANDCAT methodology (agent).</title>
                </caption>
                <graphic id="gr1" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure1.gif"/>
            </fig>
            <sec id="sec4">
                <title>Steps</title>
                <p>
                    <italic toggle="yes">First step: update Kali Linux</italic>
                </p>
                <p>In this step, we updated the Kali Linux system version 2022.1 to get the latest version of all the programs in the Kali Linux distribution; this can be done by typing the following command in the terminal &#x201c;apt-get update&#x201d;. 
                    <xref ref-type="fig" rid="f2">Figure 2</xref> shows the command used to update Kali Linux.</p>
                <fig fig-type="figure" id="f2" orientation="portrait" position="float">
                    <label>Figure 2. </label>
                    <caption>
                        <title>Updating Kali Linux.</title>
                    </caption>
                    <graphic id="gr2" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure2.gif"/>
                </fig>
                <p>
                    <italic toggle="yes">Second step: installing CALDERA</italic>
                </p>
                <p>Then, we proceeded to installing the CALDERA framework within the Kali distribution by using the following command &#x201c;apt-get -y install caldera&#x201d;. 
                    <xref ref-type="fig" rid="f3">Figure 3</xref> shows the command used to install CALDERA.</p>
                <fig fig-type="figure" id="f3" orientation="portrait" position="float">
                    <label>Figure 3. </label>
                    <caption>
                        <title>Installing CALDERA.</title>
                    </caption>
                    <graphic id="gr3" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure3.gif"/>
                </fig>
                <p>
                    <italic toggle="yes">Third step: running CALDERA</italic>
                </p>
                <p>Since CALDERA was integrated into the Kali repositories of the latest versions, this made it easy to install and run the CALDERA system from Kali Linux. After we installed CALDERA in the previous step, we then ran it via the command &#x201c;caldera&#x201d;. 
                    <xref ref-type="fig" rid="f4">Figure 4</xref> shows the command used to run CALDERA.</p>
                <fig fig-type="figure" id="f4" orientation="portrait" position="float">
                    <label>Figure 4. </label>
                    <caption>
                        <title>Running CALDERA.</title>
                    </caption>
                    <graphic id="gr4" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure4.gif"/>
                </fig>
                <p>
                    <italic toggle="yes">Fourth step</italic>
                </p>
                <p>After using the previous command, we obtained the login information to the CALDERA framework, red username and password, blue username and password. In this study, we logged in using the red user and password. Considering that CALDERA runs through port 8888, we could start CALDERA by going to the browser and typing the IP of the CALDERA device followed by the port as in the following example. 
                    <ext-link ext-link-type="uri" xlink:href="http://192.168.0.14:8888/">http://192.168.0.14:8888/</ext-link> or 
                    <ext-link ext-link-type="uri" xlink:href="http://localhost:8888/">http://localhost:8888/</ext-link>
                </p>
                <p>Red:</p>
                <p>&#x2003;USERNAME: red</p>
                <p>&#x2003;PASSWORD: je2nj8OypHUCp4fZiH08R9z0qDeKTD6vexftrOJ7Ru0</p>
                <p>&#x2003;API_TOKEN: Sugyhnx290IG3zAFgHh8wvx0zqWHh9yBICUHurcVkuk</p>
                <p>We then logged into the CALDERA framework, and selected "Navigate", then "Agent", and then we chose the orange key on the left side &#x201c;Click here to deploy an agent&#x201d;; after that, we identified the agent kind &#x201c;54ndc47. Finally, we specified the operating system as &#x201c;Windows&#x201d;. Depending on these settings, the CALDERA system generates a code to create a client to bypass the Windows Security system and get a direct connection to the victim&#x2019;s machine as shown below:</p>
                <p>($server="192.168.0.14:8888/";$url="$server/file/download";$wc=New-Object System.Net.WebClient;$wc. Headers.add("platform","windows");$wc. Headers.add("file","sandcat.go");$data=$wc. DownloadData($url);$name=$wc. ResponseHeaders["Content-Disposition"].Substring($wc. ResponseHeaders["Content-Disposition"].IndexOf("filename=")+9).Replace("`"","");get-process|? {$_.modules.filename -like "C:\Users\Public\$name.exe"}|stop-process -f;rm -force "C:\Users\Public\$name.exe" -ea ignore;[io.file]::WriteAllBytes("C:\Users\Public\$name.exe",$data)|Out-Null;Start-Process -FilePath C:\Users\Public\$name.exe -ArgumentList "-server $server -group red" -WindowStyle hidden;)).</p>
            </sec>
            <sec id="sec5">
                <title>Ethics approval</title>
                <p>This study was approved by Rabdan Academy (Homeland Security (HLS) department), Abu Dhabi, United Arab Emirates.</p>
            </sec>
        </sec>
        <sec id="sec6" sec-type="results">
            <title>Results</title>
            <sec id="sec7">
                <title>Bypassing Windows security</title>
                <p>Organizations may fail if they do not implement a solid approach against apt attack and confirmation controls may then permit an aggressor to evade verification. In addition, enemies may also evade the verification component through taking substantial victim sessions and cookies. It is also possible to avoid authentication powerlessness, which appears to permit assailants to perform different noxious processes by avoiding the device verification method. After performing this process, the primary concern is verification bypassing abuse, solely because of a powerless confirmation structure. Companies that fail to maintain a robust and secure infrastructure may create many conditions that allow an attacker to bypass verification.
                    <sup>
                        <xref ref-type="bibr" rid="ref5">5</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref6">6</xref>
                    </sup>
                    <sup>,</sup>
                    <sup>
                        <xref ref-type="bibr" rid="ref38">38</xref>
                    </sup>
                </p>
                <p>In this study, CALDERA was used to send a single command to the victim&#x2019;s device, and through it enable the opening of a session on the target device. This provided the hostname, username, privilege, group, and other sensitive information. 
                    <xref ref-type="fig" rid="f3">Figures 3</xref> and 
                    <xref ref-type="fig" rid="f4">4</xref> show the command used to bypass Microsoft Windows security and take over the victim machine through got active session over victim machine. 
                    <xref ref-type="fig" rid="f5">Figure 5</xref> shows create agent from CALDEARA server for Windows operating system.</p>
                <fig fig-type="figure" id="f5" orientation="portrait" position="float">
                    <label>Figure 5. </label>
                    <caption>
                        <title>Creating an agent for a specific OS.</title>
                    </caption>
                    <graphic id="gr5" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure5.gif"/>
                </fig>
                <p>Following this, the green color batten was utilized (agent 16232) to open the operations screen; this provides considerable possibilities with regards to implementing all the tactics and techniques used by the offensive groups&#x2019; APTs. All this was performed without any warnings or messages identifying suspicious activity on the victim&#x2019;s device, even though the version installed was recently updated to the latest one available. 
                    <xref ref-type="fig" rid="f6">Figure 6</xref> shows bypassing Windows security (connected with victim machine and take over through the agent).</p>
                <fig fig-type="figure" id="f6" orientation="portrait" position="float">
                    <label>Figure 6. </label>
                    <caption>
                        <title>Bypassing Windows Security (connected with victim machine and take over through the agent).</title>
                    </caption>
                    <graphic id="gr6" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure6.gif"/>
                </fig>
                <p>After bypassing the Windows security system by agent that was created by CALDERA and then applying the Collection tactic, gives us the ability to collect the entre files from the victim&#x2019;s machine as shown in the following figure. Taking into account that attacker can use the same device to penetrate the rest of the devices and servers in the target infrastructure trough applying lateral movement tactic, credential access tactic, credential dumping technique. 
                    <xref ref-type="fig" rid="f7">Figure 7</xref> shows results from victim machine (paths to aggregated files).</p>
                <fig fig-type="figure" id="f7" orientation="portrait" position="float">
                    <label>Figure 7. </label>
                    <caption>
                        <title>Results from the victim machine (paths to aggregated files).</title>
                    </caption>
                    <graphic id="gr7" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure7.gif"/>
                </fig>
                <p>The next figure present example of results (paths to aggregated files).</p>
                <p>As we can see, these paths have become compromised, and all files can be transferred to the attacker&#x2019;s device.</p>
                <p>C:\Users\engcn\Desktop\Results of APT Attack\</p>
                <p>C:\Users\engcn\OneDrive\Desktop\CTIA\CTIA Lab Prerequisites\CTIA Lab Prerequisites\CTIA Desktop</p>
                <p>C:\Users\engcn\OneDrive\Desktop\</p>
                <p>The following figures shows the final report that was generated by CALDERA after bypassing Windows Security and getting files from the victim&#x2019;s machine as shown earlier. The full report has been uploaded to the GitHub website as shown in the Data Availability section. The entire code for this study has been uploaded to 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/Nachaat3040/CALDERA-Code-Bypassing-Microsoft-Windows-security-/releases/tag/CALDERA">GitHub</ext-link> and archived in 
                    <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/record/6309927">Zenodo</ext-link>.</p>
                <fig fig-type="figure" id="f8" orientation="portrait" position="float">
                    <label>Figure 8. </label>
                    <caption>
                        <title>CALDERA commands/report.</title>
                    </caption>
                    <graphic id="gr8" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure8.gif"/>
                </fig>
                <fig fig-type="figure" id="f9" orientation="portrait" position="float">
                    <label>Figure 9. </label>
                    <caption>
                        <title>CALDERA commands/report.</title>
                    </caption>
                    <graphic id="gr9" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure9.gif"/>
                </fig>
            </sec>
        </sec>
        <sec id="sec8" sec-type="discussion">
            <title>Discussion</title>
            <sec id="sec9">
                <title>ATT&amp;CK</title>
                <p>Adversarial Tactics, Techniques, and Common Knowledge (ATT&amp;CK) is a database used by information security professionals to understand the methods and techniques used by high-level attack groups utilizing APT, and these groups may be funded by state actors to attack other countries or regions.
                    <sup>
                        <xref ref-type="bibr" rid="ref37">37</xref>
                    </sup> ATT&amp;CK allows to develop new methods and plans to protect against attack groups. The most important thing distinguishing ATT&amp;CK is that it is free and available for governments and private organizations.
                    <sup>
                        <xref ref-type="bibr" rid="ref35">35</xref>
                    </sup> It is possible to take advantage of this approach in order to develop interception methods and defensive plans against offensives by attack teams funded by state actors for espionage or sabotage purposes.
                    <sup>
                        <xref ref-type="bibr" rid="ref36">36</xref>
                    </sup> ATT&amp;CK contains 14 tactics and more than 500 techniques to counter the attacks of these groups.
                    <sup>
                        <xref ref-type="bibr" rid="ref38">38</xref>
                    </sup> 
                    <xref ref-type="fig" rid="f1">Figure 1</xref> shows which tactics and techniques are designed based on MITER. 
                    <xref ref-type="fig" rid="f10">Figure 10</xref> shows ATT&amp;CK tactics and techniques.</p>
                <fig fig-type="figure" id="f10" orientation="portrait" position="float">
                    <label>Figure 10. </label>
                    <caption>
                        <title>ATT&amp;CK tactics and techniques.</title>
                    </caption>
                    <graphic id="gr10" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure10.gif"/>
                </fig>
            </sec>
            <sec id="sec10">
                <title>Real scenario</title>
                <p>In this scenario, 
                    <xref ref-type="fig" rid="f8">Figures 8</xref> and 
                    <xref ref-type="fig" rid="f9">9</xref> illustrates how, through the application of CALDERA, it was possible to infiltrate all the data from the victim&#x2019;s device after achieving the initial access, credential access, and using lateral movement tactics and techniques. Through using the indicial access (IA) tactic, the enemy attempts to create a &#x2018;foothold&#x2019; inside the existing infrastructure to allow access into the target network. IA tactics are ultimately used to comprise the target infrastructure that access different passage paths, in order to establish an introductory, solid footing inside an arrangement. The strategies utilized to pick up a dependable balance incorporate a focus upon spear-phishing and abusing shortcomings on web servers&#x2019; interfaces. The privileges that were obtained at the beginning of the penetration stage greatly facilitated the upgrade process to obtain full privileges on the victim&#x2019;s device, then completely manage the victim&#x2019;s machine and use it to access the rest of the network resources. This may result in limited or no use after passwords have been changed. In this scenario, spear-phishing was conducted through normal commands, sent to the victim machine from the CALDERA attacker machine, in order to pick up get to casualty frameworks. Spear-phishing by incomes of value may be a particular variation of command execution. It is as diverse as most of the more common methods of spear phishing, in that it engages in taking advantage of third-party administrations, rather than specifically using project mail channels. All spear-phishing forms are electronically conveyed social building focused on a particular person, company, or industry. In this situation, enemies send messages through different social media administrations, individual webmail, and other non-enterprise-controlled administrations. These administrations are more likely to have a less-strict security arrangement than other undertakings. As with most types of spear-phishing, the objective is to create affinity with the target or develop the target&#x2019;s intrigue and attention in a variety of ways. 
                    <xref ref-type="fig" rid="f11">Figure 11</xref> Windows defender, and firewall cannot detect the payload when APT used ATT&amp;CK against target infrastructure.</p>
                <fig fig-type="figure" id="f11" orientation="portrait" position="float">
                    <label>Figure 11. </label>
                    <caption>
                        <title>Windows defender, and firewall cannot detect the payload.</title>
                    </caption>
                    <graphic id="gr11" orientation="portrait" position="float" xlink:href="https://f1000research-files.f1000.com/manuscripts/120614/6ffea564-1c3f-4eaa-8cc9-439f7ee58b58_figure11.gif"/>
                </fig>
            </sec>
        </sec>
        <sec id="sec11" sec-type="conclusions">
            <title>Conclusions</title>
            <p>This paper focused upon the application of the MITRE CALDERA framework to test Microsoft Windows security. Utilizing this framework, the paper explains and provides evidence on how to bypass Microsoft Windows security. A primary recommendation following this research, is that Microsoft specifically work to improve the level of security in recognizing commands written on devices by using artificial intelligence. The primary method for this can be through analyzing the commands before executing them. Finally, the paper advises researchers to study MITRE ATT&amp;CK to develop security solutions against APT attacks.</p>
        </sec>
        <sec id="sec12">
            <title>Data availability</title>
            <sec id="sec13">
                <title>Underlying data</title>
                <p>All data underlying the results are included as part of the article and no additional data are required.</p>
            </sec>
            <sec id="sec14">
                <title>Extended data</title>
                <p>Zenodo: Nachaat3040/CALDERA-Code-Bypassing-Microsoft-Windows-security-: Cyber Attack DOI: 
                    <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/record/6309927">https://zenodo.org/record/6309927</ext-link>
                </p>
                <p>This project contains the following extended data:
                    <list list-type="bullet">
                        <list-item>
                            <label>-</label>
                            <p>Agent 1.png</p>
                        </list-item>
                        <list-item>
                            <label>-</label>
                            <p>CALDERA Code - Bypassing Microsoft Windows security.txt</p>
                        </list-item>
                        <list-item>
                            <label>-</label>
                            <p>Collect info blugin 2.png</p>
                        </list-item>
                        <list-item>
                            <label>-</label>
                            <p>Files collected 4.png</p>
                        </list-item>
                        <list-item>
                            <label>-</label>
                            <p>README.txt</p>
                        </list-item>
                        <list-item>
                            <label>-</label>
                            <p>collect txt files 3.png</p>
                        </list-item>
                    </list>
                </p>
                <p>Analysis code available from: 
                    <ext-link ext-link-type="uri" xlink:href="https://github.com/Nachaat3040/CALDERA-Code-Bypassing-Microsoft-Windows-security-/releases/tag/CALDERA">https://github.com/Nachaat3040/CALDERA-Code-Bypassing-Microsoft-Windows-security-/releases/tag/CALDERA</ext-link>
                </p>
                <p>Archived analysis code as at time of publication: 
                    <ext-link ext-link-type="uri" xlink:href="https://zenodo.org/record/6309927">https://zenodo.org/record/6309927</ext-link>
                </p>
                <p>License: 
                    <ext-link ext-link-type="uri" xlink:href="https://opensource.org/licenses/MIT">MIT</ext-link>
                </p>
            </sec>
        </sec>
    </body>
    <back>
        <ack>
            <title>Acknowledgments</title>
            <p>The authors express their gratitude and thanks to the Department of Internal Security at Rabdan Academy for their financial and moral support to complete this study. At the same time, we extend our sincere thanks and appreciation to all the reviewers and readers of this study for their valuable comments and suggestions, which improve the presentation of this research effort in a wonderful way that benefits the community and researchers.</p>
        </ack>
        <ref-list>
            <title>References</title>
            <ref id="ref1">
                <label>1</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Saleem</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ullah</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kwak</surname>
                            <given-names>KS</given-names>
                        </name>
</person-group>:
                    <article-title>A study of IEEE 802.15. 4 security framework for wireless body area networks.</article-title>
                    <source>

                        <italic toggle="yes">Sensors.</italic>
</source>
                    <year>2011</year>;<volume>11</volume>(<issue>2</issue>):<fpage>1383</fpage>&#x2013;<lpage>1395</lpage>.
                    <pub-id pub-id-type="pmid">22319358</pub-id>
                    <pub-id pub-id-type="doi">10.3390/s110201383</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref2">
                <label>2</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Miller</surname>
                            <given-names>D</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Alford</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Applebaum</surname>
                            <given-names>A</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <source>

                        <italic toggle="yes">Automated adversary emulation: A case for planning and acting with unknowns.</italic>
</source>
                    <publisher-name>MITRE CORP MCLEAN VA MCLEAN</publisher-name>;<year>2018</year>.</mixed-citation>
            </ref>
            <ref id="ref3">
                <label>3</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Toorani</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <chapter-title>On vulnerabilities of the security association in the IEEE 802.15. 6 standard.</chapter-title>
                    <source>

                        <italic toggle="yes">International conference on financial cryptography and data security.</italic>
</source>
                    <publisher-loc>Berlin, Heidelberg</publisher-loc>:
                    <publisher-name>Springer</publisher-name>;<year>2015, January</year>; (pp.<fpage>245</fpage>&#x2013;<lpage>260</lpage>).</mixed-citation>
            </ref>
            <ref id="ref4">
                <label>4</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Manadhata</surname>
                            <given-names>PK</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wing</surname>
                            <given-names>JM</given-names>
                        </name>
</person-group>:
                    <article-title>An attack surface metric.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans. Softw. Eng.</italic>
</source>
                    <year>2010</year>;<volume>37</volume>(<issue>3</issue>):<fpage>371</fpage>&#x2013;<lpage>386</lpage>.
                    <pub-id pub-id-type="doi">10.1109/TSE.2010.60</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref5">
                <label>5</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Huang</surname>
                            <given-names>Y</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Arsenault</surname>
                            <given-names>D</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sood</surname>
                            <given-names>A</given-names>
                        </name>
</person-group>:
                    <chapter-title>Closing cluster attack windows through server redundancy and rotations.</chapter-title>
                    <source>

                        <italic toggle="yes">Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID&#x2019;06).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2006, May</year>; (Vol.<volume>2</volume>, pp.<fpage>12</fpage>-pp).</mixed-citation>
            </ref>
            <ref id="ref6">
                <label>6</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Tomlinson</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bryans</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Shaikh</surname>
                            <given-names>SA</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Detection of automotive CAN cyber-attacks by identifying packet timing anomalies in time windows.</chapter-title>
                    <source>

                        <italic toggle="yes">2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2018, June</year>; (pp.<fpage>231</fpage>&#x2013;<lpage>238</lpage>).</mixed-citation>
            </ref>
            <ref id="ref7">
                <label>7</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Furdek</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Natalino</surname>
                            <given-names>C</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Lipp</surname>
                            <given-names>F</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Machine learning for optical network security monitoring: A practical perspective.</article-title>
                    <source>

                        <italic toggle="yes">J. Lightwave Technol.</italic>
</source>
                    <year>2020</year>;<volume>38</volume>(<issue>11</issue>):<fpage>1</fpage>&#x2013;<lpage>2871</lpage>.
                    <pub-id pub-id-type="doi">10.1109/JLT.2020.2987032</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref8">
                <label>8</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Sambandam</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hussein</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Siddiqi</surname>
                            <given-names>N</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Network security for iot using sdn: Timely ddos detection.</chapter-title>
                    <source>

                        <italic toggle="yes">2018 IEEE Conference on Dependable and Secure Computing (DSC).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2018, December</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>2</lpage>).</mixed-citation>
            </ref>
            <ref id="ref9">
                <label>9</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Alharbi</surname>
                            <given-names>F</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Chang</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zhou</surname>
                            <given-names>Y</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Collaborative client-side DNS cache poisoning attack.</chapter-title>
                    <source>

                        <italic toggle="yes">IEEE INFOCOM 2019-IEEE Conference on Computer Communications.</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2019, April</year>; (pp.<fpage>1153</fpage>&#x2013;<lpage>1161</lpage>).</mixed-citation>
            </ref>
            <ref id="ref10">
                <label>10</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Mohamed</surname>
                            <given-names>N</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Belaton</surname>
                            <given-names>B</given-names>
                        </name>
</person-group>:
                    <article-title>SBI Model for the Detection of Advanced Persistent Threat Based on Strange Behavior of Using Credential Dumping Technique.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Access.</italic>
</source>
                    <year>2021</year>;<volume>9</volume>:<fpage>42919</fpage>&#x2013;<lpage>42932</lpage>.
                    <pub-id pub-id-type="doi">10.1109/ACCESS.2021.3066289</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref11">
                <label>11</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Li</surname>
                            <given-names>F</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Li</surname>
                            <given-names>Q</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zhang</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Detection and diagnosis of data integrity attacks in solar farms based on multilayer long short-term memory network.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans. Power Electron.</italic>
</source>
                    <year>2020</year>;<volume>36</volume>(<issue>3</issue>):<fpage>2495</fpage>&#x2013;<lpage>2498</lpage>.
                    <pub-id pub-id-type="doi">10.1109/TPEL.2020.3017935</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref12">
                <label>12</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Nazarov</surname>
                            <given-names>AN</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sychev</surname>
                            <given-names>AK</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Voronkov</surname>
                            <given-names>IM</given-names>
                        </name>
</person-group>:
                    <chapter-title>The Role of Datasets when Building Next Generation Intrusion Detection Systems.</chapter-title>
                    <source>

                        <italic toggle="yes">2019 Wave Electronics and its Application in Information and Telecommunication Systems (WECONF).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2019, June</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>5</lpage>).</mixed-citation>
            </ref>
            <ref id="ref13">
                <label>13</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Hassan</surname>
                            <given-names>WU</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bates</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Marino</surname>
                            <given-names>D</given-names>
                        </name>
</person-group>:
                    <chapter-title>Tactical provenance analysis for endpoint detection and response systems.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 IEEE Symposium on Security and Privacy (SP).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, May</year>; (pp.<fpage>1172</fpage>&#x2013;<lpage>1189</lpage>).</mixed-citation>
            </ref>
            <ref id="ref14">
                <label>14</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Ajmal</surname>
                            <given-names>AB</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Shah</surname>
                            <given-names>MA</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Maple</surname>
                            <given-names>C</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Offensive security: Towards proactive threat hunting via adversary emulation.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Access.</italic>
</source>
                    <year>2021</year>;<volume>9</volume>:<fpage>126023</fpage>&#x2013;<lpage>126033</lpage>.
                    <pub-id pub-id-type="doi">10.1109/ACCESS.2021.3104260</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref15">
                <label>15</label>
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Karuna</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Hemberg</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>O&#x2019;Reilly</surname>
                            <given-names>UM</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Automating Cyber Threat Hunting Using NLP, Automated Query Generation, and Genetic Perturbation.</article-title>
                    <source>

                        <italic toggle="yes">arXiv preprint arXiv:2104.11576.</italic>
</source>
                    <year>2021</year>.</mixed-citation>
            </ref>
            <ref id="ref16">
                <label>16</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Xiong</surname>
                            <given-names>W</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Legrand</surname>
                            <given-names>E</given-names>
                        </name>

                        <name name-style="western">
                            <surname>&#x00c5;berg</surname>
                            <given-names>O</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Cyber security threat modeling based on the MITRE Enterprise ATT&amp;CK Matrix.</article-title>
                    <source>

                        <italic toggle="yes">Softw. Syst. Model.</italic>
</source>
                    <year>2021</year>;<fpage>1</fpage>&#x2013;<lpage>21</lpage>.</mixed-citation>
            </ref>
            <ref id="ref17">
                <label>17</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Golushko</surname>
                            <given-names>AP</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zhukov</surname>
                            <given-names>VG</given-names>
                        </name>
</person-group>:
                    <chapter-title>Application of Advanced Persistent Threat ActorsTechniques aor Evaluating Defensive Countermeasures.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, January</year>; (pp.<fpage>312</fpage>&#x2013;<lpage>317</lpage>).</mixed-citation>
            </ref>
            <ref id="ref18">
                <label>18</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Hong</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kim</surname>
                            <given-names>K</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kim</surname>
                            <given-names>T</given-names>
                        </name>
</person-group>:
                    <article-title>The Design and Implementation of Simulated Threat Generator based on MITRE ATT&amp;CK for Cyber Warfare Training.</article-title>
                    <source>

                        <italic toggle="yes">Journal of the Korea Institute of Military Science and Technology.</italic>
</source>
                    <year>2019</year>;<volume>22</volume>(<issue>6</issue>):<fpage>797</fpage>&#x2013;<lpage>805</lpage>.</mixed-citation>
            </ref>
            <ref id="ref19">
                <label>19</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Enoch</surname>
                            <given-names>SY</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Huang</surname>
                            <given-names>Z</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Moon</surname>
                            <given-names>CY</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>HARMer: Cyber-attacks automation and evaluation.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Access.</italic>
</source>
                    <year>2020</year>;<volume>8</volume>:<fpage>129397</fpage>&#x2013;<lpage>129414</lpage>.
                    <pub-id pub-id-type="doi">10.1109/ACCESS.2020.3009748</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref20">
                <label>20</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Gianvecchio</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Burkhalter</surname>
                            <given-names>C</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Lan</surname>
                            <given-names>H</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Sillers</surname>
                            <given-names>A</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Closing the gap with APTs through semantic clusters and automated cybergames.</chapter-title>
                    <source>

                        <italic toggle="yes">International Conference on Security and Privacy in Communication Systems.</italic>
</source>
                    <publisher-loc>Cham</publisher-loc>:
                    <publisher-name>Springer</publisher-name>;<year>2019, October</year>; (pp.<fpage>235</fpage>&#x2013;<lpage>254</lpage>).</mixed-citation>
            </ref>
            <ref id="ref21">
                <label>21</label>
                <mixed-citation publication-type="other">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Sen</surname>
                            <given-names>&#x00d6;</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Velde</surname>
                            <given-names>D</given-names>
                            <prefix>van der</prefix>
                        </name>

                        <name name-style="western">
                            <surname>Peters</surname>
                            <given-names>SN</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>An Approach of Replicating Multi-Staged Cyber-Attacks and Countermeasures in a Smart Grid Co-Simulation Environment.</article-title>
                    <source>

                        <italic toggle="yes">arXiv preprint arXiv:2110.02040.</italic>
</source>
                    <year>2021</year>.</mixed-citation>
            </ref>
            <ref id="ref22">
                <label>22</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Brangetto</surname>
                            <given-names>P</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Veenendaal</surname>
                            <given-names>MA</given-names>
                        </name>
</person-group>:
                    <chapter-title>Influence cyber operations: The use of cyberattacks in support of influence operations.</chapter-title>
                    <source>

                        <italic toggle="yes">2016 8th International Conference on Cyber Conflict (CyCon).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2016, May</year>; (pp.<fpage>113</fpage>&#x2013;<lpage>126</lpage>).</mixed-citation>
            </ref>
            <ref id="ref23">
                <label>23</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Naveen</surname>
                            <given-names>S</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Puzis</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Angappan</surname>
                            <given-names>K</given-names>
                        </name>
</person-group>:
                    <chapter-title>Deep Learning for Threat Actor Attribution from Threat Reports.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 4th International Conference on Computer, Communication and Signal Processing (ICCCSP).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, September</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>6</lpage>).</mixed-citation>
            </ref>
            <ref id="ref24">
                <label>24</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Perry</surname>
                            <given-names>L</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Shapira</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Puzis</surname>
                            <given-names>R</given-names>
                        </name>
</person-group>:
                    <chapter-title>No-doubt: Attack attribution based on threat intelligence reports.</chapter-title>
                    <source>

                        <italic toggle="yes">2019 IEEE International Conference on Intelligence and Security Informatics (ISI).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2019, July</year>; (pp.<fpage>80</fpage>&#x2013;<lpage>85</lpage>).</mixed-citation>
            </ref>
            <ref id="ref25">
                <label>25</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Geiger</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Bauer</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Masuch</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Franke</surname>
                            <given-names>J</given-names>
                        </name>
</person-group>:
                    <chapter-title>An Analysis of Black Energy 3, Crashoverride, and Trisis, Three Malware Approaches Targeting Operational Technology Systems.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, September</year>; (Vol.<volume>1</volume>, pp.<fpage>1537</fpage>&#x2013;<lpage>1543</lpage>).</mixed-citation>
            </ref>
            <ref id="ref26">
                <label>26</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Siadati</surname>
                            <given-names>H</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Saket</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Memon</surname>
                            <given-names>N</given-names>
                        </name>
</person-group>:
                    <chapter-title>Detecting malicious logins in enterprise networks using visualization.</chapter-title>
                    <source>

                        <italic toggle="yes">2016 IEEE Symposium on Visualization for Cyber Security (VizSec).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2016, October</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>8</lpage>).</mixed-citation>
            </ref>
            <ref id="ref27">
                <label>27</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Siadati</surname>
                            <given-names>H</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Saket</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Memon</surname>
                            <given-names>N</given-names>
                        </name>
</person-group>:
                    <chapter-title>Detecting malicious logins in enterprise networks using visualization.</chapter-title>
                    <source>

                        <italic toggle="yes">2016 IEEE Symposium on Visualization for Cyber Security (VizSec).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2016</year>; pp.<fpage>1</fpage>&#x2013;<lpage>8</lpage>.</mixed-citation>
            </ref>
            <ref id="ref28">
                <label>28</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Noor</surname>
                            <given-names>U</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Anwar</surname>
                            <given-names>Z</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Rashid</surname>
                            <given-names>Z</given-names>
                        </name>
</person-group>:
                    <chapter-title>An Association Rule Mining-Based Framework for Profiling Regularities in Tactics Techniques and Procedures of Cyber Threat Actors.</chapter-title>
                    <source>

                        <italic toggle="yes">2018 International Conference on Smart Computing and Electronic Enterprise (ICSCEE).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2018, July</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>6</lpage>).</mixed-citation>
            </ref>
            <ref id="ref29">
                <label>29</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Toker</surname>
                            <given-names>FS</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Akpinar</surname>
                            <given-names>KO</given-names>
                        </name>

                        <name name-style="western">
                            <surname>&#x00d6;z&#x00e7;elik</surname>
                            <given-names>&#x0130;</given-names>
                        </name>
</person-group>:
                    <chapter-title>MITRE ICS Attack Simulation and Detection on EtherCAT Based Drinking Water System.</chapter-title>
                    <source>

                        <italic toggle="yes">2021 9th International Symposium on Digital Forensics and Security (ISDFS).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2021, June</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>6</lpage>).</mixed-citation>
            </ref>
            <ref id="ref30">
                <label>30</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Kwon</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ashley</surname>
                            <given-names>T</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Castleberry</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Cyber Threat Dictionary Using MITRE ATT&amp;CK Matrix and NIST Cybersecurity Framework Mapping.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 Resilience Week (RWS).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, October</year>; (pp.<fpage>106</fpage>&#x2013;<lpage>112</lpage>).</mixed-citation>
            </ref>
            <ref id="ref31">
                <label>31</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Yin</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wang</surname>
                            <given-names>Q</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Cao</surname>
                            <given-names>M</given-names>
                        </name>
</person-group>:
                    <chapter-title>An Attack Vector Evaluation Method for Smart City Security Protection.</chapter-title>
                    <source>

                        <italic toggle="yes">2019 International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2019, October</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>7</lpage>).</mixed-citation>
            </ref>
            <ref id="ref32">
                <label>32</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Park</surname>
                            <given-names>K</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Ahn</surname>
                            <given-names>B</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Kim</surname>
                            <given-names>J</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>An advanced persistent threat (apt)-style cyberattack testbed for distributed energy resources (der).</chapter-title>
                    <source>

                        <italic toggle="yes">2021 IEEE Design Methodologies Conference (DMC).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2021, July</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>5</lpage>).</mixed-citation>
            </ref>
            <ref id="ref33">
                <label>33</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Wang</surname>
                            <given-names>W</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Zhang</surname>
                            <given-names>X</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Dong</surname>
                            <given-names>L</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Network Attack Detection based on Domain Attack Behavior Analysis.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, October</year>; (pp.<fpage>962</fpage>&#x2013;<lpage>965</lpage>).</mixed-citation>
            </ref>
            <ref id="ref34">
                <label>34</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Fujimoto</surname>
                            <given-names>M</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Matsuda</surname>
                            <given-names>W</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Mitsunaga</surname>
                            <given-names>T</given-names>
                        </name>
</person-group>:
                    <chapter-title>Detecting abuse of domain administrator privilege using windows event log.</chapter-title>
                    <source>

                        <italic toggle="yes">2018 IEEE Conference on Application, Information and Network Security (AINS).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2018, November</year>; (pp.<fpage>15</fpage>&#x2013;<lpage>20</lpage>).</mixed-citation>
            </ref>
            <ref id="ref35">
                <label>35</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Diffenderfer</surname>
                            <given-names>PA</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Baumgartner</surname>
                            <given-names>DM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Long</surname>
                            <given-names>KM</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <chapter-title>Authentication and Authorization Challenges for Controller-Pilot Information Exchange Using Mobile Devices.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, October</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>8</lpage>).</mixed-citation>
            </ref>
            <ref id="ref36">
                <label>36</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Niakanlahiji</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Wei</surname>
                            <given-names>J</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Chu</surname>
                            <given-names>BT</given-names>
                        </name>
</person-group>:
                    <chapter-title>A natural language processing based trend analysis of advanced persistent threat techniques.</chapter-title>
                    <source>

                        <italic toggle="yes">2018 IEEE International Conference on Big Data (Big Data).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2018, December</year>; (pp.<fpage>2995</fpage>&#x2013;<lpage>3000</lpage>).</mixed-citation>
            </ref>
            <ref id="ref37">
                <label>37</label>
                <mixed-citation publication-type="journal">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Nisioti</surname>
                            <given-names>A</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Loukas</surname>
                            <given-names>G</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Laszka</surname>
                            <given-names>A</given-names>
                        </name>

                        <etal/>
</person-group>:
                    <article-title>Data-driven decision support for optimizing cyber forensic investigations.</article-title>
                    <source>

                        <italic toggle="yes">IEEE Trans. Inf. Forensics Secur.</italic>
</source>
                    <year>2021</year>;<volume>16</volume>:<fpage>2397</fpage>&#x2013;<lpage>2412</lpage>.
                    <pub-id pub-id-type="doi">10.1109/TIFS.2021.3054966</pub-id>
                </mixed-citation>
            </ref>
            <ref id="ref38">
                <label>38</label>
                <mixed-citation publication-type="book">
                    <person-group person-group-type="author">

                        <name name-style="western">
                            <surname>Al-Shaer</surname>
                            <given-names>R</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Spring</surname>
                            <given-names>JM</given-names>
                        </name>

                        <name name-style="western">
                            <surname>Christou</surname>
                            <given-names>E</given-names>
                        </name>
</person-group>:
                    <chapter-title>Learning the Associations of MITRE ATT &amp; CK Adversarial Techniques.</chapter-title>
                    <source>

                        <italic toggle="yes">2020 IEEE Conference on Communications and Network Security (CNS).</italic>
</source>
                    <publisher-name>IEEE</publisher-name>;<year>2020, June</year>; (pp.<fpage>1</fpage>&#x2013;<lpage>9</lpage>).</mixed-citation>
            </ref>
        </ref-list>
    </back>
    <sub-article article-type="reviewer-report" id="report134931">
        <front-stub>
            <article-id pub-id-type="doi">10.5256/f1000research.120614.r134931</article-id>
            <title-group>
                <article-title>Reviewer response for version 1</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Alabdulatif</surname>
                        <given-names>Abdullah</given-names>
                    </name>
                    <xref ref-type="aff" rid="r134931a1">1</xref>
                    <role>Referee</role>
                    <uri content-type="orcid">https://orcid.org/0000-0002-4400-5140</uri>
                </contrib>
                <aff id="r134931a1">
                    <label>1</label>Department of Computer, College of Sciences and Arts in Al-Rass, Qassim University, Al-Rass, Saudi Arabia</aff>
            </contrib-group>
            <author-notes>
                <fn fn-type="conflict">
                    <p>
                        <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                </fn>
            </author-notes>
            <pub-date pub-type="epub">
                <day>27</day>
                <month>4</month>
                <year>2022</year>
            </pub-date>
            <permissions>
                <copyright-statement>Copyright: &#x00a9; 2022 Alabdulatif A</copyright-statement>
                <copyright-year>2022</copyright-year>
                <license xlink:href="https://creativecommons.org/licenses/by/4.0/">
                    <license-p>This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
                </license>
            </permissions>
            <related-article ext-link-type="doi" id="relatedArticleReport134931" related-article-type="peer-reviewed-article" xlink:href="10.12688/f1000research.109148.1"/>
            <custom-meta-group>
                <custom-meta>
                    <meta-name>recommendation</meta-name>
                    <meta-value>approve-with-reservations</meta-value>
                </custom-meta>
            </custom-meta-group>
        </front-stub>
        <body>
            <p>The author has done very good work and explained the problem, which is an Advanced Persistent Threat (APT) against Microsoft Windows Security with the last versions of Windows 10 &amp;11. In this part of the paper, the author has studied and analyzed the problem clearly. However, there are some references that are a bit old. They should be updated with some references in this part.</p>
            <p> </p>
            <p> After that, the author explains the methodology for attaching APT and supports the explanation with great steps and pictures that make this part easy to follow and understand for the reader. Before the end, the author presented the results and discussed them, along with some pictures to prove these results and identify flaws in the APT. I recommend the author discuss the results in depth, as well as make the discussion more clear. Finally, the conclusion part should be expanded by adding more about the future work.</p>
            <p>Is the work clearly and accurately presented and does it cite the current literature?</p>
            <p>Partly</p>
            <p>If applicable, is the statistical analysis and its interpretation appropriate?</p>
            <p>Not applicable</p>
            <p>Are all the source data underlying the results available to ensure full reproducibility?</p>
            <p>Yes</p>
            <p>Is the study design appropriate and is the work technically sound?</p>
            <p>Yes</p>
            <p>Are the conclusions drawn adequately supported by the results?</p>
            <p>Yes</p>
            <p>Are sufficient details of methods and analysis provided to allow replication by others?</p>
            <p>Yes</p>
            <p>Reviewer Expertise:</p>
            <p>information security</p>
            <p>I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.</p>
        </body>
        <sub-article article-type="response" id="comment8175-134931">
            <front-stub>
                <contrib-group>
                    <contrib contrib-type="author">
                        <name>
                            <surname>Mohamed</surname>
                            <given-names>Nachaat</given-names>
                        </name>
                        <aff>HLS, Rabdan Academy, Abu Dhabi, Abu Dhabi, United Arab Emirates</aff>
                    </contrib>
                </contrib-group>
                <author-notes>
                    <fn fn-type="conflict">
                        <p>
                            <bold>Competing interests: </bold>No competing interests were disclosed.</p>
                    </fn>
                </author-notes>
                <pub-date pub-type="epub">
                    <day>30</day>
                    <month>4</month>
                    <year>2022</year>
                </pub-date>
            </front-stub>
            <body>
                <p>I would like to thank the reviewer for the valuable comments and I assure that first, I will change the old reference for the year 2010 with a new one. Secondly, I will add the future work to the conclusion section as recommended.</p>
                <p> </p>
                <p> Thank you</p>
            </body>
        </sub-article>
    </sub-article>
</article>
