Development and interval testing of a naturalistic driving methodology to evaluate driving behavior in clinical research

Ganesh M. Babulal; Aaron Addison; Nupur Ghoshal; Sarah H. Stout; Elizabeth K. Vernon; Mark Sellan; Catherine M. Roe

doi:10.12688/f1000research.9150.2

Home Browse Development and interval testing of a naturalistic driving methodology...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Method Article

Revised

Development and interval testing of a naturalistic driving methodology to evaluate driving behavior in clinical research

[version 2; peer review: 2 approved]

Ganesh M. Babulal¹, Aaron Addison², Nupur Ghoshal¹, [...] Sarah H. Stout¹, Elizabeth K. Vernon¹, Mark Sellan², Catherine M. Roe¹

Ganesh M. Babulal¹, Aaron Addison², [...] Nupur Ghoshal¹, Sarah H. Stout¹, Elizabeth K. Vernon¹, Mark Sellan², Catherine M. Roe¹

PUBLISHED 15 Sep 2016

Author details Author details

¹ Charles F. and Joanne Knight Alzheimer’s Disease Research Center, Department of Neurology, Washington University School of Medicine, St. Louis, USA
² University Geographic Information System, Washington University, St. Louis, USA

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

Background: The number of older adults in the United States will double by 2056. Additionally, the number of licensed drivers will increase along with extended driving-life expectancy. Motor vehicle crashes are a leading cause of injury and death in older adults. Alzheimer’s disease (AD) also negatively impacts driving ability and increases crash risk. Conventional methods to evaluate driving ability are limited in predicting decline among older adults. Innovations in GPS hardware and software can monitor driving behavior in the actual environments people drive in. Commercial off-the-shelf (COTS) devices are affordable, easy to install and capture large volumes of data in real-time. However, adapting these methodologies for research can be challenging. This study sought to adapt a COTS device and determine an interval that produced accurate data on the actual route driven for use in future studies involving older adults with and without AD.
Methods: Three subjects drove a single course in different vehicles at different intervals (30, 60 and 120 seconds), at different times of day, morning (9:00-11:59AM), afternoon (2:00-5:00PM) and night (7:00-10pm). The nine datasets were examined to determine the optimal collection interval.
Results: Compared to the 120-second and 60-second intervals, the 30-second interval was optimal in capturing the actual route driven along with the lowest number of incorrect paths and affordability weighing considerations for data storage and curation.
Discussion: Use of COTS devices offers minimal installation efforts, unobtrusive monitoring and discreet data extraction. However, these devices require strict protocols and controlled testing for adoption into research paradigms. After reliability and validity testing, these devices may provide valuable insight into daily driving behaviors and intraindividual change over time for populations of older adults with and without AD. Data can be aggregated over time to look at changes or adverse events and ascertain if decline in performance is occurring.

Keywords

naturalistic driving, interval testing, geographic information system, global positioning systems, in-vehicle technology

Corresponding author: Ganesh M. Babulal

Competing interests: No competing interests were disclosed.

Grant information: Funding for this study was provided by the National Institute on Aging [R01AG043434 (Roe), R01AG43434-03S1(Roe), P50-AG05681(Morris), P01-AG03991(Morris), P01-AG026276(Morris)]; Fred Simmons and Olga Mohan, and the Charles and Joanne Knight Alzheimer’s Research Initiative of the Washington University Knight Alzheimer’s Disease Research Center.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2016 Babulal GM et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Babulal GM, Addison A, Ghoshal N et al. Development and interval testing of a naturalistic driving methodology to evaluate driving behavior in clinical research [version 2; peer review: 2 approved]. F1000Research 2016, 5:1716 (https://doi.org/10.12688/f1000research.9150.2) First published: 15 Jul 2016, 5:1716 (https://doi.org/10.12688/f1000research.9150.1) Latest published: 15 Sep 2016, 5:1716 (https://doi.org/10.12688/f1000research.9150.2)

Revised Amendments from Version 1

We deleted part of a sentence in the introduction to reduce potential confusion on whether this method manuscript may contain biomarker information. Based on reviewer feedback, we have slightly modified the results section to clarify the main findings from this study while responding to specific comments about the methods. We thank the reviewers for their time.

See the authors' detailed response to the review by David K. Johnson

Background

Motor vehicle crashes (MVC) are a leading cause of injury among older adults (586 daily) in the United States¹ and MVC deaths have steadily climbed over the past decade, along with an increase in crash risk with each year². Coupled with the growth of the aging population and the increasing prevalence of dementias like Alzheimer disease (AD), being able to predict when driving performance will decline may prevent crashes and deaths among older adult drivers and others who share the roadway^3–5.

To this end, our research program seeks to better characterize the driving behaviors of older adults and predict the onset of driving difficulties so that we can implement appropriate interventions to maintain safety and prolong driving life⁶. We are particularly interested in the association between preclinical AD and driving.

Road tests and driving simulators are the most common and dominant measures used to assess driving performance and determine road safety⁷. Both methods have proven reliable and valid in evaluating poor driving performance and estimating crash risks for older drivers^8,9. However, driving is an overlearned task and controlled conditions like the road test and simulator may not reflect driving as it occurs on a daily basis or expose errors made by experienced or cognitively-normal drivers outside of these controlled conditions¹⁰. Other limitations of both methods include rater subjectivity, anxiety (poorer performance), Hawthorne effect, dedicated single site measures, simulator sickness and high equipment cost, maintenance, and programming^10–13.

Due to these limitations, we sought to find an objective, cost-effective method that would allow us to assess future research of driving performance longitudinally on a daily basis among hundreds of older adults in the actual environments that they drive, something that has been unavailable until now. This manuscript describes the first step in our work to adapt a commercial global positioning data acquisition system (GPDAS) and develop a methodology to evaluate driving performance. This technology is capable of collecting data at a constant rate over any determined time. However, due to the cost of data storage and greater programming time with larger volumes of data, we sought to determine the “optimal” time interval for accurate data collection using GPDAS devices.

Methods

GPDAS device

The GPDAS device (G2 Tracking Device^TM Azuga, Inc) is compact (length = 1.7”, width = 1.8”, height = 1”, weight = 1.1 ounce), plugs into the on-board diagnostic systems port (OBDII) and uses the vehicle’s own battery to supply the 12 volts required to function. Only vehicles manufactured in 1996 or later are compatible with the device. The device’s wireless capabilities include use of third generation mobile phone network (3G), jamming detection, Bluetooth, internal antenna and Firmware-Over-The-Air update for configuration of device firmware. Its global positioning system (GPS) capability includes a 56-channel receiver with a 4-Hertz acquisition rate, accuracy of 2.5 meters circular error probable (CEP) and integrated anti-jamming capability. Finally, it has a tri-axial (X, Y, Z) accelerometer with 8–13 bits of resolution on each axis. The accelerometer can detect and report changes in acceleration over +/- 16 g-force and the data can be reported at a rate ranging from 1 to 24 Hertz.

Standard protocol approvals, registrations, ethics and consents

The GPDAS device sends data at intervals of 30, 60, or 120 seconds, which shows the exact location, speed, and date/time at each interval. The optimal time interval would accurately represent the route traveled using the minimum number of data points possible in order to minimize cost and extraneous data collection. The data collected did not contain any personal or identifying information about the drivers. Ethical permission to conduct this study was sought and received via expedited review from the Washington University Human Research Protection Office who determined that this is a non-human subjects study (201412024). Informed consent was obtained from all drivers who participated in this study.

Data acquisition

After being plugged into the OBDII port, the GPDAS device extracts the signal from the vehicle speed sensor (VSS), which measures the transaxle speed, also known as the wheel speed. The VSS is the reference speed that the majority of a vehicle’s systems rely upon to achieve their specific functionality. For example, the Engine Control Module uses the VSS signal to modify engine functions and initiate specific diagnostic routines, while the variable assist power steering system uses it to regulate power steering pressure for assistance at slow speeds. The speed displayed on the speedometer is generally greater than the actual VSS signal, ranging anywhere between 1–3 mph more. The VSS signal, which the GPDAS device uses, is the most accurate reflection of the vehicle speed. Installation takes less than one minute, and once plugged in, the device accesses available satellites for orientation and then begins simultaneously transmitting data to secured servers using available cell phone towers. These data can be then accessed online in real-time or stored in a database for retrieval at a later date. If a vehicle is driven in areas where cellular signal is lost, data continue to be collected and is then re-transmitted when a stronger signal is established. When a vehicle is turned off, the device enters sleep mode but sends a signal every four hours to indicate the ignition is off but the device is still functioning. When a vehicle is turned on, the device immediately begins sending data at a specified time interval. A standard set of data is obtained during a trip, which is specified as the time between when the ignition is turned on and off. These variables include time and date stamp, drive time, idling time, miles, latitude and longitude, speeding over posted speed limit, hard braking, sudden acceleration and an alert if the device was unplugged and plugged back in. Since the device is powered by the vehicle’s battery, if the battery starts to drop below the required 12V, the device sends out a series of alerts indicating insufficient power and will stop transmitting if the power drops below 10V. Additionally, the device will detect problem codes that the vehicle’s computer may send out (e.g. check engine light indicating oxygen sensor requires replacement).

Outcome

A structured driving course of approximately seven miles (Figure 1) was designed to represent various real-world driving conditions with a comprehensive mix of stoplights, stop signs, right and left hand turns and merging into traffic. The route began at an office complex in an urban setting and continued several blocks east following a divided boulevard. Drivers then turned south to merge onto a freeway. The freeway section of the route provided driving conditions and associated data logging for highway speeds. Drivers then exited into a large park where the designated route was designed to simulate more rural driving conditions. The park section also allowed for more nuanced driving such as roundabouts where data interval logging could be analyzed for correctness to the real-world, and to simulate driving events such as a U-turn or missed turn. Finally, the drivers exited the park and returned to the office complex starting point driving on surface streets with traffic to simulate additional urban conditions.

Figure 1. Structured driving route.

A map of the route was provided to each member of the driving research team, as well as the turn-by-turn driving directions. Drivers did not navigate the route prior to data collection. All GPDAS data were logged into daily csv files with results uploaded to a secured server by Azuga. Automation scripts were used to validate files and copies stored on a secured server. A secured file transfer protocol was designed and automated to transfer the log files from Azuga’s server to our servers on a daily basis.

Protocol

Three healthy subjects drove a single course in three different vehicles. The drivers negotiated the course at three different time intervals (30 seconds, 60 seconds, 120 seconds), and at three different times of day, morning (9:00-11:59AM), afternoon (2:00-5:00PM) and night (7:00-10pm). In order to minimize bias associated with the order of driving combinations and day, the time intervals and time of day were randomized for all drivers. Depending on the time of day, data were collected over several days, including weekdays and weekends. Each driver yielded nine sets of data (i.e., all possible combinations of time interval and time of day). The device remained installed in the vehicle without removal until each driver completed the set of routes.

Statistical analyses

Data were logged into files (csv) stored in a secure Amazon S3 folder. Data were downloaded in bulk and sorted into folders based on the respective driver IDs. Secondary sorting was done for time interval and filtered for spurious data points. All data were imported to ArcGIS Desktop 10.2 software (Environmental Systems Research Institute, Redlands, CA, USA) and plotted on a map using the latitude and longitude coordinates logged by the GPDAS device (Figure 2).

Figure 2. GPS points collected.

Each dataset was queried for a specific time interval, such as 30 seconds. The resulting dataset was used as an input for the Network Analyst extension of ArcGIS. A base road network (edge network) was also loaded into the tool for the routing algorithm. Routing algorithms use an impedance to determine “cost” of travel on the network, but are often defined in terms of time needed to traverse a given section of the network or distance needed to travel the network segment. The network impedance was defined by time of travel on the base network. The routing algorithm processed all coordinate data from each driving circuit, creating a line representing the path traveled during data collection. These data were then visualized in ArcGIS as lines (Figure 3).

Figure 3. 1 minute AM routes.

Each of the time intervals was evaluated for best fit to the base road network. Best fit was determined by comparing the route generated by the routing algorithms to the actual true route of the course. ArcGIS was used to conduct spatial comparisons between the routes driven and the real-world road course or “correct” route to determine “best” fit. The results of this analysis were used to determine the preferred data collection interval for the device.

Results

Dataset 1.Determining the optimal interval collection for a Global Positioning Data Acquisition System.

Vehicle Name: Participant ID via Vehicle Make and Model; Vehicle Speed: Vehicle speed (miles per hour); Latitude: The angular distance of a place north or south of the earth's equator (degrees); Longitude: The angular distance of a place east or west of the earth's equator (degrees); Event Type: Type of event coded by the GPDAS device; Timestamp: Date (mm/dd/yyy) and Time (hh:mm:am/pm); Odometer Reading: Vehicle odometer reporting the number of miles.

The 30-second collection interval was determined by ArcGIS Desktop 10.2 software to have the strongest “goodness of fit” and lowest number of incorrect paths traveled compared to the one and two minute data collection intervals (See Figure 2). The mapped routes were displayed and symbolized by time interval for the main visualization product. In addition to identification of the preferred time interval for data collection, the visualization process also revealed data artifacts and incorrect routing of the base network. The source of these errors was explained primarily by the interval of data collection. For instance, if the data collection interval is too large (e.g. 120 seconds), the driver may travel through several turns before the next valid data point is logged. This absence of specific data points (e.g. found in 120 seconds interval) to guide the routing algorithm may lead to incorrect assumptions, resulting in incorrect path of travel produced.

As Figure 4 illustrates on the two-minute interval, an incorrect line was generated (zigzag line) due to lack of data about the actual route (red line). Other events, such as hard braking, can also add intermediate data points to aid in the process, but these points are unpredictable and cannot be relied upon for routing protocols since any stimulus from the external environment can impact driving behavior and trigger an event.

Figure 4. Incorrect routes.

Discussion

This study investigated the optimal time interval for data collection using a GPDAS device to accurately capture a driven route while weighing the considerations of cost associated with data storage and post-processing efforts. The 30-second interval was determined to be the most accurate based on goodness of fit and was affordable for our research program. Technological innovations have led to faster processors and ability to gather greater volumes of data. Yet, the challenges required to analyze big data include large statistical and computational costs, incidental homogeneity, noise, and an inherent requirement to develop newer, robust statistical models to deal with larger sample sizes^14,15. Further, given a stricter funding climate, researchers working with clinical populations cannot afford the time and cost to collect, process and analyze continuous data using existing naturalistic research paradigms. Some studies use in-vehicle data recorders that require hours of installation and extensive modification of participants’ vehicle⁸. Others use in-vehicle cameras which may modify driving behavior and also require hours of post-processing and extensive rater training^13,16. Larger studies that collect hundreds of hours of data may require participants to regularly return to the study site and have the data from their vehicle downloaded¹⁷. Studies using smart phone applications require participants to charge phones, turn the phone on and off and to remember to bring it in the vehicle when driving, thereby elevating participant burden¹⁸.

As a whole, driving research and crash prevention research is shifting toward the use of naturalistic methodologies for evaluating driving performance^19–22. Development and interval testing of a naturalistic driving methodology to evaluate driving behavior is required to measure real world driving conditions and responses²³ in a variety of clinical populations. The methodology presented here implements a non-obtrusive device installed in the OBDII port of a vehicle. The device stores locational data in the form of latitude and longitude at each time sampled, as well as driving behaviors that may occur at any time, such as hard braking, speeding, and vehicle on/off events. Since all data are tied to a spatial location, it is possible to understand the “place” of where data have been collected.

It is important to note that driving events such as “what happened at this exact moment or day?” should not be singled out. The inherent value of longitudinal data collection is to collect data to better understand changes over time for an older adult driver that may be otherwise hidden from observation. The true potential of this methodology is that data gathered could be linked to other databases to answer a number of questions. One can link weather and meteorological databases to understand the impact of the weather on driving patterns and Department of Transportation databases on road construction to explore how roadwork influences driving navigation. Driving behavior for clinical populations could also be evaluated in a pre-post design for patients who have had medication changes, surgeries, stroke, undergoing chemotherapy or radiation, a diagnosis of seizures or a range of neurological conditions that ultimately impact driving for a brief or longer period of time. Naturalistic driving research has the potential to study and aid the management of driving behavior of older adults with chronic neurological disease like dementia. The long-term goal of our program is to model driving behavior and driving risk of older adults using this naturalistic driving data to identify driving decline over time and develop educational interventions to improve driving performance, decrease vehicle crash risk while driving and structure driving retirement for older adults with a higher risk for MVCs.

Data availability

F1000Research: Dataset 1. Determining the optimal interval collection for a Global Positioning Data Acquisition System, 10.5256/f1000research.9150.d128877²⁴

Consent

Informed consent was obtained from all drivers who participated in this study from Washington University Human Research Protection Office.

Author contributions

GMB, AA and CMR designed the study, collected and analyzed the data, and drafted the manuscript. NG, SSH, EKV and MS contributed to the conception and design of the study and revised it critically for intellectual content. All authors have read and approved the final version of the manuscript for publication and agree to be accountable for all aspects of the work.

Competing interests

No competing interests were disclosed.

Grant information

Funding for this study was provided by the National Institute on Aging [R01AG043434(Roe), R01AG43434-03S1(Roe), P50-AG05681(Morris), P01-AG03991(Morris), P01-AG026276(Morris)]; Fred Simmons and Olga Mohan, and the Charles and Joanne Knight Alzheimer’s Research Initiative of the Washington University Knight Alzheimer’s Disease Research Center.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Faculty Opinions recommended

References

1. National Highway Traffic Safety Administration EDRWG: Event Data Recorders-Summary of Findings by the NHTSA EDR Working Group. 2001. Reference Source
2. National Highway Traffic Safety Administration: Traffic Safety Facts 2012 data: older population. 2014. Reference Source
3. Adler G, Rottunda SJ: Mandatory testing of drivers on the basis of age and degenerative diseases: stakeholder opinions. J Aging Soc Policy. 2010; 22(3): 304–19. PubMed Abstract | Publisher Full Text
4. Alzheimer’s Association: 2015 Alzheimer's disease facts and figures. Alzheimers Dement. 2015; 11(3): 332–84. PubMed Abstract | Publisher Full Text
5. Prevention CfDC: Web-based Injury Statistics Query and Reporting System (WISQARS)(2010). National Center for Injury Prevention and Control, CDC. 2011. Reference Source
6. Roe CM, Barco PP, Head DM, et al.: Amyloid Imaging, Cerebrospinal Fluid Biomarkers Predict Driving Performance Among Cognitively Normal Individuals. Alzheimer Dis Assoc Disord. 2016. PubMed Abstract | Publisher Full Text
7. Eby DW, Silverstein NM, Molnar LJ, et al.: Driving behaviors in early stage dementia: a study using in-vehicle technology. Accid Anal Prev. 2012; 49: 330–7. PubMed Abstract | Publisher Full Text
8. Hunt LA, Murphy CF, Carr D, et al.: Reliability of the Washington University Road Test. A performance-based assessment for drivers with dementia of the Alzheimer type. Arch Neurol. 1997; 54(6): 707–12. PubMed Abstract | Publisher Full Text
9. Odenheimer GL, Beaudet M, Jette AM, et al.: Performance-based driving evaluation of the elderly driver: safety, reliability, and validity. J Gerontol. 1994; 49(4): M153–M159. PubMed Abstract | Publisher Full Text
10. Rizzo M, Reinach S, McGehee D, et al.: Simulated car crashes and crash predictors in drivers with Alzheimer disease. Arch Neurol. 1997; 54(5): 545–551. PubMed Abstract | Publisher Full Text
11. Davis JD, Papandonatos GD, Miller LA, et al.: Road test and naturalistic driving performance in healthy and cognitively impaired older adults: does environment matter? J Am Geriatr Soc. 2012; 60(11): 2056–2062. PubMed Abstract | Publisher Full Text | Free Full Text
12. De Winter JC, Van Leuween P, Happee R: Advantages and Disadvantages of Driving Simulators: A Discussion. Citeseer: 2012; 47–50. Reference Source
13. Ott BR, Papandonatos GD, Davis JD, et al.: Naturalistic validation of an on-road driving test of older drivers. Hum Factors. 2012; 54(4): 663–674. PubMed Abstract | Publisher Full Text | Free Full Text
14. Jacobs A: The pathologies of big data. Commun ACM. 2009; 52(8): 36–44. Publisher Full Text
15. Fan J, Han F, Liu H: Challenges of Big Data Analysis. Natl Sci Rev. 2014; 1(2): 293–314. PubMed Abstract | Publisher Full Text | Free Full Text
16. Stavrinos D, Ross L, Sisiopiku V: A Naturalistic Driving Study Across the Lifespan. 2014. Reference Source
17. Dingus TA, Hankey JM, Antin JF, et al.: Naturalistic Driving Study: Technical Coordination and Quality Control. 2015. Reference Source
18. Johnson DA, Trivedi MM: Driving style recognition using a smartphone as a sensor platform. IEEE: 2011; 1609–1615. Reference Source
19. Dingus TA, Klauer S, Neale V, et al.: The 100-car naturalistic driving study, Phase II-results of the 100-car field experiment. 2006. Reference Source
20. Neale VL, Dingus TA, Klauer SG, et al.: An overview of the 100-car naturalistic study and findings. National Highway Traffic Safety Administration, Paper. 2005. Reference Source
21. Shichrur R, Sarid A, Ratzon NZ: Determining the sampling time frame for in-vehicle data recorder measurement in assessing drivers. Transp Res Part C Emerg Technol. 2014; 42: 99–106. Publisher Full Text
22. Toledo T, Musicant O, Lotan T: In-vehicle data recorders for monitoring and feedback on drivers’ behavior. Transp Res Part C Emerg Technol. 2008; 16(3): 320–331. Publisher Full Text
23. Molnar LJ, Charlton JL, Eby DW, et al.: Factors affecting self-regulatory driving practices among older adults. Traffic Inj Prev. 2014; 15(3): 262–272. PubMed Abstract | Publisher Full Text
24. Babulal G, Addison A, Ghoshal N, et al.: Dataset 1 in: Development and interval testing of a naturalistic driving methodology to evaluate driving behavior. F1000Research. 2016. Data Source

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 15 Jul 2016

Author details Author details

Competing interests

No competing interests were disclosed.

Grant information

Funding for this study was provided by the National Institute on Aging [R01AG043434 (Roe), R01AG43434-03S1(Roe), P50-AG05681(Morris), P01-AG03991(Morris), P01-AG026276(Morris)]; Fred Simmons and Olga Mohan, and the Charles and Joanne Knight Alzheimer’s Research Initiative of the Washington University Knight Alzheimer’s Disease Research Center.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 15 Sep 2016, 5:1716

https://doi.org/10.12688/f1000research.9150.2

version 1

Published: 15 Jul 2016, 5:1716

https://doi.org/10.12688/f1000research.9150.1

© 2016 Babulal GM et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Babulal GM, Addison A, Ghoshal N et al. Development and interval testing of a naturalistic driving methodology to evaluate driving behavior in clinical research [version 2; peer review: 2 approved]. F1000Research 2016, 5:1716 (https://doi.org/10.12688/f1000research.9150.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 2

VERSION 2

PUBLISHED 15 Sep 2016

Revised

Views

Reviewer Report 11 Oct 2016

David K. Johnson, Department of Psychology, Center for Computational Biology, University of Kansas, Lawrence, KS, USA

Approved

https://doi.org/10.5256/f1000research.10338.r16901

The paper looks much better. ... Continue reading

CITE

Report a concern

Respond or Comment

Version 1

VERSION 1

PUBLISHED 15 Jul 2016

Views

Reviewer Report 30 Aug 2016

Monique M. Williams, VITAS Healthcare, St. Louis, MO, USA

Approved

https://doi.org/10.5256/f1000research.9846.r15049

The authors describe the use of a commercial-off-the-shelf (COTS) device to ascertain which interval provided the most accurate data regarding the route driven. The objective is to determine the optimal interval that would be applied to future research studies involving individuals with and without Alzheimer’s disease.

A total of three participants drove a single 7-mile driving course. The course was designed to reflect real world driving conditions. Participants drove in different cars, at different times of day (morning, afternoon, and night), and different intervals (120-second, 60-second, and 30-second). The three participants were healthy and cognitively normal.

The datasets were analyzed. The preferred data collection interval was determined based on the results of best fit. Best fit was operationalized as the route generated by the routing algorithm as compared with the actual route driven. The analyses revealed that the 30-second interval provided the optimal (best fit) characterization of the route.

The ability to use a COTS device to assess driving skills has considerable public health implications in the context of an aging nation. The study is a critical initial step in determining the utility of the COTS device for prospective research studies. With the inherent limitations of driving simulators and road testing, the use of the device as described provides a cost-effective option that more closely emulates real world driving experiences.

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Respond or Comment

Views

Reviewer Report 01 Aug 2016

David K. Johnson, Department of Psychology, Center for Computational Biology, University of Kansas, Lawrence, KS, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.9846.r15332

Three subjects drove a single course in different vehicles at different intervals (30, 60 and 120 seconds), at different times of day, morning (9:00-11:59AM), afternoon (2:00-5:00PM) and night (7:00-10pm). The nine datasets were examined to determine the optimal collection interval. The 30-second interval was reported as optimal. Time of day was apparently uninformative.

I think this paper could make a significant contribution to a burgeoning and important literature. How to measure driving in older adults is a complex issue. If there were valid processes and instrumentation for the meaningful and accurate measurement of safety-relevant driver outcomes in older adults, this contribution would be huge. However, I had a hard time following the results and conclusions drawn by the investigators.

I found that most of this manuscript read like a proposal and it editing was seemingly incomplete. As it currently reads, there are significant revisions that need to be made to improve its intelligibility. For instance I had a very hard time finding any real or intelligible outcomes in the results section. The only results I could find were in the abstract (What does it mean to "capture the actual route driven along with the lowest number of incorrect paths") and then again buried in the discussion section properly. Is there really no intelligible data in the results section? Further, it is opaque to me why 30 seconds is preferred - what data supports the 30-second interval and the investigators’ reasoning behind this the acceptance of this interva? If it's GIS type data, why isn't reported and interpreted? There is an oblique reference to a GOF but where is the table and logic behind the decision to adopt one model over another? More detail and logic is needed to make the authors' writing, logic and conclusions more clear. At the end of the manuscript I felt that the abstract's conclusion that 30 seconds was an optimal time for data collection is unsupported substantively.

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 15 Sep 2016

Ganesh Babulal, Charles F. and Joanne Knight Alzheimer’s Disease Research Center, Department of Neurology, Washington University School of Medicine, St. Louis, USA

15 Sep 2016

Author Response

Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the ... Continue reading Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the available choices using GIS data by adopting a commercially off the shelf GPS device. We designed a study to examine if the 30 seconds collection interval was more accurate than the 60 and 120 seconds. We found that 30 seconds was indeed more accurate across the three drivers, days and times. Based on your feedback, we relocated the main finding at the beginning of the results section instead of the end. The background needed to understand the results is detailed in the last two paragraphs of the methods section. This is GIS data and it is explained in that section that we used a combination of ArcGIS software, the Network Analyst extension tool and routing algorithms to compare the driven routes to the actual routes. Our coauthor who is an expert in GIS, AA, indicates that unfortunately there are no p-values or confidence intervals to report with this type of data. Based on the GIS data captured from the COTS device, the 30 seconds was more accurate with fewer errors compared to the 60 and 120 seconds interval. The dataset is available for review.

Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the available choices using GIS data by adopting a commercially off the shelf GPS device. We designed a study to examine if the 30 seconds collection interval was more accurate than the 60 and 120 seconds. We found that 30 seconds was indeed more accurate across the three drivers, days and times. Based on your feedback, we relocated the main finding at the beginning of the results section instead of the end. The background needed to understand the results is detailed in the last two paragraphs of the methods section. This is GIS data and it is explained in that section that we used a combination of ArcGIS software, the Network Analyst extension tool and routing algorithms to compare the driven routes to the actual routes. Our coauthor who is an expert in GIS, AA, indicates that unfortunately there are no p-values or confidence intervals to report with this type of data. Based on the GIS data captured from the COTS device, the 30 seconds was more accurate with fewer errors compared to the 60 and 120 seconds interval. The dataset is available for review.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 15 Sep 2016

Ganesh Babulal, Charles F. and Joanne Knight Alzheimer’s Disease Research Center, Department of Neurology, Washington University School of Medicine, St. Louis, USA

15 Sep 2016

Author Response

Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the ... Continue reading Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the available choices using GIS data by adopting a commercially off the shelf GPS device. We designed a study to examine if the 30 seconds collection interval was more accurate than the 60 and 120 seconds. We found that 30 seconds was indeed more accurate across the three drivers, days and times. Based on your feedback, we relocated the main finding at the beginning of the results section instead of the end. The background needed to understand the results is detailed in the last two paragraphs of the methods section. This is GIS data and it is explained in that section that we used a combination of ArcGIS software, the Network Analyst extension tool and routing algorithms to compare the driven routes to the actual routes. Our coauthor who is an expert in GIS, AA, indicates that unfortunately there are no p-values or confidence intervals to report with this type of data. Based on the GIS data captured from the COTS device, the 30 seconds was more accurate with fewer errors compared to the 60 and 120 seconds interval. The dataset is available for review.

Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the available choices using GIS data by adopting a commercially off the shelf GPS device. We designed a study to examine if the 30 seconds collection interval was more accurate than the 60 and 120 seconds. We found that 30 seconds was indeed more accurate across the three drivers, days and times. Based on your feedback, we relocated the main finding at the beginning of the results section instead of the end. The background needed to understand the results is detailed in the last two paragraphs of the methods section. This is GIS data and it is explained in that section that we used a combination of ArcGIS software, the Network Analyst extension tool and routing algorithms to compare the driven routes to the actual routes. Our coauthor who is an expert in GIS, AA, indicates that unfortunately there are no p-values or confidence intervals to report with this type of data. Based on the GIS data captured from the COTS device, the 30 seconds was more accurate with fewer errors compared to the 60 and 120 seconds interval. The dataset is available for review.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 15 Jul 2016

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 15 Sep 16	read
Version 1 15 Jul 16	read	read

David K. Johnson, University of Kansas, Lawrence, USA
Monique M. Williams, VITAS Healthcare, St. Louis, USA

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

9 Views

11 Oct 2016 | for Version 2

David K. Johnson, Department of Psychology, Center for Computational Biology, University of Kansas, Lawrence, KS, USA

9 Views Cite this report Responses(0)

Approved

The paper looks much better. My concerns have been satisfied.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

15 Views

30 Aug 2016 | for Version 1

Monique M. Williams, VITAS Healthcare, St. Louis, MO, USA

15 Views Cite this report Responses(0)

Approved

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

25 Views

01 Aug 2016 | for Version 1

David K. Johnson, Department of Psychology, Center for Computational Biology, University of Kansas, Lawrence, KS, USA

25 Views Cite this report Responses(1)

Approved With Reservations

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

15 Sep 2016

Ganesh Babulal, Charles F. and Joanne Knight Alzheimer’s Disease Research Center, Department of Neurology, Washington University School of Medicine, St. Louis, USA

Dr. Johnson,

Thank you for your review and comments. They were helpful in our revision. This brief method article sought to determine which interval was more accurate/optimal given the available choices using GIS data by adopting a commercially off the shelf GPS device. We designed a study to examine if the 30 seconds collection interval was more accurate than the 60 and 120 seconds. We found that 30 seconds was indeed more accurate across the three drivers, days and times. Based on your feedback, we relocated the main finding at the beginning of the results section instead of the end. The background needed to understand the results is detailed in the last two paragraphs of the methods section. This is GIS data and it is explained in that section that we used a combination of ArcGIS software, the Network Analyst extension tool and routing algorithms to compare the driven routes to the actual routes. Our coauthor who is an expert in GIS, AA, indicates that unfortunately there are no p-values or confidence intervals to report with this type of data. Based on the GIS data captured from the COTS device, the 30 seconds was more accurate with fewer errors compared to the 60 and 120 seconds interval. The dataset is available for review.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Click here to access the data.

Downloaded data do not display as expected? Download the data

[1] 1. National Highway Traffic Safety Administration EDRWG: Event Data Recorders-Summary of Findings by the NHTSA EDR Working Group. 2001. Reference Source

[2] 2. National Highway Traffic Safety Administration: Traffic Safety Facts 2012 data: older population. 2014. Reference Source

[3] 3. Adler G, Rottunda SJ: Mandatory testing of drivers on the basis of age and degenerative diseases: stakeholder opinions. J Aging Soc Policy. 2010; 22(3): 304–19. PubMed Abstract | Publisher Full Text

[4] 4. Alzheimer’s Association: 2015 Alzheimer's disease facts and figures. Alzheimers Dement. 2015; 11(3): 332–84. PubMed Abstract | Publisher Full Text

[5] 5. Prevention CfDC: Web-based Injury Statistics Query and Reporting System (WISQARS)(2010). National Center for Injury Prevention and Control, CDC. 2011. Reference Source

[6] 6. Roe CM, Barco PP, Head DM, et al.: Amyloid Imaging, Cerebrospinal Fluid Biomarkers Predict Driving Performance Among Cognitively Normal Individuals. Alzheimer Dis Assoc Disord. 2016. PubMed Abstract | Publisher Full Text

[7] 7. Eby DW, Silverstein NM, Molnar LJ, et al.: Driving behaviors in early stage dementia: a study using in-vehicle technology. Accid Anal Prev. 2012; 49: 330–7. PubMed Abstract | Publisher Full Text

[8] 8. Hunt LA, Murphy CF, Carr D, et al.: Reliability of the Washington University Road Test. A performance-based assessment for drivers with dementia of the Alzheimer type. Arch Neurol. 1997; 54(6): 707–12. PubMed Abstract | Publisher Full Text

[9] 9. Odenheimer GL, Beaudet M, Jette AM, et al.: Performance-based driving evaluation of the elderly driver: safety, reliability, and validity. J Gerontol. 1994; 49(4): M153–M159. PubMed Abstract | Publisher Full Text

[10] 10. Rizzo M, Reinach S, McGehee D, et al.: Simulated car crashes and crash predictors in drivers with Alzheimer disease. Arch Neurol. 1997; 54(5): 545–551. PubMed Abstract | Publisher Full Text

[11] 11. Davis JD, Papandonatos GD, Miller LA, et al.: Road test and naturalistic driving performance in healthy and cognitively impaired older adults: does environment matter? J Am Geriatr Soc. 2012; 60(11): 2056–2062. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. De Winter JC, Van Leuween P, Happee R: Advantages and Disadvantages of Driving Simulators: A Discussion. Citeseer: 2012; 47–50. Reference Source

[13] 13. Ott BR, Papandonatos GD, Davis JD, et al.: Naturalistic validation of an on-road driving test of older drivers. Hum Factors. 2012; 54(4): 663–674. PubMed Abstract | Publisher Full Text | Free Full Text

[14] 14. Jacobs A: The pathologies of big data. Commun ACM. 2009; 52(8): 36–44. Publisher Full Text

[15] 15. Fan J, Han F, Liu H: Challenges of Big Data Analysis. Natl Sci Rev. 2014; 1(2): 293–314. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Stavrinos D, Ross L, Sisiopiku V: A Naturalistic Driving Study Across the Lifespan. 2014. Reference Source

[17] 17. Dingus TA, Hankey JM, Antin JF, et al.: Naturalistic Driving Study: Technical Coordination and Quality Control. 2015. Reference Source

[18] 18. Johnson DA, Trivedi MM: Driving style recognition using a smartphone as a sensor platform. IEEE: 2011; 1609–1615. Reference Source

[19] 19. Dingus TA, Klauer S, Neale V, et al.: The 100-car naturalistic driving study, Phase II-results of the 100-car field experiment. 2006. Reference Source

[20] 20. Neale VL, Dingus TA, Klauer SG, et al.: An overview of the 100-car naturalistic study and findings. National Highway Traffic Safety Administration, Paper. 2005. Reference Source

[21] 21. Shichrur R, Sarid A, Ratzon NZ: Determining the sampling time frame for in-vehicle data recorder measurement in assessing drivers. Transp Res Part C Emerg Technol. 2014; 42: 99–106. Publisher Full Text

[22] 22. Toledo T, Musicant O, Lotan T: In-vehicle data recorders for monitoring and feedback on drivers’ behavior. Transp Res Part C Emerg Technol. 2008; 16(3): 320–331. Publisher Full Text

[23] 23. Molnar LJ, Charlton JL, Eby DW, et al.: Factors affecting self-regulatory driving practices among older adults. Traffic Inj Prev. 2014; 15(3): 262–272. PubMed Abstract | Publisher Full Text

[24] 24. Babulal G, Addison A, Ghoshal N, et al.: Dataset 1 in: Development and interval testing of a naturalistic driving methodology to evaluate driving behavior. F1000Research. 2016. Data Source

Development and interval testing of a naturalistic driving methodology to evaluate driving behavior in clinical research

Abstract

Keywords

Revised Amendments from Version 1

Background

Methods

GPDAS device

Standard protocol approvals, registrations, ethics and consents

Data acquisition

Outcome

Figure 1. Structured driving route.

Protocol

Statistical analyses

Figure 2. GPS points collected.

Figure 3. 1 minute AM routes.

Results

Figure 4. Incorrect routes.

Discussion

Data availability

Consent

Author contributions

Competing interests

Grant information

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

The problem

How to fix it

Competing Interests Policy

Stay Updated