TicTimer Web: software for measuring tic suppression remotely

Jonathan K. Black; Jonathan M. Koller; Kevin J. Black

doi:10.12688/f1000research.26347.1

Home Browse TicTimer Web: software for measuring tic suppression remotely

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Software Tool Article

TicTimer Web: software for measuring tic suppression remotely

[version 1; peer review: 1 approved, 1 approved with reservations]

Jonathan K. Black ¹, Jonathan M. Koller^2,3, Kevin J. Black ^2-5

PUBLISHED 22 Oct 2020

Author details Author details

¹ Department of Mechanical Engineering, Brigham Young University, Provo, Utah, 84602, USA
² Department of Psychiatry, Washington University in St. Louis, St. Louis, Missouri, 63110, USA
³ Department of Radiology, Washington University in St. Louis, St. Louis, Missouri, 63110, USA
⁴ Department of Neurology, Washington University in St. Louis, St. Louis, Missouri, 63110, USA
⁵ Department of Neuroscience, Washington University in St. Louis, St. Louis, Missouri, 63110, USA

Jonathan K. Black
Roles: Methodology, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan M. Koller
Roles: Methodology, Software, Writing – Review & Editing

Kevin J. Black
Roles: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the Tics collection.

Abstract

Woods and Himle developed a standardized tic suppression paradigm (TSP) for the experimental setting, to quantify the effects of intentional tic suppression in Tourette syndrome. We previously provided a computer program to facilitate recording tic occurrence and to automate reward delivery during the several experimental conditions of the TSP. The present article describes a web-based program that performs the same functions. Implementing this program on the web allows research sessions to be performed remotely, in tandem with a video calling program. Relevant data for each session, such as the timing of tics and dispensed rewards, are stored in plain text files for later analysis. Expected applications include research on Tourette syndrome and related disorders.

Keywords

tc disorders, Tourette syndrome, reward, reinforcement (psychology), software

Corresponding authors: Jonathan K. Black, Kevin J. Black

Competing interests: No competing interests were disclosed.

Grant information: Software development and manuscript preparation were funded in part by the U.S. National Institutes of Health (NIH), grant number R01MH104030.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 Black JK et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Black JK, Koller JM and Black KJ. TicTimer Web: software for measuring tic suppression remotely [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2020, 9:1264 (https://doi.org/10.12688/f1000research.26347.1) First published: 22 Oct 2020, 9:1264 (https://doi.org/10.12688/f1000research.26347.1) Latest published: 03 Mar 2021, 9:1264 (https://doi.org/10.12688/f1000research.26347.2)

Introduction

In a previous article¹ we mentioned the tic suppression paradigm (TSP) developed by Woods and Himle for use in the experimental setting to demonstrate and quantify the effects of intentional tic suppression on tic frequency in Tourette syndrome (TS) and other tic disorders^2–6. In this paradigm, each participant is observed during several experimental conditions: usually baseline and differential reinforcement of zero-rate ticcing (DRO), and sometimes also verbal instruction to suppress tics and/or noncontingent reinforcement (NCR).

In the course of conducting a longitudinal study of children with Provisional Tic Disorder⁷, we found that tic suppression is seen within the first few months after a child’s first tic⁸. We also found that the TSP required substantial investigator effort, so we wrote a simple program to facilitate record keeping and reward delivery during research sessions¹. It required an expert observer to press a button to record each tic observed, and in the DRO and NCR conditions it delivered reward tokens at the appropriate times by connecting to a relay module and a token dispenser box. This software improved convenience for the investigator and accuracy of record-keeping.

Because of enforced social distancing in 2020, the need arose for sessions to be performed remotely. A video calling program allowed us to observe the subjects, but we still needed a way to deliver rewards during the DRO and NCR conditions of the TSP. Previously, we had created a web-based program called TicTrainer for behavioral therapy⁹, and we decided to expand the functionality of that program so it could be used for TSP research. We present the software here¹⁰ to facilitate its use by others.

Methods

Implementation

TicTimer Web¹⁰ uses the node.js server that was made for TicTrainer. It adopts that program’s structure, with user accounts for research subjects and an admin account for the rater. Details on account registration, data storage, and logging on were described previously⁹.

A new field was added to user accounts so that a research ID (different from the ID used to log on) can be set for research subjects.

Sessions for TicTimer Web use separate but simultaneous connections to a server from a “user” and from a rater. TSP DRO sessions deliver rewards after every 10-s tic-free interval. To deliver rewards at the appropriate times, the user page checks in with the server periodically to see if it has been 10 seconds since the last tic was signaled by the rater. If it has not, the server responds with the time remaining until the next reward is due, and the user page uses that number, adjusted for the lag time of the round trip, to schedule when to check back with the server. Using this method, reward timing is synchronized so that awards are delivered usually within 50 ms of the target time. The previous, in-person version of TicTimer used a token dispenser box to automatically deliver rewards at the appropriate times. In an attempt to approximate the user experience of the physical token dispenser, TicTimer Web delivers rewards by displaying coin images on the subject’s screen along with a chime sound.

At the end of each session, a summary is generated and appended at the bottom of the session log file, which is then archived with the date and time of the session in the filename.

Operation

Setup. First, node.js is installed on the server. We used an Amazon EC2 instance, but the program can operate on any computer with node.js (e.g. a laptop).

The researcher and subject need only a modern web browser to interface with TicTimer Web once the server is running. The browser must support JavaScript and HTML5. We have tested TicTimer Web with current versions of Chrome and Edge.

To perform sessions with TicTimer Web, the researcher creates a user account if needed, then if desired uses the admin interface to assign the user a research ID number to identify the subject for later data analysis.

Use. The researcher and subject sign in on their respective TicTimer session pages and the rater begins the session by selecting one of the four experimental conditions.

During a session, the rater watches the subject. We have used a separate video calling program for this observation, but a video camera or one-way mirror could be used for in-person visits. The rater records any observed tics by immediately pressing the “Tic Detected” button, the spacebar, or the letter “T”. If the session type includes rewards (DRO and NCR), they are dispensed appropriately. The session ends after the predetermined duration, or when the rater presses “End Session,” or when either rater or subject closes their browser window early.

For the NCR condition, the rater first chooses a log file previously created with the current subject, and rewards are delivered to the user at the same times (relative to the session start) that they were delivered in the specified session.

Archived session log files can be downloaded from the admin interface, or they can be copied over directly from the server itself.

Use cases

The video file (Extended data, Supplementary File 1)¹¹ demonstrates the operation of TicTimer Web¹⁰ from a researcher’s perspective. The sessions performed here were test sessions with no human subjects being observed. Extended data, Supplementary Files 2–4¹¹ are the session log files created in that video.

Conclusions

The TicTimer Web¹⁰ program allows for remote implementation of the TSP, while maintaining the benefits of earlier versions of the software¹: ease and accuracy of record keeping and automated reward delivery. TicTimer Web, while designed for our purposes in tic disorder research, may find other uses. The most obvious of these may be for research on traditional habit disorders; for instance, hair pulling and skin picking appear in the “Obsessive-compulsive and related disorders” section of DSM-5¹². The most obvious application to the clinical setting may be in documenting suppression ability before and after treatment. Another potential future modification would be to add machine detection of tics, e.g. by online video analysis, surface EMG or accelerometry; such an improvement would be quite welcome but is difficult to reduce to practice.

Data availability

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Zenodo: TicTimer Web: software for measuring tic suppression remotely: Supplementary Files. http://doi.org/10.5281/zenodo.4023134¹¹.

This project contains the following extended data:

ttw_demo.mp4. (Supplementary File 1: Video Demonstration of Operation. A video demonstrating how to operate TicTimer Web from the researcher’s perspective.)
au5_20200622-153043_baseline.ttsd. (Supplementary File 2: Sample Log, baseline. Log file for the baseline session performed during the video demonstration.)
au5_20200622-153125_DRZ.ttsd. (Supplementary File 3: Sample Log, DRO. Log file for the DRO session performed during the video demonstration.)
au5_20200622-153214_NCR.ttsd. (Supplementary File 4: Sample Log, NCR. Log file for the NCR session performed during the video demonstration.)

License: MIT License.

Software availability

The source code for TicTimer Web is available at: https://github.com/jonkb/TicTrainer-node.

Archived source code at time of publication: https://doi.org/10.5281/zenodo.3990474¹⁰.

License: MIT License.

Faculty Opinions recommended

References

1. Black JK, Koller JM, Black KJ: TicTimer software for measuring tic suppression [version 2; peer review: 2 approved]. F1000Res. 2017; 6: 1560. PubMed Abstract | Publisher Full Text | Free Full Text
2. Woods DW, Himle MB: Creating tic suppression: comparing the effects of verbal instruction to differential reinforcement. J Appl Behav Anal. 2004; 37(3): 417–20. PubMed Abstract | Publisher Full Text | Free Full Text
3. Himle MB, Woods DW: An experimental evaluation of tic suppression and the tic rebound effect. Behav Res Ther. 2005; 43(11): 1443–51. PubMed Abstract | Publisher Full Text
4. Himle MB, Woods DW, Bunaciu L: Evaluating the role of contingency in differentially reinforced tic suppression. J Appl Behav Anal. 2008; 41(2): 285–9. PubMed Abstract | Publisher Full Text | Free Full Text
5. Lyon GJ, Samar SM, Conelea C, et al.: Testing tic suppression: comparing the effects of dexmethylphenidate to no medication in children and adolescents with attention-deficit/hyperactivity disorder and Tourette’s disorder. J Child Adolesc Psychopharmacol. 2010; 20(4): 283–9. PubMed Abstract | Publisher Full Text | Free Full Text
6. Conelea CA, Wellen B, Woods DW, et al.: Patterns and Predictors of Tic Suppressibility in Youth With Tic Disorders. Front Psychiatry. 2018; 9: 188. PubMed Abstract | Publisher Full Text | Free Full Text
7. Black KJ, Kim S, Schlaggar BL, et al.: The New Tics study: A novel approach to pathophysiology and cause of tic disorders. J Psychiatr Brain Sci. 2020; 5: e200012. PubMed Abstract | Publisher Full Text | Free Full Text
8. Greene DJ, Koller JM, Robichaux-Viehoever A, et al.: Reward enhances tic suppression in children within months of tic disorder onset. Dev Cogn Neurosci. 2015; 11: 65–74. PubMed Abstract | Publisher Full Text | Free Full Text
9. Black JK, Black KJ: Software for web-based tic suppression training [version 2; peer review: 3 approved]. F1000Res. 2018; 6: 2150. PubMed Abstract | Publisher Full Text | Free Full Text
10. Black JK, Black KJ: TicTrainer and TicTimer Web v4.0.2 (Version 4.0.2).Zenodo. 2020. http://www.doi.org/10.5281/zenodo.3990474
11. Black JK, Koller JM, Black KJ: TicTimer Web: software for measuring tic suppression remotely: Supplementary Files. F1000research. Zenodo. 2020. http://www.doi.org/10.5281/zenodo.4023134
12. American Psychiatric Association: Obsessive-Compulsive and Related Disorders. In: Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition. Arlington, VA: American Psychiatric Association; 2013. Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 22 Oct 2020

Author details Author details

Jonathan K. Black
Roles: Methodology, Software, Writing – Original Draft Preparation, Writing – Review & Editing

Jonathan M. Koller
Roles: Methodology, Software, Writing – Review & Editing

Kevin J. Black
Roles: Conceptualization, Funding Acquisition, Project Administration, Supervision, Writing – Original Draft Preparation, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

Software development and manuscript preparation were funded in part by the U.S. National Institutes of Health (NIH), grant number R01MH104030.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 03 Mar 2021, 9:1264

https://doi.org/10.12688/f1000research.26347.2

version 1

Published: 22 Oct 2020, 9:1264

https://doi.org/10.12688/f1000research.26347.1

© 2020 Black JK et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Black JK, Koller JM and Black KJ. TicTimer Web: software for measuring tic suppression remotely [version 1; peer review: 1 approved, 1 approved with reservations]. F1000Research 2020, 9:1264 (https://doi.org/10.12688/f1000research.26347.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 22 Oct 2020

Views

Reviewer Report 17 Feb 2021

Valerie Brandt, Department of Psychology, Centre for Innovation in Mental Health, University of Southampton, Southampton, UK

Approved

https://doi.org/10.5256/f1000research.29084.r77056

The report describes the development of TicTimer Web, a nifty software that can be used to record tics, tic timings, and distribute reinforcement tokens during online experiments are therapeutic sessions. The authors describe that they adapted the TicTrainer programme to an online version that allows patients to login to a user account and therapists / tic raters to login as administrators. User and rater are then connected to the server simultaneously. The tic rater can record tics via button press and the software stores the timing of that tic. After every tic-free 10 second interval, the software delivers a reward. The software also delivers a summary at the end of the session.
In order for the rater to watch tics, a separate programme, or a webcam can be used. This allows an accurate reward delivery, even remotely.

The source code of TicTimer is available on GitHub, the link is provided in the manuscript. Furthermore, an explanatory video shows how TicTimer can be used. The video is simple and clear and provides step-by-step explanations. Examples of demo files are also provided on Zenodo.

The manuscript is clearly written, the software is demonstrated in a video, examples and the source code are provided.

I have no suggestions for improvement.

Is the rationale for developing the new software tool clearly explained?

Yes
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Tic disorders.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 23 Feb 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

23 Feb 2021

Author Response F1000Research Advisory Board Member

We thank Prof. Brandt for her kind comments.
Competing Interests: No competing interests were disclosed.
We thank Prof. Brandt for her kind comments.
We thank Prof. Brandt for her kind comments.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 23 Feb 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

23 Feb 2021

Author Response F1000Research Advisory Board Member

We thank Prof. Brandt for her kind comments.
Competing Interests: No competing interests were disclosed.
We thank Prof. Brandt for her kind comments.
We thank Prof. Brandt for her kind comments.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 20 Jan 2021

Jennifer Schild, Suffolk University, Boston, MA, USA

Matthew W. Specht, Weill Cornell Medicine, New York City, NY, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.29084.r73543

This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research. The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.
A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.
In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”
In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.
A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.
Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

Is the rationale for developing the new software tool clearly explained?

Partly
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Partly
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: I have 15 years of research regarding Tourette's disorder and have utilized the tic detector paradigm in my own research.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however we have significant reservations, as outlined above.

CITE

Report a concern

Author Response (F1000Research Advisory Board Member) 23 Feb 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

23 Feb 2021

Author Response F1000Research Advisory Board Member

We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
Competing Interests: No competing interests were disclosed.
We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Author Response (F1000Research Advisory Board Member) 03 Mar 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

03 Mar 2021

Author Response F1000Research Advisory Board Member
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable ... Continue reading
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research.

> Thank you for these positive comments.

The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.

> Done (first 2 paragraphs).

A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.

> Done (last paragraph of Intro).

In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”

> Thank you. Corrected.

In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.

> Done (in first paragraph of Intro).

A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.

> I elaborate on these in the first paragraph of the revised Conclusions section.

Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

> Done (last paragraph of Conclusions).

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

> Thank you.
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research.

> Thank you for these positive comments.

The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.

> Done (first 2 paragraphs).

A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.

> Done (last paragraph of Intro).

In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”

> Thank you. Corrected.

In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.

> Done (in first paragraph of Intro).

A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.

> I elaborate on these in the first paragraph of the revised Conclusions section.

Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

> Done (last paragraph of Conclusions).

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

> Thank you.
Competing Interests: None Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response (F1000Research Advisory Board Member) 23 Feb 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

23 Feb 2021

Author Response F1000Research Advisory Board Member

We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
Competing Interests: No competing interests were disclosed.
We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
We thank Ms. Schild and Dr. Specht for their thoughtful review, and will edit the report as they suggest.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Author Response (F1000Research Advisory Board Member) 03 Mar 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

03 Mar 2021

Author Response F1000Research Advisory Board Member
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable ... Continue reading
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research.

> Thank you for these positive comments.

The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.

> Done (first 2 paragraphs).

A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.

> Done (last paragraph of Intro).

In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”

> Thank you. Corrected.

In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.

> Done (in first paragraph of Intro).

A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.

> I elaborate on these in the first paragraph of the revised Conclusions section.

Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

> Done (last paragraph of Conclusions).

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

> Thank you.
Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research.

> Thank you for these positive comments.

The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.

> Done (first 2 paragraphs).

A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.

> Done (last paragraph of Intro).

In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”

> Thank you. Corrected.

In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.

> Done (in first paragraph of Intro).

A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.

> I elaborate on these in the first paragraph of the revised Conclusions section.

Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

> Done (last paragraph of Conclusions).

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

> Thank you.
Competing Interests: None Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 22 Oct 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 2 (revision) 03 Mar 21	read
Version 1 22 Oct 20	read	read

Jennifer Schild, Suffolk University, Boston, USA

Matthew W. Specht, Weill Cornell Medicine, New York City, USA
Valerie Brandt, University of Southampton, Southampton, UK

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

6 Views

23 Mar 2021 | for Version 2

Matthew W. Specht, Weill Cornell Medicine, New York City, NY, USA

Jennifer Schild, Suffolk University, Boston, MA, USA

6 Views Cite this report Responses(0)

Approved

The authors appear to have responded to our prior suggested edits and comments, we have no further comments at this time.

Competing Interests

Dr. Specht has previously received grant funding for research from the Tourette Association of America (TAA). He has also utilized the Tic Detector paradigm in prior research.

Reviewer Expertise

I have 15 years of research regarding Tourette's disorder and have utilized the tic detector paradigm in my own research.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

11 Views

17 Feb 2021 | for Version 1

Valerie Brandt, Department of Psychology, Centre for Innovation in Mental Health, University of Southampton, Southampton, UK

11 Views Cite this report Responses(1)

Approved

Is the rationale for developing the new software tool clearly explained?

Yes
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Yes
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Yes

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Tic disorders.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Back to all reports

Reviewer Report

22 Views

20 Jan 2021 | for Version 1

Jennifer Schild, Suffolk University, Boston, MA, USA

Matthew W. Specht, Weill Cornell Medicine, New York City, NY, USA

22 Views Cite this report Responses(2)

Approved With Reservations

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.
A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.
In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”
In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.
A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.
Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

Is the rationale for developing the new software tool clearly explained?

Partly
Is the description of the software tool technically sound?

Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?

Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?

Partly
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?

Partly

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

I have 15 years of research regarding Tourette's disorder and have utilized the tic detector paradigm in my own research.

Respond to this report

Responses (2)

Author Response F1000Research Advisory Board Member

03 Mar 2021

Kevin J Black, Department of Psychiatry, Washington University in St. Louis, St. Louis, 63110, USA

Author responses are below, marked by the > sign and bold face, intercalated between sections of the original review.

APPROVED WITH RESERVATIONS
This paper describes a timely and valuable tool for assessing tic suppression remotely in clinical and research settings. This web-based program is based on Woods and Himle’s standardized tic suppression paradigm (TSP) and performs the same function as a computer program previously developed by the authors to record tics and partially automate delivery of rewards. This paper is particularly useful during this time in which remote data collection for research is relevant and necessary, and it provides a helpful overview of a program that can be easily understood by other tic researchers to apply in their own research.

> Thank you for these positive comments.

The paper could benefit from elaboration in multiple areas to help readers better understand the broader relevance of TSP as well as the research utility of TicTimer Web. Elaboration in the introduction section, as well as a more detailed description of potential applications of this software and how this software helps to overcome previous limitations of the TSP are needed. We recommend that this article be revised and resubmitted. Below are some suggestions for the authors to consider.

Please provide a clearer, more detailed explanation of the broad importance of TSP and related concepts in the introduction section. This will be helpful for readers who are less familiar with tic suppression to understand what insights this paradigm have yielded and how this software program could be beneficial to future investigation. For example, the TSP is described as requiring “substantial investigator effort,” but it is not clear what exactly is entailed. Those who used the old tic-detector boxes know how laborious running a participant was, but those less familiar will not appreciate the leap forward TicTimer Web provides.

> Done (first 2 paragraphs).

A short description of the TicTrainer web-based program is also needed in the introduction section. This will be helpful for orienting the reader to the new program that is described in this article, especially considering how similar this older program is to TicTimer Web.

> Done (last paragraph of Intro).

In the methods section, there is interchangeable usage of the term “reward” and “award.” The word “awards” in paragraph 3 of the implementation subsection of the methods should be replaced with “rewards.”

> Thank you. Corrected.

In the methods section, elaboration is needed on what each experimental condition is (e.g., differential reinforcement of zero-rate ticcing (DRO), noncontingent reinforcement (NCR). Alternatively, this can be elaborated upon in the introduction section as it is also mentioned there.

> Done (in first paragraph of Intro).

A separate section before the conclusion discussing the strengths and future applications of this software would be useful. This is somewhat addressed in the conclusion section, but further elaboration in a devoted section would do a better job of highlighting the potential of this software program.

> I elaborate on these in the first paragraph of the revised Conclusions section.

Next steps: It would be helpful to provide another sentence or two regarding the potential benefits of using artificial intelligence (AI) and machine learning (ML) to do this job as well as barriers to taking this approach.

> Done (last paragraph of Conclusions).

Overall, this paper provides a concise and useful overview of a web-based program for remote implementation of the TSP. With some clarification in the areas highlighted above, this paper offers a useful contribution to tic research and is especially applicable given the current relevance of remote methods for data collection. We strongly encourage the authors to revise and resubmit this manuscript for consideration.

> Thank you.

View more View less

Competing Interests

None

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Black JK, Koller JM, Black KJ: TicTimer software for measuring tic suppression [version 2; peer review: 2 approved]. F1000Res. 2017; 6: 1560. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. Woods DW, Himle MB: Creating tic suppression: comparing the effects of verbal instruction to differential reinforcement. J Appl Behav Anal. 2004; 37(3): 417–20. PubMed Abstract | Publisher Full Text | Free Full Text

[3] 3. Himle MB, Woods DW: An experimental evaluation of tic suppression and the tic rebound effect. Behav Res Ther. 2005; 43(11): 1443–51. PubMed Abstract | Publisher Full Text

[4] 4. Himle MB, Woods DW, Bunaciu L: Evaluating the role of contingency in differentially reinforced tic suppression. J Appl Behav Anal. 2008; 41(2): 285–9. PubMed Abstract | Publisher Full Text | Free Full Text

[5] 5. Lyon GJ, Samar SM, Conelea C, et al.: Testing tic suppression: comparing the effects of dexmethylphenidate to no medication in children and adolescents with attention-deficit/hyperactivity disorder and Tourette’s disorder. J Child Adolesc Psychopharmacol. 2010; 20(4): 283–9. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Conelea CA, Wellen B, Woods DW, et al.: Patterns and Predictors of Tic Suppressibility in Youth With Tic Disorders. Front Psychiatry. 2018; 9: 188. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Black KJ, Kim S, Schlaggar BL, et al.: The New Tics study: A novel approach to pathophysiology and cause of tic disorders. J Psychiatr Brain Sci. 2020; 5: e200012. PubMed Abstract | Publisher Full Text | Free Full Text

[8] 8. Greene DJ, Koller JM, Robichaux-Viehoever A, et al.: Reward enhances tic suppression in children within months of tic disorder onset. Dev Cogn Neurosci. 2015; 11: 65–74. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Black JK, Black KJ: Software for web-based tic suppression training [version 2; peer review: 3 approved]. F1000Res. 2018; 6: 2150. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Black JK, Black KJ: TicTrainer and TicTimer Web v4.0.2 (Version 4.0.2).Zenodo. 2020. http://www.doi.org/10.5281/zenodo.3990474

[11] 11. Black JK, Koller JM, Black KJ: TicTimer Web: software for measuring tic suppression remotely: Supplementary Files. F1000research. Zenodo. 2020. http://www.doi.org/10.5281/zenodo.4023134

[12] 12. American Psychiatric Association: Obsessive-Compulsive and Related Disorders. In: Diagnostic and Statistical Manual of Mental Disorders. Fifth Edition. Arlington, VA: American Psychiatric Association; 2013. Publisher Full Text

TicTimer Web: software for measuring tic suppression remotely

Abstract

Keywords

Introduction

Methods

Implementation

Operation

Use cases

Conclusions

Data availability

Underlying data

Extended data

Software availability

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated