Ocular disconjugacy cannot be measured without establishing a solid spatial reference

This correspondence points out a need for clarification concerning the methodology utilized in the study “Eye tracking detects disconjugate eye movements associated with structural traumatic brain injury and concussion”, recently published in Journal of Neurotrauma. The authors of the paper state that binocular eye movements were recorded using a single-camera video-oculography technique and that binocular disconjugate characteristics were analyzed without calibration of eye orientation. It is claimed that a variance-based disconjugacy metric was found to be sensitive to the severity of a concussive brain injury and to the status of recovery after the original injury. However, the reproducibility of the paper’s findings may be challenged simply by the paucity of details in the methodological description. More importantly, from the information supplied or cited in the paper, it is difficult to evaluate the validity of the potentially interesting conclusions of the paper.

Correspondence I wish to point out a need for clarification concerning the methodology utilized in the study "Eye tracking detects disconjugate eye movements associated with structural traumatic brain injury and concussion" by Samadani et al., 2015 1 . The authors state that binocular eye movements were recorded using a single-camera infraredbased video-oculography technique (EyeLink 1000, SR Research, Ontario, Canada) and that binocular disconjugate characteristics were analyzed without calibration of eye orientation. The authors claim that their variance-based disconjugacy metric was sensitive to the severity of a concussive brain injury and to the status of recovery after the original injury. The EyeLink 1000 system is an excellent eye tracker with a single high resolution camera and an infrared light source affixed to a side of the camera. This system uses a dark pupil-corneal reflection principle for tracking eye movements. A typical recording setup consists of a computer monitor with which the visual stimuli are presented to the subject and the camera unit (placed in front of the monitor base) with which the eye movement is recorded monocularly or binocularly. The system has an option of easily outputting image-based, uncalibrated eye coordinates with hundreds of units representing 1° of eye rotation.
The concern I would raise with the Samadani et al. paper is the unclearness of the relationship between their metric and binocular disconjugacy. Logically, for an identical amount of eye rotation, any asymmetry in the spatial relationship that the camera or the infrared light source has with the two fellow eyes would result in different extents of relocation of the images of the pupils or corneal reflections. Asymmetries exist because there is a physical separation between the two eyes as well as between the camera and the infrared light source. For instance, the camera may be centered in front of the subject to obtain a symmetric view of the pupils for a gaze along the midline, but the infrared light source cannot be centered simultaneously, the consequence of which is an asymmetric view of the corneal reflections.
The individually variable physical separation of the two eyes contributes to variations in the extent of asymmetry in the camera view of the landmark features of the eyes. It may be asserted that the group-wise comparisons utilized by Samadani et al. are robust to effects of subject-wise asymmetry and the equally constant lack of calibration. However, a systematic between-group bias may be created when the demographic composition of the subject groups are different. For example, having a larger male-to-female ratio in one group could increase the extent of binocular asymmetry in uncalibrated data since men tend to have a larger interocular distance 2,3 . There are still other variables that confound the relationship between eye rotation and changes in pixel coordinates. Although the biometric characteristics of eyes are highly symmetrical within individuals, they are not perfectly symmetrical 4 and a 1-2% nonconformity in corneal curvature or axial length is not uncommon. Each of the two fellow eyes has its own function that maps pixel movement to the eye rotation, and this mapping is not linear. Thus, the arithmetic difference between the uncalibrated coordinates of the two eyes is quite removed from a physical representation of gaze misalignment.
Beyond the factors associated with the raw data, the analytic methods in the paper also do not seem to be constructed with a clear intent. It is puzzling why the disconjugacy metric is represented by the variance of the left-right differences after independently averaging for each eye the uncalibrated coordinates over several cycles for a given stimulus position, as opposed to the straightforward variance of the left-right differences at all sample points. Furthermore, the ranges of outcome values presented in the series of figures run from 0 to at most 0.25, but how the value 0 could have been obtained is not clear. The question arises because in the two eyes' uncalibrated coordinates there must be a constant bias related to the interocular distance. Lastly, what the high end of the outcome range represents is not clear. Since one unit in EyeLink's uncalibrated data output is smaller than 0.01° of eye rotation, being able to report differences in 0.25 square units or less seems implausible. If the raw data were numerically centered or scaled, the procedure should have been noted in the text.
Since disconjugacy is spatial in nature, a solid reference is needed. The authors discuss some valid points regarding potential pitfalls associated with calibration and phoria. However, these points can be directly addressed by implementing a calibration procedure under monocular viewing 5,6 . A comparison between the results from thus calibrated and uncalibrated data, and a demonstration of test-retest reliability could have improved the paper.
In summary, the reproducibility of the paper's findings may be challenged simply by the paucity of details in the methodological description. More importantly, however, from the information supplied or cited in the paper it is difficult to evaluate the validity of the potentially interesting conclusion that deficits in conjugacy of eye movements may quantitate physiologic impact of brain injury.

Competing interests
The author holds stock option in SyncThink, Inc.

Grant information
The author(s) declared that no grants were involved in supporting this work.

REVISED Amendments from Version 1
The main goals of this revision were: 1) to emphasize that disconjugacy is spatial in nature, and thus a solid spatial reference is needed; and 2) to add details on the forms of asymmetries that may exist because of the geometries among the camera, the light source, and the eyes. That said, it is uncertain to what extent the methodological aspects influence the differences between the experimental groups found by Samadani The correspondence would therefore benefit from a et al. quantitative investigation addressing the practical significance of (at least some of) the issues raised.
I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed. The title of the piece is not logically correct, and needs further modification. Although the author's clarification explains how the asymmetry of the light source could, in principle lead to discrepancies in the measurement of ocular disconjugacy, it is based on a long chain of reasoning that is far from establishing the claim of the title, that "Ocular disconjugacy cannot be measured without establishing a solid spatial reference." What the Correspondence establishes is that ocular disconjugacy measures could, in principle, be biased by asymmetries in the geometry of eye tracking camera illumination. To my mind, however, he has not yet established that the ocular disconjugacy measures were in fact biased in this way. He admits that, if the male-female ratios were equated, there would be no basis for a bias between the TBI and the control groups, even given the asymmetry of the illumination. Thus, even if the present result were significantly biased by the small differences in the male-female ratios of the present study, it would still not be the case that "ocular disconjugacy cannot be measured without establishing a solid spatial reference," since the any bias introduced by interpupillary differences could be compensated by matching the interpupilllary distances of the controls to those of the test subjects. matching the interpupilllary distances of the controls to those of the test subjects.
The added phrase should be "affixed to one side of the camera". The sentence "Asymmetries exist because there is a physical separation between the two eyes as well as between the camera and the infrared light source." is logically incorrect. Asymmetries exist because the light source is asymmetrically placed with respect to the symmetry axis of the two eyes. If the displaced infrared source were either above or below the camera lens, i.e. if it were on the symmetry axis, there would be no asymmetry in the view of the eyes (assuming that the eyes themselves were symmetric). Of course, the first half of the sentence is definitely not true on its own: "asymmetries exist because there is a physical separation between the two eyes".
The concept of a bias introduced by consistent male-to-female ratio differences developed in the response goes some way to addressing the issue of bias introduced by the asymmetry, but it has to be established for the groups in the study in order to be a relevant criticism. The male-to-female ratio differences between groups were small, and were not statistically significant for these small groups. Can the author show that these small ratio differences were quantitatively sufficient to account for the Samadani results (in terms of the reported range of male-female interpupillary distances in the et al. subject populations that they used)?

I have read this submission. I believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
No competing interests were disclosed.

Competing Interests:
Author Response 29 Apr 2015 , Brain Trauma Foundation, USA

Jun Maruta
It seems that this Correspondence is somehow viewed as an attempt to explain the findings of Samadani . This responsibility rests with Samadani . In my view, that ocular disconjugacy et al et al cannot be measured without a solid spatial reference is so by virtue of what defines ocular disconjugacy, rather than needing to be established. In any individual measurement, non-pathologically-related asymmetries exist between the uncalibrated coordinates of the two eyes; thus, their arithmetic difference does not represent gaze misalignment. The physical meanings of the metrics that Samadani . claim to be associated with TBI are not clear, and the et al reported values associated with these metrics do not follow the explanation provided in the text. It is possible that some output of a magical black box should indeed be found to be associated with TBI. However, my argument would still be, "What is the science behind it?" Regarding the sentence "Asymmetries exist because there is a physical separation between the two eyes as well as between the camera and the infrared light source," it is made clear that, in the context, the relative displacement of the infrared light source from the camera is in the lateral direction. The first half of the sentence is not meant to stand alone. The author does not specify what kinds of asymmetries exist due to the camera/source geometry. Usually the subject is centered in the Eyelink viewing aperture, so the view of the eyes would be symmetric. The attempt to argue that a 1-2% asymmetry between the eyes would significantly degrade the assessment capability does not seem plausible.
The criticism of methodological unclarity may be justified in itself, but it has no bearing on the outcome of the paper since all patients would be equally subject to the same degree by the effects of asymmetry and lack of calibration. As stated, none of the criticisms suggest a systematic bias between the different patient categories. The significant differences among categories cannot therefore be attributed to any of the factors raised by the author, and controlling these factors should only improve the significance of the Samadani results. The author's challenge to the findings of the paper is thus easily refuted. et al.
Moreover, despite the fact that this Correspondence is entirely concerned with calibration, it neglects to mention the fact that a core motivation for the Samadani paper is the issue of a misleading spatial et al. calibration metric. They are making the point that restrictions of eye movement may restrict the array of calibration positions, providing a false assessment that the subsequent test movements are normal. Their goal is thus to use the differences between the two eyes (which they term "temporal uncalibrated calibration") as a probe for deficits that would be masked by the spatial calibration procedure. It is this strategy that the author should have addressed directly.
On a stylistic note, I do not favor using the English possessive "s" on the Latin phrase "et al(ia)" and recommend the form above, as in "the Samadani . paper". et al I have read this submission. I believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.
No competing interests were disclosed.

Competing Interests:
Author Response 10 Apr 2015 , Brain Trauma Foundation, USA Jun Maruta I thank Dr. Tyler for his comments and the opportunity to think through what I had written again. One revision I can make in the second paragraph of the original text is to add "a side of" to "an infrared light source affixed to the camera" so that the phrase reads "an infrared light source affixed to a side of the camera." This revision would make it clearer that the camera may be centered in front of the subject but the infrared light source cannot be centered simultaneously, the consequence of which is an asymmetric view of the corneal reflections. I can also specify that EyeLink 1000 uses a dark pupil-corneal reflection principle for tracking eye movements.
Another clarification I can offer is to point out that the physical separation of the two eyes is individually variable [1,2]. This variability contributes to variations in the extent of asymmetry in the