F1000Research F1000Research 2046-1402 F1000 Research Limited London, UK 10.12688/f1000research.108770.1 Research Article Articles A re-analysis of data from Sullins, Rosik, and Santero (2021): are sexual orientation change efforts (SOCE) helpful for those who don't change?

[version 1; peer review: 2 not approved]

 Schumm Walter R. Conceptualization Data Curation Formal Analysis Investigation Methodology Project Administration Software Writing – Original Draft Preparation Writing – Review & Editing https://orcid.org/0000-0003-3097-3551 a 1 2 Department of Applied Human Sciences, Kansas State University, Manhattan, Kansas, 66506-1403, USA Department of Math, Highland Community College, Wamego, Kansas, 66547, USA schumm@ksu.edu

Competing interests: The author has been a substitute pastor since 2010 at Ashland Community Church, Manhattan, Kansas USA, though he has no formal theological training. The regular pastor at that church is Dr. Paul E. Barkey, D. Miss. (School of World Missions, Fuller Theological Seminary, Pasadena, California, dissertation: The Russian Orthodox Church’s Mission to the Aleuts, 1988, UMI No. 8827520). Dr. Barkey is also a retired chaplain (rank of colonel), U.S. Army. Dr. Schumm, is a retired colonel, U. S. Army. Both Dr. Barkey and Dr. Schumm are graduates of the U. S. Army War College. Dr. Schumm received his Ph.D. in family studies from Purdue University and taught for 42 years at Kansas State University prior to his retirement in May 2021.

26 5 2022 2022 11 580 6 5 2022 Copyright: © 2022 Schumm WR 2022 This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Background. Sullins, Rosik, and Santero (2021) evaluated a convenience sample of 125 men who had undergone sexual orientation change efforts (SOCE) and they made their data available for independent analysis.

Methods: Data from Sullins et al. (2021) were reanalyzed in ten new ways. Several new typologies and variables were created. Paired sample t-tests, correlations, regression analyses, repeated measures analyses with time x group interactions, and chi-square tests were used to evaluate ten novel hypotheses.

Results: Using parametric statistics, we found similar results to Sullins et al. (2021) where they had used nonparametric statistics. Regression analyses found that lower initial SSI and higher SSB predicted more change. Using a new typology for marital status, it was found that those who became engaged/got married during SOCE had the largest gains in OSB compared to other categories of marital status. Using the typology of change, it was found that some men who became stronger for SSA, SSI, and SSB nevertheless reported high levels of helpfulness for SOCE. Harms from SOCE seem to be minimal compared to the positives reported for young adults. SOCE effectiveness did not appear to change with time since therapy, lending less support to a recall bias argument. Congruence between SSA and SSI, may, in some cases, reduce the apparent effectiveness of SOCE

Conclusion: While the sample’s results cannot be generalized outside of highly religious men with initially higher levels of SSA and SSB than SSI, these results differ from many contemporary assertions that SOCE cannot ever be effective and is inherently harmful, or that SSA is inherently immutable. Therefore, calls to ban SOCE legally appear to be founded upon incomplete or inaccurate data and thus premature, while more higher quality research is yet needed regarding SOCE.

 sexual orientation change SOCE marriage ex-gay harms immutability gay lesbian bisexual The author(s) declared that no grants were involved in supporting this work. Introduction

The general purpose of this report is to explore the data of Sullins, Rosik, and Santero (2021) in more depth. Their data set is one of the larger ones available for independent analysis and it also has a plethora of measures. Given the controversy surrounding SOCE (American Psychological Association, 2021; Freedman, 2020; Haldeman, 2022; Przeworski et al., 2021; Rosik & Popper, 2014; Sprigg, 2021), independent analysis of such data sets may be especially important. Some criticisms of SOCE are quite strident. Salway et al. (2020) described SOCE as “pseudoscientific practices intended to suppress or deny unwanted feelings of sexual attraction to members of the same gender/sex”, that the “failure rate of SOCE has been estimated at ≥97%” and that SOCE was “associated with numerous negative health outcomes including self-hatred, depression, and suicidal ideation and suicide attempts” (p. 503). Goodyear et al. (2021) argue that “the impacts of [SOCE] are predominately negative and severe, to the point of being life threatening.” (p. 4) such that they approve of international attempts to ban SOCE. Likewise, Kinitz et al. (2021) report that SOCE involves “a set of scientifically discredited practices” associated with “significant adverse health and social outcomes” (p. 1). Bradshaw et al. (2015) assume that sexual attraction is immutable, and that sexual orientation has a biological origin (p. 409), although many would disagree on both counts. On the other hand, some research has found few significant or substantial differences between SSA individuals who reject an LGB identity and those who accept an LGB identity ( Rosik, Lefevor, & Beckstead, 2021). Karten and Wade (2010) found a significant and substantial (eta-squared of.57) reported decrease in homosexual feelings and behavior in their study of 117 men. Jones and Yarhouse (2011) found effect sizes ranging between 1.67 and 4.25 ( Table 1, p. 414, my calculations) for reductions in same-sex sexual attraction, fantasy, and infatuation, while Pela and Sutton (2021, p. 74, my calculations) found effect size decreases over two years for SOCE subjects of 0.28 for SSA and 0.52 for SSI, suggesting that same-sex attractions (and identity) might not be “immutable”. Whitehead and Whitehead (1999) conclude from their extensive review of the literature that “it is clear that sexual orientation is fluid, not fixed, so that it is impossible to argue it is genetically pre-determined. There is a good possibility that various degrees of change will happen with the right support, including therapy of various kinds. The problem in the present social climate may be finding such support” (p. 228).

Because some have cited Sullins et al. (2021) as one of the “two strongest studies methodologically” ( Sprigg, 2021, p. 30) of the 79 he reviewed and because it was not cited as evidence by the American Psychological Association (2021; Haldeman, 2022), it may be especially important to take a second look into this particular data set. It is not uncommon for scholars to claim that SOCE is based on flawed assumptions, religious bias, might do harm, and is ineffective ( del Rio-Gonzalez et al., 2021; Haldeman, 2022). It appears that even most conservative scholars recognize that some subjects have been harmed by SOCE or at least some unprofessional versions of so-called SOCE, but few therapeutic approaches appear to be exempt from being harmful to a few subjects ( Rosik & Popper, 2014; Sprigg, 2021). The situation is further complicated by severe backlash often taken against conservative scholars ( Rosik & Popper, 2014, p. 231). Nevertheless, some research seems to have found SOCE to be effective for some subjects with low levels of harm ( Jones & Yarhouse, 2011). Regardless of their politics, most scholars and professional therapists are opposed to forms of SOCE that are involuntary, punitive, or that amount to torture ( Drescher, 2022). Glassgold (2022, p. 20) has made a clear distinction between older, more aversive forms of SOCE and more modern, recent “verbal” approaches. Even Haldeman (2022) recognizes that his APA book did not address what he has called “more recent iterations of SOCE” or “conversion therapy lite” (p. 8). Accordingly, careful analysis of the most recent iterations of SOCE should be viewed as especially important.

There were several more specific objectives for this reanalysis of their data. In terms of measurement, I wanted to create new variables that might better explain the results and allow the use of scales rather than merely single items as variables. In terms of analyses, I wished to focus on possible effects of SOCE in terms of effect sizes, expressed as Cohen’s d. With N > 100, it might be possible to attain statistical significance in the absence of large effect sizes; without clear indications of effect sizes, SOCE might be granted credibility too easily. Furthermore, I wanted to take advantage of repeated measures analyses of variance, thus permitting a test of any interactions between apparent changes over time and other between-subjects grouping factors. I also wanted to see which, if any, prior factors were predictive of overall outcomes. Would a client’s initial situation be predictive of later outcomes? Even if research found that SOCE was helpful on average for some subjects, that might not help the therapist much for that new client who has just walked in the door with his/her own unique set of circumstances. Finally, I wanted to create a variable that might allow approximation of a control group for this data set and check if indeed changes in sexual orientation variables would parallel client reports of the degree of helpfulness of their SOCE. In other words, therapists might be rated as more or less helpful on other factors than perceived change in sexual orientation variables; therapy might be rated highly even if there were no changes in apparent sexual orientation. Thus, one ought not to assume that “more change” would equal “more helpful SOCE”; rather such an idea should be tested empirically.

 Methods Participants

The participants in this study were recruited through an online survey administered pursuant to the doctoral dissertation of Paul Santero (2011) at Southern California Seminary, which, as Sullins et al. (2021) have noted, “contains a more complete description of the survey methods, administration, and question wording.” (p. 3). Sullins et al. (2021) also reported that participants were “contacted through religious organizations and therapist networks who offered services including talk therapy, retreats, and support groups that serve this population” (pp. 3-4). The original participants included 25 men from countries outside the United States, but those men were not included in the analyses here or in Sullins et al. (2021). Respondents were distributed across the United States: west coast (24.0%), mountain west (27.2%), southwest (6.4%), south (8.8%), northeast (4.8%), east coast (14.4%), central (3.2%), with one response “having lived all over the USA” with no missing data for that information. The percentages here for location differ from those reported in Sullins et al. (2021) because two variables concerning residence were merged into one variable to eliminate apparent missing data. For example, in one case, the primary variable was not answered but the second variable answer was “Utah”, which was counted as “mountain west.” No women were among the 125 participants studied here, although 8 women had participated originally. Most of the sample was white (91%) and 73% had a college education or higher, while 58% reported a household income above $50,000. With respect to religious attendance, 88% reported weekly or more often.

 Measurement

For the items and scales used in this report, Appendix A lists the mean, median, standard deviation, minimum, maximum, and total number of non-missing cases. All data used here were obtained from the data set provided by Sullins et al. (2021). The data set did not include any measures of social desirability response bias; however, a more detailed discussion of different forms of social desirability measurement is available elsewhere ( Schumm, 2015).

 Scales

Generally, multi-item scales are preferable to single item measures, at the very least because of the potential for increased measurement reliability in terms of internal consistency. There were three questions that concerned sexual behavior, kissing, and daydreaming about sex for both heterosexuality and homosexuality as well as prior to SOCE and during/after SOCE, a total of 12 items. Items about desiring emotional intimacy were added to each of the four scales to see if an additional, but possibly unrelated item, would improve reliability. Adequate-to-fair Cronbach alphas were obtained as follows: prior same-sex sexuality, 0.68; prior heterosexuality, 0.76; post same-sex sexuality, 0.68; and post heterosexuality, 0.79. Adding the fourth item changed the reliabilities, respectively, to 0.58 (a decrease), 0.76 (no change), 0.76 (an increase), and 0.82 (an increase).

 Overall helpfulness of SOCE

There were 10 items that asked how helpful different forms of SOCE had been, with answers from not at all/none, slightly, moderately, markedly, and extremely. The forms of SOCE listed included psychiatrist, psychologist, social worker, mental health counselor, pastoral counselor, religious peer group, nonreligious peer group, weekend retreat, personal study, and mentoring. Some subjects used only one form of SOCE while others used multiple forms. To create an overall measure for each client, regardless of how many forms they had used or experienced, the highest rating given to any of the ten forms of SOCE was used to create one overall measure of SOCE helpfulness. Taking the opposite approach, using the worst results to create a scale, would have overlooked forms of SOCE that had been effective for the subjects – someone might try a lot of things to find help, but even if some don’t help, the one(s) that did help, would hopefully be those that mattered the most. For the 123 subjects who answered at least one of the helpfulness questions, the percentage results were none (1.6%), slightly (8.9%), moderately (11.4%), markedly (19.5%), and extremely (58.5%), with the values coded from 1 to 5, respectively. The mean score for overall helpfulness was 4.24 (SD = 1.07) with a median of 5.0. The item was significantly skewed, -1.27 (SE of skew = 0.22) with the distribution of responses more strongly favorable than unfavorable. Furthermore, a trinary measure was created by recoding the none/slightly, moderately/markedly, and extremely responses into three groups, which was intended to provide a type of control group (SOCE was not effective, as perhaps no treatment control group would have been) along with two levels of relative SOCE effectiveness. The results for this new item reflect what Sullins et al. (2021) concluded – that some subjects were not benefitted by SOCE but that the majority did seem to report beneficial outcomes.

 Typology of sexual orientation before SOCE

Sexual orientation is usually expressed or described in terms of attraction, behavior, and identity. Would it be possible to develop a typology of sexual orientation of subjects prior to SOCE? Using median splits for sexual attraction, same-sex sexual behavior, and sexual identity, a typology of the three aspects of sexual orientation prior to SOCE was created. Six of the eight possible combinations were obtained for 123 subjects: low behavior, low attraction, and low identity (7.3%); low behavior, high attraction, low identity (30.9%); low behavior, high attraction, high identity (21.1%); high behavior, low attraction, low identity (4.1%); high behavior, high attraction, low identity (15.4%); and high behavior, high attraction, high identity (21.1%). A one-sample Chi-squared test ( df = 5) of 36.2 indicated that the types were not equally distributed ( p < .001).

 Changes in marital status

Marital status was assessed for the time prior to SOCE and after/during SOCE. Sullins et al. (2021) reported that the men in their study were more likely to be single than in the United States in general but less likely to be divorced; their study seemed to assume that all marriages were heterosexual. Some subjects were single, married, or divorced/widowed before SOCE; some got married from the single state; some were divorced from the married status after/during SOCE. A new item was created with four categories: single (before and later), married (before and later), got married (changed from single to married), and divorced/widowed (either stayed in that category or changed to that category). The percentages of subjects (N = 125) in each category, respectively, were 52.8%, 26.4%, 15.2%, and 5.6%.

 Maximum number of SOCE sessions

Respondents reported the number of SOCE sessions for each of the types of SOCE listed, which included sessions with a psychiatrist, a psychologist, a social worker, a mental health counselor, a pastoral counselor, with an ex-gay support group, and with a nonreligious support group. To create a summary variable, the maximum number of sessions for any of the types of SOCE was noted. In other words, if pastoral counseling involved 30 sessions and mental health counseling involved 50 sessions, the coding would be for 50 sessions. For two subjects, none of the seven types of counseling sessions were used. Thus, the breakdown for maximum number of sessions for the most used type of counseling was none (1.6%), 1-10 (4.8%), 11-25 (8.8%), 26-50 (16.0%), 51-100 (29.6%), 101-200 (12.0%), and more than 200 (27.2%). The logic was that the most used type was probably the one that subjects felt was most useful and therefore those number of sessions probably were most related to any favorable outcomes. Another measure of the maximum number of sessions was created by crediting the variable with the upper limit of each category and crediting ratings of over 200 sessions with a score of 250. While that approach might seem to overestimate the number of sessions, it is counterbalanced by the fact that many SOCE participants had sessions in multiple therapeutic approaches.

 Research questions

The first research question concerned finding the Cohen’s d effect sizes for most of the key before/after variables dealing with different aspects of sexual orientation, to be analyzed with paired samples t-tests ( Table 1).

The second research question concerned how changes would occur in sexual orientation (attraction, identity, same-sex sexual behavior, and different-sex sexual behavior) as a function of a typology of pre-SOCE sexual orientation conditions. In other words, would certain combinations of sexual attraction, behavior, and identity be associated with lesser or greater changes in sexual orientation during or after SOCE? Paired samples t-tests were used to detect apparent change over time for each of six conditions from the pre-SOCE sexual orientation typology ( Tables 2– 7).

The third research question concerned whether pre-SOCE levels of same-sex sexual attraction, identity, and behavior, as well as marital status would predict overall level of reported helpfulness of SOCE in positive or negative directions, to be analyzed using ordinary least squares regression.

The fourth research question concerned how marital status might moderate changes in sexual attraction, identity, or behavior (both same-sex and different-sex), using a repeated measures analysis of variance using marital status as a between-subjects factor. To assess the changes in terms of effect sizes, paired samples t-tests will also be performed for each pair of outcome variables for each level of marital status ( Table 8).

The fifth research question involved predicting the reported helpfulness of SOCE as a linear and/or quadratic function of the maximum number of sessions, using the curve estimation program of SPSS’s version 26.0 linear regression program.

The sixth research question concerned a possible interaction effect between our trinary measure of reported helpfulness of SOCE as a between-subjects variable and change over time for our four key measures of sexual orientation, using repeated measures analyses of variance, with a focus on observing how effect sizes would change as a function of the three levels of reported helpfulness ( Tables 9 through 12).

The seventh research question concerns how the reported helpfulness of SOCE would be related to the reported changes in both sexual attraction and sexual identity in terms of degree and consistency, using a chi-square test to compare levels of helpfulness against a typology of changes.

The eighth research question reflects a concern from some that SOCE results might be a function of recall bias or recency bias; recall bias might lead respondents to report more favorable results the longer the time since therapy while recency bias might lead them to report more favorable results the more recent therapy.

The ninth research question concerns congruence of sexual attraction and sexual identity before and after SOCE. Sullins et al. (2021) found that measures of sexual attraction, identity, and behavior became more congruent after SOCE; Bondy (2021) found congruence of attraction and identity to be related to less effective outcomes of SOCE in his study of 156 participants. As noted by Bondy (2021, p. 93), identity theory may need further consideration ( Schumm, 2020). Because previous SOCE studies have included a number of persons who report same-sex attraction but less or no same-sex sexual orientation identity, one could anticipate that more participants would report higher same-sex attraction than sexual orientation identity both before and after SOCE; based on Sullins et al. (2021), one might expect a stronger correlation after SOCE than before for those two variables. When comparing the two variables before and then after SOCE, one might expect a significant difference using paired samples t-tests. Furthermore, the difference before SOCE might be less than after SOCE. It is possible that high congruence of SSA and SSI (before and/or after SOCE) might be associated with greater resistance to SOCE.

The tenth research question focused on the participants ages 18 to 25. Some have claimed that SOCE is very harmful for minors and this age group was the closest we could come to assessing SOCE’s potential impact on minors, since some had been in therapy for some time. Notably, Ryan et al. (2020) used a sample of youth ages 21 to 25 to assess retrospective accounts of SOCE, an older age than used here. The goal was to compare positives and harms attributed to SOCE in the areas of self-esteem, depression, suicidality, and social functioning and then to assess on a case-by-case basis how positives and harms compared for each of the four outcomes.

Analysis

Paired samples t-tests, linear regression, Chi-squared tests, and repeated measures analyses of variance will be used to evaluate these research questions, using SPSS version 26 (RRID: SCR 002865). An open access alternative is JASP (RRID: SCR 015823); however, numerous alternative free website sources for statistical analysis have been listed elsewhere ( Schumm et al., 2021).

 Results First question: effect sizes in terms of Cohen’s <italic toggle="yes">d</italic>

Sullins et al. (2021) reported nonparametric tests to compare client conditions before and after SOCE. Here, t-tests with Cohen’s d as a measure of effect size are reported. While all the variables used, including the scales, had significantly non-normal distributions, t-tests are generally robust with respect to such violations. However, the main point of the analyses here was to provide a widely recognized measure of effect size that also considered the correlated nature of the variables being compared to avoid underestimating the true effect sizes involved. Cohen (1992) indicated that an effect size of 0.50 or greater would be “visible to the naked eye of a careful observer” (p. 156), though Funder and Ozer (2019) noted that even small effect sizes ( d = 0.20) were important (also see VanVoorhis & Morgan, 2007). Results are presented in Table 1.

Basic paired samples comparisons over time for SOCE.
Variables Pre-test Mean (SD) Post-test Mean (SD) R t df p Cohen’s d
Sexual Attraction 5.73 (1.10) 4.14(1.76) .380 *** 10.49 123 < .001 0.94
Daydreaming about gay sex 4.56 (0.85) 3.16(1.38) .138 10.27 123 < .001 0.92
Daydreaming about hetero-sexual sex 1.79 (1.13) 2.80 (1.43) .458 *** 8.27 122 < .001 0.74
Desired hetero-sexual emotional intimacy 2.45 (1.47) 3.39 (1.54) .576 *** 7.51 121 < .001 0.68
Desired gay emotional intimacy 4.00 (1.42) 2.97 (1.46) .345 *** 6.98 123 < .001 0.63
Sexual identity 4.80 (1.96) 3.58 (1.96) .464 *** 6.75 124 < .001 0.60
Gay sexual behavior 2.41 (1.57) 1.50 (1.57) .267 ** 6.20 122 < .001 0.56
Gay kissing 1.79 (1.20) 1.37 (0.92) .227 * 3.51 122 .001 0.31
Heterosexual kissing 1.82 (1.22) 2.18 (1.48) .572 *** 3.15 120 .002 0.28
Heterosexual sexual behavior 1.69 (1.17) 1.97 (1.36) .577 *** 2.61 122 .010 0.24
Gay Scale (3 items) 8.77 (2.91) 6.05 (2.62) .097 8.08 121 < .001 0.73
Heterosexual Scale (3 items) 5.28 (2.88) 6.93 (3.57) .628 *** 6.38 120 < .001 0.58

Note: Effect sizes calculated from https://memory.psych.mun.ca/models/stats/effect_size.shtml.

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

It appears that the effect sizes detected range from 0.24 to 0.94, falling into the small to large effect size range. Same-sex sexual behavior decreased by more than twice as much as the increase in different-sex behavior (0.56 versus 0.24). Of the three main components of sexual orientation, attraction appeared to decrease the most (0.94) while same-sex sexual behavior (0.56) and same-sex sexual identity (0.60) appeared to decrease by about the same amount. Desired heterosexual emotional intimacy increased by about the same amount (0.68) as desired same-sex emotional intimacy decreased (.63). By Cohen’s (1992) definition, same-sex sexual attraction decreased to a large extent (≥ .80) while the other two aspects decreased by medium size effects (≥ .50). By Cohen’s definition, all three effects would be observable to a trained naked eye.

 Second question: pre-SOCE conditions

Sullins et al. (2021) compared SOCE scores before and during/after but did not assess a typology of pre-SOCE sexual orientation. Tables 2- 7 display the results obtained for apparent changes in four key aspects of sexual orientation: attraction, identity, same-sex behavior, and different-sex behavior.

Tables 2- 7 predicting basic changes as a function of a typology of six pre-SOCE conditions.

Initial condition: Relatively low on same-sex attraction, behavior, and identity (N = 9).
Variables Pre-test Mean (SD) Post-test Mean (SD) R t df p Cohen’s d
Sexual Attraction 3.56 (0.73) 2.56 (1.33) .029 2.00 8 .081 0.67
Homosexual Behavior 1.33 (0.50) 1.22 (0.67) .500 0.56 8 .594 0.18
Heterosexual Behavior 2.67 (1.41) 2.89 (1.45) .892 ** 1.00 8 .347 0.33
Sexual Identity 3.22 (1.09) 2.22 (1.30) .225 2.00 8 .081 0.67

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Initial condition: relatively high on same-sex attraction, but low on behavior and on identity (N = 38).
Variables Pre-test Mean (SD) Post-test Mean (SD) R t df p Cohen’s d
Sexual Attraction 5.54 (0.61) 4.27 (1.56) .076 4.75 36 < .001 0.78
Homosexual Behavior 1.16 (0.37) 1.24 (0.68) -0.15 -0.60 37 .556 0.10
Heterosexual Behavior 1.55 (1.18) 2.00 (1.40) .444 ** 2.01 37 .051 0.33
Sexual Identity 3.53 (1.84) 3.37 (1.90) .462 ** 0.50 37 .619 0.08

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Initial condition: relatively high on same-sex attraction and identity, but low on behavior (N = 26).
Variables Pre-test Mean (SD) Post-test Mean (SD) r t df p Cohen’s d
Sexual Attraction 6.58 (0.50) 4.92 (1.85) .349 + 4.84 25 < .001 0.95
Homosexual Behavior 1.24 (0.44) 1.48 (1.05) .377 + 1.24 24 .228 0.25
Heterosexual Behavior 1.12 (0.60) 1.60 (1.19) .070 1.85 24 .076 0.37
Sexual Identity 6.58 (0.50) 4.58 (2.08) .509 ** 5.44 25 < .001 1.07

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Initial condition: relatively high on same-sex behavior, but low attraction and on identity (N = 5).
Variables Pre-test Mean (SD) Post-test Mean (SD) r t df p Cohen’s d
Sexual Attraction 3.80 (0.45) 3.00 (1.00) .000 1.63 4 .178 0.73
Homosexual Behavior 3.80 (1.10) 2.00 (1.41) .968 ** 9.00 4 .001 4.07
Heterosexual Behavior 3.40 (0.89) 4.00 (0.71) .791 2.45 4 .070 1.10
Sexual Identity 3.20 (0.84) 2.80 (1.30) .504 0.78 4 .477 0.35

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Initial condition: relatively high on same-sex behavior and on attraction, but low on identity (N = 19).
Variables Pre-test Mean (SD) Post-test Mean (SD) R t df p Cohen’s d
Sexual Attraction 5.42 (0.61) 3.84 (1.34) .495 * 5.88 18 < .001 1.36
Homosexual Behavior 4.05 (0.78) 1.68 (1.16) .143 7.94 18 < .001 1.82
Heterosexual Behavior 2.21 (1.18) 1.89 (1.33) .581 ** 1.19 18 .250 0.28
Sexual Identity 3.89 (1.49) 3.05 (1.51) .696 ** 3.15 18 .006 0.72

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Initial condition: relatively high on same-sex attraction, behavior, and identity (N = 26).
Variables Pre-test Mean (SD) Post-test Mean (SD) R t df p Cohen’s d
Sexual Attraction 6.62 (0.50) 4.31 (2.00) -.037 5.67 25 < .001 1.11
Homosexual Behavior 4.31 (0.74) 1.77 (1.28) .036 8.94 25 < .001 1.75
Heterosexual Behavior 1.32 (0.80) 1.40 (0.82) .814 *** 0.81 24 .425 0.16
Sexual Identity 6.46 (0.51) 4.04 (2.05) .136 6.05 25 < .001 1.18

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

In Table 2, the low/low/low group was small, and these subjects reported higher levels of heterosexual behavior than homosexual sexual behavior prior to SOCE. Homosexual behavior was very low before SOCE. Both sexual attraction and sexual identity were, on average, in the lower bisexual to heterosexual range. Though not significant with a two-tailed test ( p < .10) due to the small sample size, decreases in homosexual sexual attraction and identity were in Cohen’s medium range (both 0.67). If one were to assume that SOCE would only decrease same-sex factors, such that a one-tailed test would be appropriate, then the results for sexual attraction and sexual identity would be statistically significant ( p < .05).

In Table 3, pre-SOCE sexual attraction scores were high while same-sex sexual behavior was lower and lower than different-sex sexual behavior; sexual identity fell between the levels for sexual attraction and sexual behavior. Sexual identity changed little with SOCE for this group while sexual attraction changed substantially ( d = 0.78, p < .001); the next largest effect size was for heterosexual behavior increasing ( d = 0.33, p < .06), a small-to-medium effect that was not quite significant at the. 05 level. These subjects were probably more concerned about their levels of same-sex sexual attraction than anything else and, if so, the results suggest that their expectations were met, at least to some extent.

In Table 4, the subjects initially were high on same-sex attraction and identity but lower on same-sex sexual behavior. While same-sex sexual behavior changed little with SOCE, changes in both attraction and identity were well above the limit for large effect sizes (≥ .80) per Cohen (1992) with 0.95 for attraction and 1.07 for identity, both significant ( p < .001). Changes in either same-sex or different-sex sexual behavior were not significant ( p < .05), although their effect sizes were in the small-to-medium range (0.25 to 0.37).

In Table 5, the subjects before SOCE were lower on attraction and identity (being in a low bisexual range) but higher on same-sex sexual behavior. With only five cases, only the changes in sexual behavior were significant or near significant, even though the effect sizes were well over the limit for large effect sizes per Cohen (1992), at a remarkable 4.07 for homosexual behavior and 1.10 for heterosexual sexual behavior. Effect sizes for attraction and identity were, respectively, nearly large (0.73) and in the small-to-medium range (0.35), though not significant statistically. It is possible that these subjects entered SOCE with more concern with their sexual behavior than with their attractions or identities; hence, the SOCE may have focused more on behavior than the other factors.

In Table 6, the subjects began higher on same-sex sexual attraction than on identity or behavior; however, in terms of change, effect sizes were very large for both same-sex sexual attraction (1.36) and for gay sexual behavior (1.82), both significant at p < .001. Smaller changes occurred for heterosexual behavior (0.28) and for identity (0.72, p < .01). Notably, same-sex sexual behavior was at a higher level ( d = 1.84) than different-sex behavior before SOCE but was lower than heterosexual behavior during/after SOCE ( d = 0.17).

In Table 7, subjects entered SOCE with higher scores on all three aspects of homosexual sexuality. While changes in heterosexual sexual behavior were minimal (0.16), there were very large changes in sexual attraction (1.11, p < .001), sexual identity (1.18, p < .001), and in sexual behavior (1.75, p < .001). While homosexual behavior remained higher than heterosexual behavior at both time points, it declined from an effect size advantage of 3.88 to only 0.34, less than a tenth of the initial effect size difference.

 Third question: initial conditions as main effects on SOCE helpfulness

Including prior sexual attraction, sexual identity, and same-sex sexual behavior as independent variables led to a non-significant overall degree of explained variance. After deleting sexual attraction as one of the independent variables, the adjusted R-squared was 0.04, with F(2, 199) = 3.52 ( p = .033), with significant or near-significant results for the independent variables, prior same-sex sexual behavior ( b = .17, p = .06) and sexual identity ( b = -.19, p < .04). This would indicate that prior conditions had relatively little impact on therapeutic outcomes as reported by the subjects in this study, although there was a slight tendency, for higher levels of prior same-sex behavior (controlling for sexual identity) and lower levels of prior same-sex sexual identity (controlling for same-sex behavior) to predict slightly higher levels of reported therapeutic helpfulness.

 Fourth question: effects of marital status

Sullins et al. (2021) also indicated that marital status played an important role in SOCE outcomes but I wanted to test that hypothesis more carefully, using a repeated-measures design and establishing a more detailed breakdown of marital status, including four categories: never married (N = 66), married (N = 33), got married or engaged (N = 19), and was divorced/widowed and stayed divorced/widowed or got divorced (N = 7).

The primary concern was whether there would be an interaction effect between apparent change in sexual orientation variables and this new version of marital status. For sexual identity, the interaction effect was not significant ( p = .393). The main effect of marital status was not significant ( p < .07). For sexual attraction, the interaction effect was not significant ( p = .246) while the main effect of marital status was significant ( p < .01). For same-sex sexual behavior, the interaction effect was not significant ( p = .07) while the main effect of marital status was significant ( p = .01). However, in terms of heterosexual sexual behavior, the main effect of time was less significant ( p = .04) than the interaction term ( p < .001) while the main effect of marital status was also very significant ( p < .001). Therefore, the detailed results for heterosexual sexual behavior are presented in more detail ( Table 8).

Outcomes in terms of heterosexual sexual behavior as a function of marital status.
Marital Groups Pre-test Mean (SD) Post-test Mean (SD) R t Df p Cohen’s d
Single (N = 64) 1.00 (0.00) 1.02 (0.13) .000 1.00 63 .321 0.16
Married (N = 33) 3.21 (1.05) 3.27 (1.10) .678 *** 0.40 32 .690 0.07
Got Married or Engaged (N = 19) 1.11 (0.32) 3.26 (1.37) -.197 6.43 18 < .001 1.47
Divorced/Widowed (N = 7) 2.43 (1.40) 1.00 (0.00) .000 2.71 6 .035 -1.02

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Fifth question: predicting helpfulness ascribed to most helpful SOCE from maximum sessions

Sullins et al. (2021) did not appear to have developed or used the two measures featured with our fifth research question. The curve estimation program under regression in SPSS was used to predict the perceived helpfulness of SOCE from the two measures of the maximum number of SOCE sessions. For the ordinal measure of sessions, the linear and quadratic model featured F(2, 120) = 4.51, p < .02 with an adjusted R-squared of 0.054. The linear standardized beta coefficient was 1.00 ( p = .01) while the quadratic beta was -.84 ( p < .03). The pattern suggested that maximum overall helpfulness peaked between 51-100 and 101-200 sessions. For the ratio measures of sessions, the linear and quadratic model featured F(2, 120) = 5.00 ( p < .01), with an adjusted R-squared of 0.062. The linear standardized beta coefficient was 1.32 ( p = .003) while the quadratic beta coefficient was -1.21 ( p = .006). The pattern suggested that the maximum helpfulness peaked at about 160 sessions. For both measures, the stronger increase in helpfulness occurred between no sessions and the maximum while the decrease in helpfulness was not as strong after the maximum.

 Sixth question: repeated measures analyses using a trinary measure of client-rated SOCE helpfulness

As noted in the introduction, it remains an open question if better SOCE ratings for providers will reflect change in sexual orientation. Perhaps, ratings will increase with increased changes in the direction of heterosexuality; perhaps, the reverse might occur. There are three groups with respect to client-rated helpfulness: none or slightly (N = 13), moderately or markedly (N = 38), and extremely (N = 71), labeled respectively as A, B, and C in the Tables 9 through 12.

Changes in sexual attraction as a function of level of perceived helpfulness of SOCE.
Group Pre-test Mean (SD) Post-test Mean (SD) r Main effect, Time ( p) d Interaction effect, time × group ( p)
A (N = 13) None to slight helpfulness 6.15 (0.90) 6.38 (0.87) .878 *** F(1, 119) = 36.73 (<.001) -.53 F(2, 119) = 24.96 (< .001)
B (N = 38) Moderate to marked helpfulness 5.55 (1.06) 4.68 (1.30) .525 ** .74
C (N = 72) Extreme helpfulness 5.73 (1.16) 3.39 (1.65) .347 ** 1.41

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Changes in same-sex sexual behavior as a function of level of perceived helpfulness of SOCE.
Group Pre-test Mean (SD) Post-test Mean (SD) r Main effect, Time ( p) d Interaction effect, time × group ( p)
A (N = 13) None to slight helpfulness 2.00 (1.29) 2.31 (1.49) .735 *** F(1, 118) = 10.05 (.002) -.30 F(2, 118) = 5.19 ( .007)
B (N = 38) Moderate to marked helpfulness 2.29 (1.43) 1.47 (0.89) .249 .55
C (N = 72) Extreme helpfulness 2.59 (1.69) 1.37 (0.94) .255 * .71

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Changes in heterosexual sexual behavior as a function of level of perceived helpfulness of SOCE.
Group Pre-test Mean (SD) Post-test Mean (SD) r Main effect, Time ( p) d Interaction effect, time × group ( p)
A (N = 13) None to slight helpfulness 1.50 (1.17) 1.75 (1.36) .775 ** F(1, 118) = 2.40 (.124) .29 F(2, 118) = 2.19 (.116)
B (N = 38) Moderate to marked helpfulness 1.82 (1.23) 1.76 (1.17) .814 *** -.08
C (N = 72) Extreme helpfulness 1.68 (1.16) 2.11 (1.46) .471 *** .31

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Changes in sexual identity as a function of level of perceived helpfulness of SOCE.
Group Pre-test Mean (SD) Post-test Mean (SD) r Main effect, Time ( p) d Interaction effect, time × group ( p)
A (N = 13) None to slight helpfulness 5.69 (1.75) 6.08 (1.71) .874 *** F(1, 120) = 12.00 (.001) -.45 F(2, 120) = 15.79 (< .001)
B (N = 38) Moderate to marked helpfulness 4.79 (1.76) 4.32 (1.58) .716 *** .37
C (N = 72) Extreme helpfulness 4.69 (2.05) 2.69 (1.61) .381 ** .97

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

For sexual attraction, the mean scores were as in the following Table 9.

For same-sex sexual behavior, the mean scores were as in the following Table 10.

For heterosexual sexual behavior, the mean scores were as in the following Table 11.

For sexual identity, the mean scores were as in the following Table 12.

For heterosexual sexual behavior, changes from SOCE were small and not significant statistically. However, for the three same-sex sexuality factors, the main effects over time and the interaction between time and the three levels of reported helpfulness were always significant ( p < .01). For group A, the reported changes were consistently in a pro-gay direction, suggesting that, on average, changes in that direction were not perceived as helpful in terms of SOCE effectiveness. For groups B and C, greater changes in a heterosexual direction were perceived as more helpful. Both of these results suggest that SOCE’s effectiveness was judged on the basis of its association with desired changes in a heterosexual direction, even though changes in the direction of heterosexual behavior were smaller. For group B, changes tended to be from small-to-large; for group C, changes tended to be large, in terms of Cohen’s (1992) criteria.

 Question 7: typology of change and reported helpfulness of SOCE based on sexual attraction and sexual identity before SOCE and after/during SOCE

A typology was created of the main types of change reported by participants crossed with a five-category measure of reported helpfulness of SOCE. The results are shown in Table 13.

Reported helpfulness of SOCE associated with size and consistency of changes in various outcomes.
Typology N None Slightly Moderately Markedly Extremely
Changes towards Heterosexual Attraction and Identity (or stayed heterosexual)
Consistently mostly heterosexual (1,2) 1 100.0%
Strongly gay (6,7) to mostly heterosexual (1,2)(consistent) 8 12.5% 87.5%
Gay or bisexual (5, 6, 7) to mostly heterosexual (1, 2) 1 100.0%
Bisexual (4, 5) to mostly heterosexual (1,2) (consistent) 11 9.1% 90.9%
Strongly gay (6,7) to bisexual (3, 4, 5)(consistent) 14 7.1% 92.9%
Strong gay or bisexual (5, 6) to weaker bisexual (3, 4, 5)(consistent) 23 4.3% 34.8% 60.9%
Slightly less gay (7 to 6, 6 to 5) (consistent) 7 14.3% 42.9% 42.9%
 Subtotals 65 6.2% 18.5% 75.4%
No changes/Inconsistent
Difficult to categorize 7 14.3% 85.7%
At least 3 points inconsistent both before and during/after SOCE 9 11.1% 11.1% 22.2% 55.6%
Consistently bisexual (3, 4, 5) both times 13 15.4% 38.5% 30.8% 15.4%
Gay or bisexual (3 to 7) across both times (inconsistently) 2 100.0%
Strongly gay (6,7) both times 17 5.9% 41.2% 23.5% 11.8% 17.6%
 Subtotals 48 2.1% 20.8% 20.8% 18.8% 37.5%
Changes towards Gay Attraction and Identity
More gay but inconsistent 1 100.0%
More gay but consistent 6 16.7% 16.7% 33.3% 33.3%
 Subtotals 7 14.3% 14.3% 42.9% 28.6%

For all categories, a Chi-square test ( df = 56) = 90.96, p = .002, indicating differences among the several categories. Comparing the three subgroup counts, a Chi-square ( df = 8) = 36.99, p < .001, r = .45, p < .001. Comparing the two largest subgroups, a Chi-square ( df = 4) = 26.38, p < .001, r = .48, p < .001. A comparison of the two smallest subgroups was not statistically significant. A comparison of the largest subgroup with the smallest subgroup yielded a chi-square ( df = 4) = 22.76, p < .001, r = .43, p < .001. For this last comparison Cramer’s V = 0.24, which represents a large effect size, as calculated using a website [ https://www.statology.org/effect-size-chi-square/]. Notably, while those men who became more gay in general (91.7%) reported at least slight or better degrees of helpfulness of SOCE, a small percentage (8.3%) of that group of men reported that SOCE was not helpful at all.

 Question 8: will reports of therapy outcome change as a function of recency of or time since SOCE?

Any time a respondent reports on past events, there may be recall bias or distortions of memory. Social desirability may contaminate reports of past or current events. Here the question was whether the time elapsed since having experienced SOCE would be associated with ratings of SOCE, which might be expected if recall bias was substantial ( Table 14).

Typology of SOCE ratings as a function of time since SOCE?.
Time since SOCE Percent slightly or less effective Percent moderately or markedly effective Percent extremely effective Subgroup sizes/total group sizes
In SOCE now 5.9 33.3 60.8 51
Less than one month since SOCE ended 0.0 0.0 100.0 3
More than one month but less than a year 16.7 29.2 54.2 24
One to two years 0.0 37.5 62.5 8
Two to five years 17.6 29.4 52.9 17
More than five years 13.3 33.3 53.3 15
Missing data 20.0 20.0 60.0 5
Total percentages 10.6 30.9 58.5 100
Total N 13 38 72 123

Chi-Square Test ( df = 12) = 6.98, p = .859; zero-order correlation r = .09, p = .302.

Results indicate that neither recall bias nor recency bias are predominant; the negative correlation suggests a slight trend in favor of recency bias. It is possible that both biases are operant but canceled each other out.

 Ninth research question: what role does congruence between sexual attraction and sexual identity play in SOCE?

Before SOCE, SSA scores (mean, 5.74; SD, 1.10) were significantly greater ( t 124 = 6.00, p < .001) than scores for SSI (mean, 4.80; SD, 1.76) with the scores correlated, r = .465, p < .001. After SOCE, SSA scores (mean, 4.14; SD, 1.76) were significantly greater ( t 123 = 5.62, p < .001) than scores for SSI (mean, 3.60; SD, 1.95) with the scores correlated, r = .839, p < .001. After SOCE, SSA and SSI scores were closer and more highly correlated.

SSASSI congruence were positive when SSA > SSI and negative when SSA < SSI. Before SOCE, the scores ranged between -4 and 6, with a mean of 0.936, a median of zero, and SD = 1.74; after SOCE, the scores ranged between -1 and 4, with a mean of 0.540, a median of zero, and SD = 1.07. Before SOCE, 57.6% of the participants had scores of zero; after SOCE, it was 65.6%. Comparing congruence before versus after SOCE, t 123 = 2.95, p = .004, with a mean of 0.895 (SD = 1.69) before and a mean of 0.540 (SD = 1.07) after, indicating that congruence scores decreased after, although remaining highly correlated, r = 0.611 ( p < .001).

To investigate whether or how congruence scores might matter, four subgroups of the total sample were created: no exact (SSI = SSA) congruence before or after SOCE, congruence before but not after, congruence after but not before, and congruence both before and after. Chi-square tests were performed for each of the four groups, crossing our trinary outcome variable versus post-SOCE sexual orientation identity split at bisexual to heterosexual orientation versus the more homosexual sexual identity orientations. None of the tests were significant except for the before and after congruence group. For that group, all the participants who reported no to only slight effectiveness, 100% (N = 11) were homosexually oriented. For those who reported moderate effectiveness, 22.7% (N = 5) were more heterosexually oriented compared to 77.3% (N = 17) who were homosexually oriented. For those who reported extremely effective, the respective percentages were 85.7% (N = 24) and 14.3% (N = 4). The chi-square test ( df = 2) = 31.76, p < .001, with r = -0.697 ( p < .001). Another way of reporting the results would be that 34.4% (11/32) of those who had stronger and congruent gay identities after SOCE indicated that SOCE was of only slight or no effectiveness compared to 82.8% (24/29) of those who had stronger and congruent more heterosexual identities indicated that their SOCE was extremely effective.

 Tenth research question: positives and harms for the youngest participants, ages 18-25

There were 18 participants between the ages of 18 and 25 at the time of the survey. Of those 18, 55.6% were currently in SOCE, 22.2% had finished SOCE within the past year, 5.6% within one to two years, and 16.7% within the past two to five years. In terms of maximum sessions for the longest type of SOCE, 27.8% had been involved in a maximum of 50 sessions, 33.3% for a maximum of 100 sessions, and 38.9% for a maximum of 200 or more sessions. Thus, it is likely that some of these participants had begun SOCE before age 18 ( Tables 15, 16).

Positive and negative or harmful outcomes of SOCE for self-esteem, depression, suicidality, and social functioning.
Self-esteem Depression Suicidality Social functioning
Positives 3.83 (SD = 1.15) 2.94 (SD = 1.47) 2.88 (SD = 1.89) 3.22 (SD = 1.48)
Negatives 1.28 (SD = 0.46) 1.44 (SD = 0.71) 1.25 (SD = 0.58) 1.44 (SD = 0.86)
t-test 7.85 3.85 3.37 3.92
Df 17 17 15 17
P < .001 = .001 = .004 = .001
Cohen’s d 2.91 1.30 1.17 1.47
Case wise (N = 18) assessment of positives and negatives associated with four SOCE outcomes for the youngest participants in SOCE ( <xref ref-type="bibr" rid="ref35">Sullins <italic toggle="yes">et al</italic>., 2021</xref>).
Self-esteem Depression Suicidality Social functioning
Positives Negatives Positives Negatives Positives Negatives Positives Negatives
Only positives (66.7%) Only positives (55.6%) Only positives (44.4%) Only positives (66.7%)
Slightly (1) Slightly (2) Slightly (1) Slightly (2)
Moderately (1) Moderately (3) Moderately (1) Moderately (3)
Markedly (6) Markedly (2) Markedly (1) Markedly (3)
Extremely (4) Extremely (3) Extremely (5) Extremely (4)
No positives or negatives (5.6%) No positives or negatives (11.1%) No positives or negatives (44.4%) No positives or negatives (5.6%)
Positives > Negatives (22.2%) Positives > Negatives (11.1%) Positives > Negatives (5.6%) Positives > Negatives (11.1%)
Moderately Slightly Moderately Slightly Extremely Moderately Moderately Slightly
Moderately Slightly Extremely Moderately Extremely Moderately
Moderately Slightly
Extremely Slightly
Positives = Negatives (5.6%) Positives = Negatives (11.1%) Positives = Negatives (5.6%) Positives = Negatives (5.6%)
Slightly Slightly Slightly Slightly Slightly Slightly Slightly Slightly
Moderately Moderately
Negatives > Positives (0%) Negatives > Positives (11.1%) Negatives > Positives (5.6%) Negatives > Positives (11.1%)
None Slightly NA Slightly None Slightly
NA Slightly None Markedly
No harm 72.2% No harm 66.7% No harm 83.3% No harm 72.2%
Slight harm 27.8% Slight harm 22.2% Slight harm 11.1% Slight harm 16.7%
> Slight harm 0.0% > Slight harm 11.1% > Slight harm 5.6% > Slight harm 5.6%

For all four outcomes used here, results were significant ( p < .01) and indicated that positive effects of SOCE exceeded negative effects or harms, as reported by the young participants themselves.

A supplementary analysis that was not planned along with the others was to evaluate the relative predictive strength of positive versus negative outcomes with respect to the reported helpfulness of SOCE, as shown in Table 17. In general, positive changes seemed to impact reported helpfulness more than negative changes did. The strongest effect occurred for self-esteem, which also featured the strongest impact of a positive change. Although the results for suicidality were limited by a large amount of missing data, the positive and negative changes were equal in magnitude and the negative change’s impact was the largest for all five outcomes, suggesting that SOCE providers need to be acutely aware of and continue to check on participants’ feelings and intentions in that area and also with respect to changes in depression.

Predicting maximum helpfulness with SOCE from positive and negative outcomes of SOCE.
Outcomes F df P B (positive) B (negative) Adj R-squared
Self-esteem 51.50 2, 113 < .001 .673 *** -.048 .468
Suicidality 6.16 2, 71 .003 .257 * -.249 * .124
Depression 12.98 2, 100 < .001 .347 *** -.244 ** .190
Self-harmful behavior 9.86 2, 120 < .001 .396 *** -.053 .127
Social functioning 15.21 2, 109 < .001 .382 *** -.239 ** .204

Significance levels are indicated by *** (p < .001), ** (p < .01), * (p < .05), and + (p < .10).

 Limitations

This report has many of the same limitations as Sullins et al. (2021). The sample is nonrandom, purposive, did not include women, and the data are over a decade old. The sample included men who were, as Sullins et al. (2021) stated, most often “white, affluent, well-educated, highly religious, and overrepresented the Mormon faith” (p. 14). In addition, our analyses used parametric statistics, but the assumption of normality was violated for most of our variables. However, many of our effect sizes are large enough that even if they were positively biased, they would usually still be of small-to-medium size. In general, more favorable results have been found for SOCE when the subjects were eager for treatment, highly religious, and married or planning to get married, as in this sample; less favorable, even harmful results have been found for SOCE when the subjects were pressured into experiencing SOCE, were not religious, were not married, and were high on same-sex sexual orientation identity at the time of the survey about past SOCE attempts. It is unlikely that these different populations will experience SOCE in the same way; results from one group should not be generalized to the other groups. Sullins et al. (2021) have presented other limitations that would also apply to this research.

 Opportunities for independent analysis

Data sets held by Bondy (2021) and Pela and Sutton (2021), if released by their authors, might provide similar opportunities for independent analysis. Compared to Sullins et al. (2021), Bondy’s data features a larger sample size (N = 156) and includes more data on SSASSI congruence, extrinsic motivations to try SOCE, and client reported sources of SSA (including 44 reports of child sexual abuse; would those reports correlate with initial levels of SSA?). Bondy found that extrinsic motivations for SOCE predicted poorer ratings of SOCE experience, a result like other areas of counseling (e.g., Schumm & Denton (1979). Pela and Sutton’s (2021) data could be explored by age group to see how the youngest (age 18-25) responded to SOCE, by level of religiosity (did those with more frequent church attendance or citing religion as a reason to change report the most change?), or by level of desire for marriage (did a higher level of desire for marriage seem to affect the reported changes?). Even though the data indicated no average decline in mental health with SOCE, it might be useful to create a typology of those whose mental health did deteriorate, those whose remained similar, and those whose increased substantially and investigate if that typology would be associated with different SOCE motivations or outcomes.

 Discussion

SOCE is a controversial intervention. Some scholars have concluded that it is not effective and might even be unethical ( Haldeman, 2022; Przeworski et al., 2021). In other controversial areas, however, scholarly consensus has been incorrect ( Schumm, 2018). Sullin’s (2022) analysis of SOCE harms found little evidence of harms even when SOCE seemed to have been ineffective and even though the persons who were exposed to SOCE had experienced greater levels of childhood adversity, including having been bullied on account of their sexual orientation than those with no SOCE experience; scholarly “consensus” may be incorrect with respect to SOCE.

Here an attempt was undertaken to independently evaluate a data set previously analyzed by Sullins et al. (2021). Our overall assessment is that Sullins et al. (2021) correctly concluded that SOCE was often effective, though change in this area is very difficult, and that in some cases, SOCE was not effective at creating change. However, our assessment may add further knowledge in the following areas:

Measurement. New measures were introduced, including four scales from the Santero data set, as well as a measure of overall helpfulness of SOCE, a typology of sexual orientation conditions prior to SOCE, a more differentiated measure of marital status, and a measure of the number of sessions of SOCE undertaken.

Comparing a variety of measures of sexual orientation before and during/after SOCE, we found effect sizes from 0.24 to 0.94, all of which were significant statistically ( p < .01). Changes in attraction and daydreaming were large (generally > .80) but even changes in attraction and behavior were larger than Cohen’s medium effect size. The smallest effect sizes occurred for heterosexual kissing and sexual behavior, though they were still of small or greater size, still important for social science (Funder and Ozer, 2019).

Among six conditions involving the three aspects of sexual orientation prior to SOCE, it was found that the most consistent results for sexual attraction with three medium and three large effect sizes. Changes in same-sex sexual behavior varied more, with three large effect sizes but also one small effect size change. Changes in sexual identity included two large, two medium, and one small effect size. All the changes were in the expected direction, towards less same-sex interest or activity. The weakest changes occurred in terms of heterosexual behavior with one large effect size and four small effect sizes. In general, SOCE seemed more effective when initial levels of sexual orientation were higher. When levels were higher than others before SOCE, the initially higher levels seemed to change more, with larger effect sizes obtained.

A slight trend was observed for higher levels of identity (controlling for behavior) to be associated with smaller levels of reported helpfulness of SOCE; likewise, a small effect was noted for higher levels of behavior (controlling for identity) to be associated with stronger levels of reported helpfulness of SOCE.

Marital status appeared to interact with changes in heterosexual sexual behavior before and during/after SOCE, with larger changes for those who got married after SOCE began and negative changes for those who were divorced or widowed.

Significant linear and quadratic effects were observed, predicting reported helpfulness from the number of sessions of SOCE, although the level of explained variance was small (0.051) and the peak benefit seemed to be around 150 sessions, though most help occurred per session in the first 50 sessions.

Using three levels of reported helpfulness, it was found that for the 13 subjects who reported changes towards homosexuality their sense of helpfulness was lower while the 109 subjects who reported changes in the other direction reported higher levels of helpfulness. Effect sizes increased across the three groups of reported helpfulness. The smallest and most inconsistent effects were observed for changes in heterosexual sexual behavior.

Similar results were found as above when perceived helpfulness was cross tabulated with typologies of reported changes in terms of size and consistency. High levels of reported helpfulness were obtained when changes occurred as desired, but lower levels occurred when changes were towards homosexuality. However, the results for the subjects who reported changes towards homosexuality were bifurcated – some subjects reported lower levels of helpfulness, but a larger percentage of that group reported high levels of helpfulness, suggesting that even though they didn’t change towards heterosexuality, they still found SOCE very helpful. Perhaps these subjects found helpful affirmation of their homosexual identity through SOCE or, as some critics might surmise, despite SOCE. The percentage of men who became more gay and found SOCE to be not helpful was small (8.3%) compared to those (91.7%) who became more gay but said SOCE was nevertheless slightly, moderately, or extremely helpful.

The self-reported efficacy of SOCE was not significantly related to the time since SOCE, which may suggest that recall bias is not as great as often feared; it’s also possible that recency bias and recall bias cancelled each other out.

Congruence between SSA and SSI was investigated. In most cases, SSA > SSI. Congruence seemed to become greater after SOCE compared to before SOCE. When SSA and SSI were congruent both before and after SOCE, self-reported efficacy was strongly related to sexual orientation.

Self-reported positives for self-esteem, depression, suicidality, and social functioning were significantly greater than negatives. On average, positives were greater than negatives for nearly 70% of the respondents. Only 5.6% of the time were reports of harms greater than the slightly negative level and never rated greater than moderately negative. In this sample, harms appeared to be mostly of a minor nature.

Positive reported changes in SOCE outcomes appeared, in general, to predict reported helpfulness of SOCE more than negative changes did. The strongest overall effect was for changes in self-esteem. The apparent effects of positive and negative changes were nearly equal for suicidality and for depression, which may warn SOCE providers to be careful to assess and prevent adverse changes in suicidality and/or depression among their SOCE participants.

Challenging identity theories

From this author’s perspective, there may be at least two “elephants in the living room.” First, there seems to be an assumption by some scholars that if you experience any degree, slight to large, of SSA, then you must adopt a gay, lesbian, or bisexual identity (SSI), assuming (1) there should be absolute congruence or else one risks becomes inauthentic as a human being and (2) that determination is not up to the individual but to a variety of others wielding some sort of social influence. It seems that for some, SOCE is in part about disagreeing with, breaking, or reframing those assumptions ( Bondy, 2021). Some religious groups may interpret SSA as a “temptation” rather than an intrinsic part of oneself, relying on the religious premises that everyone (even Jesus) has been tempted to do unusual, strange, even wrong things but that only verbal or nonverbal behavior involves moral culpability ( Sutton, 2019). As Karten & Wade (2010) noted, “there are men who are sexually attracted to other men but do not identify as gay; rather, they experience their homosexual orientation and behavior as at odds with who they really are” (p. 86). Furthermore, as the number of persons identifying as nonheterosexual has increased in recent decades ( Pellicane & Ciesla, 2022), perhaps a greater number of individuals might describe themselves as mostly heterosexual or somewhat bisexual – and might find SOCE more relevant to their needs than those who would identify as exclusively gay or lesbian.

Basing one’s identity around a particular type of issue might be counterproductive for some, even if it provided self-justification or social justification for others. As an extreme example, believing in a flat earth might provide a person with an identity of sorts and a community of fellow believers, but is basing an identity on an incorrect belief helpful in the long run? An identity might inhibit a person’s perception of their ability to change when presented with more accurate facts. There is some evidence that a stronger identity as gay or lesbian may be associated with SOCE being ineffective, even counterproductive ( Karten & Wade, 2010). The data here weakly suggested the same outcome, with a stronger LGB identity predicting less chance of change in attraction or behavior. My hypothesis here is that encouraging persons to assume nearly any identity (gay, Muslim, Christian, transgender, evangelical, tea party, professional, etc.) may be motivated for sociological or political purposes as much or more than for the personal or psychological benefit of those persons. For example, if a person is same-sex attracted but does not want to identify as gay/lesbian, what right would I have to force him (her) to do so or to try to prohibit him (her) from doing so? Should it not be his (her) choice on how to identify? For an SSA person to identify accordingly, may not necessarily be in their best interest ( Pela & Sutton, 2021, p. 63; Diamond, 2008). While I think it’s fair to discuss the possible or likely advantages and disadvantages of any given identification, I don’t think it would be right for me to impose my desires about the other person’s potential identifications on them, even if it would make me feel better if they identified in a way I might prefer. Moreover, I am not sure the other person should assume a right to demand that I accept their identification, in part because many of the labels we might attach to ourselves can change over short or long times for a variety of both anticipated and unanticipated reasons.

The second elephant is something both sides in the debate over SOCE seem to overlook. Lesbian, gay, and bisexual behavior and identity can have many positive advantages, especially from a near-term perspective ( Schumm, 2018, pp. 31-46; Schumm, 2020). As one gay youth said, “We have all the fun!” (Russell, 2003, p. 1253) or as a famous actress, Cynthia Nixon, said, “I’ve been straight and I’ve been gay, and gay is better” ( Schumm, 2018, p. 39, citing from Diamond & Rosky, 2016, p. 382). A recent study found that lesbian and gay persons, especially those married, had higher socioeconomic status than heterosexual persons ( Elwood et al., 2020), at least in California. In my view, these advantages should be openly discussed in SOCE; it may well be that some SOCE subjects may realize that they enjoy and want the advantages of a nonheterosexual lifestyle and should feel free to continue or move in that direction, without apology. Given such numerous advantages, LGB persons may experience guilt from being “over benefitted”, a prediction from social exchange theory, guilt that might account for internalized homophobia or lowered psychosocial health rather than or in addition to sexual minority stress.

It may also be that the difficulty found in SOCE is due in part to a failure to recognize and consider such advantages, as well as any disadvantages. Even if one believes that SSA is genetic or inborn, it might still be possible that the many advantages of being lesbian, gay, or bisexual work to reinforce those identities and behaviors over time. The advantages might lead a person to become LGB even without SSA as was found for some of the children of lesbian mothers, as detailed elsewhere ( Schumm, 2004). In other words, beneficial experiences may reinforce attraction, behavior, and identity. At the same time, there may be some persons who feel that some negatives, perhaps especially long-term risks, of a gay, lesbian, or bisexual lifestyle might outweigh the positives. In general, my view is that when a person weighs short-term versus long-term advantages and disadvantages of a decision, change will be more difficult when the former have many positives and the latter (whether advantages or disadvantages) may not accrue for years, even decades. Even if change of sexual orientation were not possible, such subjects may find professional assistance helpful in evaluating their situation and understanding their own (accurate or inaccurate) perceptions of advantages and disadvantages, from both a short-term and a long-term perspective.

 Asymmetric standards

One possible interpretation of this research is that because SOCE did not lead to all participants attaining a completely heterosexual outcome in terms of attraction, behavior, or identity it must have failed. Such a conclusion may reveal implicit bias in terms of standards used for evaluating “success” in therapy. Using some raw data drawn up in only a few minutes, let’s suppose that a therapist was running a marriage therapy program, using the scores from one spouse to assess change. The data used here used five 1’s, four 2’s, four 3’s, four 4’s, three 5’s, and four 6’s at pre-test, with the following sets of scores for pretest 1’s (7, 2, 3, 4, 5), 2’s (1, 1, 5, 5), 3’s (6, 6, 5, 4), 4’s (5, 3, 5, 6), 5’s (6, 5, 6), and 6’s (7, 6, 6, 7), so anyone is welcome to replicate the analyses. There were 24 spouses assessed at pre-test and post-test on an item whose score ranged between 1 and 7, with higher scores indicating greater marital satisfaction. With the data used, the pre-test mean was 3.33 (SD = 1.79) and the post-test mean was 4.83 (SD = 1.74). The mean difference was 1.50 and the standard deviation of the difference was 1.69. Depending on which website calculator is used, Cohen’s d was between 0.84 and 0.89, a large effect size. The results were very significant, t(23) = 4.23 ( p < .001). Using a Wilcoxin signed-ranks test, z = 3.45, p = .001, so the results would be similar using either parametric or nonparametric statistics. It is likely that most therapists would consider the results impressive, both substantial in effect size and very significant statistically.

However, SOCE critics could argue that in the raw data only one spouse changed from a 1 to a 7 and only three ended up at a 7 while three scored lower at post-test and three more were unchanged at post-test with many (n = 11) changing by only one or two points in a positive direction (so that the majority of the clients (n = 14) either did not change at all or only changed a “little”. One might claim that of the 24 clients, seven were divorced during or after the program, which might be taken as failure, harm, or success ( Moxley et al., 1987). Both explanations of the results are technically correct.

While most scientists would present the first set of results and claim “success”, SOCE critics are more likely to take issue with the results by focusing on the second set of results, assuming the outcome measured was sexual orientation (i.e., few (n = 3) clients became completely heterosexual (and most - two of three - of them started as “mostly” heterosexual so their change was small), only one changed from completely gay to completely heterosexual, and most remained more or less bisexual (i.e., started out as bisexual and ended up as bisexual), while three became “more” gay. SOCE critics would probably conclude from the data that SOCE was not effective, despite the “impressive” first set of results. Furthermore, SOCE critics could argue that the program was “harmful” because some clients got “worse”, some did not change at all, and for all the time and expense lost to the participants, a majority got worse or got little benefit from the program. Perhaps marital therapy should be banned, given such poor results! It is also interesting that recent research has found a number of interventions (other than SOCE) to be ineffective ( Williams et al., 2020), even more harmful than effective, and yet we are unaware of calls for their termination by major professional organizations, or at least not with the same fervor as for SOCE.

We believe that the same standards should be used for SOCE as for other types of therapeutic interventions rather than carving out a special set of standards for SOCE not used elsewhere for evaluating therapeutic interventions. In other words, we do not think it’s logically coherent to apply different standards statistically just because the outcome measure is different. In other words, SOCE critics are apt to use a double standard or special pleading when evaluating SOCE results.

 Conclusions

While the results here confirm those reported by Sullins et al. (2021), they also add to our knowledge about SOCE. For most of the subjects in this study, SOCE appeared to reduce same-sex attractions, identity, and behavior, while SOCE seemed less effective at increasing heterosexual behaviors. A small minority of some subjects did not appear to find SOCE helpful, but most subjects reported greater helpfulness of SOCE to the extent that it did help them reduce same-sex attractions, identity, and behavior. Remarkably, some subjects reported SOCE as very helpful even when their same-sex behaviors, identity, and attractions did not change or when they increased over time. Thus, SOCE may be capable of affirming even increases in same-sex behaviors, identity, and attractions when that is the overall outcome. Harms from SOCE seem to be minimal compared to the positives reported for young adults, reporting on their SOCE experience. SOCE effectiveness did not appear to change with time since therapy, lending less support to a recall bias argument. Congruence between SSA and SSI, may, in some cases, reduce the apparent effectiveness of SOCE, although our limited findings may not replicate. Since the findings here differ from many contemporary assertions that SOCE cannot be effective under any circumstances and is inherently harmful (especially for youth), or that SSA is inherently immutable. In essence, recent criticisms of SOCE ( Haldeman, 2022) are beating a dead horse, criticizing former aversive methods condemned even by those researching current “talk” versions of SOCE. In other words, much of the criticism, even condemnation, of older versions of SOCE are irrelevant to current versions of SOCE. Furthermore, even the best therapies can lead to harm to some clients and therapies that have substantial and statistically significant effect sizes may not be successful for many of their clients. Therefore, calls to ban SOCE legally appear to be founded upon incomplete or inaccurate data, as well as biased standards for therapeutic success, and thus are very premature. Future research needs to be done more carefully ( Rosik, 2020a, b; Sprigg, 2021) and focus on current versions of SOCE rather than memories of versions of SOCE that may date back a few decades.

 Data availability

A complete data availability statement may be found in Sullins et al. (2021).

 Ethics

As noted in Sullins et al. (2021, p. 4), “The original study and protocols were approved by the Southern California Seminary Institutional Review Board. Written informed consent was obtained from all study subjects prior to participation ( Santero, 2011, p. 154). As a secondary analysis of pre-existing data, the Catholic University of America Institutional Review Board has certified this study as exempt from human subject ethical review under 45 CFR 46.101.” On 25 April 2022, the Committee on Research Involving Human Subjects, Kansas State University, proposal number IRB-11175, declared this research activity exempt under the criteria set forth in the Federal Policy for the Protection of Human Subjects, 45 CFR 104(d), category: Exempt Category 4 Subsection 1, signed electronically by Chair, Dr. Rick Scheidt, 26 April 2022.

 Acknowledgments

Special gratitude is extended to Dr. Paul Sullins, Paul Santero, and Christopher Rosik for making the data available for independent analysis; without their prior work with this data and its initial collection, this secondary analysis would have been impossible.

 Appendix A.

Items/Scales Mean Median Standard Deviation Minimum Maximum Number of non-missing cases
Homosexual sex before SOCE 2.40 2.00 1.57 1 5 124
Homosexual passionate kissing before SOCE 1.78 1.00 1.20 1 5 125
Homosexual daydreaming before SOCE 4.54 5.00 0.86 1 5 125
Pre-gay scale3 (sum of previous three items) 8.73 7.50 2.90 3 15 124
Desiring homosexual intimacy before SOCE 4.00 5.00 1.42 1 5 124
Pre-gay scale4 (sum of previous four items) 12.74 12.00 3.44 4 20 123
Heterosexual sex before SOCE 1.69 1.00 1.17 1 4 123
Heterosexual passionate kissing before SOCE 1.81 1.00 1.22 1 5 122
Heterosexual daydreaming before SOCE 1.79 1.00 1.13 1 5 123
Pre-het scale3 (sum of previous three items) 5.26 4.00 2.88 3 14 122
Desiring heterosexual intimacy before SOCE 2.45 2.00 1.47 1 5 122
Pre-het scale4 (sum of previous four items) 7.63 7.00 3.75 4 17 121
Sexual attraction before SOCE 5.74 6.00 1.10 2 7 125
Homosexual sex currently 1.49 1.00 1.02 1 5 124
Homosexual passionate kissing currently 1.37 1.00 0.92 1 5 123
Post-gay3 (sum of previous three items) 6.04 6.00 2.61 3 13 123
Homosexual daydreaming currently 3.16 4.00 1.38 1 5 124
Post-gay4 (sum of previous four items) 9.01 9.00 3.70 4 18 123
Desiring homosexual intimacy currently 2.95 3.00 1.46 1 5 125
Heterosexual sex currently 1.97 1.00 1.36 1 5 123
Heterosexual passionate kissing currently 2.19 1.00 1.48 1 5 122
Heterosexual daydreaming currently 2.80 3.00 1.43 1 5 123
Post-het3 (sum of previous three items) 6.98 6.00 3.58 3 15 122
Desiring heterosexual intimacy currently 3.41 4.00 1.54 1 5 123
Post-het4 (sum of previous four items) 10.40 10.00 4.66 4 19 122
Current sexual attraction 4.14 4.00 1.76 1 7 124
Sexual identity before SOCE 4.80 5.00 1.96 1 7 125
Current sexual identity 3.58 3.00 1.96 1 7 125
Maximum sessions for longest therapy, ratio measure 131.88 100.00 88.49 0 250 125
Maximum sessions for longest therapy, ordinal measure 4.12 4.00 1.55 0 6 125
Helpfulness of most helpful type of SOCE 4.24 5.00 1.07 1 5 123
Positive changes: self-esteem 3.69 4.00 1.22 0 5 124
Positive changes: depression 3.02 3.00 1.38 0 5 109
Positive changes: self-harm 2.91 3.00 1.54 0 5 80
Positive changes: suicidality 2.62 2.00 1.65 0 5 78
Positive changes: social functioning 3.37 4.00 1.19 0 5 122
Positive changes: drug abuse 2.45 1.50 1.76 0 5 42
Negative changes: self-esteem 1.30 1.00 0.63 0 4 117
Negative changes: depression 1.59 1.00 0.93 0 5 112
Negative changes: self-harm 1.27 1.00 0.77 0 5 92
Negative changes: suicidality 1.34 1.00 0.78 0 5 95
Negative changes: social functioning 1.36 1.00 0.79 0 4 113
Negative changes: drug abuse 1.12 1.00 0.51 0 4 67
Currently in SOCE 1.57 2.00 0.51 0 2 125
How long ago did you stop SOCE? 3.26 3.00 1.29 1 5 69
Age 2.99 3.00 1.34 1 6 125
How many children? 1.24 0.00 1.66 0 5 125
Annual household income 3.90 4.00 1.86 0 7 125
Highest level of education 5.10 5.00 1.14 0 7 125
Attending religious services 4.22 4.00 0.94 1 6 125
SSA/SSI congruence before SOCE 0.94 0.00 1.74 -4 6 125
SSA/SSI congruence after SOCE 0.54 0.00 1.07 -1 4 124
References American Psychological Association: APA resolution on sexual orientation change efforts. Washington, DC: Author;1994. February 2021. Reference Source Bondy G : Subjective experiences in sexual orientation change efforts: a mixed-method analysis. J. Hum. Sex. 2021;12:89–122. Bradshaw K Dehlin JP Crowell KA : Sexual orientation change efforts through psychotherapy for LGBQ individuals affiliated with the Church of Jesus Christ of Latter-Day Saints. J. Sex Marital Ther. 2015;41(4):391–412. 24814980 10.1080/0092623X.2014.915907 Cohen J : A power primer. Psychol. Bull. 1992;112:155–159. 19565683 10.1037/0033-2909.112.1.155 Del Rio-Gonzalez AM Zea MC Florez-Donado J : Sexual orientation and gender identity change efforts and suicide morbidity among sexual and gender minority adults in Colombia. LGBT Health. 2021;8(7):463–472. 34242517 10.1089/lgbt.2020.0490 Diamond LM : Sexual fluidity: understanding women’s love and desire. Cambridge, MA: Harvard University Press;2008. Diamond LM Rosky CJ : Scrutinizing immutability: research on sexual orientation and US legal advocacy for sexual minorities. J. Sex. Res. 2016;53(4-5):363–391. Drescher J : Foreword. The case against conversion “therapy”: evidence, ethics, and alternatives. Haldeman DC , editor. Washington, DC: The American Psychological Association;2022. Elwood WN Irvin VL Liu B : Health-related influences of extending marital benefits to same-sex couples: results from The California Health Interview Survey. Fam. Relat. 2020;69:934–943. 10.1111/fare.12507 Freedman E : Critique of the Report of the American Psychological Association Task Force on Appropriate Therapeutic Responses to Sexual Orientation (2009). J. Hum. Sex. 2020;11:34–59. Funder DC: Ozer DJ : Evaluating effect size in psychological research: Sense and nonsense. Adv. Methods Pract. Psychol. Sci. 2019;2(2):156–168. 10.1177/2515245919847202 Glassgold JM : Research on sexual orientation change efforts: a summary. Haldeman DC , editor The case against conversion “therapy”: evidence, ethics, and alternatives. Washington, DC: The American Psychological Association;2022; (pp.19–50) Goodyear T Kinitz DJ Dromer E : “They want you to kill your inner queer but somehow leave the human alive”: Delineating the impacts of sexual orientation and gender identity and expression change efforts. J. Sex Res. 2021;1–11. online advance. 33871297 10.1080/00224499.2021.1910616 Haldeman DC : The case against conversion “therapy”: evidence, ethics, and alternatives. Washington, DC: American Psychological Association;2022. Jones SL Yarhouse MA : A longitudinal study of attempted religiously mediated sexual orientation change. J. Sex Marital Ther. 2011;37:404–427. 21961446 10.1080/0092623X.2011.607052 Karten EY Wade JC : Sexual orientation change efforts in men: a client perspective. J. Mens Stud. 2010;18(1):84–102. 10.3149/jms.1801.84 Kinitz DJ Salway T Dromer E : The scope and nature of sexual orientation and gender identity and expression change efforts: a systematic review protocol. Syst. Rev. 2021;10(1):1–8. Moxley V Eggeman K Schumm WR : An evaluation of the “Recovery of Hope” program. J. Divorce. 1987;10(1-2):241–261. 10.1300/J279v10n01_17 Pela C Sutton PM : Sexual attraction fluidity and well-being in men: a therapeutic outcome study. J. Hum. Sex. 2021;12:61–86. Pellicane MJ Ciesla JA : Temporal trends in rates of depression, anxiety, and suicidality among cisgender sexual minority and heterosexual college students. Psychol. Sex. Orientat. Gend. Divers. 2022. online advance. Przeworski A Peterson E Piedra A : A systematic review of the efficacy, harmful effects, and ethical issues related to sexual orientation change efforts. Clin. Psychol. Sci. Pract. 2021;28(1):81–100. Rosik CH : The creation and inflation of prevalence statistics: the case of “conversion therapy.”. J. Hum. Sex. 2020a;11:66–86. Rosik CH : A critical review of 2020 research on harms from efforts to change sexual attractions and behaviors: minimal advancement of science, maximal advancement of agendas. J. Hum. Sex. 2020b;11:87–108. Rosik CH Lefevor GT Beckstead AL : Sexual minorities who reject an LGB identity: who are they and why does it matter?. Issues Law Med. 2021;36(1):27–43. 33939341 Rosik CH Popper P : Clinical approaches to conflicts between religious values and same-sex attractions: contrasting gay-affirmative, sexual identity, and change-oriented models of therapy. Couns. Values. 2014;59:222–237. 10.1002/j.2161-007X.2014.00053.x Ryan C Toomey RB Diaz RM : Parent-initiated sexual orientation change efforts with LGBT adolescents: Implications for young adult mental health and adjustment. J. Homosex. 2020;67(2):159–173. 30403564 10.1080/00918369.2018.1538407 Salway T Ferlatte O Gesink D : Prevalence of exposure to sexual orientation change efforts and associated sociodemographic characteristics and psychosocial health outcomes among Canadian sexual minority men. Can. J. Psychiatry. 2020;65(7):502–509. 31984758 10.1177/0706743720902629 Santero P : Change Effects in U.S. Men with Unwanted Same Sex Attraction after Therapy. [ PsyD Dissertation]. Southern California Seminary. 2011. Schumm WR : What was really learned from Tasker and Golombok’s (1995) study of the children of lesbian and single parent mothers?. Psychol. Rep. 2004;94:422–424. 15154166 10.2466/pr0.94.2.422-424 Schumm WR : Navigating treacherous waters – one researcher’s 40 years of experience with controversial scientific research. Comprehensive. Psychology. 2015;4(4):17.CP.4.24. 1-40. 10.2466/17.CP.4.24 Schumm WR : Same-sex parenting research: A critical assessment. London, UK: Wilberforce;2018. Schumm WR : Avenues for future LGBT theory and research. JSM J. Sex. Med. 2020;4(2):1–7. Schumm WR Dugan M Nauman W : Using free websites to perform statistical calculations in basic statistics courses at high school or college levels. Sunkrist Sociology and Research Journal. 2021;2(1):1009. 1-7. Schumm WR Denton W : Trends in premarital counseling. J. Marital Fam. Ther. 1979;5(4):23–32. 10.1111/j.1752-0606.1979.tb01278.x Sprigg PS : Searching for evidence of harm: 79 key studies do not demonstrate that sexual orientations change efforts (SOCE) are more harmful than other counseling. J. Hum. Sex. 2021;12:6–40. Sullins DP : Absence of behavioral harm following non-efficacious sexual orientation change efforts: A retrospective study of United States sexual minority adults, 2016-2018. Front. Psychol. 2022;13(823647):1–12. 10.3389/fpsyg.2022.823647 Sullins DP Rosik CH Santero P : Efficacy and risk of sexual orientation change efforts: a retrospective Analysis of 125 exposed men [version 2; peer review: 2 approved]. F1000Res. 2021;10:222. 33968367 10.12688/f1000research.51209.2 Sutton PM : Serving persons with (unwanted) same-sex attraction and behavior (SSA) from the Roman Catholic tradition. Journal of Human Sexuality. 2019;10:17–52. VanVoorhis CRW Morgan BL : Understanding power and rules of thumb for determining sample sizes. Tutor. Quant. Methods Psychol. 2007;3(2):43–50. 10.20982/tqmp.03.2.p043 Whitehead N Whitehead B : My genes made me do it. Lafayette, LA: Huntington House Publishers;1999. Williams AJ Botanov Y Kilshaw RE ; Potentially harmful therapies: a meta-scientific review of evidential value. Clin. Psychol. Sci. Pract. 2020;28(1):5. 1-16. 10.5256/f1000research.120192.r200847 Reviewer response for version 1 Lefevor G Tyler 1 Referee https://orcid.org/0000-0003-4510-7306 Utah State University, Utah State University, Logan, Utah, USA

Competing interests: No competing interests were disclosed.

1 9 2023 Copyright: © 2023 Lefevor GT 2023 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation reject

I am very concerned about the adequacy of this paper as a scientific endeavor. Below I detail two major concerns and several minor concerns.

Major concern #1) I am concerned that the premise of this piece is statistically unsound. The authors write, “Our analyses used parametric statistics, but the assumption of normality was violated for most of our variables.” If assumptions of normality are violated, then nonparametric statistics should have been used. As such, the authors’ findings rest on an entirely unfounded statistical foundation.

Major concern #2) The authors routinely generalize past what their data allows them to say. The sample they used differs from the general population in many ways, including being entirely cisgender men, being mostly religious, and being recruited because of its interest in pursuing SOCE. The authors final statement in their abstract, “Therefore, calls to ban SOCE legally appear to be founded upon incomplete or inaccurate data and thus premature, while more higher quality research is yet needed regarding SOCE,” reaches as broad as possible (to all Americans); however, the authors do not have data to support this assertion. This article cannot be indexed with such a politically motivated interpretation of the authors’ findings that goes beyond the data in the abstract.

Method:

The authors should redescribe the sampling procedure so that the reader can understand the population from which the current sample was taken. This population is going to introduce substantial interpretive constraints on this paper.

Generally, obtaining a Cronbach’s alpha that is lower than .7 is seen as grounds for not using a scale in scientific research. I would encourage the authors to not use the “prior same-sex sexuality” scale for this reason. With an alpha as low as .58 it is unclear what is being measured here.

I am concerned about the helpfulness measure. Because participants were able to select in or out of the measure, it is unclear how to understand the participants who did not respond to one (or more) of the helpfulness questions. As such, it does not make conceptual sense to create a mean “helpfulness” score because participants likely only responded to items that they perceived as helpful. Better analyses might look at the items that participants saw as helpful versus those not responded to.

What does a median split look like on a dichotomous variable (sexual identity)? Also, please provide the general descriptive statistics for sexual attraction, behavior, and identity. Does “low attraction” actually mean a substantial degree of same-gender attraction?

Results:

It appears that you are measuring same-sex identity on a continuous scale? This seems unlikely, but I am otherwise confused how this variable was assessed with a t-test.

You did not provide rationale for your median split, which makes it harder to interpret results that use it. I am not confident that these groups are substantially different or how to interpret what low vs. high is.

What was the relationship between sexual attraction, behavior, and identity? My guess is that on your third question, you are running into some multicollinearity problems and should perhaps only include one indicator of sexual orientation. This would explain why same-sex identity and behavior had different signs.

Is the work clearly and accurately presented and does it cite the current literature?

No

If applicable, is the statistical analysis and its interpretation appropriate?

No

Are all the source data underlying the results available to ensure full reproducibility?

No source data required

Is the study design appropriate and is the work technically sound?

No

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

LGBTQ+ health, religion/spirituality

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

10.5256/f1000research.120192.r139899 Reviewer response for version 1 Forsythe Anna 1 Referee Value and Access, Cytel Inc., Waltham, MA, USA

Competing interests: No competing interests were disclosed.

 30 6 2022 Copyright: © 2022 Forsythe A 2022 This is an open access peer review report distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. recommendation reject

Overall, this is a 3 rd re-analysis of a survey conducted in 2011 among a sample of 125 highly religious adult males.

The author did not provide any information on the survey development or validation, putting into question the validity of the results – measuring relevant concepts or accurately detecting change. The reliability of the results of the analyses must be considered in light of the very small samples (mostly <30 patients). The results of the analyses should be interpreted with extreme caution and cannot be used to derive the conclusions that the author has reached. Considering the many methodological flaws of this paper and the outdated timing of data collection, it is not clear how this research can add any value the scientific community. I would not recommend indexing.

The study was conducted in a convenience sample of highly religious, predominantly white, largely college educated, and generally economically secure adult men who freely consented to undergo SOCE. Importantly, based on this selection bias if this paper is meant to inform the public discussion on policies regarding SOCE, then it is essential to highlight the multitude of ways in which the study population does not represent those who are the focus of bans and proposed bans on SOCE: minors under the age of 18 from all socioeconomic, ethnic, and racial groups without regard for religious affiliation and who may be coerced by families and faith groups to undergo SOCE. As such, any attempts to broaden the implications of the analyses beyond the specific population initially surveyed are merely opinions of the author and not substantiated by the research in the paper.

The original study referenced in this article demonstrated that in the majority (55%) of the original sample, no change (complete or partial) in sexual behavior followed SOCE. Most changes were reported with magnitude of change ranging from a quarter to ½ standard deviation of the original values, without any evidence of validation whether such change is meaningful. Neither the original study nor the current re-analysis establish a minimally important difference (MID) for the reported outcomes.

This study is the third analysis of the original survey conducted in 2011. Despite the availability of multiple, larger, more methodologically robust studies with more diverse and representative populations, the author chose to return to a >10-year-old sample. Also notable is that while still a largely heteronormative dominated culture, significant social change has occurred as evident with the recognition of marriage equality, awareness of gender identity and dysphoria. Considering the rapidly evolving social and cultural landscape regarding LGBTQ issues, and the availability of newer data on SOCE, it is questionable whether another re-analysis of a limited, biased small-sample-size, older study merits publication.

The selection of recent publications referenced in this study appears to be incomplete, as it does not include several important, recently published papers, most covering much larger and more representative/generalizable populations:

Blosnich et al. (2020 1 ).

Turban et al. (2020 2 ).

UCLA School of Law Williams Institute. Conversion therapy and LGBT youth. Published June 2019. Accessed March 1, 2021.  https://williamsinstitute.law.ucla.edu/publications/conversion-therapy-and-lgbt-youth/

The Trevor Project. The Trevor Project National Survey on LGBTQ Youth Mental Health 2020. Published 2020. Accessed March 1, 2021.  https://www.thetrevorproject.org/wp-content/uploads/2020/07/The-Trevor-Project-National-Survey-Results-2020.pdf

The 77-question survey (originally used in the 2011 study and re-analyzed for the second time in this manuscript) was not a methodologically robust survey: it did not follow FDA Patient-Reported Outcomes guidance, select patient-relevant concepts, or validate the questions to evaluate if a meaningful change could be measured. Conducting additional statistical analyses on an exploratory, unvalidated questionnaire does not make the original results any more reliable. The author should add information on the creation and validation of the original questionnaire, including the methodological limitations.

Tables 2 through 7, as well as analysis to support question 10, report statistical analyses based on very small sample size (all <30 patients). The study was not prospectively powered to find meaningful results in these small samples, and thus the results can only be considered non-significant trends.

Since the time of the survey, gay marriage has become legal in all 50 states. Thus, the assumptions behind the fourth question (marital status, with marriage being defined as “between a man and a woman”) are no longer valid or generalizable. A key limitation of the research is that individuals now have the opportunity to be in a legally and socially recognized homosexual marriage, which may impact their willingness to undergo—and their experience of—SOCE. 

The author understates the level of concern expressed by the international medical community regarding SOCE. The introduction and discussions sections of this paper should explicitly state that the issue goes beyond one of scholarly controversy: the American Academy of Pediatrics, the American Psychological Association, the American Academy of Child and Adolescent Psychiatry, the American Medical Association, the American Counseling Association, the American Psychiatric Association, and the United Nations Human Rights Office of the High Commissioner, among other medical, mental health, and human rights organizations, formally oppose SOCE.

Is the work clearly and accurately presented and does it cite the current literature?

No

If applicable, is the statistical analysis and its interpretation appropriate?

No

Are all the source data underlying the results available to ensure full reproducibility?

Partly

Is the study design appropriate and is the work technically sound?

No

Are the conclusions drawn adequately supported by the results?

No

Are sufficient details of methods and analysis provided to allow replication by others?

No

Reviewer Expertise:

Systematic Literature review, Meta-analysis, Humanistic and Economic burden of disease, Health Economic analyses of interventions.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.

References : Sexual Orientation Change Efforts, Adverse Childhood Experiences, and Suicide Ideation and Attempt Among Sexual Minority Adults, United States, 2016-2018. Am J Public Health .2020; 10.2105/AJPH.2020.305637 e1-e7 32437277 10.2105/AJPH.2020.305637 : Association Between Recalled Exposure to Gender Identity Conversion Efforts and Psychological Distress and Suicide Attempts Among Transgender Adults. JAMA Psychiatry .2020;77(1) : 10.1001/jamapsychiatry.2019.2285 10.1001/jamapsychiatry.2019.2285