Blinding in trials of interventional procedures is possible and worthwhile

Karolina Wartolowska; David Beard; Andrew Carr

doi:10.12688/f1000research.12528.2

Home Browse Blinding in trials of interventional procedures is possible and worthwhile

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Review

Revised

Blinding in trials of interventional procedures is possible and worthwhile

[version 2; peer review: 3 approved]

Karolina Wartolowska^1,2, David Beard², Andrew Carr²

PUBLISHED 30 Jan 2018

Author details Author details

¹ Nuffield Department of Primary Health Care Sciences, Radcliffe Observatory Quarter, Woodstock Road, Oxford, OX2 6GG, UK
² Botnar Research Centre, Nuffield Department of Orthopaedics Rheumatology and Musculoskeletal Sciences (NDORMS), Windmill Road, Oxford, OX3 7LD, UK

Karolina Wartolowska
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

David Beard
Roles: Writing – Review & Editing

Andrew Carr
Roles: Funding Acquisition, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

In this paper, we use evidence from our earlier review of surgical randomised controlled trials with a placebo arm to show that blinding in trials of interventional procedures is feasible. We give examples of ingenious strategies that have been used to simulate the active procedure and to make the placebo control indistinguishable from the active treatment. We discuss why it is important to blind of patients, assessors, and caregivers and what types of bias that may occur in interventional trials. Finally, we describe the benefits of blinding, from the obvious ones such as avoiding bias, as well as less evident benefits such as avoiding patient drop out in the control arm.

Keywords

blinding, RCT, randomised controlled trials, placebo, surgery, procedures, non-pharmacological

Corresponding author: Karolina Wartolowska

Competing interests: No competing interests were disclosed.

Grant information: This work has received funding from the NIHR Oxford Musculoskeletal Biomedical Research Unit.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2018 Wartolowska K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Wartolowska K, Beard D and Carr A. Blinding in trials of interventional procedures is possible and worthwhile [version 2; peer review: 3 approved]. F1000Research 2018, 6:1663 (https://doi.org/10.12688/f1000research.12528.2) First published: 08 Sep 2017, 6:1663 (https://doi.org/10.12688/f1000research.12528.1) Latest published: 30 Jan 2018, 6:1663 (https://doi.org/10.12688/f1000research.12528.2)

Revised Amendments from Version 1

The differences between these two versions consist of added definitions and references - as requested by the Reviewers, and some minor edits of spelling, grammar, and punctuation that were missed on the original submission.

See the authors' detailed response to the review by Helene Moustgaard and Asbjørn Hróbjartsson
See the authors' detailed response to the review by Harald Walach
See the authors' detailed response to the review by Ian A Harris

Introduction

The aim of a trial is to produce unbiased evidence. As randomised controlled trials (RCTs) with a placebo arm control for many types of bias and have high internal validity they are regarded as a reliable method of demonstrating treatment efficacy^1,2. RCTs of interventional procedures are rare^3–6, partly because they are challenging⁷; however, they are not impossible to perform, even if they involve a placebo arm⁸. In this paper we discuss why trials should be blinded and summarise methods that have been used to achieve blinding in the published placebo-controlled trials of interventional procedures.

Blinding in interventional trials is often necessary because, nowadays, many procedures are performed to reduce pain and improve function, and quality of life. Pain, function and quality of life are sometimes regarded as preferable outcome measures because they reflect patients’ needs and point of view⁹. However, as these outcomes depend on patients’ subjective perception, they are prone to bias and may lead to an exaggerated treatment effect in open-label trials^10–13. Using subjective outcomes in an open-label study undermines its internal validity because it makes it impossible to determine how much of the reported effect is related to the investigated treatment and how much is related to various forms of bias.

It is important to note that controlling for bias comes at a price. Because of the standardised conditions under which they are performed, the uncertainty of treatment allocation, and the presence of assessors, RCTs differ from everyday patient care; therefore, they often have low external validity¹⁴. Moreover, the size of the effect in blinded trials tends to be smaller than in clinical practice because of the inherent uncertainty of the treatment allocation in a trial¹⁵.

Definitions

In this paper, we have defined surgery like in our previous publication⁸ as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”⁸. Also the type of outcomes was defined in the same way as in our earlier paper¹⁶ i.e., “Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”

Blinding of patients, surgeons, outcome assessors, and care-givers

Blinding of patients

Blinding means concealing the treatment allocation from patients and any other people involved in the trial who may bias the results of the trial by knowing which groups the patients were randomised to.

Blinding of patients prevents reporting bias in patient-reported measures. For example, it has been demonstrated that non-blinded patients may exaggerate the effects size by 0.56 standard deviations and that the effect is even larger in studies on interventional procedures, such as acupuncture^17,18. This bias may be caused by patients’ expectations of treatment effect and information given to them before the treatment¹⁹. Patients may also report symptoms depending on their “hunches” about treatment being effective or they may give answers they believe are “correct” or expected from them, for example, because it would have been impolite not to report improvement¹⁷. Therefore, it has been suggested that patients should be blinded whenever possible¹⁷.

Blinding of patients also reduces adherence bias, i.e. patients in the control group not following the protocol. It may also prevent so called “contamination of the control group”, i.e. seeking additional treatment outside the trial and receiving concomitant treatment. Blinding also improves patient retention in the trial. Risk of attrition in blinded trials is about 4% whereas in non-blinded trials it is 7%¹⁷. Specifically, in placebo-controlled surgical trials, subject retention is often reported as “excellent”²⁰, and in our analysis the withdrawal rate was low (4%) and comparable between the treatment and the placebo arm²¹.

Blinding of surgeons

Unlike drug trials, in which the physician gives a tablet prepared somewhere else, the surgeon has to perform a specific procedure considered to be therapeutic; therefore, blinding of surgeons may not always be possible.

There have been attempts to blind surgeons, for example, a surgeon inserted a catheter under fluoroscopic guidance and handed over the procedure to a technician who delivered the radiofrequency energy (or not) according to the allocation²². In other trials, a palatal implant delivery system was prepared by the manufacturer to either contain the implant or not, which allowed for blinding of surgeons^23,24.

Blinding of outcome assessors

In 81% of placebo-controlled surgical trials both patients and assessors were blinded²¹. It has been demonstrated that, non-blinded assessors of subjective outcomes cause less bias in trials than non-blinded patients reporting their symptoms²⁵. Blinding of assessors prevents observer-related bias, detection bias, and the Pygmalion effect. The Pygmalion effect, also called the Rosenthal effect, refers to a situation when investigators looking for a particular response are predisposed to interpret the result in a way that shows the response they expect, even if it is objectively absent²⁶. Interestingly, a study by Hróbjartsson and colleagues demonstrated that non-blinded assessors were over-optimistic and “over-rated” patients in the treatment group rather than “under-rated” patients in the control group²⁷. To minimise the assessor-related bias, in some trials the assessment was done by people not involved in the surgery, for example, by staff at another hospital²⁸, or by a pathologist blinded to the treatment allocation²⁹.

Blinding of care-givers

Apart from blinding patients and assessors, it is important that care-givers and clinical or research staff also do not know patient treatment allocation, because their behaviour and attitudes may influence patient responses^30–32. Patient-clinician interaction plays an important role in treatment response, and patients in trials do better as they get more attention and time from clinical staff than patients receiving standard care^33,34. Therefore, the interactions between patients and the trial team should be standardised so that the “treatment context” (similar attention from doctors, expectations, and settings) is comparable between the groups.

Strategies used to maintain blinding in interventional placebo-controlled trials

A placebo-controlled RCT is a special type of a trial, in which one of the control arms involves an imitation procedure that seems identical but does not involve the crucial element believed to be "the cure". A placebo arm is necessary to demonstrate that the observed improvement is really caused by the investigated procedure as it controls for the effects of receiving treatment other than the crucial surgical element.

It is often difficult to determine what is a specific and what is a non-specific effect in a trial^34,35, and to disentangle a placebo response from response bias or the effect of patient-doctor interactions³⁶. It is beyond the scope of this review to discuss definitions of placebo^1,36. Whether something is or is not a placebo depends on the intervention and chosen outcome variables¹, but in order for blinding to be successful, the control procedure has to be as similar as possible to the investigated procedure³⁴. Interventional trials differ from drug trials as they require access to the anatomical structure of interest; therefore, they involve a skin incision or an insertion of a scope.

In many published trials, blinding during the surgery was straightforward because patients were under general anaesthesia or heavy sedation and, therefore; unaware of the details of surgical procedures. In such trials, only the surgical wound had to be similar in both groups. Some studies did not add any placebo procedure but simply omitted part of interventional procedure, for example, in the trial by Stone and colleagues, all patients underwent a percutaneous coronary intervention and maximal medical therapy but only patients in the active arm also had percutaneous transmyocardial revascularisation³⁷.

When light sedation or local anaesthesia were used, surgical staff had to simulate the actual intervention to preserve the blinding. The complexity of a surgical procedure made blinding challenging, and ingenious ideas were required to make the real and placebo interventions indistinguishable.

Imitation of incision/ surgical access point

If a procedure requires open surgery, then it leaves an obvious mark where the incision has been made, which has to be imitated in the placebo group. There have been very few trials involving full-skin incision, in both the surgical and placebo arms. In the seminal trials on internal mammary artery ligation^38,39 a skin incision was made to expose the arteries in all patients, but no ligation was made in patients in the placebo group. Similarly, Guyuron and colleagues used a skin incision to expose superficial nerves and muscles, which were cut during the active surgery, but in the placebo group, the integrity of these structures was maintained⁴⁰. Trials investigating transplantation of dopaminergic neurones as a treatment for Parkinson’s disease not only required skin incision but also burr holes in the skull, but in the placebo group the burr holes did not penetrate the dura matter^41,42.

Most of the published placebo-controlled surgical trials used minimally-invasive methods to access the structure of interest. For example, the placebo procedure involved laparoscopy but without ablation⁴³, endoscopy without radiofrequency energy delivery⁴⁴, bronchoscopy without radiofrequency energy delivery⁴⁵, or bronchoscopy without a valve placement⁴⁶. Therefore, most of the studies required a small incision to mimic the portals created during the laparoscopy or arthroscopy, or to mimic the incision through which an intravascular catheter was inserted⁴⁷. Interestingly, Sutton and colleagues used three incisions in both groups, so that patients could not tell apart a diagnostic laparoscopy from a laparoscopic surgery; even though the third instrument port was not necessary in the placebo group⁴⁸. Trials using endoscopy and bronchoscopy were even easier to blind as natural orifices were used to insert the scope, and the incision or actual procedure site was not visible to patients, care-givers, and assessors.

Simulation of an interventional procedure

Typically, the preparation for the placebo and the active procedure was as similar as possible and imitated the visual, auditory, and physical cues^49–52. In order to mimic the sounds, surgeons were required to talk through the procedure steps⁵³, ask for instruments^54,55, use suction⁵⁵ or ask for a laser or other device to be activated, even though it was not used in the placebo group^56–60.

Clinical staff performing the intervention were screened from the patients’ view⁶¹, either by a surgical drape⁵⁹ or by arranging the operating room in a way that the patient could not see the procedure⁵¹. In a trial by Stone and colleagues, patients were heavily sedated and wore opaque goggles³⁷. In a trial by Maurer and colleagues, the manufacturer delivered tools that looked identical but those for the placebo group did not contain an implant, which allowed for blinding of patients and clinical staff²⁴.

Surgeons also attempted to imitate sensory cues, for example, by manipulating the knee as if the actual arthroscopy were performed⁵⁵, injecting saline to imitate tidal irrigation²⁰, or by splashing saline on the knee to simulate lavage⁵². In a trial on meniscectomy, Sihvonen and colleagues used a mechanised shaver (without the blade) pushing it firmly against the patella to simulate the sensations the patient would experience during the surgery⁵⁵. In a trial on intragastric balloon for obesity, operators manipulated the endoscope as during the balloon insertion to create the sensation of resistance in the stomach⁶⁰.

Even smell during the surgery was imitated to make the placebo procedure indistinguishable from surgery. For example, in the trial by Deviere and colleagues there were concerns that patients could have known the allocation because the copolymer used in the active arm had a distinct smell⁶². In trials on vertebroplasty, a container with cement was opened during placebo procedure to help with blinding by imitating the smell^49,63.

Inactive nature of the placebo

It is important that the procedure used for blinding does not have any therapeutic effect. For example, the results of the vertebroplasty trials^43,63 were criticised because the elements of placebo procedure could have had an effect on the reported pain, namely, a potential pharmacological anaesthesia due to injection of an anaesthetic into the facet capsule and periosteum⁶⁴.

On the other hand, the procedure used for blinding may have diagnostic use, as with diagnostic laparoscopy^43,48,65 or diagnostic laparoscopy with biopsy⁶⁶. In the trial by Sihvonen and colleagues, all participants underwent diagnostic arthroscopy, but only after they had been confirmed to be eligible for inclusion in the trial was the envelope with the assignment opened and the assignment revealed to the surgeon⁵⁵.

Duration of the procedure

Many trials specifically stated that the duration of procedure in the surgical and control arms were matched, either by imitating the elements of the surgical procedure or by keeping all patients in the operation room for the same duration of time^{41,45,52,54,55,67,68}. However, in some trials, the placebo procedure was shortened in comparison to the actual surgery because it was believed it would have been ethically unacceptable to prolong the placebo intervention^56,62.

Additional procedures that may reveal allocation

Interventional treatment often requires additional procedures, such as diagnostic scans or medication to prevent infection^50,63,69, blood clots⁴⁷, transplant rejection⁷⁰, or epileptic fits⁴². For example, in the trial by Freed and colleagues, both groups received identical preoperative evaluation, intraoperative sedation and pain control, underwent two PET scans and a MRI scan, and received phenytoin⁴². In some trials, the same medication was given in both groups, whereas in others unnecessary treatment was omitted or imitated, for example, by injecting saline instead of antibiotics⁷¹.

Standardisation of interventions and care

The active and placebo procedure have to be indistinguishable but they also have to be stable and standardised. Standardisation of the procedure itself may be difficult but is important because surgeons vary in their experience and gain experience throughout the trial.

Some of the changes observed in a trial may not be related to the treatment or the placebo intervention, but may be caused by the natural course of the disease, spontaneous remissions or fluctuations in the severity of symptoms or regression to the mean^34,72. Some changes may be a result of just being in a trial either because of lifestyle changes that are part of the protocol such as self-monitoring, using diaries, or avoiding alcohol, or due to so called “Hawthorne effect”, which refers to change in the behaviour when people, both patients and doctors, know that they are being observed³⁴. Finally, it has been demonstrated that adhering to a protocol improves the performance of doctors, and that patients who adhere to treatment regimens have better outcomes⁷³. Therefore, it is important to standardise pre- and post-operative care, and the explanations given to the patients. For example, in a trial by Sihvonen and colleagues, all procedures were standardised and recorded on video; the post-operative care was also standardised, and all patients received the same exercise program and walking aids⁵⁵.

Blinding after the surgery

Most trials blinded the assessors while the surgeon and other staff in the operating room were aware of the group assignment, and did not participate in further treatment, post-operative care or follow-up of the patient^43,55,74. In a trial by Thomsen and colleagues, the post-operative care and assessment was done at a different hospital than the surgery²⁸. In a trial by Cotton and colleagues, the post-operative care was provided by the referring physician, who was blinded when deciding on treatment, and when this was not sufficient, by the evaluating physician at the study site (who was also blinded)⁷⁴.

Bias specific to surgical trials

There are other types of bias that are specific to surgical trials. They are mostly related to patients not being entered into the trial because their symptoms are too severe or not severe enough to justify surgery or because the anatomical conditions or technical difficulties make the surgery impossible to perform. For example, in trials on upper gastrointestinal tract bleeding, the endoscopic procedure was not performed if the rate of blood loss was too fast, or the endoscopy was judged to be life-threatening and posed an unacceptable risk^75–77. This meant that only patients with less severe symptoms were included in the study. In other trials, some patients were excluded because they could not tolerate endoscopy, or due to anatomical conditions that made the surgery impossible to carry out, for example, not being able to aim a laser at the bleeding arteries⁷⁷. Alternatively, some patients were not included in a trial because they were no longer eligible, for example because the bleeding had stopped⁷⁷ or they no longer reported the symptoms on the day of the study⁷⁸. These confounders are difficult to predict and control for.

Conclusions

Blinding in trials of interventional procedures is possible and many creative methods have been used to maintain the blinding. Interventional procedures are challenging to blind, but the effort is worthwhile because of the obvious benefits, such as avoiding bias, as well as the less evident benefits, such as avoiding patient drop-out in the control arm.

Competing interests

No competing interests were disclosed.

Grant information

This work has received funding from the NIHR Oxford Musculoskeletal Biomedical Research Unit.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

F1000 recommended

References

1. Gøtzsche PC: Is there logic in the placebo? Lancet. 1994; 344(8927): 925–6. PubMed Abstract | Publisher Full Text
2. Walach H, Loef M: Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence. J Clin Epidemiol. 2015; 68(11): 1251–60. PubMed Abstract | Publisher Full Text
3. McLeod RS: Issues in surgical randomized controlled trials. World J Surg. 1999; 23(12): 1210–4. PubMed Abstract | Publisher Full Text
4. Wente MN, Seiler CM, Uhl W, et al.: Perspectives of evidence-based surgery. Dig Surg. 2003; 20(4): 263–9. PubMed Abstract | Publisher Full Text
5. Ergina PL, Cook JA, Blazeby JM, et al.: Challenges in evaluating surgical innovation. Lancet. 2009; 374(9695): 1097–104. PubMed Abstract | Publisher Full Text | Free Full Text
6. Cook JA: The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials. 2009; 10: 9. PubMed Abstract | Publisher Full Text | Free Full Text
7. Fairbank J: Randomised controlled trials in surgery. Lancet. 1999; 354(9174): 257. PubMed Abstract | Publisher Full Text
8. Wartolowska K, Judge A, Hopewell S, et al.: Use of placebo controls in the evaluation of surgery: systematic review. BMJ. 2014; 348: g3253. PubMed Abstract | Publisher Full Text | Free Full Text
9. Ware JE: Measuring patients’ views: the optimum outcome measure. BMJ. 1993; 306(6890): 1429–30. PubMed Abstract | Publisher Full Text | Free Full Text
10. Hróbjartsson A, Gøtzsche PC: Is the placebo powerless? Update of a systematic review with 52 new randomized trials comparing placebo with no treatment. J Intern Med. 2004; 256(2): 91–100. PubMed Abstract | Publisher Full Text
11. Wood L, Egger M, Gluud LL, et al.: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008; 336(7644): 601–5. PubMed Abstract | Publisher Full Text | Free Full Text
12. Page MJ, Higgins JP, Clayton G, et al.: Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies. PLoS One. 2016; 11(7): e0159267. PubMed Abstract | Publisher Full Text | Free Full Text
13. Hróbjartsson A, Boutron I: Blinding in randomized clinical trials: Imposed impartiality. Clin Pharmacol Ther. 2011; 90(5): 732–6. PubMed Abstract | Publisher Full Text
14. Deaton A, Cartwright N: Understanding and misunderstanding randomized controlled trials. Soc Sci Med. 2017; pii: S0277-9536(17)30735-9. PubMed Abstract | Publisher Full Text
15. Enck P, Klosterhalfen S, Weimer K, et al.: The placebo response in clinical trials: more questions than answers. Philos Trans R Soc Lond B Biol Sci. 2011; 366(1572): 1889–95. PubMed Abstract | Publisher Full Text | Free Full Text
16. Wartolowska KA, Feakins BG, Collins GS, et al.: The magnitude and temporal changes of response in the placebo arm of surgical randomized controlled trials: a systematic review and meta-analysis. Trials. 2016; 17(1): 589. PubMed Abstract | Publisher Full Text | Free Full Text
17. Hróbjartsson A, Emanuelsson F, Skou Thomsen AS, et al.: Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014; 43(4): 1272–83. PubMed Abstract | Publisher Full Text | Free Full Text
18. Nüesch E, Reichenbach S, Trelle S, et al.: The importance of allocation concealment and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis Rheum. 2009; 61(12): 1633–41. PubMed Abstract | Publisher Full Text
19. Desharnais R, Jobin J, Côté C, et al.: Aerobic exercise and the placebo effect: a controlled study. Psychosom Med. 1993; 55(2): 149–54. PubMed Abstract | Publisher Full Text
20. Bradley JD, Heilman DK, Katz BP, et al.: Tidal irrigation as treatment for knee osteoarthritis: a sham-controlled, randomized, double-blinded evaluation. Arthritis Rheum. 2002; 46(1): 100–8. PubMed Abstract | Publisher Full Text
21. Wartolowska K, Collins GS, Hopewell S, et al.: Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review. BMJ Open. 2016; 6(3): e010194. PubMed Abstract | Publisher Full Text | Free Full Text
22. Andersson GB, Mekhail NA, Block JE: A randomized, double-blind, controlled trial: intradiscal electrothermal therapy versus placebo for the treatment of chronic discogenic low back pain. Spine (Phila Pa 1976). 2006; 31(14): 1637–8; author reply 1638. PubMed Abstract | Publisher Full Text
23. Gillespie MB, Wylie PE, Lee-Chiong T, et al.: Effect of palatal implants on continuous positive airway pressure and compliance. Otolaryngol Head Neck Surg. 2011; 144(2): 230–6. PubMed Abstract | Publisher Full Text
24. Maurer JT, Sommer JU, Hein G, et al.: Palatal implants in the treatment of obstructive sleep apnea: a randomised, placebo-controlled single-centre trial. Eur Arch Otorhinolaryngol. 2012; 269(7): 1851–6. PubMed Abstract | Publisher Full Text
25. Hróbjartsson A, Thomsen AS, Emanuelsson F, et al.: Observer bias in randomized clinical trials with measurement scale outcomes: A systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013; 185(4): E201–11. PubMed Abstract | Publisher Full Text | Free Full Text
26. Rosenthal R, Jacobson L: Pygmalion in the classroom. Urban Rev. 1968; 3(1): 16–20. Publisher Full Text
27. Hróbjartsson A, Thomsen AS, Emanuelsson F, et al.: Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012; 344: e1119. PubMed Abstract | Publisher Full Text
28. Thomsen J, Bretlau P, Tos M, et al.: Ménière's disease: endolymphatic sac decompression compared with sham (placebo) decompression. Ann N Y Acad Sci. 1981; 374: 820–30. PubMed Abstract | Publisher Full Text
29. Shaheen NJ, Sharma P, Overholt BF, et al.: Radiofrequency ablation in Barrett’s esophagus with dysplasia. N Engl J Med. 2009; 360(22): 2277–88. PubMed Abstract | Publisher Full Text
30. Gracely RH, Dubner R, Deeter WR, et al.: Clinicians' expectations influence placebo analgesia. Lancet. 1985; 325(8419): 43. PubMed Abstract | Publisher Full Text
31. Benedetti F: How the doctor's words affect the patient's brain. Eval Health Prof. 2002; 25(4): 369–86. PubMed Abstract | Publisher Full Text
32. Thomas KB: General practice consultations: is there any point in being positive? Br Med J (Clin Res Ed). 1987; 294(6581): 1200–2. PubMed Abstract | Publisher Full Text | Free Full Text
33. Kaptchuk TJ, Kelley JM, Conboy LA, et al.: Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome. BMJ (Clin Res Ed). 2008; 336(7651): 999–1003. PubMed Abstract | Publisher Full Text | Free Full Text
34. Braunholtz DA, Edwards SJ, Lilford RJ: Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect”. J Clin Epidemiol. 2001; 54(3): 217–24. PubMed Abstract | Publisher Full Text
35. Doherty M, Dieppe P: The “placebo” response in osteoarthritis and its implications for clinical practice. Osteoarthr Cartil. 2009; 17(10): 1255–62. PubMed Abstract | Publisher Full Text
36. Hróbjartsson A: What are the main methodological problems in the estimation of placebo effects? J Clin Epidemiol. 2002; 55(5): 430–5. PubMed Abstract | Publisher Full Text
37. Stone GW, Teirstein PS, Rubenstein R, et al.: A prospective, multicenter, randomized trial of percutaneous transmyocardial laser revascularization in patients with nonrecanalizable chronic total occlusions. J Am Coll Cardiol. 2002; 39(10): 1581–7. PubMed Abstract | Publisher Full Text
38. Cobb LA, Thomas GI, Dillard DH, et al.: An evaluation of internal-mammary-artery ligation by a double-blind technic. N Engl J Med. 1959; 260(22): 1115–8. PubMed Abstract | Publisher Full Text
39. Dimond EG, Kittle CF, Crockett JE: Comparison of internal mammary artery ligation and sham operation for angina pectoris. Am J Cardiol. 1960; 5(4): 483–6. PubMed Abstract | Publisher Full Text
40. Guyuron B, Reed D, Kriegler JS, et al.: A placebo-controlled surgical trial of the treatment of migraine headaches. Plast Reconstr Surg. 2009; 124(2): 461–8. PubMed Abstract | Publisher Full Text
41. Gross RE, Watts RL, Hauser RA, et al.: Intrastriatal transplantation of microcarrier-bound human retinal pigment epithelial cells versus sham surgery in patients with advanced Parkinson’s disease: a double-blind, randomised, controlled trial. Lancet Neurol. 2011; 10(6): 509–19. PubMed Abstract | Publisher Full Text
42. Freed CR, Greene PE, Breeze RE, et al.: Transplantation of Embryonic Dopamine Neurons for Severe Parkinson’s Disease. N Engl J Med. 2001; 344(10): 710–9. PubMed Abstract | Publisher Full Text
43. Abbott J, Hawe J, Hunter D, et al.: Laparoscopic excision of endometriosis: a randomized, placebo-controlled trial. Fertil Steril. 2004; 82(4): 878–84. PubMed Abstract | Publisher Full Text
44. Arts J, Bisschops R, Blondeau K, et al.: A double-blind sham-controlled study of the effect of radiofrequency energy on symptoms and distensibility of the gastro-esophageal junction in GERD. Am J Gastroenterol. 2012; 107(2): 222–30. PubMed Abstract | Publisher Full Text
45. Castro M, Rubin AS, Laviolette M, et al.: Effectiveness and safety of bronchial thermoplasty in the treatment of severe asthma: a multicenter, randomized, double-blind, sham-controlled clinical trial. Am J Respir Crit Care Med. 2010; 181(2): 116–24. PubMed Abstract | Publisher Full Text | Free Full Text
46. Davey C, Zoumot Z, Jordan S, et al.: Bronchoscopic lung volume reduction with endobronchial valves for patients with heterogeneous emphysema and intact interlobar fissures (the BeLieVeR-HIFi trial): study design and rationale. Thorax. 2015; 70(3): 288–90. PubMed Abstract | Publisher Full Text | Free Full Text
47. Dowson A, Mullen MJ, Peatfield R, et al.: Migraine Intervention With STARFlex Technology (MIST) trial: a prospective, multicenter, double-blind, sham-controlled trial to evaluate the effectiveness of patent foramen ovale closure with STARFlex septal repair implant to resolve refractory migraine headache. Circulation. 2008; 117(11): 1397–404. PubMed Abstract | Publisher Full Text
48. Sutton CJ, Ewen SP, Whitelaw N, et al.: Prospective, randomized, double-blind, controlled trial of laser laparoscopy in the treatment of pelvic pain associated with minimal, mild, and moderate endometriosis. Fertil Steril. 1994; 62(4): 696–700. PubMed Abstract | Publisher Full Text
49. Kallmes DF, Comstock BA, Heagerty PJ, et al.: A randomized trial of vertebroplasty for osteoporotic spinal fractures. N Engl J Med. 2009; 361(6): 569–79. PubMed Abstract | Publisher Full Text | Free Full Text
50. Pauza KJ, Howell S, Dreyfuss P, et al.: A randomized, placebo-controlled trial of intradiscal electrothermal therapy for the treatment of discogenic low back pain. Spine J. 2004; 4(1): 27–35. PubMed Abstract | Publisher Full Text
51. Kapural L, Vrooman B, Sarwar S, et al.: A randomized, placebo-controlled trial of transdiscal radiofrequency, biacuplasty for treatment of discogenic lower back pain. Pain Med. 2013; 14(3): 362–73. PubMed Abstract | Publisher Full Text
52. Moseley JB, O’Malley K, Petersen NJ, et al.: A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002; 347(2): 81–8. PubMed Abstract | Publisher Full Text
53. Rothstein R, Filipi C, Caca K, et al.: Endoscopic full-thickness plication for the treatment of gastroesophageal reflux disease: A randomized, sham-controlled trial. Gastroenterology. 2006; 131(3): 704–12. PubMed Abstract | Publisher Full Text
54. Koutsourelakis I, Georgoulopoulos G, Perraki E, et al.: Randomised trial of nasal surgery for fixed nasal obstruction in obstructive sleep apnoea. Eur Respir J. 2008; 31(1): 110–7. PubMed Abstract | Publisher Full Text
55. Sihvonen R, Paavola M, Malmivaara A, et al.: Arthroscopic partial meniscectomy versus sham surgery for a degenerative meniscal tear. N Engl J Med. 2013; 369(26): 2515–24. PubMed Abstract | Publisher Full Text
56. Fleischer D: Endoscopic Nd:YAG laser therapy for active esophageal variceal bleeding. A randomized controlled study. Gastrointest Endosc. 1985; 31(1): 4–9. PubMed Abstract | Publisher Full Text
57. Leon MB, Kornowski R, Downey WE, et al.: A blinded, randomized, placebo-controlled trial of percutaneous laser myocardial revascularization to improve angina symptoms in patients with severe coronary disease. J Am Coll Cardiol. 2005; 46(10): 1812–9. PubMed Abstract | Publisher Full Text
58. Nease CJ, Krempl GA: Radiofrequency treatment of turbinate hypertrophy: a randomized, blinded, placebo-controlled clinical trial. Otolaryngol Head Neck Surg. 2004; 130(3): 291–9. PubMed Abstract | Publisher Full Text
59. Roehrborn CG, Gange SN, Shore ND, et al.: The prostatic urethral lift for the treatment of lower urinary tract symptoms associated with prostate enlargement due to benign prostatic hyperplasia: the L.I.F.T. Study. J Urol. 2013; 190(6): 2161–7. PubMed Abstract | Publisher Full Text
60. Mathus-Vliegen EM, Tytgat GN, Veldhuyzen-Offermans EA: Intragastric balloon in the treatment of super-morbid obesity. Double-blind, sham-controlled, crossover evaluation of 500-milliliter balloon. Gastroenterology. 1990; 99(2): 362–9. PubMed Abstract | Publisher Full Text
61. Salem M, Rotevatn S, Stavnes S, et al.: Usefulness and safety of percutaneous myocardial laser revascularization for refractory angina pectoris. Am J Cardiol. 2004; 93(9): 1086–91. PubMed Abstract | Publisher Full Text
62. Devière J, Costamagna G, Neuhaus H, et al.: Nonresorbable copolymer implantation for gastroesophageal reflux disease: a randomized sham-controlled multicenter trial. Gastroenterology. 2005; 128(3): 532–40. PubMed Abstract | Publisher Full Text
63. Buchbinder R, Osborne RH, Ebeling PR, et al.: A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. N Engl J Med. 2009; 361(6): 557–68. PubMed Abstract | Publisher Full Text
64. Bono CM, Heggeness M, Mick C, et al.: North American Spine Society: Newly released vertebroplasty randomized controlled trials: a tale of two trials. Spine J. 2010; 10(3): 238–40. PubMed Abstract | Publisher Full Text
65. Swank DJ, Swank-Bordewijk SC, Hop WC, et al.: Laparoscopic adhesiolysis in patients with chronic abdominal pain: a blinded randomised controlled multi-centre trial. Lancet. 2003; 361(9365): 1247–51. PubMed Abstract | Publisher Full Text
66. Jarrell J, Mohindra R, Ross S, et al.: Laparoscopy and reported pain among patients with endometriosis. J Obstet Gynaecol Can. 2005; 27(5): 477–85. PubMed Abstract | Publisher Full Text
67. Eid GM, McCloskey CA, Eagleton JK, et al.: StomaphyX vs a sham procedure for revisional surgery to reduce regained weight in Roux-en-Y gastric bypass patients : a randomized clinical trial. JAMA Surg. 2014; 149(4): 372–9. PubMed Abstract | Publisher Full Text
68. Montgomery M, Hakanson B, Ljungqvist O, et al.: Twelve months’ follow-up after treatment with the EndoCinch endoscopic technique for gastro-oesophageal reflux disease: a randomized, placebo-controlled study. Scand J Gastroenterol. 2006; 41(12): 1382–9. PubMed Abstract | Publisher Full Text
69. Friedman M, Schalch P, Lin HC, et al.: Palatal implants for the treatment of snoring and obstructive sleep apnea/hypopnea syndrome. Otolaryngol Head Neck Surg. 2008; 138(2): 209–16. PubMed Abstract | Publisher Full Text
70. Olanow CW, Goetz CG, Kordower JH, et al.: A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Ann Neurol. 2003; 54(3): 403–14. PubMed Abstract | Publisher Full Text
71. Fockens P, Cohen L, Edmundowicz SA, et al.: Prospective randomized controlled trial of an injectable esophageal prosthesis versus a sham procedure for endoscopic treatment of gastroesophageal reflux disease. Surg Endosc. 2010; 24(6): 1387–97. PubMed Abstract | Publisher Full Text | Free Full Text
72. Ernst E, Resch KL: Concept of true and perceived placebo effects. BMJ. 1995; 311(7004): 551–3. PubMed Abstract | Publisher Full Text | Free Full Text
73. Simpson SH, Eurich DT, Majumdar SR, et al.: A meta-analysis of the association between adherence to drug therapy and mortality. BMJ. 2006; 333(7557): 15. PubMed Abstract | Publisher Full Text | Free Full Text
74. Cotton PB, Durkalski V, Romagnuolo J, et al.: Effect of endoscopic sphincterotomy for suspected sphincter of Oddi dysfunction on pain-related disability following cholecystectomy: the EPISOD randomized clinical trial. JAMA. 2014; 311(20): 2101–9. PubMed Abstract | Publisher Full Text | Free Full Text
75. Hartigan P: Sclerotherapy for male alcoholic cirrhotic patients who have bled from esophageal varices: Results of a randomized, multicenter clinical trial. The Veterans Affairs Cooperative Variceal Sclerotherapy Group. Hepatology. 1994; 20(3): 618–25. PubMed Abstract | Publisher Full Text
76. Freitas D, Donato A, Monteiro JG: Controlled Trial of Liquid Monopolar Electrocoagulation in Bleeding Peptic Ulcers. Am J Gastroenterol. 1985; 80(11): 853–7. PubMed Abstract
77. MacLeod IA, Mills PR, MacKenzie JF, et al.: Neodymium yttrium aluminium garnet laser photocoagulation for major haemorrhage from peptic ulcers and single vessels: a single blind controlled study. Br Med J (Clin Res Ed). 1983; 286(6362): 345–8. PubMed Abstract | Publisher Full Text | Free Full Text
78. Scolapio JS, Gostout CJ, Schroeder KW, et al.: Dysphagia without endoscopically evident disease: to dilate or not? Am J Gastroenterol. 2001; 96(2): 327–30. PubMed Abstract | Publisher Full Text

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 08 Sep 2017

Author details Author details

Karolina Wartolowska
Roles: Writing – Original Draft Preparation, Writing – Review & Editing

David Beard
Roles: Writing – Review & Editing

Andrew Carr
Roles: Funding Acquisition, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

This work has received funding from the NIHR Oxford Musculoskeletal Biomedical Research Unit.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (2)

version 2

Revised

Published: 30 Jan 2018, 6:1663

https://doi.org/10.12688/f1000research.12528.2

version 1

Published: 08 Sep 2017, 6:1663

https://doi.org/10.12688/f1000research.12528.1

© 2018 Wartolowska K et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Wartolowska K, Beard D and Carr A. Blinding in trials of interventional procedures is possible and worthwhile [version 2; peer review: 3 approved]. F1000Research 2018, 6:1663 (https://doi.org/10.12688/f1000research.12528.2)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 08 Sep 2017

Views

Reviewer Report 04 Dec 2017

Helene Moustgaard, Centre for Evidence-Based Medicine, Odense University Hospital (OUH), University of Southern Denmark, Odense, Denmark

Asbjørn Hróbjartsson, Centre for Evidence-Based Medicine, Odense University Hospital (OUH), University of Southern Denmark, Odense, Denmark

Approved

https://doi.org/10.5256/f1000research.13566.r25901

The overall topic of this paper is important: Blinding facilitates unbiased trial results, but blinding is less used in surgical trials (and less frequently discussed) than in trials of pharmacological interventions. A prevalent preconception is that blinding is not possible, or not relevant, in surgical trials. The main strength of the paper is that it illustrates well this to be ill founded.

However, we wondered whether the manuscript could be further improved. First, we were in doubt what the specific aim of the paper was. We assumed that the authors aimed to discuss the methodological role of binding in surgical trials (and not primarily to present and reflect on their previous review of surgical placebo trials)? If so, it would have strengthened the paper if it had provided some reflections on the barriers to the use of blinding in surgical trials, the ethical issues involved (and how we can best balance the need for reliable evidence for effects of surgery against the risks or discomforts patients may be exposed to in blinded surgical trials), as wells as discuss what measures could be taken to remedy the current situation where blinding is used infrequently in surgical trials. In general, the reader would be better prepared if the paper clearly stated whether the topic it addresses is blinding in surgical trials or use of placebo in surgical trials and whether the main focus is providing practical examples or providing a general overview of the topic.
It would be helpful if the paper defined central terms used. For example, what is meant by “interventional procedures”? Does that include acupuncture, radiotherapy, and physiotherapy? Also, how do the authors conceptualize “subjective outcomes vs. objective outcomes” [1].

The central section Bias specific to surgical trials appears compressed. It would help the reader if the section could more clearly show how the example provided introduces bias, and in what way this bias is specific too surgery (rather than just more frequently encountered).

Of minor importance was that we were puzzled by some of the references used. For instance, in the Introduction, there is a reference to a study by Wood et al [2], but several more recent studies have looked at this issue [3]. Also, in the section Blinding of outcome assessors the authors refer to a study by one of the undersigned, to the effect that the study documents a “Pygmalion effect” in un-blinded studies [4]. The term, however, was not used in the original publication. It would possibly help the reader if the authors referred to publications that provide a general overview of blinding in trials (and not only surgical trials).

Is the topic of the review discussed comprehensively in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

1. Moustgaard H, Bello S, Miller FG, Hróbjartsson A: Subjective and objective outcomes in randomized clinical trials: definitions differed in methods publications and were often absent from trial reports.J Clin Epidemiol. 2014; 67 (12): 1327-34 PubMed Abstract | Publisher Full Text
2. Wood L, Egger M, Gluud LL, Schulz KF, et al.: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study.BMJ. 2008; 336 (7644): 601-5 PubMed Abstract | Publisher Full Text
3. Page MJ, Higgins JP, Clayton G, Sterne JA, et al.: Empirical Evidence of Study Design Biases in Randomized Trials: Systematic Review of Meta-Epidemiological Studies.PLoS One. 2016; 11 (7): e0159267 PubMed Abstract | Publisher Full Text
4. Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, et al.: Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors.BMJ. 2012; 344: e1119 PubMed Abstract

Competing Interests: No competing interests were disclosed.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes ... Continue reading Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes are subjective. The need to control for bias in surgical trials is often not recognised. We agree with the Reviewer that there is a preconception/assumption that blinding in surgical trials is not possible or not necessary. Surgery may mean an open surgery undertaken to preserve life. In such cases the outcome is objective, binary and with low risk of bias. However, nowadays, many invasive procedures are performed not to save lives but to improve the quality of life, function or to reduce pain. As a consequence, their outcomes are highly subjective, and as such, prone to bias. This problem seems to be unappreciated in clinical and research community. In this paper, we wanted to demonstrate that blinding can be achieved in surgical trials and that it is necessary if the outcomes are subjective.

Placebo-controlled surgical trials represent a single type of trial that requires blinding, but similar arguments apply to trials comparing two different surgical methods.

In this paper, we did not discuss the placebo because our main focus was blinding as a way to control bias. We used placebo-controlled surgical trials as an example but similar arguments apply to trials comparing two different surgical methods. We have already written about barriers in completing placebo-controlled surgical trials (Wartolowska et al., 2016), about the balance between harms and benefits of such trials (Wartolowska et al., 2014) and about the ethical implications of placebo in surgery (Savulescu, Wartolowska and Carr, 2016)(George et al., 2016).

We have used acupuncture as an example of an experimental study on placebo effect because acupuncture is the only type of invasive procedure used in such studies.

We have included radiofrequency procedures because they change anatomy. In this paper, we have defined surgery as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”.(Wartolowska et al., 2014) We have added these definitions to the manuscript.

“Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”(Wartolowska et al., 2016)

As suggested by the Reviewers, we have explained in the section titled “Bias specific to surgical trials” that by including only patients with non-life-threatening symptoms or with sufficiently severe symptoms on the day of surgery a bias is created, which is difficult to control for.

As suggested by the Reviewers, we have added the paper by Page et al. as a reference in this paragraph. We have also added a reference that provides a general overview of blinding in trials (Hróbjartsson and Boutron, 2011) and provided references to Rosenthal’s papers to explain the “Pygmalion effect”.(Rosenthal and Jacobson, 1968)

REFERENCES
George, A. et al. (2016) ‘When should placebo surgery as a control in clinical trials be carried out?’, The Bulletin of the Royal College of Surgeons of England, 98(2), pp. 75–79.
Hróbjartsson, A. and Boutron, I. (2011) ‘Blinding in randomized clinical trials: Imposed impartiality’, Clinical Pharmacology and Therapeutics. Nature Publishing Group, 90(5), pp. 732–736.
Rosenthal, R. and Jacobson, L. (1968) ‘Pygmalion in the classroom’, The Urban Review, 3(1), pp. 16–20.
Savulescu, J., Wartolowska, K. A. and Carr, A. (2016) ‘Randomised Placebo-Controlled Trials of Surgery: Ethical Analysis and Guidelines’, J Med Ethics, 42(12), pp. 776–783.
Wartolowska, K. et al. (2014) ‘Use of placebo controls in the evaluation of surgery: systematic review.’, BMJ (Clinical research ed.), 348(May), p. g3253.
Wartolowska, K. et al. (2016) ‘Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review’, BMJ Open, 6(3), p. e010194.
Wartolowska, K. A. et al. (2016) ‘The Magnitude and Temporal Changes of Response in the Placebo Arm of Surgical Randomized Controlled Trials - A Systematic Review and Meta-Analysis’, Trials. Trials, 17(1), p. 589.
Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes are subjective. The need to control for bias in surgical trials is often not recognised. We agree with the Reviewer that there is a preconception/assumption that blinding in surgical trials is not possible or not necessary. Surgery may mean an open surgery undertaken to preserve life. In such cases the outcome is objective, binary and with low risk of bias. However, nowadays, many invasive procedures are performed not to save lives but to improve the quality of life, function or to reduce pain. As a consequence, their outcomes are highly subjective, and as such, prone to bias. This problem seems to be unappreciated in clinical and research community. In this paper, we wanted to demonstrate that blinding can be achieved in surgical trials and that it is necessary if the outcomes are subjective.

Placebo-controlled surgical trials represent a single type of trial that requires blinding, but similar arguments apply to trials comparing two different surgical methods.

In this paper, we did not discuss the placebo because our main focus was blinding as a way to control bias. We used placebo-controlled surgical trials as an example but similar arguments apply to trials comparing two different surgical methods. We have already written about barriers in completing placebo-controlled surgical trials (Wartolowska et al., 2016), about the balance between harms and benefits of such trials (Wartolowska et al., 2014) and about the ethical implications of placebo in surgery (Savulescu, Wartolowska and Carr, 2016)(George et al., 2016).

We have used acupuncture as an example of an experimental study on placebo effect because acupuncture is the only type of invasive procedure used in such studies.

We have included radiofrequency procedures because they change anatomy. In this paper, we have defined surgery as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”.(Wartolowska et al., 2014) We have added these definitions to the manuscript.

“Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”(Wartolowska et al., 2016)

As suggested by the Reviewers, we have explained in the section titled “Bias specific to surgical trials” that by including only patients with non-life-threatening symptoms or with sufficiently severe symptoms on the day of surgery a bias is created, which is difficult to control for.

As suggested by the Reviewers, we have added the paper by Page et al. as a reference in this paragraph. We have also added a reference that provides a general overview of blinding in trials (Hróbjartsson and Boutron, 2011) and provided references to Rosenthal’s papers to explain the “Pygmalion effect”.(Rosenthal and Jacobson, 1968)

REFERENCES
George, A. et al. (2016) ‘When should placebo surgery as a control in clinical trials be carried out?’, The Bulletin of the Royal College of Surgeons of England, 98(2), pp. 75–79.
Hróbjartsson, A. and Boutron, I. (2011) ‘Blinding in randomized clinical trials: Imposed impartiality’, Clinical Pharmacology and Therapeutics. Nature Publishing Group, 90(5), pp. 732–736.
Rosenthal, R. and Jacobson, L. (1968) ‘Pygmalion in the classroom’, The Urban Review, 3(1), pp. 16–20.
Savulescu, J., Wartolowska, K. A. and Carr, A. (2016) ‘Randomised Placebo-Controlled Trials of Surgery: Ethical Analysis and Guidelines’, J Med Ethics, 42(12), pp. 776–783.
Wartolowska, K. et al. (2014) ‘Use of placebo controls in the evaluation of surgery: systematic review.’, BMJ (Clinical research ed.), 348(May), p. g3253.
Wartolowska, K. et al. (2016) ‘Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review’, BMJ Open, 6(3), p. e010194.
Wartolowska, K. A. et al. (2016) ‘The Magnitude and Temporal Changes of Response in the Placebo Arm of Surgical Randomized Controlled Trials - A Systematic Review and Meta-Analysis’, Trials. Trials, 17(1), p. 589.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes ... Continue reading Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes are subjective. The need to control for bias in surgical trials is often not recognised. We agree with the Reviewer that there is a preconception/assumption that blinding in surgical trials is not possible or not necessary. Surgery may mean an open surgery undertaken to preserve life. In such cases the outcome is objective, binary and with low risk of bias. However, nowadays, many invasive procedures are performed not to save lives but to improve the quality of life, function or to reduce pain. As a consequence, their outcomes are highly subjective, and as such, prone to bias. This problem seems to be unappreciated in clinical and research community. In this paper, we wanted to demonstrate that blinding can be achieved in surgical trials and that it is necessary if the outcomes are subjective.

Placebo-controlled surgical trials represent a single type of trial that requires blinding, but similar arguments apply to trials comparing two different surgical methods.

In this paper, we did not discuss the placebo because our main focus was blinding as a way to control bias. We used placebo-controlled surgical trials as an example but similar arguments apply to trials comparing two different surgical methods. We have already written about barriers in completing placebo-controlled surgical trials (Wartolowska et al., 2016), about the balance between harms and benefits of such trials (Wartolowska et al., 2014) and about the ethical implications of placebo in surgery (Savulescu, Wartolowska and Carr, 2016)(George et al., 2016).

We have used acupuncture as an example of an experimental study on placebo effect because acupuncture is the only type of invasive procedure used in such studies.

We have included radiofrequency procedures because they change anatomy. In this paper, we have defined surgery as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”.(Wartolowska et al., 2014) We have added these definitions to the manuscript.

“Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”(Wartolowska et al., 2016)

As suggested by the Reviewers, we have explained in the section titled “Bias specific to surgical trials” that by including only patients with non-life-threatening symptoms or with sufficiently severe symptoms on the day of surgery a bias is created, which is difficult to control for.

As suggested by the Reviewers, we have added the paper by Page et al. as a reference in this paragraph. We have also added a reference that provides a general overview of blinding in trials (Hróbjartsson and Boutron, 2011) and provided references to Rosenthal’s papers to explain the “Pygmalion effect”.(Rosenthal and Jacobson, 1968)

REFERENCES
George, A. et al. (2016) ‘When should placebo surgery as a control in clinical trials be carried out?’, The Bulletin of the Royal College of Surgeons of England, 98(2), pp. 75–79.
Hróbjartsson, A. and Boutron, I. (2011) ‘Blinding in randomized clinical trials: Imposed impartiality’, Clinical Pharmacology and Therapeutics. Nature Publishing Group, 90(5), pp. 732–736.
Rosenthal, R. and Jacobson, L. (1968) ‘Pygmalion in the classroom’, The Urban Review, 3(1), pp. 16–20.
Savulescu, J., Wartolowska, K. A. and Carr, A. (2016) ‘Randomised Placebo-Controlled Trials of Surgery: Ethical Analysis and Guidelines’, J Med Ethics, 42(12), pp. 776–783.
Wartolowska, K. et al. (2014) ‘Use of placebo controls in the evaluation of surgery: systematic review.’, BMJ (Clinical research ed.), 348(May), p. g3253.
Wartolowska, K. et al. (2016) ‘Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review’, BMJ Open, 6(3), p. e010194.
Wartolowska, K. A. et al. (2016) ‘The Magnitude and Temporal Changes of Response in the Placebo Arm of Surgical Randomized Controlled Trials - A Systematic Review and Meta-Analysis’, Trials. Trials, 17(1), p. 589.
Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes are subjective. The need to control for bias in surgical trials is often not recognised. We agree with the Reviewer that there is a preconception/assumption that blinding in surgical trials is not possible or not necessary. Surgery may mean an open surgery undertaken to preserve life. In such cases the outcome is objective, binary and with low risk of bias. However, nowadays, many invasive procedures are performed not to save lives but to improve the quality of life, function or to reduce pain. As a consequence, their outcomes are highly subjective, and as such, prone to bias. This problem seems to be unappreciated in clinical and research community. In this paper, we wanted to demonstrate that blinding can be achieved in surgical trials and that it is necessary if the outcomes are subjective.

Placebo-controlled surgical trials represent a single type of trial that requires blinding, but similar arguments apply to trials comparing two different surgical methods.

In this paper, we did not discuss the placebo because our main focus was blinding as a way to control bias. We used placebo-controlled surgical trials as an example but similar arguments apply to trials comparing two different surgical methods. We have already written about barriers in completing placebo-controlled surgical trials (Wartolowska et al., 2016), about the balance between harms and benefits of such trials (Wartolowska et al., 2014) and about the ethical implications of placebo in surgery (Savulescu, Wartolowska and Carr, 2016)(George et al., 2016).

We have used acupuncture as an example of an experimental study on placebo effect because acupuncture is the only type of invasive procedure used in such studies.

We have included radiofrequency procedures because they change anatomy. In this paper, we have defined surgery as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”.(Wartolowska et al., 2014) We have added these definitions to the manuscript.

“Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”(Wartolowska et al., 2016)

As suggested by the Reviewers, we have explained in the section titled “Bias specific to surgical trials” that by including only patients with non-life-threatening symptoms or with sufficiently severe symptoms on the day of surgery a bias is created, which is difficult to control for.

As suggested by the Reviewers, we have added the paper by Page et al. as a reference in this paragraph. We have also added a reference that provides a general overview of blinding in trials (Hróbjartsson and Boutron, 2011) and provided references to Rosenthal’s papers to explain the “Pygmalion effect”.(Rosenthal and Jacobson, 1968)

REFERENCES
George, A. et al. (2016) ‘When should placebo surgery as a control in clinical trials be carried out?’, The Bulletin of the Royal College of Surgeons of England, 98(2), pp. 75–79.
Hróbjartsson, A. and Boutron, I. (2011) ‘Blinding in randomized clinical trials: Imposed impartiality’, Clinical Pharmacology and Therapeutics. Nature Publishing Group, 90(5), pp. 732–736.
Rosenthal, R. and Jacobson, L. (1968) ‘Pygmalion in the classroom’, The Urban Review, 3(1), pp. 16–20.
Savulescu, J., Wartolowska, K. A. and Carr, A. (2016) ‘Randomised Placebo-Controlled Trials of Surgery: Ethical Analysis and Guidelines’, J Med Ethics, 42(12), pp. 776–783.
Wartolowska, K. et al. (2014) ‘Use of placebo controls in the evaluation of surgery: systematic review.’, BMJ (Clinical research ed.), 348(May), p. g3253.
Wartolowska, K. et al. (2016) ‘Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review’, BMJ Open, 6(3), p. e010194.
Wartolowska, K. A. et al. (2016) ‘The Magnitude and Temporal Changes of Response in the Placebo Arm of Surgical Randomized Controlled Trials - A Systematic Review and Meta-Analysis’, Trials. Trials, 17(1), p. 589.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 04 Dec 2017

Harald Walach, Department of Pediatric Gastroenterology, Medical University Poznan, Poznań, Poland

Approved

https://doi.org/10.5256/f1000research.13566.r28006

This review is, as its self-set goal goes, very well substantiated, balanced and timely. There is no factual correction in itself that is necessary in my view.

However, I wish to give a few critical feedbacks to the authors that they may wish to integrate. I would prefer if they did, but in my view it is unfair to formally demand such an extension. However, the debate itself would profit.

My point is regarding the presuppositions of the current model the authors implicitly subscribe to, for instance when they say randomised trials “provide the highest level of evidence” (1^st para). This is true, but only from a - debated – point of departure that accepts the hierarchisation of evidence as a given and true proposal. As it happens, we, and others, have debated this [1,2]. The hierarchisation of evidence accepts – without further reflection – that internal and external validity are compatible, linear concepts, where first internal evidence can be gleaned and is prior and superior to external validity. This I find an unsubstantiated point of view that needs at least acknowledgement. All the arguments in this paper address the improvement of internal validity, and as such they are very good.

The argument that “specific” effects have to be tested against “non-specific” or placebo-effects and therefore blinding is necessary also makes presuppositions that are not clarified and critically debated. The presupposition is that there is such a thing as separability of these effects. This separability argument is highly speculative, very little researched and where and when researched found to be wrong [3, 4]. One can, of course, try and disentangle various components of interventions. And especially where interventions are costly or fraught with side-effects this is even an ethical requirement, because else patients would be submitted to problematic procedures with little benefit or where other interventions might be just as good, all taken together. But one should be clear about the fact that this is not something that automatically gets rid of all problems in evaluating therapeutic procedures.

Therefore, I think the arguments the authors present are valid within the context of improving internal validity of intervention trials that offer costly and potentially dangerous or problematic interventions. But I think the authors should be clear themselves that this is not a universally valid strategy and it would be wise to integrate some caveats and restrictions in their argument.

Is the topic of the review discussed comprehensively in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

1. Walach H, Loef M: Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence.J Clin Epidemiol. 2015; 68 (11): 1251-60 PubMed Abstract | Publisher Full Text
2. Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance Group: Evidence based medicine: a movement in crisis?. BMJ. 2014; 348: g3725 PubMed Abstract
3. Walach H: Placebo controls: historical, methodological and general aspects.Philos Trans R Soc Lond B Biol Sci. 2011; 366 (1572): 1870-8 PubMed Abstract | Publisher Full Text
4. Meissner K, Fässler M, Rücker G, Kleijnen J, et al.: Differential effectiveness of placebo treatments: a systematic review of migraine prophylaxis.JAMA Intern Med. 2013; 173 (21): 1941-51 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Research methodology, clinical and other trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise ... Continue reading We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise bias due to confounding factors”. We have also added the suggested reference (Walach and Loef, 2015).

We agree that RCTs have high internal validity because they are performed in a standardised way, and because they attempt to control for confounding factors. However, owing to the controlled conditions under which they are performed, RCTs often have low external validity. In this paper, we wanted to draw attention to the fact that surgical trials often use subjective outcomes, and are therefore prone to bias, which has to be minimised or controlled for.

We agree with the Reviewer, that the “specific” and “non-specific” effects are difficult to separate or to unequivocally define. As surgical procedures are complex and consist of many elements, it is sometimes difficult to identify the crucial surgical element that is believed to be therapeutic. We have described the CSAW trial as an example. The aim of this trial was to investigate whether the improvement was due to change of anatomy (arthroscopic removal of a bony spur on the acromion) or it was not related to the change in anatomy and the spur removal was not necessary because a placebo procedure (arthroscopy only) results in similar change in pain and function.

We wanted to draw attention to the fact that in the case of surgical trials, with their high costs and risks, one would expect to see a much larger effect after a surgical procedure than after a placebo or no treatment.

We are familiar with the debate in the literature regarding specific and non-specific effects. In the context of surgical trials, by “specific effects” we meant changes directly related to change in anatomy, for example, a bony spur removal in the trial of arthroscopic shoulder decompression surgery for subacromial pain. All other possible factors resulting in improvement, such as the effect of arthroscopy, the placebo effect, influence of patient-doctor interactions, symptoms fluctuation as well as spontaneous improvement were regarded as “non-specific”. We also agree with Enck and colleagues that the effect in the placebo and active arms may not be additive; therefore, the improvement related to the crucial surgical element may not equal the difference between the effect in the active and placebo arms.(Enck et al., 2011) We would like to clarify that blinding and placebo control help to minimise bias and aid in the interpretation of trial results; especially if the outcomes are subjective and the symptoms fluctuate or may spontaneously improve.
REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Walach, H. and Loef, M. (2015) ‘Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence’, Journal of Clinical Epidemiology. Elsevier Inc, 68(11), pp. 1251–1260.
We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise bias due to confounding factors”. We have also added the suggested reference (Walach and Loef, 2015).

We agree that RCTs have high internal validity because they are performed in a standardised way, and because they attempt to control for confounding factors. However, owing to the controlled conditions under which they are performed, RCTs often have low external validity. In this paper, we wanted to draw attention to the fact that surgical trials often use subjective outcomes, and are therefore prone to bias, which has to be minimised or controlled for.

We agree with the Reviewer, that the “specific” and “non-specific” effects are difficult to separate or to unequivocally define. As surgical procedures are complex and consist of many elements, it is sometimes difficult to identify the crucial surgical element that is believed to be therapeutic. We have described the CSAW trial as an example. The aim of this trial was to investigate whether the improvement was due to change of anatomy (arthroscopic removal of a bony spur on the acromion) or it was not related to the change in anatomy and the spur removal was not necessary because a placebo procedure (arthroscopy only) results in similar change in pain and function.

We wanted to draw attention to the fact that in the case of surgical trials, with their high costs and risks, one would expect to see a much larger effect after a surgical procedure than after a placebo or no treatment.

We are familiar with the debate in the literature regarding specific and non-specific effects. In the context of surgical trials, by “specific effects” we meant changes directly related to change in anatomy, for example, a bony spur removal in the trial of arthroscopic shoulder decompression surgery for subacromial pain. All other possible factors resulting in improvement, such as the effect of arthroscopy, the placebo effect, influence of patient-doctor interactions, symptoms fluctuation as well as spontaneous improvement were regarded as “non-specific”. We also agree with Enck and colleagues that the effect in the placebo and active arms may not be additive; therefore, the improvement related to the crucial surgical element may not equal the difference between the effect in the active and placebo arms.(Enck et al., 2011) We would like to clarify that blinding and placebo control help to minimise bias and aid in the interpretation of trial results; especially if the outcomes are subjective and the symptoms fluctuate or may spontaneously improve.
REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Walach, H. and Loef, M. (2015) ‘Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence’, Journal of Clinical Epidemiology. Elsevier Inc, 68(11), pp. 1251–1260.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise ... Continue reading We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise bias due to confounding factors”. We have also added the suggested reference (Walach and Loef, 2015).

We agree that RCTs have high internal validity because they are performed in a standardised way, and because they attempt to control for confounding factors. However, owing to the controlled conditions under which they are performed, RCTs often have low external validity. In this paper, we wanted to draw attention to the fact that surgical trials often use subjective outcomes, and are therefore prone to bias, which has to be minimised or controlled for.

We agree with the Reviewer, that the “specific” and “non-specific” effects are difficult to separate or to unequivocally define. As surgical procedures are complex and consist of many elements, it is sometimes difficult to identify the crucial surgical element that is believed to be therapeutic. We have described the CSAW trial as an example. The aim of this trial was to investigate whether the improvement was due to change of anatomy (arthroscopic removal of a bony spur on the acromion) or it was not related to the change in anatomy and the spur removal was not necessary because a placebo procedure (arthroscopy only) results in similar change in pain and function.

We wanted to draw attention to the fact that in the case of surgical trials, with their high costs and risks, one would expect to see a much larger effect after a surgical procedure than after a placebo or no treatment.

We are familiar with the debate in the literature regarding specific and non-specific effects. In the context of surgical trials, by “specific effects” we meant changes directly related to change in anatomy, for example, a bony spur removal in the trial of arthroscopic shoulder decompression surgery for subacromial pain. All other possible factors resulting in improvement, such as the effect of arthroscopy, the placebo effect, influence of patient-doctor interactions, symptoms fluctuation as well as spontaneous improvement were regarded as “non-specific”. We also agree with Enck and colleagues that the effect in the placebo and active arms may not be additive; therefore, the improvement related to the crucial surgical element may not equal the difference between the effect in the active and placebo arms.(Enck et al., 2011) We would like to clarify that blinding and placebo control help to minimise bias and aid in the interpretation of trial results; especially if the outcomes are subjective and the symptoms fluctuate or may spontaneously improve.
REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Walach, H. and Loef, M. (2015) ‘Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence’, Journal of Clinical Epidemiology. Elsevier Inc, 68(11), pp. 1251–1260.
We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise bias due to confounding factors”. We have also added the suggested reference (Walach and Loef, 2015).

We agree that RCTs have high internal validity because they are performed in a standardised way, and because they attempt to control for confounding factors. However, owing to the controlled conditions under which they are performed, RCTs often have low external validity. In this paper, we wanted to draw attention to the fact that surgical trials often use subjective outcomes, and are therefore prone to bias, which has to be minimised or controlled for.

We agree with the Reviewer, that the “specific” and “non-specific” effects are difficult to separate or to unequivocally define. As surgical procedures are complex and consist of many elements, it is sometimes difficult to identify the crucial surgical element that is believed to be therapeutic. We have described the CSAW trial as an example. The aim of this trial was to investigate whether the improvement was due to change of anatomy (arthroscopic removal of a bony spur on the acromion) or it was not related to the change in anatomy and the spur removal was not necessary because a placebo procedure (arthroscopy only) results in similar change in pain and function.

We wanted to draw attention to the fact that in the case of surgical trials, with their high costs and risks, one would expect to see a much larger effect after a surgical procedure than after a placebo or no treatment.

We are familiar with the debate in the literature regarding specific and non-specific effects. In the context of surgical trials, by “specific effects” we meant changes directly related to change in anatomy, for example, a bony spur removal in the trial of arthroscopic shoulder decompression surgery for subacromial pain. All other possible factors resulting in improvement, such as the effect of arthroscopy, the placebo effect, influence of patient-doctor interactions, symptoms fluctuation as well as spontaneous improvement were regarded as “non-specific”. We also agree with Enck and colleagues that the effect in the placebo and active arms may not be additive; therefore, the improvement related to the crucial surgical element may not equal the difference between the effect in the active and placebo arms.(Enck et al., 2011) We would like to clarify that blinding and placebo control help to minimise bias and aid in the interpretation of trial results; especially if the outcomes are subjective and the symptoms fluctuate or may spontaneously improve.
REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Walach, H. and Loef, M. (2015) ‘Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence’, Journal of Clinical Epidemiology. Elsevier Inc, 68(11), pp. 1251–1260.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 20 Sep 2017

Ian A Harris, Whitlam Orthopaedic Research Centre, Ingham Institute for Applied Medical Research, South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW, Australia

Approved

https://doi.org/10.5256/f1000research.13566.r25898

This manuscript is a narrative overview of methods of blinding in trials of procedures, using the previous review by these authors as the source of data. It explains the rationale for blinding and gives examples (subgrouped) of how blinding was achieved in previous trials. The authors wisely avoid a detailed discussion of the placebo effect and concentrate on the need for blinding based on empirical evidence of bias resulting from not blinding.
The paper reinforces the need for blinding in procedural trials which is a reasonable demand that is increasingly being heeded. In other words, the message is important.
The paper is very well written. I only have minor comments, below:

Page 2, second last paragraph, “crucial element of believed to be the cure” should read “crucial element believed to be the cure”
The evidence of bias resulting from not blinding may be one sided. Opposing evidence should be discussed, such as Berkmann ND, The Empirical Evidence of Bias in Trials Measuring Treatment Differences¹ which shows that the effect of blinding may even be in the opposite direction and the classic Schultz article from 1995¹ which gives a lower estimate of effect exaggeration from unblinding. There are several other summaries and meta-epidemiological studies in this area.

Is the topic of the review discussed comprehensively in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

1. Berkman ND, Santaguida PL, Viswanathan M, Morton SC: The Empirical Evidence of Bias in Trials Measuring Treatment Differences. PubMed Abstract
2. Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.JAMA. 1995; 273 (5): 408-12 PubMed Abstract

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Orthopaedics, Epidemiology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

CITE

Report a concern

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they ... Continue reading We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they become very different from usual clinical practice. They are also liable to have study populations that are so unique that the results of the trial cannot be easily generalised.

In the case of blinding, the magnitude of effect may be lower in a blinded trial than in everyday practice, because blinding causes uncertainty about treatment allocation. In clinical practice, there is usually no doubt as the patient-doctor relationship is based on trust and on the assumption that doctors offer patients the best available treatment. In an RCT, there is an inherent uncertainty as to which treatment group a patient has been allocated.(Enck et al., 2011)

REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they become very different from usual clinical practice. They are also liable to have study populations that are so unique that the results of the trial cannot be easily generalised.

In the case of blinding, the magnitude of effect may be lower in a blinded trial than in everyday practice, because blinding causes uncertainty about treatment allocation. In clinical practice, there is usually no doubt as the patient-doctor relationship is based on trust and on the assumption that doctors offer patients the best available treatment. In an RCT, there is an inherent uncertainty as to which treatment group a patient has been allocated.(Enck et al., 2011)

REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

30 Jan 2018

Author Response

We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they ... Continue reading We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they become very different from usual clinical practice. They are also liable to have study populations that are so unique that the results of the trial cannot be easily generalised.

In the case of blinding, the magnitude of effect may be lower in a blinded trial than in everyday practice, because blinding causes uncertainty about treatment allocation. In clinical practice, there is usually no doubt as the patient-doctor relationship is based on trust and on the assumption that doctors offer patients the best available treatment. In an RCT, there is an inherent uncertainty as to which treatment group a patient has been allocated.(Enck et al., 2011)

REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they become very different from usual clinical practice. They are also liable to have study populations that are so unique that the results of the trial cannot be easily generalised.

In the case of blinding, the magnitude of effect may be lower in a blinded trial than in everyday practice, because blinding causes uncertainty about treatment allocation. In clinical practice, there is usually no doubt as the patient-doctor relationship is based on trust and on the assumption that doctors offer patients the best available treatment. In an RCT, there is an inherent uncertainty as to which treatment group a patient has been allocated.(Enck et al., 2011)

REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (0)

Version 2

VERSION 2 PUBLISHED 08 Sep 2017

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 30 Jan 18
Version 1 08 Sep 17	read	read	read

Ian A Harris, University of New South Wales, Liverpool, Australia
Harald Walach, Medical University Poznan, Poznań, Poland
Helene Moustgaard, University of Southern Denmark, Odense, Denmark

Asbjørn Hróbjartsson, University of Southern Denmark, Odense, Denmark

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

16 Views

04 Dec 2017 | for Version 1

Helene Moustgaard, Centre for Evidence-Based Medicine, Odense University Hospital (OUH), University of Southern Denmark, Odense, Denmark

Asbjørn Hróbjartsson, Centre for Evidence-Based Medicine, Odense University Hospital (OUH), University of Southern Denmark, Odense, Denmark

16 Views Cite this report Responses(1)

Approved

Is the topic of the review discussed comprehensively in the context of the current literature?

Partly
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

Competing Interests

No competing interests were disclosed.

We confirm that we have read this submission and believe that we have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

Our aim was to discuss the methodological role of blinding in surgical trials and to demonstrate that blinding can and should be done in surgical trials, especially, if the outcomes are subjective. The need to control for bias in surgical trials is often not recognised. We agree with the Reviewer that there is a preconception/assumption that blinding in surgical trials is not possible or not necessary. Surgery may mean an open surgery undertaken to preserve life. In such cases the outcome is objective, binary and with low risk of bias. However, nowadays, many invasive procedures are performed not to save lives but to improve the quality of life, function or to reduce pain. As a consequence, their outcomes are highly subjective, and as such, prone to bias. This problem seems to be unappreciated in clinical and research community. In this paper, we wanted to demonstrate that blinding can be achieved in surgical trials and that it is necessary if the outcomes are subjective.

Placebo-controlled surgical trials represent a single type of trial that requires blinding, but similar arguments apply to trials comparing two different surgical methods.

In this paper, we did not discuss the placebo because our main focus was blinding as a way to control bias. We used placebo-controlled surgical trials as an example but similar arguments apply to trials comparing two different surgical methods. We have already written about barriers in completing placebo-controlled surgical trials (Wartolowska et al., 2016), about the balance between harms and benefits of such trials (Wartolowska et al., 2014) and about the ethical implications of placebo in surgery (Savulescu, Wartolowska and Carr, 2016)(George et al., 2016).

We have used acupuncture as an example of an experimental study on placebo effect because acupuncture is the only type of invasive procedure used in such studies.

We have included radiofrequency procedures because they change anatomy. In this paper, we have defined surgery as “any interventional procedure that changes the anatomy and requires a skin incision or the use of endoscopic techniques; dental studies were excluded. We used the term placebo to refer to a surgical placebo, a sham surgery, or an imitation procedure intended to mimic the active intervention; including the scenario when a scope was inserted and nothing was done but patients were sedated or under general anaesthesia and could not distinguish whether or not they underwent the actual surgery”.(Wartolowska et al., 2014) We have added these definitions to the manuscript.

“Outcomes were classified as “subjective”, i.e., patient-reported and depending on the patients’ perception and cooperation, “assessed”, i.e., subjective ratings by external assessors and depending upon their judgment, and “objective”, i.e., measured using devices or laboratory tests and independent of patients’ or observers’ perception, for example, weight.”(Wartolowska et al., 2016)

As suggested by the Reviewers, we have explained in the section titled “Bias specific to surgical trials” that by including only patients with non-life-threatening symptoms or with sufficiently severe symptoms on the day of surgery a bias is created, which is difficult to control for.

As suggested by the Reviewers, we have added the paper by Page et al. as a reference in this paragraph. We have also added a reference that provides a general overview of blinding in trials (Hróbjartsson and Boutron, 2011) and provided references to Rosenthal’s papers to explain the “Pygmalion effect”.(Rosenthal and Jacobson, 1968)

REFERENCES
George, A. et al. (2016) ‘When should placebo surgery as a control in clinical trials be carried out?’, The Bulletin of the Royal College of Surgeons of England, 98(2), pp. 75–79.
Hróbjartsson, A. and Boutron, I. (2011) ‘Blinding in randomized clinical trials: Imposed impartiality’, Clinical Pharmacology and Therapeutics. Nature Publishing Group, 90(5), pp. 732–736.
Rosenthal, R. and Jacobson, L. (1968) ‘Pygmalion in the classroom’, The Urban Review, 3(1), pp. 16–20.
Savulescu, J., Wartolowska, K. A. and Carr, A. (2016) ‘Randomised Placebo-Controlled Trials of Surgery: Ethical Analysis and Guidelines’, J Med Ethics, 42(12), pp. 776–783.
Wartolowska, K. et al. (2014) ‘Use of placebo controls in the evaluation of surgery: systematic review.’, BMJ (Clinical research ed.), 348(May), p. g3253.
Wartolowska, K. et al. (2016) ‘Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review’, BMJ Open, 6(3), p. e010194.
Wartolowska, K. A. et al. (2016) ‘The Magnitude and Temporal Changes of Response in the Placebo Arm of Surgical Randomized Controlled Trials - A Systematic Review and Meta-Analysis’, Trials. Trials, 17(1), p. 589.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

9 Views

04 Dec 2017 | for Version 1

Harald Walach, Department of Pediatric Gastroenterology, Medical University Poznan, Poznań, Poland

9 Views Cite this report Responses(1)

Approved

Is the topic of the review discussed comprehensively in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Research methodology, clinical and other trials

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

We are grateful for the comment regarding the hierarchy of evidence. We have rephrased the sentence so that it now reads “high level of evidence because they attempt to minimise bias due to confounding factors”. We have also added the suggested reference (Walach and Loef, 2015).

We agree that RCTs have high internal validity because they are performed in a standardised way, and because they attempt to control for confounding factors. However, owing to the controlled conditions under which they are performed, RCTs often have low external validity. In this paper, we wanted to draw attention to the fact that surgical trials often use subjective outcomes, and are therefore prone to bias, which has to be minimised or controlled for.

We agree with the Reviewer, that the “specific” and “non-specific” effects are difficult to separate or to unequivocally define. As surgical procedures are complex and consist of many elements, it is sometimes difficult to identify the crucial surgical element that is believed to be therapeutic. We have described the CSAW trial as an example. The aim of this trial was to investigate whether the improvement was due to change of anatomy (arthroscopic removal of a bony spur on the acromion) or it was not related to the change in anatomy and the spur removal was not necessary because a placebo procedure (arthroscopy only) results in similar change in pain and function.

We wanted to draw attention to the fact that in the case of surgical trials, with their high costs and risks, one would expect to see a much larger effect after a surgical procedure than after a placebo or no treatment.

We are familiar with the debate in the literature regarding specific and non-specific effects. In the context of surgical trials, by “specific effects” we meant changes directly related to change in anatomy, for example, a bony spur removal in the trial of arthroscopic shoulder decompression surgery for subacromial pain. All other possible factors resulting in improvement, such as the effect of arthroscopy, the placebo effect, influence of patient-doctor interactions, symptoms fluctuation as well as spontaneous improvement were regarded as “non-specific”. We also agree with Enck and colleagues that the effect in the placebo and active arms may not be additive; therefore, the improvement related to the crucial surgical element may not equal the difference between the effect in the active and placebo arms.(Enck et al., 2011) We would like to clarify that blinding and placebo control help to minimise bias and aid in the interpretation of trial results; especially if the outcomes are subjective and the symptoms fluctuate or may spontaneously improve.
REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.
Walach, H. and Loef, M. (2015) ‘Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence’, Journal of Clinical Epidemiology. Elsevier Inc, 68(11), pp. 1251–1260.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

13 Views

20 Sep 2017 | for Version 1

Ian A Harris, Whitlam Orthopaedic Research Centre, Ingham Institute for Applied Medical Research, South Western Sydney Clinical School, University of New South Wales, Liverpool, NSW, Australia

13 Views Cite this report Responses(1)

Approved

Page 2, second last paragraph, “crucial element of believed to be the cure” should read “crucial element believed to be the cure”
The evidence of bias resulting from not blinding may be one sided. Opposing evidence should be discussed, such as Berkmann ND, The Empirical Evidence of Bias in Trials Measuring Treatment Differences¹ which shows that the effect of blinding may even be in the opposite direction and the classic Schultz article from 1995¹ which gives a lower estimate of effect exaggeration from unblinding. There are several other summaries and meta-epidemiological studies in this area.

Is the topic of the review discussed comprehensively in the context of the current literature?

Yes
Are all factual statements correct and adequately supported by citations?

Yes
Is the review written in accessible language?

Yes
Are the conclusions drawn appropriate in the context of the current research literature?

Yes

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Orthopaedics, Epidemiology

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (1)

Author Response

30 Jan 2018

Karolina Wartolowska, University of Oxford, UK

We are grateful for the comment and we have added an explanation that trials have limited external validity, as by controlling for confounding factors and standardising the trial conditions, they become very different from usual clinical practice. They are also liable to have study populations that are so unique that the results of the trial cannot be easily generalised.

In the case of blinding, the magnitude of effect may be lower in a blinded trial than in everyday practice, because blinding causes uncertainty about treatment allocation. In clinical practice, there is usually no doubt as the patient-doctor relationship is based on trust and on the assumption that doctors offer patients the best available treatment. In an RCT, there is an inherent uncertainty as to which treatment group a patient has been allocated.(Enck et al., 2011)

REFERENCES
Enck, P. et al. (2011) ‘The placebo response in clinical trials: more questions than answers.’, Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 366, pp. 1889–1895.

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. Gøtzsche PC: Is there logic in the placebo? Lancet. 1994; 344(8927): 925–6. PubMed Abstract | Publisher Full Text

[2] 2. Walach H, Loef M: Using a matrix-analytical approach to synthesizing evidence solved incompatibility problem in the hierarchy of evidence. J Clin Epidemiol. 2015; 68(11): 1251–60. PubMed Abstract | Publisher Full Text

[3] 3. McLeod RS: Issues in surgical randomized controlled trials. World J Surg. 1999; 23(12): 1210–4. PubMed Abstract | Publisher Full Text

[4] 4. Wente MN, Seiler CM, Uhl W, et al.: Perspectives of evidence-based surgery. Dig Surg. 2003; 20(4): 263–9. PubMed Abstract | Publisher Full Text

[5] 5. Ergina PL, Cook JA, Blazeby JM, et al.: Challenges in evaluating surgical innovation. Lancet. 2009; 374(9695): 1097–104. PubMed Abstract | Publisher Full Text | Free Full Text

[6] 6. Cook JA: The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials. 2009; 10: 9. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. Fairbank J: Randomised controlled trials in surgery. Lancet. 1999; 354(9174): 257. PubMed Abstract | Publisher Full Text

[8] 8. Wartolowska K, Judge A, Hopewell S, et al.: Use of placebo controls in the evaluation of surgery: systematic review. BMJ. 2014; 348: g3253. PubMed Abstract | Publisher Full Text | Free Full Text

[9] 9. Ware JE: Measuring patients’ views: the optimum outcome measure. BMJ. 1993; 306(6890): 1429–30. PubMed Abstract | Publisher Full Text | Free Full Text

[10] 10. Hróbjartsson A, Gøtzsche PC: Is the placebo powerless? Update of a systematic review with 52 new randomized trials comparing placebo with no treatment. J Intern Med. 2004; 256(2): 91–100. PubMed Abstract | Publisher Full Text

[11] 11. Wood L, Egger M, Gluud LL, et al.: Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008; 336(7644): 601–5. PubMed Abstract | Publisher Full Text | Free Full Text

[12] 12. Page MJ, Higgins JP, Clayton G, et al.: Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies. PLoS One. 2016; 11(7): e0159267. PubMed Abstract | Publisher Full Text | Free Full Text

[13] 13. Hróbjartsson A, Boutron I: Blinding in randomized clinical trials: Imposed impartiality. Clin Pharmacol Ther. 2011; 90(5): 732–6. PubMed Abstract | Publisher Full Text

[14] 14. Deaton A, Cartwright N: Understanding and misunderstanding randomized controlled trials. Soc Sci Med. 2017; pii: S0277-9536(17)30735-9. PubMed Abstract | Publisher Full Text

[15] 15. Enck P, Klosterhalfen S, Weimer K, et al.: The placebo response in clinical trials: more questions than answers. Philos Trans R Soc Lond B Biol Sci. 2011; 366(1572): 1889–95. PubMed Abstract | Publisher Full Text | Free Full Text

[16] 16. Wartolowska KA, Feakins BG, Collins GS, et al.: The magnitude and temporal changes of response in the placebo arm of surgical randomized controlled trials: a systematic review and meta-analysis. Trials. 2016; 17(1): 589. PubMed Abstract | Publisher Full Text | Free Full Text

[17] 17. Hróbjartsson A, Emanuelsson F, Skou Thomsen AS, et al.: Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies. Int J Epidemiol. 2014; 43(4): 1272–83. PubMed Abstract | Publisher Full Text | Free Full Text

[18] 18. Nüesch E, Reichenbach S, Trelle S, et al.: The importance of allocation concealment and patient blinding in osteoarthritis trials: A meta-epidemiologic study. Arthritis Rheum. 2009; 61(12): 1633–41. PubMed Abstract | Publisher Full Text

[19] 19. Desharnais R, Jobin J, Côté C, et al.: Aerobic exercise and the placebo effect: a controlled study. Psychosom Med. 1993; 55(2): 149–54. PubMed Abstract | Publisher Full Text

[20] 20. Bradley JD, Heilman DK, Katz BP, et al.: Tidal irrigation as treatment for knee osteoarthritis: a sham-controlled, randomized, double-blinded evaluation. Arthritis Rheum. 2002; 46(1): 100–8. PubMed Abstract | Publisher Full Text

[21] 21. Wartolowska K, Collins GS, Hopewell S, et al.: Feasibility of surgical randomised controlled trials with a placebo arm: a systematic review. BMJ Open. 2016; 6(3): e010194. PubMed Abstract | Publisher Full Text | Free Full Text

[22] 22. Andersson GB, Mekhail NA, Block JE: A randomized, double-blind, controlled trial: intradiscal electrothermal therapy versus placebo for the treatment of chronic discogenic low back pain. Spine (Phila Pa 1976). 2006; 31(14): 1637–8; author reply 1638. PubMed Abstract | Publisher Full Text

[23] 23. Gillespie MB, Wylie PE, Lee-Chiong T, et al.: Effect of palatal implants on continuous positive airway pressure and compliance. Otolaryngol Head Neck Surg. 2011; 144(2): 230–6. PubMed Abstract | Publisher Full Text

[24] 24. Maurer JT, Sommer JU, Hein G, et al.: Palatal implants in the treatment of obstructive sleep apnea: a randomised, placebo-controlled single-centre trial. Eur Arch Otorhinolaryngol. 2012; 269(7): 1851–6. PubMed Abstract | Publisher Full Text

[25] 25. Hróbjartsson A, Thomsen AS, Emanuelsson F, et al.: Observer bias in randomized clinical trials with measurement scale outcomes: A systematic review of trials with both blinded and nonblinded assessors. CMAJ. 2013; 185(4): E201–11. PubMed Abstract | Publisher Full Text | Free Full Text

[26] 26. Rosenthal R, Jacobson L: Pygmalion in the classroom. Urban Rev. 1968; 3(1): 16–20. Publisher Full Text

[27] 27. Hróbjartsson A, Thomsen AS, Emanuelsson F, et al.: Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012; 344: e1119. PubMed Abstract | Publisher Full Text

[28] 28. Thomsen J, Bretlau P, Tos M, et al.: Ménière's disease: endolymphatic sac decompression compared with sham (placebo) decompression. Ann N Y Acad Sci. 1981; 374: 820–30. PubMed Abstract | Publisher Full Text

[29] 29. Shaheen NJ, Sharma P, Overholt BF, et al.: Radiofrequency ablation in Barrett’s esophagus with dysplasia. N Engl J Med. 2009; 360(22): 2277–88. PubMed Abstract | Publisher Full Text

[30] 30. Gracely RH, Dubner R, Deeter WR, et al.: Clinicians' expectations influence placebo analgesia. Lancet. 1985; 325(8419): 43. PubMed Abstract | Publisher Full Text

[31] 31. Benedetti F: How the doctor's words affect the patient's brain. Eval Health Prof. 2002; 25(4): 369–86. PubMed Abstract | Publisher Full Text

[32] 32. Thomas KB: General practice consultations: is there any point in being positive? Br Med J (Clin Res Ed). 1987; 294(6581): 1200–2. PubMed Abstract | Publisher Full Text | Free Full Text

[33] 33. Kaptchuk TJ, Kelley JM, Conboy LA, et al.: Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome. BMJ (Clin Res Ed). 2008; 336(7651): 999–1003. PubMed Abstract | Publisher Full Text | Free Full Text

[34] 34. Braunholtz DA, Edwards SJ, Lilford RJ: Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect”. J Clin Epidemiol. 2001; 54(3): 217–24. PubMed Abstract | Publisher Full Text

[35] 35. Doherty M, Dieppe P: The “placebo” response in osteoarthritis and its implications for clinical practice. Osteoarthr Cartil. 2009; 17(10): 1255–62. PubMed Abstract | Publisher Full Text

[36] 36. Hróbjartsson A: What are the main methodological problems in the estimation of placebo effects? J Clin Epidemiol. 2002; 55(5): 430–5. PubMed Abstract | Publisher Full Text

[37] 37. Stone GW, Teirstein PS, Rubenstein R, et al.: A prospective, multicenter, randomized trial of percutaneous transmyocardial laser revascularization in patients with nonrecanalizable chronic total occlusions. J Am Coll Cardiol. 2002; 39(10): 1581–7. PubMed Abstract | Publisher Full Text

[38] 38. Cobb LA, Thomas GI, Dillard DH, et al.: An evaluation of internal-mammary-artery ligation by a double-blind technic. N Engl J Med. 1959; 260(22): 1115–8. PubMed Abstract | Publisher Full Text

[39] 39. Dimond EG, Kittle CF, Crockett JE: Comparison of internal mammary artery ligation and sham operation for angina pectoris. Am J Cardiol. 1960; 5(4): 483–6. PubMed Abstract | Publisher Full Text

[40] 40. Guyuron B, Reed D, Kriegler JS, et al.: A placebo-controlled surgical trial of the treatment of migraine headaches. Plast Reconstr Surg. 2009; 124(2): 461–8. PubMed Abstract | Publisher Full Text

[41] 41. Gross RE, Watts RL, Hauser RA, et al.: Intrastriatal transplantation of microcarrier-bound human retinal pigment epithelial cells versus sham surgery in patients with advanced Parkinson’s disease: a double-blind, randomised, controlled trial. Lancet Neurol. 2011; 10(6): 509–19. PubMed Abstract | Publisher Full Text

[42] 42. Freed CR, Greene PE, Breeze RE, et al.: Transplantation of Embryonic Dopamine Neurons for Severe Parkinson’s Disease. N Engl J Med. 2001; 344(10): 710–9. PubMed Abstract | Publisher Full Text

[43] 43. Abbott J, Hawe J, Hunter D, et al.: Laparoscopic excision of endometriosis: a randomized, placebo-controlled trial. Fertil Steril. 2004; 82(4): 878–84. PubMed Abstract | Publisher Full Text

[44] 44. Arts J, Bisschops R, Blondeau K, et al.: A double-blind sham-controlled study of the effect of radiofrequency energy on symptoms and distensibility of the gastro-esophageal junction in GERD. Am J Gastroenterol. 2012; 107(2): 222–30. PubMed Abstract | Publisher Full Text

[45] 45. Castro M, Rubin AS, Laviolette M, et al.: Effectiveness and safety of bronchial thermoplasty in the treatment of severe asthma: a multicenter, randomized, double-blind, sham-controlled clinical trial. Am J Respir Crit Care Med. 2010; 181(2): 116–24. PubMed Abstract | Publisher Full Text | Free Full Text

[46] 46. Davey C, Zoumot Z, Jordan S, et al.: Bronchoscopic lung volume reduction with endobronchial valves for patients with heterogeneous emphysema and intact interlobar fissures (the BeLieVeR-HIFi trial): study design and rationale. Thorax. 2015; 70(3): 288–90. PubMed Abstract | Publisher Full Text | Free Full Text

[47] 47. Dowson A, Mullen MJ, Peatfield R, et al.: Migraine Intervention With STARFlex Technology (MIST) trial: a prospective, multicenter, double-blind, sham-controlled trial to evaluate the effectiveness of patent foramen ovale closure with STARFlex septal repair implant to resolve refractory migraine headache. Circulation. 2008; 117(11): 1397–404. PubMed Abstract | Publisher Full Text

[48] 48. Sutton CJ, Ewen SP, Whitelaw N, et al.: Prospective, randomized, double-blind, controlled trial of laser laparoscopy in the treatment of pelvic pain associated with minimal, mild, and moderate endometriosis. Fertil Steril. 1994; 62(4): 696–700. PubMed Abstract | Publisher Full Text

[49] 49. Kallmes DF, Comstock BA, Heagerty PJ, et al.: A randomized trial of vertebroplasty for osteoporotic spinal fractures. N Engl J Med. 2009; 361(6): 569–79. PubMed Abstract | Publisher Full Text | Free Full Text

[50] 50. Pauza KJ, Howell S, Dreyfuss P, et al.: A randomized, placebo-controlled trial of intradiscal electrothermal therapy for the treatment of discogenic low back pain. Spine J. 2004; 4(1): 27–35. PubMed Abstract | Publisher Full Text

[51] 51. Kapural L, Vrooman B, Sarwar S, et al.: A randomized, placebo-controlled trial of transdiscal radiofrequency, biacuplasty for treatment of discogenic lower back pain. Pain Med. 2013; 14(3): 362–73. PubMed Abstract | Publisher Full Text

[52] 52. Moseley JB, O’Malley K, Petersen NJ, et al.: A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med. 2002; 347(2): 81–8. PubMed Abstract | Publisher Full Text

[53] 53. Rothstein R, Filipi C, Caca K, et al.: Endoscopic full-thickness plication for the treatment of gastroesophageal reflux disease: A randomized, sham-controlled trial. Gastroenterology. 2006; 131(3): 704–12. PubMed Abstract | Publisher Full Text

[54] 54. Koutsourelakis I, Georgoulopoulos G, Perraki E, et al.: Randomised trial of nasal surgery for fixed nasal obstruction in obstructive sleep apnoea. Eur Respir J. 2008; 31(1): 110–7. PubMed Abstract | Publisher Full Text

[55] 55. Sihvonen R, Paavola M, Malmivaara A, et al.: Arthroscopic partial meniscectomy versus sham surgery for a degenerative meniscal tear. N Engl J Med. 2013; 369(26): 2515–24. PubMed Abstract | Publisher Full Text

[56] 56. Fleischer D: Endoscopic Nd:YAG laser therapy for active esophageal variceal bleeding. A randomized controlled study. Gastrointest Endosc. 1985; 31(1): 4–9. PubMed Abstract | Publisher Full Text

[57] 57. Leon MB, Kornowski R, Downey WE, et al.: A blinded, randomized, placebo-controlled trial of percutaneous laser myocardial revascularization to improve angina symptoms in patients with severe coronary disease. J Am Coll Cardiol. 2005; 46(10): 1812–9. PubMed Abstract | Publisher Full Text

[58] 58. Nease CJ, Krempl GA: Radiofrequency treatment of turbinate hypertrophy: a randomized, blinded, placebo-controlled clinical trial. Otolaryngol Head Neck Surg. 2004; 130(3): 291–9. PubMed Abstract | Publisher Full Text

[59] 59. Roehrborn CG, Gange SN, Shore ND, et al.: The prostatic urethral lift for the treatment of lower urinary tract symptoms associated with prostate enlargement due to benign prostatic hyperplasia: the L.I.F.T. Study. J Urol. 2013; 190(6): 2161–7. PubMed Abstract | Publisher Full Text

[60] 60. Mathus-Vliegen EM, Tytgat GN, Veldhuyzen-Offermans EA: Intragastric balloon in the treatment of super-morbid obesity. Double-blind, sham-controlled, crossover evaluation of 500-milliliter balloon. Gastroenterology. 1990; 99(2): 362–9. PubMed Abstract | Publisher Full Text

[61] 61. Salem M, Rotevatn S, Stavnes S, et al.: Usefulness and safety of percutaneous myocardial laser revascularization for refractory angina pectoris. Am J Cardiol. 2004; 93(9): 1086–91. PubMed Abstract | Publisher Full Text

[62] 62. Devière J, Costamagna G, Neuhaus H, et al.: Nonresorbable copolymer implantation for gastroesophageal reflux disease: a randomized sham-controlled multicenter trial. Gastroenterology. 2005; 128(3): 532–40. PubMed Abstract | Publisher Full Text

[63] 63. Buchbinder R, Osborne RH, Ebeling PR, et al.: A randomized trial of vertebroplasty for painful osteoporotic vertebral fractures. N Engl J Med. 2009; 361(6): 557–68. PubMed Abstract | Publisher Full Text

[64] 64. Bono CM, Heggeness M, Mick C, et al.: North American Spine Society: Newly released vertebroplasty randomized controlled trials: a tale of two trials. Spine J. 2010; 10(3): 238–40. PubMed Abstract | Publisher Full Text

[65] 65. Swank DJ, Swank-Bordewijk SC, Hop WC, et al.: Laparoscopic adhesiolysis in patients with chronic abdominal pain: a blinded randomised controlled multi-centre trial. Lancet. 2003; 361(9365): 1247–51. PubMed Abstract | Publisher Full Text

[66] 66. Jarrell J, Mohindra R, Ross S, et al.: Laparoscopy and reported pain among patients with endometriosis. J Obstet Gynaecol Can. 2005; 27(5): 477–85. PubMed Abstract | Publisher Full Text

[67] 67. Eid GM, McCloskey CA, Eagleton JK, et al.: StomaphyX vs a sham procedure for revisional surgery to reduce regained weight in Roux-en-Y gastric bypass patients : a randomized clinical trial. JAMA Surg. 2014; 149(4): 372–9. PubMed Abstract | Publisher Full Text

[68] 68. Montgomery M, Hakanson B, Ljungqvist O, et al.: Twelve months’ follow-up after treatment with the EndoCinch endoscopic technique for gastro-oesophageal reflux disease: a randomized, placebo-controlled study. Scand J Gastroenterol. 2006; 41(12): 1382–9. PubMed Abstract | Publisher Full Text

[69] 69. Friedman M, Schalch P, Lin HC, et al.: Palatal implants for the treatment of snoring and obstructive sleep apnea/hypopnea syndrome. Otolaryngol Head Neck Surg. 2008; 138(2): 209–16. PubMed Abstract | Publisher Full Text

[70] 70. Olanow CW, Goetz CG, Kordower JH, et al.: A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Ann Neurol. 2003; 54(3): 403–14. PubMed Abstract | Publisher Full Text

[71] 71. Fockens P, Cohen L, Edmundowicz SA, et al.: Prospective randomized controlled trial of an injectable esophageal prosthesis versus a sham procedure for endoscopic treatment of gastroesophageal reflux disease. Surg Endosc. 2010; 24(6): 1387–97. PubMed Abstract | Publisher Full Text | Free Full Text

[72] 72. Ernst E, Resch KL: Concept of true and perceived placebo effects. BMJ. 1995; 311(7004): 551–3. PubMed Abstract | Publisher Full Text | Free Full Text

[73] 73. Simpson SH, Eurich DT, Majumdar SR, et al.: A meta-analysis of the association between adherence to drug therapy and mortality. BMJ. 2006; 333(7557): 15. PubMed Abstract | Publisher Full Text | Free Full Text

[74] 74. Cotton PB, Durkalski V, Romagnuolo J, et al.: Effect of endoscopic sphincterotomy for suspected sphincter of Oddi dysfunction on pain-related disability following cholecystectomy: the EPISOD randomized clinical trial. JAMA. 2014; 311(20): 2101–9. PubMed Abstract | Publisher Full Text | Free Full Text

[75] 75. Hartigan P: Sclerotherapy for male alcoholic cirrhotic patients who have bled from esophageal varices: Results of a randomized, multicenter clinical trial. The Veterans Affairs Cooperative Variceal Sclerotherapy Group. Hepatology. 1994; 20(3): 618–25. PubMed Abstract | Publisher Full Text

[76] 76. Freitas D, Donato A, Monteiro JG: Controlled Trial of Liquid Monopolar Electrocoagulation in Bleeding Peptic Ulcers. Am J Gastroenterol. 1985; 80(11): 853–7. PubMed Abstract

[77] 77. MacLeod IA, Mills PR, MacKenzie JF, et al.: Neodymium yttrium aluminium garnet laser photocoagulation for major haemorrhage from peptic ulcers and single vessels: a single blind controlled study. Br Med J (Clin Res Ed). 1983; 286(6362): 345–8. PubMed Abstract | Publisher Full Text | Free Full Text

[78] 78. Scolapio JS, Gostout CJ, Schroeder KW, et al.: Dysphagia without endoscopically evident disease: to dilate or not? Am J Gastroenterol. 2001; 96(2): 327–30. PubMed Abstract | Publisher Full Text

Blinding in trials of interventional procedures is possible and worthwhile

Abstract

Keywords

Revised Amendments from Version 1

Introduction

Definitions

Blinding of patients, surgeons, outcome assessors, and care-givers

Blinding of patients

Blinding of surgeons

Blinding of outcome assessors

Blinding of care-givers

Strategies used to maintain blinding in interventional placebo-controlled trials

Imitation of incision/ surgical access point

Simulation of an interventional procedure

Inactive nature of the placebo

Duration of the procedure

Additional procedures that may reveal allocation

Standardisation of interventions and care

Blinding after the surgery

Bias specific to surgical trials

Conclusions

Competing interests

Grant information

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated