Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia: protocol for a quasi-experimental, mixed-methods research study

Feisul Mustapha; Michael Calopietro; Karoline Kragelund Nielsen; Jens Aagaard-Hansen; Shiang Cheng Lim; Ulla Bjerre-Christensen

doi:10.12688/f1000research.21127.1

Home Browse Impact evaluation of the Steno REACH Certificate Course in Clinical...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Study Protocol

Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia: protocol for a quasi-experimental, mixed-methods research study

[version 1; peer review: 2 approved with reservations]

Feisul Mustapha ¹, Michael Calopietro², Karoline Kragelund Nielsen³, Jens Aagaard-Hansen^3,4, Shiang Cheng Lim⁵, Ulla Bjerre-Christensen³

Feisul Mustapha ¹, Michael Calopietro², [...] Karoline Kragelund Nielsen³, Jens Aagaard-Hansen^3,4, Shiang Cheng Lim⁵, Ulla Bjerre-Christensen³

PUBLISHED 10 Feb 2020

Author details Author details

¹ Disease Control Division, Ministry of Health Malaysia, Putrajaya, Putrajaya, 62590, Malaysia
² Novo Nordisk, Copenhagen, Denmark
³ Steno Diabetes Center, Copenhagen, Denmark
⁴ University of the Witwatersrand, Johannesburg, South Africa
⁵ United Nations University - International Institute for Global Health, Kuala Lumpur, Malaysia

Feisul Mustapha
Roles: Conceptualization, Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Michael Calopietro
Roles: Conceptualization, Methodology, Project Administration, Writing – Review & Editing

Karoline Kragelund Nielsen
Roles: Conceptualization, Methodology, Writing – Review & Editing

Jens Aagaard-Hansen
Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Shiang Cheng Lim
Roles: Project Administration, Writing – Review & Editing

Ulla Bjerre-Christensen
Roles: Funding Acquisition, Supervision, Writing – Review & Editing

OPEN PEER REVIEW

REVIEWER STATUS

Abstract

The burden of diabetes continues to increase in Malaysia, and the public primary health sector has an insufficient number of health care providers well-trained in diabetes care. The Ministry of Health Malaysia collaborated with Steno Diabetes Center to educate primary care doctors and nurses on the fundamentals of clinical diabetes care using a competency-based approach that blends e-learning, classroom-based learning, and clinic-based group work. This programme is called Steno REACH Certificate Course in Clinical Diabetes Care (SRCC).

The aim of this study was to assess the effectiveness of the SRCC intervention in improving diabetes-related knowledge, attitudes, skills and clinical practices among non-specialised doctors and general nurses working in public health clinics in Malaysia. This paper presents the study protocol.

A quasi-experimental, mixed-methods study based on Solomon’s Four Group Design was applied. Non-specialist doctors and general nurses from ten health clinics were randomly selected to receive the educational intervention. Comparison clinics were purposive selected matching on proxy indicators for quality of diabetes care. The intervention consisted of 50 hours of e-learning, 48 hours of classroom-based learning and approximately 25 hours of work-based learning that covered all main aspects of clinical diabetes care and delivered over a six-month period. Primary outcomes were changes in diabetes-related knowledge, attitudes, skills, and clinical practice. Patients’ perceptions regarding the quality of care provided were classified as a secondary outcome. Other outcome measures included patients' assessment of their chronic disease care and providers' perceptions, attitudes and perceived barriers in care delivery.

Results from this study will inform future educational approaches within the Malaysian health system. The study is unique because it evaluated a pertinent public health topic using a very robust methodology.

Keywords

Continuing medical education, diabetes, healthcare providers, Malaysia, mixed methods, Solomon’s Four Group Design

Corresponding author: Feisul Mustapha

Competing interests: No competing interests were disclosed.

Grant information: This research was fully funded by the Novo Nordisk Foundation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Copyright: © 2020 Mustapha F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite: Mustapha F, Calopietro M, Nielsen KK et al. Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia: protocol for a quasi-experimental, mixed-methods research study [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:98 (https://doi.org/10.12688/f1000research.21127.1) First published: 10 Feb 2020, 9:98 (https://doi.org/10.12688/f1000research.21127.1) Latest published: 10 Feb 2020, 9:98 (https://doi.org/10.12688/f1000research.21127.1)

Introduction

The burden of diabetes continues to increase in Malaysia. Data from the 2015 population-based National Health and Morbidity Survey (NHMS) showed a prevalence of 17.5% among adults aged 18 years and above, which translated to about 3.5 million adults¹. The overall prevalence of diabetes is amongst the highest in the Asia-Pacific region, excluding the pacific island nations².

Primary health care is at the center of the Malaysian healthcare system and is supported by secondary and tertiary care. Malaysia has a well-established, public primary health care structure, with 1,061 health clinics, 1,810 community clinics, and 307 “out-reach” health clinics (called 1Malaysia clinics)³. Although Malaysia has a parallel public and private system, the majority of treatment for chronic diseases is provided by the public health system which is heavily subsidised by the government.

The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes. The latest data showed that about 80% of diagnosed diabetes patients seek treatment at MOH healthcare facilities and this proportion is expected to continue to increase¹. As the burden of diabetes increases, the public health sector is faced with an insufficient number of well-trained diabetes medical practitioners to handle the increasing number of diabetes patients⁴. This has contributed to a treatment gap that requires MOH healthcare professionals to be better equipped to provide patient-centered diabetes services at the primary care level.

A large part of addressing this treatment gap focuses on human resource capacity building in the management of non-communicable diseases (NCDs)^5,6. Continuing medical education (CME) is a cornerstone in developing clinical skills and ensuring high-quality patient care by nurses and medical doctors. Cervero and Gaines (2015) conclude in a synthesis of systematic reviews, that CME improves physician performance and patient outcomes⁷, and while studies on CME in general have reported a positive effect on physician performance and patient outcomes, several have expressed the need for further studies on the implementation of knowledge, skills, and attitudes, as well as the impact of contextual and implementation factors⁷. Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting^8–10. These studies have shown varying results^8,10–13. Studies have also shown that CME is associated with increased satisfaction and a better psychosocial wellbeing of diabetes patients¹¹, and that it is very well received among participating healthcare providers (HCPs)¹⁴.

The SRCC intervention

As part of its efforts to build diabetes management capacity, MOH Malaysia collaborated with Steno Diabetes Center to educate primary care doctors and nurses on the fundamentals of clinical diabetes care using a competency-based approach that blends e-learning, classroom-based learning, and clinic-based group work. This programme is called the Steno REACH Certificate Course in Clinical Diabetes Care (SRCC). The goal of SRCC is to improve the knowledge and skills of participating HCPs in clinical diabetes management, thereby empowering them to provide high-quality diabetes care. The course was designed for primary care doctors and nurses with little training in diabetes care and was facilitated by a team of Malaysian health care professionals trained to deliver the programme by experts from Steno Diabetes Center Copenhagen.

The curriculum included ten modules:

1. Diagnosis and Pathophysiology
2. Patient Engagement
3. Non-Pharmacological Treatment
4. Pharmacological Treatment
5. Insulin Therapy
6. Acute Complications
7. Microvascular Complications
8. Macrovascular Complications
9. Diabetes and Pregnancy
10. Clinical Quality

Two versions of each module were developed, one for doctors and one for nurses, with content tailored to their individual job roles. The intervention was designed to have learners cycle through a schedule of approximately 50 hours of independent online study and 48 hours of face-to-face classroom time. The e-learning content included all foundational materials delivered in an interactive learning environment. Participants completed the first five modules before coming to the classroom to reinforce key learning outcomes through interactive learning activities. This experience repeated itself for modules 6 to 10. In addition, participants were assigned work-based learning activities to complete in the periods between classroom-based sessions. Work-based learning activities included defining and reflecting on personal learning goals, patient journaling, article discussions with clinic peers, case discussions with clinic peers, and medical record reviews with clinic leaders. Together, work-based learning activities took an additional 25 to 30 hours of learning time.

The intervention was piloted twice between 10 October 2015 and 4 December 2016 to ensure its proper functioning and make any necessary curriculum improvements. Pilot course participants were recruited using a clinic-based recruitment approach. Ten interested clinics were invited to send at least one doctor and one nurse in order to reinforce the team-based care model essential to good diabetes clinical care. A total of 22 non-specialist doctors and 40 general nurses participated in the pilot studies.

This paper describes the protocol of the impact evaluation of the SRCC.

Study objectives

General objective

The impact evaluation aims to assess the effect of participation of non-specialised medical doctors and general nurses in the SRCC in selected public health clinics in Malaysia.

Specific objectives

(1) To measure changes in diabetes-related knowledge, attitudes and clinical skills before and after course participation;
(2) To assess whether course participation and improved diabetes-related knowledge and clinical skills translate into changes in an individual’s clinical practice; and
(3) To examine the influence of contextual factors at facility, health system and sociocultural levels on individual diabetes-related clinical practice changes.

The primary hypothesis of this research was that participation in the SRCC will result in increased knowledge about diagnosing diabetes, the role of diet and exercise in the treatment of diabetes, oral and injectable therapies used in the treatment of diabetes, common diabetes complications, and approaches to patient engagement. It was also the hypothesis that improved knowledge would translate into improved clinical skills in these aspects of care and that providers’ attitudes about people with diabetes would be positively impacted.

In addition, it was hoped that 1) patients’ perceptions of their care experience would improve with the intervention, and 2) primary care providers’ perceptions, attitudes, experiences and perceived barriers in implementing the intervention were explored.

Methods

Study design

A quasi-experimental, mixed-methods approach was applied in this study, based on Solomon’s Four Group Design, which is ideally used in evaluation of educational interventions that contain pre- and post-assessments. This design is recommended when it is possible that the pre-test could influence later tests, for example by learning or priming effects. This design contain four groups. In addition to the basic pre-test/intervention/post-test groups, three additional groups are included: one that received both tests, but not the intervention; one that received the intervention without the pre-test only; and one with neither pre-test nor intervention. This design serves to reduce the influence of confounding variables and allows the researcher to test whether the pretest itself has an effect on the subjects. Doctors and nurses from comparison clinics not participating in SRCC were also assessed to isolate individual and clinic level changes that may be attributable to the educational intervention.

Each of the four study arms were subjected to a different set of data collection methods at different points of the intervention. This is shown diagrammatically in Figure 1.

Figure 1. Overview of research design showing the different points and methods of data collection.

Selection of clinics and participants

From a pre-determined sampling frame, health clinics in the states of Kuala Lumpur and Selangor were randomly selected for the four arms of this research. These two states were selected as they have the full range of the diverse types of health clinics located in Malaysia. The four arms were selected to conform to Solomon’s Four Group Design. This design allows researchers to control for the possibility of a test-effect.

Arm 1: Intervention clinics with pre and post investigations
Arm 2: Intervention clinics with post investigation only
Arm 3: Comparison clinics with pre and post investigations (no intervention)
Arm 4: Comparison clinics with a single investigation only (no intervention)

The pilot test of the educational intervention included ten clinics, and based on this experience, ten clinics were deemed an appropriate number of clinics to support a diverse learning experience among participants while still limiting class size to an appropriate level for delivering a heavily facilitated and interactive learning experience. All MOH health clinics in Selangor and Kuala Lumpur meeting the inclusion criteria were subjected to a two-stage stratified random sampling for selection of the ten intervention clinics. Clinics meeting the following criteria were eligible for random selection:

Inclusion criteria were: MOH health clinics with more than 1,000 registered active diabetes patients (as defined and registered in the National Diabetes Registry).

Exclusion criteria were: MOH health clinics that have participants enrolled in the two SRCC pilot classes.

A total of 70 possible clinics were reduced to 43 eligible clinics based on the above criteria, and these 43 were included in the stratified, random selection process. For the first stage, the 43 health clinics were divided into the following three categories based on the number of registered active diabetes patients:

Between 1,000 and 1,999 patients
Between 2,000 and 2,999 patients
≥3,000 patients

For the second stage the health clinics were divided into the following two categories based on the variety and complexity of medical services they provide as defined by MOH:

Intermediate clinic
Advanced clinic

Based on the above criteria, the number of eligible clinics is summarised in Table 1. Ten clinics were then randomly selected for inclusion as the intervention clinics as shown in Table 2.

Table 1. Distribution of clinics within each category after the 2-level selections (n=43).

	1,000 to 1,999 patients	2,000 to 2,999 patients	≥3,000 patients
Intermediate clinic	8	8	8
Advanced clinic	11	4	4

Table 2. Distribution of clinics randomly enrolled into the intervention arm (n=10).

	1,000 to 1,999 patients	2,000 to 2,999 patients	≥3,000 patients
Intermediate clinic	2	2	2
Advanced clinic	2	1	1

These ten clinics were then randomly allocated to one of the two intervention arms. Six clinics were allocated to Arm 1 and four were allocated to Arm 2. The six-to-four split was to allow for more pre- and post-data to be included for the intervention arm of research, as per Solomon’s Four Group Design, only Arm 1 of the intervention clinics has both pre- and post-data available.

Comparison clinics were then selected from the remaining post-stratification clinics. Six comparison clinics were selected for Arm 3 using a purposive approach in which comparison clinics were matched based on a proxy indicator for quality of diabetes care. Using data from the National Diabetes Registry (NDR), the mean HbA₁c value for all patients with diabetes in each intervention clinic was calculated and a comparison clinic with the closest mean HbA₁c value for all diabetes patients was matched and selected for inclusion in Arm 3. The same process was used to select the four comparison clinics in Arm 4, which were chosen to match the four intervention clinics in Arm 2.

From each of the ten health clinics selected to receive the intervention, the participants were chosen by the Family Medicine Specialists (FMS) who are the clinical heads of the health clinic, based on the following criteria:

(1) Each clinic nominated at least four participants, two medical officers and two nurses.
(2) The medical officers had to be a non-specialist without advanced training in diabetes care.
(3) The nurses should not have undergone post-basic or advanced training in diabetes care.
(4) The participants at the clinic had to be able to interact as a team at their workplace during the six-month SRCC.
(5) The participants were either already managing or interacting with diabetes patients prior to enrolment, or if the participants were not currently managing or interacting with diabetes patients, that opportunity had to be provided to them during the six-month study period of the SRCC.

Participants from the intervention clinics were enrolled in the programme. Participants from the comparison clinics were then selected so that their clinical experience with diabetes patients matched the participants in the intervention arm as much as possible. Participants from the comparison clinics were offered enrolment in an SRCC class after the completion of the data collection.

The number of participants in each arm of the research is shown in Table 3.

Table 3. Distribution of Participants by Arm (n=77).

Arm	Number of Medical Officers	Number of Nurses	Total Number of Participants
1	11	12	23
2	6	10	16
3	9	13	22
4	6	10	16

Sample size considerations

Despite the fairly small number of participants in each pre and post-test group, the validity of the pre and post-test results can be achieved through a meta-analysis of the data. A statistical treatment of the quantitative data in Solomon’s Four Group Design is possible with a meta-analytical approach¹⁵. With this approach, the results of different statistical tests (2x2 ANOVA, repeated measure ANOVA and double tailed t-test) are combined to create statistical power without the need for a large sample. Meta-analysis demonstrates how the results from disparate, independent tests of the same hypotheses may be statistically combined even when the significance tests arise through different statistical techniques¹⁶.

Data collection methods

This study utilised both quantitative and qualitative methods to achieve the specific research objectives. The frequency and timing of these methods were applied depending on the research arm as described in Figure 1. A summary of the different data collection methods and purpose is shown in Table 4. All interviewers, investigators and examiners were trained regarding the study procedures prior to the conduct of the research to minimise variability in the method of data collection.

Table 4. Summary of methods of data collection, target and purpose.

	Data collection method	Target	Purpose
1.	Examination & Diabetes Attitude Scale	SRCC participants	To assess the level of knowledge and attitude of participants
2.	Objective structured clinical examination (OSCE)	SRCC participants	To assess applied clinical reasoning and actual skills of participants in a standardised setting
3.	In-depth interviews	SRCC participants	To describe how participants are using increased knowledge and improved skills in their individual clinical practice
4.	Key informant interviews	Family Medicine Specialist	To examine the contextual factors that may impact the application of clinical skills
5.	Direct observation	SRCC participants and clinics	To evaluate how well participants are using increased knowledge and improved skills in their individual clinical practice
6.	Patient exit interview	Patients	To describe the patients' experience and perception of the clinical encounter (post-direct observation of participants)
7.	National Diabetes Registry	Secondary data	To assess if there is early indication that individual clinical practice affects clinic level performance

Quantitative data collection. All course participants were subjected to the same core data collection methods that aim to describe and measure the effectiveness of the educational programme. The quantitative study tools are as follows. The majority of data collection instruments are available as Extended data¹⁷; however, since the multiple choice questions (MCQs) are still being used to assess doctors and nurses, these are only available on request on a case-by-case basis.

Multiple choice questions assessing knowledge

The SRCC test of clinical diabetes-related knowledge was developed by a team of endocrinologist and diabetes nurses based on a review of the full course curriculum for medical officers and nurses. The medical officers’ version included 79 MCQs made available in English and the nurses’ version included 67 MCQs and was available in English and Malay language. Since these questions are still being used to assess candidates, they cannot be made openly available.

Diabetes Attitude Scale

The third version of Diabetes Attitude Scale 3 (DAS-3) is a previously validated general measure of diabetes-related attitudes developed by the University of Michigan Diabetes Research and Training Center¹⁸. This method has been shown to be suitable for evaluation of professional education programmes provided that they include the specific topic areas measured by the five DAS-3 sub-scales¹⁸, including the attitude of HCPs on the need for special training in education, seriousness of type 2 diabetes, the overall value of tight glucose control in diabetes care, psychosocial impact of diabetes on patients, and attitude toward patient autonomy, which is the case with SRCC.

The test was administered to participants in the pilot study and their results were evaluated by an independent medical education consultancy firm who evaluated each version of the test for reliability using Chronbach’s α. The resulting α of 0.61 was slightly lower than the gold standard of 0.8, but deemed sufficiently reliable for a non-high stakes examination, particularly given the relatively small sample size of the pilot cohort.

Objective structured clinical examination (OSCE)

An OSCE is a short circuit of stations at which each participant is examined individually by one or more experienced examiners using real or simulated (actor) patients. The OSCE stations were developed by a team of endocrinologists and diabetes nurses, based on a review of the full course curriculum for medical officers and nurses. Here, medical officers were examined at nine stations and nurses at eight stations. At each station, participants were asked to perform a limited number of clinical tasks within a specified time period (15 minutes) during which two trained examiners used a marking scheme to differentiate good performance from poor performance. The examiners were FMS for the medical officers, and senior diabetes educators for the nurses.

For each observed competency, each examiner assigned a competency ranking (very poor, poor, acceptable, good, very good). Each participant also received an overall station score for each station (1–<2 = fail, 2–<3 = borderline, 3–<4 = pass, 4–<5 = good, 5 = outstanding) as well as domain scores representing the different measured skills within that station.

Qualitative data collection. Additional qualitative investigations were also carried out to better understand how knowledge is being acquired and applied in clinical practice and to create a reliable description of the clinical context into which learnings are being integrated. Qualitative investigations were conducted in Arms 1, 2 and 3 clinics, using the following qualitative tools.

In-depth interviews with SRCC participants

Pre- and post-intervention in-depth interviews were conducted with all participants in Arm 1 clinics (23 in total), in their respective clinics, in order to better understand their history with diabetes related training, typical clinical encounters with people with diabetes and daily clinic life. Interviewed participants were also asked about the process of learning from the various learning platforms and each explored the application of new knowledge and skills to clinical practice. These interviews were audio recorded.

Key informant interviews with FMS

FMSs from the intervention (Arm 1 and 2) and comparison (Arm 3) clinics were interviewed pre- and post-interaction to obtain information on the organisation of clinical services for diabetes patients in the 16 health clinics of Arm 1, 2 and 3, in their respective clinics. Post-intervention FMS interviews for Arm 1 and 2 clinics included themes of their involvement and interactions with the SRCC participants during the intervention period. These interviews were audio recorded.

Interviews with patients receiving care from SRCC participants

Short semi-structured interviews lasting between 10 and 15 minutes with 162 randomly selected patients directly after an observed clinical encounter were conducted in Arm 1 clinics, in the respective clinics. Interview data were used to investigate the patients’ experiences and perceptions of the particular observed clinical interaction. The inclusions of patient interviews provided an important opportunity to triangulate data from the direct observations, the HCP interviews, and the patients' experience of the clinical encounter. These interviews were audio recorded.

Direct clinical observation of participants and clinics

Lastly, to investigate the translation of new knowledge into actual clinical practice two days of direct observation of clinical practice of the healthcare professionals from the six clinics participating in Arm 1 (23 in total) were conducted by an experienced clinician of the research team. This researcher assessed the level of diabetes related clinical proficiency demonstrated in each observed clinical consultation. In addition to a narrative assessment of the clinical encounter, each observed session was assigned one of five proficiency levels based on criteria established by the National Institutes of Health Proficiency Scale¹⁹.

An observation-based assessment of the clinical environment was completed and focused on the general clinical environment, interactions between patient and HCPs, and the nature of interactions between health staff working in the clinic.

Data management

In order to manage research data effectively and efficiently as well as to ensure confidentiality, all participating clinics and respondents (participants, FMS and patients) were anonymised and only able to be identified by a sequentially generated ID-number during data collection, follow-up, data processing, analyses and publication. All collected information and data, such as consent forms, background information of respondents, and audio files were stored according to the generated ID-number. Paper records were securely stored in locked file cabinets and also scanned in digital form, while digital records of the files including the audio files were stored and backed-up in a password-protected external thumbdrive and hardisk. All data were only accessed by the principal investigator, research project manager and key research assistants. Data entry and transcriptions were conducted by the research assistants and quality checked by the research project manager and principal investigator.

Data analysis plan

Data were analysed using a convergent parallel design approach, a commonly used mixed-methods analytical approach^20,21. The purpose of the convergent design is to “obtain different, but complementary data on the same topic” to best understand the research questions²². This design is used to triangulate the methods by comparing and contrasting quantitative statistical results with qualitative findings for corroboration and validation purposes. It is also used to synthesise complementary quantitative and qualitative results to develop a more complete understanding of a phenomenon²⁰.

Independent data analysis

Pre and posttest examination, DAS and OSCE scores were analysed using both the 2x2 ANOVA, repeated measure ANOVA test and independent t-test according to the Solomon’s Four Group Design¹⁵.

The interviews were transcribed and analysed using thematic content analysis²⁰. The lead investigator coded the transcripts according to recurrent themes. As the main purpose of this analysis was to identify how participation in the SRCC impacted clinical competency and whether or not there are contextual factors that support or hinder the translation of new knowledge into clinical practice, these formed the main a priori domains into which sub-categories can be grouped. However, the analysis also allowed for the inclusion of themes not pre-defined in the template.

Data from the observations were in the form of descriptive field notes evaluating various aspects of each clinical encounter. Since observational data were collected pre- and post-intervention, the short descriptions of each observed clinical encounter were assessed for observed similarities and differences using an inductive approach in which many observations were analysed to find more abstract generalisations and to build a picture of the phenomenon being studied (i.e. whether the acquisition of new knowledge about diabetes clinical care was translated into practice).

Merging and triangulation of data

The observational data were combined with the in-depth interviews and key-informant interviews to understand more thoroughly how participation in the educational intervention facilitated the demonstration of new clinical skills, and to further explore the contextual factors that enabled or inhibited the translation of new knowledge into practice. The data were further explored to identify emerging and recurrent trends within and between qualitative data and to the incidence of themes within and across qualitative data types.

Quantitative and qualitative data were merged using common mixed-methods strategies²⁰. Firstly, content areas that were represented in both data types were identified, compared, contrasted and synthesised as findings. Secondly, differences in one set of results were further examined based on defined dimensions found in the other data set in an attempt to identify and explore data that complements or contradicts the findings from the shared content areas. The exact methods and procedures of the merging process would be dependent on the results from each data collection method.

Interpretation of merged data focused on a discussion of the ways in which the quantitative and qualitative data converged, diverged, related to each other and produced a more complete understanding of whether and how the educational intervention results in changes in diabetes care competence.

Expected outcomes

Primary outcomes were changes in diabetes-related knowledge, attitudes, skills, and clinical practice. Patients’ perceptions regarding the quality of care provided were classified as a secondary outcome. Other data included patients' assessment of their chronic disease care and providers' perceptions, attitudes and perceived barriers in care delivery.

Research ethics

The Medical Research and Ethics Committee (MREC) of the MOH approved the study protocol (reference: NMRR-16-449-29909 (IIR), dated 7 April 2016). Permissions from the Family Health Development Division (FHDD) of the MOH and the respective Health District Offices were also obtained prior to the study. The study was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice (GCP) requirements.

Study information sheets were distributed to all course participants and observed patients, and informed consent was obtained from all participants and patients prior to their study enrolment. Informed consent forms were made available in English and the Malay language. Confidentiality of personal information was ensured at all times. Participation of the HCP and patients was voluntary.

Since the study included direct observation of clinical encounters, the possibility of observing sub-standard care existed. If diabetes and/or medical care was witnessed that could lead to acute hospitalisation and/or death, the data collector(s) were instructed to immediately raise this concern to the HCP(s), in order to avoid this risk. For substandard diabetes and/or medical care not likely to lead to acute hospitalisation and/or death, investigators were instructed to avoid intervention, so as not to lead to punitive actions toward the involved HCPs, nor to compromise the HCP-patient relationship.

Discussion

This study has two key strengths. Firstly, the use of the robust Solomon’s Four Group Design enabled the researchers not only to assess the intervention effect, but also the presence of pretest sensitisation and the interaction effects between intervention and pretest. The Solomon’s Four Group design has not been extensively used in recent studies, partly because it is a complicated design and partly because the statistical analysis is rather complicated. Secondly, the use of mixed methods combining both quantitative and qualitative methodologies provided the researchers with data on intervention effects on knowledge, attitudes, clinical skills and practices, and allowed triangulation of the data.

There are also inherent limitations in the protocol. Observational data collection methods may have limited validity, including the Hawthorne effect²³; regarding the interviews, there may be courtesy bias, where participants provide responses that they think the interviewer would like to hear. These potential biases can be addressed through triangulation of data from the different data collection methods²⁴.

Part of the statistical analysis required the transformation of qualitative data for the clinical practice into quantitative data. While we acknowledge the concerns of the limitations of quantitised qualitative data for statistical measurement, the transformation process will be clearly described and only simple statistical measures for differences employed.

Data availability

Underlying data

No data are associated with this article.

Extended data

Figshare: Data collection tools for Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia. https://doi.org/10.6084/m9.figshare.11764146.v2¹⁷.

This project contains the following extended data:

OSCE Doctors Stations 1–8 Examiners copy (DOCX). OSCE examination guidelines and scoring rubrics for all OSCE stations (doctors).
OSCE Nurse Stations 1–8 Examiners copy (DOCX). OSCE examination guidelines and scoring rubrics for all OSCE stations (nurses).
Pre- and post-intervention interview guide, patients (DOCX).
Pre-intervention interview guide, doctors and nurses (DOCX)
Pre-intervention interview guide, FMS (DOCX)
Pre- and post-intervention observation format, doctors and nurses (DOCX).
Post-intervention interview guide, FMS (DOCX).
Post-intervention interview guide, doctors and nurses (DOCX).

Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

The multiple choice questions are currently being used as part of the assessment of doctors and nurses undergoing the Steno REACH Certificate Course in Clinical Diabetes Care organised by MOH Malaysia. We would however be willing to share a copy on a case-by-case basis, if we are assured that the confidentiality of the multiple choice questions is maintained. Any queries can be directed to Dr Feisul Mustapha (email: dr.feisul@moh.gov.my).

Acronyms and abbreviations

CME Continuing Medical Education

CPG Clinical Practice Guideline

DAS Diabetes Attitude Scale

FHDD Family Health Development Division

FMS Family Medicine Specialist

GCP Good Clinical Practice

HCP Healthcare Provider

LMIC Low and Middle-income Countries

MCQ Multiple choice questionnaire

MOH Ministry of Health Malaysia

MREC Medical Research and Ethics Committee

NCD Non-Communicable Disease

NDR National Diabetes Registry

NHMS National Health and Morbidity Survey

OSCE Objective Structured Clinical Examination

SRCC Steno REACH Certificate Course in Clinical Diabetes Care

T2D Type 2 Diabetes

Author contributions

FIM, MC, JA-H and UB-C conceived and contributed to the design of the study. FIM and LSC contributed to the acquisition of data. FIM, KKN and MC each wrote first drafts of parts of the manuscript. All authors critically revised the manuscript for intellectual content, and have read and approved the final manuscript.

Acknowledgements

We would like to thank the Director General of Health, Malaysia for his permission to publish this article.

Faculty Opinions recommended

References

1. MOH: National Health and Morbidity Survey 2015 (NHMS 2015). Volume II: Non-Communicable Diseases, Risk Factors & Other Health Problems. 2015; Institute for Public Health. Reference Source
2. WHO: Global status report on noncommunicable diseases 2014. 2014. Reference Source
3. MOH: Health Facts. 2015. Reference Source
4. Ramli A, Taher S: Managing Chronic Diseases in the Malaysian Primary Health Care – a Need for Change. Malays Fam Physician. 2008; 3(1): 7–13. PubMed Abstract | Free Full Text
5. Ali MK, Rabadán-Diehl C, Flanigan J, et al.: Systems and capacity to address noncommunicable diseases in low- and middle-income countries. Sci Transl Med. 2013; 5(181): 181cm4. PubMed Abstract | Publisher Full Text
6. Samb B, Desai N, Nishtar S, et al.: Prevention and management of chronic disease: a litmus test for health-systems strengthening in low-income and middle-income countries. Lancet. 2010; 376(9754): 1785–97. PubMed Abstract | Publisher Full Text
7. Cervero RM, Gaines JK: The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Contin Educ Health Prof. 2015; 35(2): 131–138. PubMed Abstract | Publisher Full Text
8. Renders CM, Valk GD, Griffin SJ, et al.: Interventions to improve the management of diabetes in primary care, outpatient, and community settings: a systematic review. Diabetes Care. 2001; 24(10): 1821–33. PubMed Abstract | Publisher Full Text
9. Reutens AT, Hutchinson R, Van Binh T, et al.: The GIANT study, a cluster-randomised controlled trial of efficacy of education of doctors about type 2 diabetes mellitus management guidelines in primary care practice. Diabetes Res Clin Pract. 2012; 98(1): 38–45. PubMed Abstract | Publisher Full Text
10. Shojania KG, Ranji SR, McDonald KM, et al.: Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA. 2006; 296(4): 427–40. PubMed Abstract | Publisher Full Text
11. Smith S, Bury G, O'Leary M, et al.: The North Dublin randomized controlled trial of structured diabetes shared care. Fam Pract. 2004; 21(1): 39–45. PubMed Abstract | Publisher Full Text
12. Kirkman MS, Williams SR, Caffrey HH, et al.: Impact of a program to improve adherence to diabetes guidelines by primary care physicians. Diabetes Care. 2002; 25(11): 1946–51. PubMed Abstract | Publisher Full Text
13. Carney T, Helliwell C: Effect of structured postgraduate medical education on the care of patients with diabetes. Br J Gen Pract. 1995; 45(392): 149–51. PubMed Abstract | Free Full Text
14. Stead JW, Dudbridge SB, Hall MS, et al.: The Exeter Diabetic Project: an acceptable district-wide education programme for general practitioners. Diabet Med. 1991; 8(9): 866–9. PubMed Abstract | Publisher Full Text
15. Braver MW, Braver Sanford L: Statistical Treatment of the Solomon Four-Group Design: A Meta-Analytic Approach. Psychol Bull. 1988; 104(1): 150–154. Publisher Full Text
16. Rosenthal R: Combining results of independent studies. Psychol Bull. 1978; 85(1): 185–193. Publisher Full Text
17. Mustapha F, Calopietro M, Nielsen KK, et al.: Data collection tools for Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia. figshare. 2020; Dataset. https://www.doi.org/10.6084/m9.figshare.11764146.v2
18. Anderson RM, Fitzgerald JT, Funnell MM, et al.: The third version of the Diabetes Attitude Scale. Diabetes Care. 1998; 21(9): 1403–1407. PubMed Abstract | Publisher Full Text
19. National Institutes of Health: Competencies Proficiency Scale. 5 February 2017. Reference Source
20. Creswell JW, Clark VLP: Designing and Conducting Mixed Methods Research. 2nd ed. 2011: Sage Publications, Inc. Reference Source
21. Collins KMT: Advanced sampling designs in mixed research: Current practices and emerging trends in the social and behavioral sciences. In: SAGE handbook of mixed methods in social and behavioral research., Tashakkori A, Teddlie C, Editors. Sage: Thousand Oaks, CA. 2010; 353–377. Publisher Full Text
22. Morse JM: Approaches to qualitative-quantitative methodological triangulation. Nurs Res. 1991; 40(2): 120–123. PubMed Abstract | Publisher Full Text
23. Monahan T, Fisher JA: Benefits of "Observer Effects": Lessons from the Field. Qual Res. 2010; 10(3): 357–376. PubMed Abstract | Publisher Full Text | Free Full Text
24. Jonsen K, Jehn KA: Using triangulation to validate themes in qualitative studies. Qual Res Org. 2009; 4(2): 123–150. Publisher Full Text

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 10 Feb 2020

Author details Author details

Feisul Mustapha
Roles: Conceptualization, Investigation, Methodology, Project Administration, Writing – Original Draft Preparation, Writing – Review & Editing

Michael Calopietro
Roles: Conceptualization, Methodology, Project Administration, Writing – Review & Editing

Karoline Kragelund Nielsen
Roles: Conceptualization, Methodology, Writing – Review & Editing

Jens Aagaard-Hansen
Roles: Conceptualization, Methodology, Writing – Original Draft Preparation, Writing – Review & Editing

Shiang Cheng Lim
Roles: Project Administration, Writing – Review & Editing

Ulla Bjerre-Christensen
Roles: Funding Acquisition, Supervision, Writing – Review & Editing

Competing interests

No competing interests were disclosed.

Grant information

This research was fully funded by the Novo Nordisk Foundation.
The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Article Versions (1)

version 1

Published: 10 Feb 2020, 9:98

https://doi.org/10.12688/f1000research.21127.1

© 2020 Mustapha F et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Mustapha F, Calopietro M, Nielsen KK et al. Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia: protocol for a quasi-experimental, mixed-methods research study [version 1; peer review: 2 approved with reservations]. F1000Research 2020, 9:98 (https://doi.org/10.12688/f1000research.21127.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Version 1

VERSION 1

PUBLISHED 10 Feb 2020

Views

Reviewer Report 11 May 2020

Kamaliah Mohamad Noh, Faculty of Medicine, Cyberjaya University College of Medical Sciences, Cyberjaya, Malaysia

Approved with Reservations

https://doi.org/10.5256/f1000research.23255.r63034

This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support claim of representativeness to the whole country. The title should reflect this.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

This study is an interesting one, with a potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

1. Mustapha F, Calopietro M: Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia. Diabetes Research and Clinical Practice. 2016; 120: S163-S164 Publisher Full Text
2. Lim SC, Mustapha FI, Aagaard-Hansen J, Calopietro M, et al.: Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices.Med Educ Online. 2020; 25 (1): 1710330 PubMed Abstract | Publisher Full Text
3. Borg S, Eeg-Olofsson K, Palaszewski B, Svedbo Engström M, et al.: Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open. 2019; 9 (1). Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: Primary health care delivery system

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 18 May 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

18 May 2020

Author Response

This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff ... Continue reading This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

Thank you.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support the claim of representativeness to the whole country. The title should reflect this.

Point well taken, however, this was clearly stated in the paper and we have not made any claim on the representativeness to the whole country.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

As the reviewer rightly points out, a previous conference abstract as well as a full paper have given some concise outline of the study design. However, in this paper a much more thorough description is provided. We contend that this has merit in its own right as a potential source of inspiration for other MOH’s researchers who would like to conduct a state of the art evaluation of their training activities – something that unfortunately is not always the case.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

We thank the reviewer for drawing our attention to this reference, which we will use in our future work.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

Thank you. We agree that the findings are of relevance outside the Malaysian context, and that a thorough evaluation of post-graduate on the job training provides a sound basis for decision making.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

Though Steno Diabetes Center Copenhagen has extensive experience from many years of diabetes-related training of health care professionals in all parts of the world, the SRCC was specifically developed for the Malaysian context. From Steno’s perspective, the SRCC is a novel training module that they have never done before.

This study is an interesting one, with the potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

We hope that the comments above have clarified the position of this study protocol. As for the results, another paper is currently on the way that very much takes into account the various issues raised by the reviewer – which we agree are valid concerns.
This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

Thank you.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support the claim of representativeness to the whole country. The title should reflect this.

Point well taken, however, this was clearly stated in the paper and we have not made any claim on the representativeness to the whole country.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

As the reviewer rightly points out, a previous conference abstract as well as a full paper have given some concise outline of the study design. However, in this paper a much more thorough description is provided. We contend that this has merit in its own right as a potential source of inspiration for other MOH’s researchers who would like to conduct a state of the art evaluation of their training activities – something that unfortunately is not always the case.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

We thank the reviewer for drawing our attention to this reference, which we will use in our future work.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

Thank you. We agree that the findings are of relevance outside the Malaysian context, and that a thorough evaluation of post-graduate on the job training provides a sound basis for decision making.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

Though Steno Diabetes Center Copenhagen has extensive experience from many years of diabetes-related training of health care professionals in all parts of the world, the SRCC was specifically developed for the Malaysian context. From Steno’s perspective, the SRCC is a novel training module that they have never done before.

This study is an interesting one, with the potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

We hope that the comments above have clarified the position of this study protocol. As for the results, another paper is currently on the way that very much takes into account the various issues raised by the reviewer – which we agree are valid concerns.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reviewer Response 02 Feb 2021

Kamaliah Noh, Faculty of Medicine, Cyberjaya University College of Medical Sciences, Cyberjaya, Malaysia

02 Feb 2021

Reviewer Response

I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised ... Continue reading I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised paper is of an acceptable scientific standard.
I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised paper is of an acceptable scientific standard.
Competing Interests: No competing interest declared Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 18 May 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

18 May 2020

Author Response

This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff ... Continue reading This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

Thank you.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support the claim of representativeness to the whole country. The title should reflect this.

Point well taken, however, this was clearly stated in the paper and we have not made any claim on the representativeness to the whole country.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

As the reviewer rightly points out, a previous conference abstract as well as a full paper have given some concise outline of the study design. However, in this paper a much more thorough description is provided. We contend that this has merit in its own right as a potential source of inspiration for other MOH’s researchers who would like to conduct a state of the art evaluation of their training activities – something that unfortunately is not always the case.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

We thank the reviewer for drawing our attention to this reference, which we will use in our future work.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

Thank you. We agree that the findings are of relevance outside the Malaysian context, and that a thorough evaluation of post-graduate on the job training provides a sound basis for decision making.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

Though Steno Diabetes Center Copenhagen has extensive experience from many years of diabetes-related training of health care professionals in all parts of the world, the SRCC was specifically developed for the Malaysian context. From Steno’s perspective, the SRCC is a novel training module that they have never done before.

This study is an interesting one, with the potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

We hope that the comments above have clarified the position of this study protocol. As for the results, another paper is currently on the way that very much takes into account the various issues raised by the reviewer – which we agree are valid concerns.
This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

Thank you.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support the claim of representativeness to the whole country. The title should reflect this.

Point well taken, however, this was clearly stated in the paper and we have not made any claim on the representativeness to the whole country.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

As the reviewer rightly points out, a previous conference abstract as well as a full paper have given some concise outline of the study design. However, in this paper a much more thorough description is provided. We contend that this has merit in its own right as a potential source of inspiration for other MOH’s researchers who would like to conduct a state of the art evaluation of their training activities – something that unfortunately is not always the case.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

We thank the reviewer for drawing our attention to this reference, which we will use in our future work.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

Thank you. We agree that the findings are of relevance outside the Malaysian context, and that a thorough evaluation of post-graduate on the job training provides a sound basis for decision making.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

Though Steno Diabetes Center Copenhagen has extensive experience from many years of diabetes-related training of health care professionals in all parts of the world, the SRCC was specifically developed for the Malaysian context. From Steno’s perspective, the SRCC is a novel training module that they have never done before.

This study is an interesting one, with the potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

We hope that the comments above have clarified the position of this study protocol. As for the results, another paper is currently on the way that very much takes into account the various issues raised by the reviewer – which we agree are valid concerns.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reviewer Response 02 Feb 2021

Kamaliah Noh, Faculty of Medicine, Cyberjaya University College of Medical Sciences, Cyberjaya, Malaysia

02 Feb 2021

Reviewer Response

I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised ... Continue reading I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised paper is of an acceptable scientific standard.
I confirm that I have read the responses by the author which I have found that these responses have satisfactorily addressed my comments. I am of the opinion that the revised paper is of an acceptable scientific standard.
Competing Interests: No competing interest declared Close
Report a concern

Views

Reviewer Report 20 Apr 2020

Prashanth N Srinivas, Institute of Public Health Bengaluru, Bengaluru, India

Approved with Reservations

https://doi.org/10.5256/f1000research.23255.r61088

Thank you for the opportunity to review this paper. This is a study protocol of a study to assess the effectiveness of an educational intervention to non-specialist doctors and nurses in public facilities in two cities in Malaysia, that provide Diabetes care to improve their knowledge, skills and practice with respect to the treatment of Diabetes. By proposing a mixed-methods study, they also aim to understand the processes through which this might occur and the specific contextual characteristics of the facilities that may facilitate improvements. By using the four-group design, they also aim to address concerns of classical pre-post test comparisons often used in the evaluation of the effectiveness of training/educational interventions.

Overall comments

Please revise carefully for grammatical errors. There are several minor grammatical errors.
The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
The current status and clarity on the timelines of the study need to be added.
The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 2004¹). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.

Minor/optional comments/suggestions

Abstract

Please consider providing numbers of doctors and nurses recruited, word limit permitting

Introduction

The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
“Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.

Methods

Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Expected outcomes do not report on any outcomes from the qualitative data component.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

1. Smith S, Bury G, O'Leary M, Shannon W, et al.: The North Dublin randomized controlled trial of structured diabetes shared care.Fam Pract. 2004; 21 (1): 39-45 PubMed Abstract | Publisher Full Text

Competing Interests: No competing interests were disclosed.

Reviewer Expertise: I have experience in the design of health policy and program evaluation especially on capacity-building programs in local health systems at district/regional levels. However, I do not have sufficient expertise in assessing the adequacy of the sample size for the proposed research method.

CITE

Report a concern

Author Response 07 Jul 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

07 Jul 2020

Author Response

General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully ... Continue reading General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully for grammatical errors. There are several minor grammatical errors.
Thank you, we will review and amend accordingly.

2. The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
By the time this article was accepted for publication, the study has been completed. This study was done a few years ago.

3. The current status and clarity on the timelines of the study need to be added.
We will include a footnote on this.

4. The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 20041). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
This is a good comment, with fair points. Our intervention is a very specific scope/definition of CME, and we hope this will assist others in their future work. We agree that readers would benefit from a more thorough discussion on CMEs

5. Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The first pilot was on a much smaller scale – for proof of concept – that it was doable with current available resources and work environment. The lecturers and facilitators for this pilot were all from Steno Denmark, and training local facilitators was part of the work.
The second pilot built on the first pilot (having addressed implementation, mostly logistic issues), to ensure that we were able to manage known issues. This second pilot was run by local lecturers and facilitators, observed and guided by colleagues from Steno Denmark.

6. The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Point well taken. Effectiveness would be a better terminology for the title.

7. Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.
Point well taken. We didn’t look at all aspects. We only focused on 2 specific aspects i.e. process of diabetes service delivery at the clinic, and patients' perspective or perception. On the use of the word "hope" - will revise to "... we contend that..."

8. Abstract - Please consider providing numbers of doctors and nurses recruited, word limit permitting
Point well taken, will amend

9. Introduction: The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
Excluding the Pacific Island nations (in the Asia Pacific region) as stated in the text – Malaysia does have the highest prevalence of diabetes.

10. Introduction - “Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Acknowledge it is a loose term, and we could remove the quotation marks.

11. Introduction - Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Our error of omission. The reference is #1 of the existing reference. Data is obtained from a community-based survey – NHMS 2015

12. Introduction - Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.
Point well taken. Will review to clarify. The pressure comes from all of the above.

13. Methods - Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
This has actually been written up as a separate paper: Mustapha, F.I., et al., Variations in the Delivery of Primary Diabetes Care in Malaysia: Lessons to Be Learnt and Potential for Improvement. Health Services Research and Managerial Epidemiology, 2020. 7: p. 2333392820918744.

14. Methods - Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Point taken, however, we still feel that sufficient detail has been provided.

15. Methods - Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
Point taken, however for we felt that the level of detail is adequate for a protocol paper, as the analysis will be elaborated further in the results paper.

16. Methods - While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Thank you for this suggestion. Has this been a full qualitative study per se, such a framework will be useful and appropriate. We contend that the present paper can study these aspects of the health care system and reach relevant conclusions without such framework. Nevertheless, we do agree with your opinion that it could be useful to guide discussions.

17. Methods - Expected outcomes do not report on any outcomes from the qualitative data component.
Is the rationale for, and objectives of, the study clearly described?
Point taken. This will be clarified further.
General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully for grammatical errors. There are several minor grammatical errors.
Thank you, we will review and amend accordingly.

2. The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
By the time this article was accepted for publication, the study has been completed. This study was done a few years ago.

3. The current status and clarity on the timelines of the study need to be added.
We will include a footnote on this.

4. The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 20041). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
This is a good comment, with fair points. Our intervention is a very specific scope/definition of CME, and we hope this will assist others in their future work. We agree that readers would benefit from a more thorough discussion on CMEs

5. Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The first pilot was on a much smaller scale – for proof of concept – that it was doable with current available resources and work environment. The lecturers and facilitators for this pilot were all from Steno Denmark, and training local facilitators was part of the work.
The second pilot built on the first pilot (having addressed implementation, mostly logistic issues), to ensure that we were able to manage known issues. This second pilot was run by local lecturers and facilitators, observed and guided by colleagues from Steno Denmark.

6. The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Point well taken. Effectiveness would be a better terminology for the title.

7. Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.
Point well taken. We didn’t look at all aspects. We only focused on 2 specific aspects i.e. process of diabetes service delivery at the clinic, and patients' perspective or perception. On the use of the word "hope" - will revise to "... we contend that..."

8. Abstract - Please consider providing numbers of doctors and nurses recruited, word limit permitting
Point well taken, will amend

9. Introduction: The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
Excluding the Pacific Island nations (in the Asia Pacific region) as stated in the text – Malaysia does have the highest prevalence of diabetes.

10. Introduction - “Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Acknowledge it is a loose term, and we could remove the quotation marks.

11. Introduction - Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Our error of omission. The reference is #1 of the existing reference. Data is obtained from a community-based survey – NHMS 2015

12. Introduction - Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.
Point well taken. Will review to clarify. The pressure comes from all of the above.

13. Methods - Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
This has actually been written up as a separate paper: Mustapha, F.I., et al., Variations in the Delivery of Primary Diabetes Care in Malaysia: Lessons to Be Learnt and Potential for Improvement. Health Services Research and Managerial Epidemiology, 2020. 7: p. 2333392820918744.

14. Methods - Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Point taken, however, we still feel that sufficient detail has been provided.

15. Methods - Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
Point taken, however for we felt that the level of detail is adequate for a protocol paper, as the analysis will be elaborated further in the results paper.

16. Methods - While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Thank you for this suggestion. Has this been a full qualitative study per se, such a framework will be useful and appropriate. We contend that the present paper can study these aspects of the health care system and reach relevant conclusions without such framework. Nevertheless, we do agree with your opinion that it could be useful to guide discussions.

17. Methods - Expected outcomes do not report on any outcomes from the qualitative data component.
Is the rationale for, and objectives of, the study clearly described?
Point taken. This will be clarified further.
Competing Interests: None to declare. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 07 Jul 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

07 Jul 2020

Author Response

General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully ... Continue reading General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully for grammatical errors. There are several minor grammatical errors.
Thank you, we will review and amend accordingly.

2. The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
By the time this article was accepted for publication, the study has been completed. This study was done a few years ago.

3. The current status and clarity on the timelines of the study need to be added.
We will include a footnote on this.

4. The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 20041). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
This is a good comment, with fair points. Our intervention is a very specific scope/definition of CME, and we hope this will assist others in their future work. We agree that readers would benefit from a more thorough discussion on CMEs

5. Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The first pilot was on a much smaller scale – for proof of concept – that it was doable with current available resources and work environment. The lecturers and facilitators for this pilot were all from Steno Denmark, and training local facilitators was part of the work.
The second pilot built on the first pilot (having addressed implementation, mostly logistic issues), to ensure that we were able to manage known issues. This second pilot was run by local lecturers and facilitators, observed and guided by colleagues from Steno Denmark.

6. The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Point well taken. Effectiveness would be a better terminology for the title.

7. Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.
Point well taken. We didn’t look at all aspects. We only focused on 2 specific aspects i.e. process of diabetes service delivery at the clinic, and patients' perspective or perception. On the use of the word "hope" - will revise to "... we contend that..."

8. Abstract - Please consider providing numbers of doctors and nurses recruited, word limit permitting
Point well taken, will amend

9. Introduction: The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
Excluding the Pacific Island nations (in the Asia Pacific region) as stated in the text – Malaysia does have the highest prevalence of diabetes.

10. Introduction - “Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Acknowledge it is a loose term, and we could remove the quotation marks.

11. Introduction - Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Our error of omission. The reference is #1 of the existing reference. Data is obtained from a community-based survey – NHMS 2015

12. Introduction - Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.
Point well taken. Will review to clarify. The pressure comes from all of the above.

13. Methods - Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
This has actually been written up as a separate paper: Mustapha, F.I., et al., Variations in the Delivery of Primary Diabetes Care in Malaysia: Lessons to Be Learnt and Potential for Improvement. Health Services Research and Managerial Epidemiology, 2020. 7: p. 2333392820918744.

14. Methods - Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Point taken, however, we still feel that sufficient detail has been provided.

15. Methods - Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
Point taken, however for we felt that the level of detail is adequate for a protocol paper, as the analysis will be elaborated further in the results paper.

16. Methods - While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Thank you for this suggestion. Has this been a full qualitative study per se, such a framework will be useful and appropriate. We contend that the present paper can study these aspects of the health care system and reach relevant conclusions without such framework. Nevertheless, we do agree with your opinion that it could be useful to guide discussions.

17. Methods - Expected outcomes do not report on any outcomes from the qualitative data component.
Is the rationale for, and objectives of, the study clearly described?
Point taken. This will be clarified further.
General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully for grammatical errors. There are several minor grammatical errors.
Thank you, we will review and amend accordingly.

2. The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
By the time this article was accepted for publication, the study has been completed. This study was done a few years ago.

3. The current status and clarity on the timelines of the study need to be added.
We will include a footnote on this.

4. The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 20041). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
This is a good comment, with fair points. Our intervention is a very specific scope/definition of CME, and we hope this will assist others in their future work. We agree that readers would benefit from a more thorough discussion on CMEs

5. Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The first pilot was on a much smaller scale – for proof of concept – that it was doable with current available resources and work environment. The lecturers and facilitators for this pilot were all from Steno Denmark, and training local facilitators was part of the work.
The second pilot built on the first pilot (having addressed implementation, mostly logistic issues), to ensure that we were able to manage known issues. This second pilot was run by local lecturers and facilitators, observed and guided by colleagues from Steno Denmark.

6. The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Point well taken. Effectiveness would be a better terminology for the title.

7. Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.
Point well taken. We didn’t look at all aspects. We only focused on 2 specific aspects i.e. process of diabetes service delivery at the clinic, and patients' perspective or perception. On the use of the word "hope" - will revise to "... we contend that..."

8. Abstract - Please consider providing numbers of doctors and nurses recruited, word limit permitting
Point well taken, will amend

9. Introduction: The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
Excluding the Pacific Island nations (in the Asia Pacific region) as stated in the text – Malaysia does have the highest prevalence of diabetes.

10. Introduction - “Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Acknowledge it is a loose term, and we could remove the quotation marks.

11. Introduction - Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Our error of omission. The reference is #1 of the existing reference. Data is obtained from a community-based survey – NHMS 2015

12. Introduction - Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.
Point well taken. Will review to clarify. The pressure comes from all of the above.

13. Methods - Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
This has actually been written up as a separate paper: Mustapha, F.I., et al., Variations in the Delivery of Primary Diabetes Care in Malaysia: Lessons to Be Learnt and Potential for Improvement. Health Services Research and Managerial Epidemiology, 2020. 7: p. 2333392820918744.

14. Methods - Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Point taken, however, we still feel that sufficient detail has been provided.

15. Methods - Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
Point taken, however for we felt that the level of detail is adequate for a protocol paper, as the analysis will be elaborated further in the results paper.

16. Methods - While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Thank you for this suggestion. Has this been a full qualitative study per se, such a framework will be useful and appropriate. We contend that the present paper can study these aspects of the health care system and reach relevant conclusions without such framework. Nevertheless, we do agree with your opinion that it could be useful to guide discussions.

17. Methods - Expected outcomes do not report on any outcomes from the qualitative data component.
Is the rationale for, and objectives of, the study clearly described?
Point taken. This will be clarified further.
Competing Interests: None to declare. Close
Report a concern

Comments on this article Comments (0)

Version 1

VERSION 1 PUBLISHED 10 Feb 2020

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2
Version 1 10 Feb 20	read	read

Prashanth N Srinivas, Institute of Public Health Bengaluru, Bengaluru, India
Kamaliah Mohamad Noh, Cyberjaya University College of Medical Sciences, Cyberjaya, Malaysia

Comments on this article

All Comments(0)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

27 Views

11 May 2020 | for Version 1

Kamaliah Mohamad Noh, Faculty of Medicine, Cyberjaya University College of Medical Sciences, Cyberjaya, Malaysia

27 Views Cite this report Responses(2)

Approved With Reservations

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

Primary health care delivery system

Respond to this report

Responses (2)

Author Response

18 May 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

This is a well written paper, easy to comprehend and to follow the logical thoughts of the author. The paper addresses a relevant topic i.e. training of front line staff in resource-constrained public primary health care clinics in providing quality diabetic care. The research question, the objectives, the methodology including the statistical analysis and the discussion are in alignment and the writing style contributes to a smooth flow of logical thought. What is interesting is the use of patient reported experience measure (PREMS) as an outcome measure, not a frequent measure used in this country context.

Thank you.

The title reflects the content of the paper as written. However, being a pilot study and the purpose sampling frame of clinics from a region, the sampling methodology would not support the claim of representativeness to the whole country. The title should reflect this.

Point well taken, however, this was clearly stated in the paper and we have not made any claim on the representativeness to the whole country.

The abstract has clear research question and objective and stops at the methodology because this is a study protocol paper.

This study was implemented in 2016 and the results of this study has already been published through a poster presentation, as in this case entitled “Pilot implementation of a novel post-graduate medical education program: Steno REACH certificate course in clinical diabetes care – Malaysia” by the same first and second authors of this paper at DOI:https://doi.org/10.1016/S0168-8227(16)31353-5[re-1]; as well as a full paper publication in MEDICAL EDUCATION ONLINE 2019, VOL. 25, 1710330 https://doi.org/10.1080/10872981.2019.1710330 entitled “Impact of continuing medical education for primary healthcare providers in Malaysia on diabetes knowledge, attitudes, skills and clinical practices” by Shiang Cheng Lim, Feisul Idzwan Mustapha, Jens Aagaard-Hansen, Michael Calopietro, Tahir Aris and Ulla Bjerre-Christensen². While it is agreed that a study protocol paper can fill a knowledge gap, the methodology has been well described in the published paper by Lim, S.C. et al. It is suggested the author demonstrate how this current paper can add value to scientific knowledge by being different from the previous publication by Lim, S.C. et al.

As the reviewer rightly points out, a previous conference abstract as well as a full paper have given some concise outline of the study design. However, in this paper a much more thorough description is provided. We contend that this has merit in its own right as a potential source of inspiration for other MOH’s researchers who would like to conduct a state of the art evaluation of their training activities – something that unfortunately is not always the case.

What is unique in this current paper under review is the use of PREMS as a secondary outcome measure. The author may want to review the paper to reflect a focus on the PREMS component of this study which has not been described elsewhere. A relevant reference for the author would be Borg S, Eeg-Olofsson K, Palaszewski B, et al. Patient-reported outcome and experience measures for diabetes: development of scale models, differences between patient groups and relationships with cardiovascular and diabetes complication risk factors, in a combined registry and survey study in Sweden. BMJ Open 2018;9:e025033. doi:10.1136/ bmjopen-2018-025033³.

We thank the reviewer for drawing our attention to this reference, which we will use in our future work.

This study has relevance to other countries with similar context on the challenges faced in upscaling the pilot into a nationwide roll out. The results of the analysis of the post intervention interviews of the trainees as well as their clinic leads may lead to discussion points of interest.

Thank you. We agree that the findings are of relevance outside the Malaysian context, and that a thorough evaluation of post-graduate on the job training provides a sound basis for decision making.

It would be helpful if the author clarifies if the intervention i.e. the training module, is original or is it an adaptation of the Steno REACH education programme implemented in China, India, the Middle East, South East Asia and Latin America. A review of references for the interventions in different country settings would be useful for the discussion.

Though Steno Diabetes Center Copenhagen has extensive experience from many years of diabetes-related training of health care professionals in all parts of the world, the SRCC was specifically developed for the Malaysian context. From Steno’s perspective, the SRCC is a novel training module that they have never done before.

This study is an interesting one, with the potential of relevant lessons for other countries with the same context, trying to improve the quality of care of diabetes challenged with resource constraints. However, for the sake of originality, the authors are suggested to review the focus of the paper.

We hope that the comments above have clarified the position of this study protocol. As for the results, another paper is currently on the way that very much takes into account the various issues raised by the reviewer – which we agree are valid concerns.

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

22 Views

20 Apr 2020 | for Version 1

Prashanth N Srinivas, Institute of Public Health Bengaluru, Bengaluru, India

22 Views Cite this report Responses(1)

Approved With Reservations

Please revise carefully for grammatical errors. There are several minor grammatical errors.
The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
The current status and clarity on the timelines of the study need to be added.
The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 2004¹). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.

Minor/optional comments/suggestions

Abstract

Please consider providing numbers of doctors and nurses recruited, word limit permitting

Introduction

The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
“Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.

Methods

Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Expected outcomes do not report on any outcomes from the qualitative data component.

Is the rationale for, and objectives of, the study clearly described?

Yes
Is the study design appropriate for the research question?

Yes
Are sufficient details of the methods provided to allow replication by others?

Yes
Are the datasets clearly presented in a useable and accessible format?

Not applicable

References

1. Smith S, Bury G, O'Leary M, Shannon W, et al.: The North Dublin randomized controlled trial of structured diabetes shared care.Fam Pract. 2004; 21 (1): 39-45 PubMed Abstract | Publisher Full Text

Competing Interests

No competing interests were disclosed.

Reviewer Expertise

I have experience in the design of health policy and program evaluation especially on capacity-building programs in local health systems at district/regional levels. However, I do not have sufficient expertise in assessing the adequacy of the sample size for the proposed research method.

Respond to this report

Responses (1)

Author Response

07 Jul 2020

Feisul Mustapha, Disease Control Division, Ministry of Health Malaysia, Putrajaya, 62590, Malaysia

General comment: Thank you for the comments - on most parts, your comments are well accepted. We will await any further reviewers before amending the manuscript.

1. Please revise carefully for grammatical errors. There are several minor grammatical errors.
Thank you, we will review and amend accordingly.

2. The entire article is written in the past tense and appears to indicate the study is now completed. However, I believe that is not the case as this is a study protocol. Please review.
By the time this article was accepted for publication, the study has been completed. This study was done a few years ago.

3. The current status and clarity on the timelines of the study need to be added.
We will include a footnote on this.

4. The background briefly summarises evidence around the effectiveness of CME. Authors state “Despite the importance of CME, few studies have measured the clinical impact of CME on diabetes in a real-world setting”. While I agree with this assertion, the definition of "CME" can be so broad so that the lack of evidence is also to do with the nature of various educational interventions that may not be going under the name CME. Overall, the background oversimplifies the complexity of educational intervention to professionals (doctors, nurses, health workers). The few interventions and reviews that the authors refer to had educational inputs as one among various other inputs within a complex intervention setting (for example Smith et al. 20041). Whereas other programs such as the GIANT study examined an intervention of training on the use of existing guidelines. Arguably, the interventions being cited in support are very diverse and complex and difficult to characterize as "…few studies have measured the clinical impact of CME". Indeed many of the studies cited do not use the term CME…..this section requires better synthesis of the literature cited and/or integrating specific systematic review evidence if available.
This is a good comment, with fair points. Our intervention is a very specific scope/definition of CME, and we hope this will assist others in their future work. We agree that readers would benefit from a more thorough discussion on CMEs

5. Two pilots are mentioned. The rationale behind having two pilots, how the first one shaped the next one and how/what were the overall lessons learned from the pilots that informed the intervention design are not stated and appear to be useful contextual information in the protocol.
The first pilot was on a much smaller scale – for proof of concept – that it was doable with current available resources and work environment. The lecturers and facilitators for this pilot were all from Steno Denmark, and training local facilitators was part of the work.
The second pilot built on the first pilot (having addressed implementation, mostly logistic issues), to ensure that we were able to manage known issues. This second pilot was run by local lecturers and facilitators, observed and guided by colleagues from Steno Denmark.

6. The characterization of this study as impact evaluation, as opposed to being a study on the effectiveness may need some reflection. While these are - to some extent - overlapping and perhaps a matter of semantics in some cases, the objectives appear to be a study on the effectiveness of the intervention rather than an impact evaluation. Impact refers to the ability of the program to meet its ultimate goals of reducing DM burden, whereas the protocol largely sticks to proximate outcomes at the level of the direct participants of the program with some effort to go to patent-level and contextual factors. Suggest to revise this or else provide clearer justification in case impact evaluation is being retained.
Point well taken. Effectiveness would be a better terminology for the title.

7. Objective 3 mentions the objective to examine the influence of contextual factors at the level of the health system and sociocultural levels. The protocol provides insufficient detail to understand how this is done. For example, which are the health system building blocks that they assess and how? What is clearly established is the factors at the level of health service (and not system). Similarly, insufficient details as to the framework/approaches to be used to assess socio-cultural levels (either in their quantitative component or in their qualitative component). A further minor point under study objectives is their characterization of a program objective/intention as a “hope”. Consider/review.
Point well taken. We didn’t look at all aspects. We only focused on 2 specific aspects i.e. process of diabetes service delivery at the clinic, and patients' perspective or perception. On the use of the word "hope" - will revise to "... we contend that..."

8. Abstract - Please consider providing numbers of doctors and nurses recruited, word limit permitting
Point well taken, will amend

9. Introduction: The authors cite WHO: Global status report on noncommunicable diseases 2014 in support of their assertion for the highest prevalence of Diabetes in Malaysia in the Asia-pacific region. On p.244 of the report cited in table titled "Raised blood glucose (fasting glucose ≥7.0 mmol/l (126 mg/dl) or on medication for raised blood glucose or with a history of diagnosis of diabetes), the Crude adjusted estimates for Malaysia do not support this assertion. Clarify.
Excluding the Pacific Island nations (in the Asia Pacific region) as stated in the text – Malaysia does have the highest prevalence of diabetes.

10. Introduction - “Out-reach” has been put in quotes. Unclear as to the purpose. Does this mean that these are not really functioning or are not actually providing outreach services.
Acknowledge it is a loose term, and we could remove the quotation marks.

11. Introduction - Authors’ assertion on majority of care for Diabetes occurring in public services should be supported with a reference. In other mixed health systems, evidence often is ambiguous on this.
Our error of omission. The reference is #1 of the existing reference. Data is obtained from a community-based survey – NHMS 2015

12. Introduction - Authors state “The Malaysian healthcare delivery system is facing increasing pressure to provide quality care to patients with diabetes.”. Unclear as to pressure is being faced by whom? By communities/patients, funders, politically? Review and sharpen.
Point well taken. Will review to clarify. The pressure comes from all of the above.

13. Methods - Methods are well described. Consider adding a section on the study setting to help readers understand the nature of populations that these clinics cater to and perhaps also brief overview of a typical clinic being included in the study in terms of its patient load and other health services characteristics.
This has actually been written up as a separate paper: Mustapha, F.I., et al., Variations in the Delivery of Primary Diabetes Care in Malaysia: Lessons to Be Learnt and Potential for Improvement. Health Services Research and Managerial Epidemiology, 2020. 7: p. 2333392820918744.

14. Methods - Adequacy of sample size for four-group design has been addressed, but this may require further details, especially if test scores and other quantitative variables do not show sufficient variation between groups (which may be the case?)
Point taken, however, we still feel that sufficient detail has been provided.

15. Methods - Further details of the quantitative analysis in terms of variables to be used and how they will be analysed will be useful.
Point taken, however for we felt that the level of detail is adequate for a protocol paper, as the analysis will be elaborated further in the results paper.

16. Methods - While this is not absolutely essential, the study does not seem to clarify the possible pathways through which they expect the change to occur (where it does). Given the use of qualitative methods and convergent study design, it is useful to include a framework or a theory of change or a program logic that clarifies the possible pathways through which they expect the change to occur. Please treat this as a suggestion and not a comment that needs to be addressed.
Thank you for this suggestion. Has this been a full qualitative study per se, such a framework will be useful and appropriate. We contend that the present paper can study these aspects of the health care system and reach relevant conclusions without such framework. Nevertheless, we do agree with your opinion that it could be useful to guide discussions.

17. Methods - Expected outcomes do not report on any outcomes from the qualitative data component.
Is the rationale for, and objectives of, the study clearly described?
Point taken. This will be clarified further.

View more View less

Competing Interests

None to declare.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

[1] 1. MOH: National Health and Morbidity Survey 2015 (NHMS 2015). Volume II: Non-Communicable Diseases, Risk Factors & Other Health Problems. 2015; Institute for Public Health. Reference Source

[2] 2. WHO: Global status report on noncommunicable diseases 2014. 2014. Reference Source

[3] 3. MOH: Health Facts. 2015. Reference Source

[4] 4. Ramli A, Taher S: Managing Chronic Diseases in the Malaysian Primary Health Care – a Need for Change. Malays Fam Physician. 2008; 3(1): 7–13. PubMed Abstract | Free Full Text

[5] 5. Ali MK, Rabadán-Diehl C, Flanigan J, et al.: Systems and capacity to address noncommunicable diseases in low- and middle-income countries. Sci Transl Med. 2013; 5(181): 181cm4. PubMed Abstract | Publisher Full Text

[6] 6. Samb B, Desai N, Nishtar S, et al.: Prevention and management of chronic disease: a litmus test for health-systems strengthening in low-income and middle-income countries. Lancet. 2010; 376(9754): 1785–97. PubMed Abstract | Publisher Full Text

[7] 7. Cervero RM, Gaines JK: The impact of CME on physician performance and patient health outcomes: an updated synthesis of systematic reviews. J Contin Educ Health Prof. 2015; 35(2): 131–138. PubMed Abstract | Publisher Full Text

[8] 8. Renders CM, Valk GD, Griffin SJ, et al.: Interventions to improve the management of diabetes in primary care, outpatient, and community settings: a systematic review. Diabetes Care. 2001; 24(10): 1821–33. PubMed Abstract | Publisher Full Text

[9] 9. Reutens AT, Hutchinson R, Van Binh T, et al.: The GIANT study, a cluster-randomised controlled trial of efficacy of education of doctors about type 2 diabetes mellitus management guidelines in primary care practice. Diabetes Res Clin Pract. 2012; 98(1): 38–45. PubMed Abstract | Publisher Full Text

[10] 10. Shojania KG, Ranji SR, McDonald KM, et al.: Effects of quality improvement strategies for type 2 diabetes on glycemic control: a meta-regression analysis. JAMA. 2006; 296(4): 427–40. PubMed Abstract | Publisher Full Text

[11] 11. Smith S, Bury G, O'Leary M, et al.: The North Dublin randomized controlled trial of structured diabetes shared care. Fam Pract. 2004; 21(1): 39–45. PubMed Abstract | Publisher Full Text

[12] 12. Kirkman MS, Williams SR, Caffrey HH, et al.: Impact of a program to improve adherence to diabetes guidelines by primary care physicians. Diabetes Care. 2002; 25(11): 1946–51. PubMed Abstract | Publisher Full Text

[13] 13. Carney T, Helliwell C: Effect of structured postgraduate medical education on the care of patients with diabetes. Br J Gen Pract. 1995; 45(392): 149–51. PubMed Abstract | Free Full Text

[14] 14. Stead JW, Dudbridge SB, Hall MS, et al.: The Exeter Diabetic Project: an acceptable district-wide education programme for general practitioners. Diabet Med. 1991; 8(9): 866–9. PubMed Abstract | Publisher Full Text

[15] 15. Braver MW, Braver Sanford L: Statistical Treatment of the Solomon Four-Group Design: A Meta-Analytic Approach. Psychol Bull. 1988; 104(1): 150–154. Publisher Full Text

[16] 16. Rosenthal R: Combining results of independent studies. Psychol Bull. 1978; 85(1): 185–193. Publisher Full Text

[17] 17. Mustapha F, Calopietro M, Nielsen KK, et al.: Data collection tools for Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia. figshare. 2020; Dataset. https://www.doi.org/10.6084/m9.figshare.11764146.v2

[18] 18. Anderson RM, Fitzgerald JT, Funnell MM, et al.: The third version of the Diabetes Attitude Scale. Diabetes Care. 1998; 21(9): 1403–1407. PubMed Abstract | Publisher Full Text

[19] 19. National Institutes of Health: Competencies Proficiency Scale. 5 February 2017. Reference Source

[20] 20. Creswell JW, Clark VLP: Designing and Conducting Mixed Methods Research. 2nd ed. 2011: Sage Publications, Inc. Reference Source

[21] 21. Collins KMT: Advanced sampling designs in mixed research: Current practices and emerging trends in the social and behavioral sciences. In: SAGE handbook of mixed methods in social and behavioral research., Tashakkori A, Teddlie C, Editors. Sage: Thousand Oaks, CA. 2010; 353–377. Publisher Full Text

[22] 22. Morse JM: Approaches to qualitative-quantitative methodological triangulation. Nurs Res. 1991; 40(2): 120–123. PubMed Abstract | Publisher Full Text

[23] 23. Monahan T, Fisher JA: Benefits of "Observer Effects": Lessons from the Field. Qual Res. 2010; 10(3): 357–376. PubMed Abstract | Publisher Full Text | Free Full Text

[24] 24. Jonsen K, Jehn KA: Using triangulation to validate themes in qualitative studies. Qual Res Org. 2009; 4(2): 123–150. Publisher Full Text

Impact evaluation of the Steno REACH Certificate Course in Clinical Diabetes Care for health care providers in Malaysia: protocol for a quasi-experimental, mixed-methods research study

Abstract

Keywords

Introduction

The SRCC intervention

Study objectives

General objective

Specific objectives

Methods

Study design

Figure 1. Overview of research design showing the different points and methods of data collection.

Selection of clinics and participants

Table 1. Distribution of clinics within each category after the 2-level selections (n=43).

Table 2. Distribution of clinics randomly enrolled into the intervention arm (n=10).

Table 3. Distribution of Participants by Arm (n=77).

Sample size considerations

Data collection methods

Table 4. Summary of methods of data collection, target and purpose.

Data management

Data analysis plan

Expected outcomes

Research ethics

Discussion

Data availability

Underlying data

Extended data

Acronyms and abbreviations

Author contributions

Acknowledgements

References

Comments on this article Comments (0)

Open Peer Review

Comments on this article Comments (0)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

Competing Interests Policy

Stay Updated