Pay-for-performance in resource-constrained settings: Lessons learned from Thailand’s Quality and Outcomes Framework

Data collection

Document review. Key international publications and unpublished research reports related to the UK QOF and relevant documents related to the Thai QOF, including published literature, research reports, and health policy documents, were reviewed. International publications were searched in PubMed (https://www.ncbi.nlm.nih.gov/pubmed/). The search was conducted using key words, including ‘QOF’ and ‘quality indicators’. Papers published between January 1, 2004 and July, 2015 were considered. Unpublished research reports were identified by UK researchers (University of Birmingham). The NHSO and its regional offices offered relevant documents in Thai, including published literature, research reports, and health policy documents.

Key informant interviews. In June 2015, in-depth interviews were conducted with 11 key informants using semi-structured interview guides (Supplementary File 1). Purposive sampling was performed; the informants were recruited according to the criteria that they were responsible for formulating QOF policy or managing the programme at the national and regional levels. Recruitment was performed under the supervision of the QOF programme managers in the NHSO. The informants included five executives and two programme managers at the national level, and three executives of NHSO offices and one programme manager at the regional level (Table 1). All QOF managers at national level were interviewed. However, at regional level, we categorised 13 NHSO regional offices into 5 groups. The first group, Bangkok, was selected due to differences in context, such as population density and mobility, patient’s help seeking behaviours, and lifestyle, as well as primary care system. Therefore, Bangkok has implemented different QOF indicators, payment criteria and management strategies. Regional offices in the south, north, north-eastern and central part of Thailand were randomly selected (1 office per region) for interviews.

Focus group discussions. Two separate focus group discussion sessions were convened in July 2015 (one for 7 representatives and other for 8 representatives). A researcher (ST) led the discussion. The discussions were guided by sets of predefined questions covering the benefits of the QOF to health facilities, the relationship between QOF performance, scores and payment, QOF payment allocation, barriers, and suggestions to improve the QOF programme. Purposive sampling was performed. The recruitment of participants was carried out in two steps. First, two provinces in each region (12 regions, excluding Bangkok Metropolitan) with the highest and lowest QOF scores were selected. Second, different types of health facilities (e.g. district hospitals and health-promoting hospitals), as well as provincial and district health offices from the 24 provinces, were selected based on consultations with NHSO staff in the regions. As such, 24 representatives from these organisations, who had been involved in the QOF introduction either as supervisors or primary care providers, were invited to participate in the meetings. Fifteen informants agreed to participate (Table 2).

Data analysis

Interviews and focus group discussions were audio-recorded, transcribed, and analysed using qualitative content analysis, a research method for the interpretation of the content of text data through the systematic classification process of identifying themes or patterns⁶. A single researcher read through the transcripts repeatedly to derive the key issues of the text data that were then sorted into categories and themes based on how different issues were related and linked. After that, the research team discussed the emerging categories, made themes, and changed categories where appropriate.

Quality assurance

Two consultation meetings were organised in July 2015. The aims of these meetings were for the research team to check that key information had been collected and to verify preliminary findings. The first meeting involved 27 key stakeholders at the national level, including policy makers from the MOPH, researchers from universities, as well as representatives from the NHSO and its regional offices. At the second meeting, 31 participants attended, including representatives from the NHSO, provincial and district health offices, health facilities, non-governmental organisations, and patient groups. Six informants in this study attended the consultation meetings.

Policy formulation

Although P4P policies were implemented in many countries with all types of income levels (high, middle, low), the Thai QOF was based on the UK model, since the NHSO believes that the UK QOF is the most famous and is implemented on the largest scale. In addition, Thai policy makers are familiar with the UK UHC model due to several staff exchanges and study visits. Nevertheless, Thai authorities were aware of significant differences in the health delivery systems of the two countries, and also anticipated the poor performance of the Thai QOF as a result of health system limitations, such as insufficient human and financial resources, unreliable databases, and conflict in some policy areas between the MOPH and the NHSO. Importantly, the newly-established initiative had never been piloted. Despite this, the UCS executives maintained that they were confident in introducing this policy since the NHSO, health providers, and officers had gained experience in UCS management from the earlier stage of the P4P programme. Furthermore, QOF managers argued that they adopted a ‘learning by doing’ approach, meaning that different stakeholders could learn through their experiences and adapt to the programme accordingly. The QOF managers also perceived the policy implementation as a capacity strengthening exercise for NHSO staff and providers’ networks.

Regarding the development of quality indicators, the NHSO formulated indicators and associated policy without consultation from key stakeholders, such as the MOPH, health providers, and professional organisations. However, personal consultation with respective experts in the Ministry was carried out for some indicators. Two broad criteria were used to select indicators: 1) an indicator would not increase workload on data entry and reporting because the performance could be measured by using data in the existing MOPH database; and 2) the focus of indicators was placed on the performance at CUP level, as opposed to the outputs and outcomes of each health facility.

Programme implementation: plan vs. reality

The NHSO did not start to disseminate QOF information to its regional offices until October 2013, which was the planned start date for the programme. After this initial dissemination, regional health boards were required to develop local indicators and criteria for the QOF budget allocation. As pointed out by some informants, this resulted in a delay of approximately three to four months in the development of the regional guidelines and its dissemination to respective institutes, including primary care providers. As such, the QOF could not begin until January 2014 in some regions. Aiming to allocate the QOF budget to all CUPs by September 2014, the NHSO decided to use data that the health facilities submitted to the MOPH between April 2013 and March 2014 to analyse QOF performance for the core indicators. Hence, the first-year QOF was based on the information during that period because it took some time for preparation, and the programme began later than expected. This means that the measurement in the first year of the QOF was mainly based on the performance of approximately 9 to 10 months before the providers had been informed about the QOF indicators.

“… QOF performance was measured based on the data of the 3rd and 4th trimester of last year [April to September 2013] and 1st and 2nd trimester of current year [October 2013 to March 2014]. The scores were not associated with current work performances. Additionally, the measurement did not align with the fiscal year, increasing difficulties in creating work plans.” (Health practitioner).

As mentioned above, two disparities between the NHSO’s QOF and MOPH’s KPI exist: 1) there are different indicators; and 2) the same indicators have different templates and timeframes for data entry and measurements. These two factors thus affected the planning and working process of data submission from health facilities.

Key informants asserted that QOF monies were not consistently allocated according to provider achievement, and high scores might not be associated with high quality primary care delivery due to two reasons. First, some local indicators did not involve healthcare quality or the performance of health providers, e.g. the amount of research conducted annually in particular facilities, and the use of data from IT systems to inform health service planning and delivery. Second, the criteria for QOF payments in different regions were not standardised, as the regional health board had discretionary power to make decisions.

An analysis by the NHSO suggested that the QOF budget was allocated according to the QOF scores in only 6 out of 13 regions⁵. In the other regions, the QOF scores were taken into account together with other criteria, such as population size and deprivation level of the catchment area (e.g. hardship and conflict areas).

“The allocation of the QOF budget was based on the financial status of the CUP. In practice, the amount depended on the performance of the health facilities in the CUP. I thought that explicit criteria or guidance for the CUP were necessary.” (Health practitioner).

“In my region, the regional health board agreed to allocate the QOF budget based on [the] number of people registered in the area (40%), and QOF scores (60%).” (Regional NHSO officer).

Impediments in QOF implementation

During the first year of the Thai QOF programme, the crucial impeding factors were the inherent conflicts between the NHSO and MOPH, and weaknesses in the existing IT system. Consequently, there was miscommunication and inadequate coordination between healthcare workers, MOPH and NHSO staff, as well as inaccurately-reported data, lack of capacity for data entry and management, and errors in data transfer from peripheral offices to the national authority.

As health workers pointed out, the QOF was perceived as an NHSO-owned initiative, while the MOPH had developed its own policies to deal with priority health problems and improve primary care services. The lack of engagement between the NHSO and MOPH in the policy formulation phase resulted in miscommunication, culminating in misinformation and inadequate coordination for the translation of QOF policy into action. The focus group discussions made it quite apparent that senior health officers in the MOPH and its central departments were not aware of the NHSO QOF. Although some MOPH general inspectors acknowledged the introduction of the QOF in their regions, they paid attention only to the performance of healthcare delivery according to the MOPH’s direction and associated indicators. Hence, technical supervision and administrative guidance provided by the inspectors for QOF-related activities in most regions were limited.

The other critical problem that surfaced in the QOF’s introduction was the unreliable IT system, resulting in inaccurately-reported data on service delivery. Interviewees asserted that the capacity for data entry and management at CUPs, PCUs, and district health offices was inadequate. Moreover, interviewees in different institutions argued that health workers in some settings intentionally made up data in order to gain high QOF scores and payments. The worst case, as discussed during a focus group meeting, involved attempts at data manipulation.

“Some practitioners entered data accurately, while others intentionally made up data. So, they [the data] did not really reflect the true performance.” (Health practitioner).

“I had a field visit at a province. In this province, they [provincial health officers] were not concerned about under-recording, but over-recording. This was really happening.” (Data manager of a CUP).

Informants also reported technical errors in data transfer from peripheral offices to the national authority.

“I did not realise what was going on. Our performance last year was zero for 5 indicators. Moreover, the OP visits were marked zero. I believe that the problems originated from the data linkage system between our hospital and the regional health data centre.” (Health practitioner).

“Once we submitted our data to the health data centre, we found that our performance did not meet the indicator targets. Only 5 indicators, related to service provision, passed the assessment. When comparing with other health facilities, they also passed approximately 5–6 indicators, which I did not believe was representative of service delivery. I thought the issue was the database.” (Health practitioner).

The poor reliability of data has also been illustrated by comparing results from different databases and surveys on certain indicators. Table 3 shows that the coverage of some primary care services as QOF indicators significantly differed from the findings of a survey conducted by Mahidol university and a report from the MOPH during the same period, as suggested by a published document of the NHSO⁷. Owing to the limited capacity of the IT system, health personnel in PCUs could not obtain feedback on their performance from the provincial data centres. Without such information, it was difficult for PCUs to confirm the accuracy of their QOF scores and payments with CUP managers.

Provider perceptions, inadequate communication, and management issues

Most interviewees in this study agreed with the QOF policy principle that monetary incentives would be effective in enhancing service quality in the Thai UHC context where financing and other resources are scarce. Some also pointed out that the policy helped strengthen teamwork among district hospitals and health-promoting hospitals in terms of patient transfer to receive proper care at the secondary level. They stated that in order to be high achievers, health facilities in a particular CUP would need to cooperate, set service delivery plans together, and improve their referral system. However, besides the problems arising from the MOPH-NHSO conflict and unreliability of data, many weaknesses and associated implementation gaps were observed. First, health workers perceived that the programme was managed in an unfair manner. Given that the QOF budget was deducted from the annual capitation payment for ambulatory care, the policy was not justified as primary care providers did not receive full subsidisation of their service delivery.

“The budget for the QOF programme was deducted from the OP/PP budget [budget for out-patient/health promotion and disease prevention services]. It should have been from other sources. The OP/PP budget is actually aimed at subsidising service deliveries, so it isn’t fair for health providers [since they may receive less money than they had received previously and this may not be sufficient for providing ambulatory services].” (Health facility director).

“…the calculation of the budget for ambulatory services was based on the number of people registered in the catchment areas. Allocation of this budget based on the performance of health providers isn’t fair. The allocation should have been based on the same principle [payments based on the number of registered population].” (Regional NHSO officer).

Additionally, providers perceived that allocating a budget based on performance was unfair because the NHSO collected a proportion of the budget for ambulatory services, and allocated this on the basis of each CUP performance instead of the registered population. While the total amount remained the same, CUPs with high QOF scores were able to receive higher QOF payments, thereby taking a proportion of other CUPs’ budgets in the same region.

“It [QOF budget allocation] was unfair… [and] inappropriate because a province that achieved high QOF scores would receive a larger budget and [would be] taking from other provinces’ UC budget.” (Health practitioner).

As also perceived by hospital administrators and health workers, the unfairness was exacerbated when the allocation of the QOF budget did not rely on actual quality improvement in service delivery at the CUP and PCU levels. Furthermore, there was perceived prejudice in performance measurements owing to the fact that some QOF indicators and targets were relatively difficult to achieve in certain circumstances, e.g. providing screening tests for non-communicable diseases in an area with a large population, urbanised culture and lifestyle, and high rate of labour migration to other areas.

Second, inefficient transfer of information from policy makers to programme managers and practitioners was emphasised by interviewees as an important drawback of the Thai QOF. Traditional, bureaucratic communication approaches that involved lengthy official documents and formal, face-to-face meetings with executives crucially impeded the dissemination of messages. As maintained by some informants from PCUs, they did not have an in-depth understanding about the QOF because CUP representatives who attended the regional meetings did not relay the obtained information. It was also argued that the information deficit resulted in poor attitudes and non-compliance with the policy among service providers.

Third, most QOF indicators, such as coverage of screening or occurrence of complications as a result of poor disease management, were designed to measure the performance at the CUP level, even though district hospitals and health-promoting hospitals are different entities with different financial management. As such, the QOF budget received by the CUP needs to be divided among health facilities. During the interviews and focus group discussions, it was evident that without guidance for QOF budget allocation within the CUP, staff at health-promoting hospitals found it difficult to negotiate a fair share of the budget with the directors of district hospitals and perceived that they were the worse off under this programme. Although some CUPs demonstrated the improved collaboration expected from the QOF implementation between district hospitals and health-promoting hospitals, other CUPs witnessed conflicts between the two entities on account of the QOF budget allocation.

“It was not told how much each indicator would deliver. So it was not sure how the budget should be further allocated to the health-promoting hospitals. We had to find [a] consensus on explicit criteria for budget allocation in order to avoid conflicts.” (District health officer).

“This method of allocation [allocation of the QOF budget through CUP] caused disputes among the health facilities because there were no explicit criteria. Each CUP board could create their own way to allocate the budget.” (Health practitioner from a health-promoting hospital).