Health technology assessment capacity at national level in sub-Saharan Africa: an initial survey of stakeholders

Background: Health technology assessment (HTA) is an effective tool for supporting priority setting (PS) in health. Stakeholder groups need to understand HTA appropriate to their role and to interpret and critique the evidence produced. We aimed to rapidly assess current health system priorities and policy areas of demand for HTA in Sub-Saharan Africa, and identify key gaps in data and skills to inform targeted capacity building. Methods: We revised an existing survey, delivered it to 357 participants, then analysed responses and explored key themes. Results: There were 51 respondents (14%, 30 full completions) across 14 countries. HTA was considered an important and valuable PS tool with a key role in the design of health benefits packages, clinical guideline development, and service improvement. Medicines were identified as a technology type that would especially benefit from the application of HTA. Using HTA to address safety issues (e.g. low-quality medicines) and value for money concerns was particularly highlighted. The perceived availability and accessibility of suitable local data to support HTA varied widely but was mostly considered inadequate and limited. Respondents also noted a need for training support in research methodology and data gathering. Conclusions: While important in raising awareness of HTA as a tool for PS, this study had a low response rate, and that respondents were self-selected. A more refined survey will be developed to support engagement strategies and capacity building.


Introduction
Many countries have committed to universal health coverage (UHC) in the context of the sustainable development goals 1 for affordable access to essential medicines and other health technologies. Many countries in sub-Saharan Africa (SSA) have established national health insurance systems (or are planning to) 2 but they require governments to set health priorities within fiscal limits 3 .
Health technology assessment (HTA) provides a structured approach to synthesising evidence of clinical and cost effectiveness to inform priority-setting activities. Institutionalised HTA systems involve active participation from a range of stakeholders 4 including (government) decision makers, clinicians, academics, consumers, development partners, and HTA knowledge brokers 5 . The International Decision Support Initiative (iDSI) is a global network of health, policy and economic expertise, which seeks to support countries make better decisions about efficient spending on healthcare. iDSI has been working in SSA since 2013 to develop local capacity and support implementation of robust HTA processes 6,7 . There is growing interest across the continent but the current HTA landscape is fragmented and undocumented. We aimed to assess the current health system priorities and policy areas that need HTA, the demand for HTA, and the supply of HTA efforts to identify gaps in data and skills.

Methods
We used an existing framework to examine the need, demand, and supply of HTA in an anonymous survey comprising 12 questions 8 . We used an online survey tool with purposive sampling; participants were contacted by email from the membership lists of the iDSI network and the African Health Economics and Policy Association (AfHEA). The survey was opened in June 2018. There were no inclusion or exclusion criteria regarding the participants. We analysed scale and ranking questions as the mean, and thematically analysed responses to open questions using an inductive approach 9 .

Consent
This survey was conducted in accordance with the Helsinki Declaration of 1975, as revised in 2008. All participants acknowledged written consent by starting the survey (as noted in the survey information section). No survey respondents can be identified in this paper as data has been aggregated. We do not have access to respondent's details -data is fully anonymized. Respondents could leave the survey at any time. We used a scientific society distribution list to send emails to participants but we did not have access to the list.

Results
Of the 357 recipients, 14% responded with 30 fully completing the survey. Half were from a research institute or university, or from within the Ministry of Health (27%). Respondents were from 14 countries (see dataset) but many were from Ghana (40%). The individual-level results are available as Underlying data 9 .
I. Need for HTA In relation to the importance of particular attributes of HTA, respondents highlighted (mean rating out of 10): allocative efficiency (8.9), improving the quality of health care (8.8), transparency in decision making (8.4), budget control (8.0), and equity (7.9). They ranked the six main policy areas for which HTA was considered as urgently needed: 1) informing the design of the basic health benefits package (HBP); 2) producing clinical guidelines or disease management pathways; 3) informing the design of health service delivery; 4) coverage or reimbursement of individual health technologies; 5) provider payment or pay for performance schemes; and 6) registration of health technologies. A key consideration in HBP design was identifying services and technologies that should be covered but in a way that is financially sustainable.
Medicines were considered the most important technology type that would benefit from HTA approaches, followed by vaccines, public health programs, medical devices and diagnostics, service delivery initiatives or incentives, screening or referral programs, and other (e.g. surgical) interventions. Medicines were prioritised because of their relatively large budget impact and their use for many high-burden diseases. Improving the availability and management of vaccines would help to reduce the burden from communicable diseases.
Respondents emphasised three key decision problem areas that could be informed by HTA: health system financing; burden of disease considerations; and health service provision. Challenges in financing included inefficient financing structures (funding schemes and management); lack of funds; sustainability of national insurance; and the costs of medicines. In terms of burden of disease, respondents concerns included antibiotic resistance, and the impact of a growing dual burden of disease (communicable and non-communicable).
The respondents identified a need for research in: 1) health system financing (financing schemes, medicine pricing, and the design of sustainable essential benefits packages); 2) health service provision taking into account equity, efficiency, quality; 3) burden of disease (antibiotic medicine resistance, noncommunicable disease, childhood immunisation); and 4) health policy research.

Amendments from Version 1
We have made amendments to the paper to address the reviewer's comments. These changes include additional text and references: an acknowledgement of some further limitations and biases including that potential collaborators were likely survey respondents; citing two recent studies considering HTA in Ghana  and Ethiopia ; including a recent study in Nigeria that extended our survey with a more detailed understanding of the local context ; citing a recent literature review of HTA in Sub-Saharan Africa ; including an iDSI report that the Government of Ghana has recently launched the Ghana Health Technology Strategy (1st Edition).

II. Demand for HTA
The respondents provided information on who they considered to be the key users of HTA information and the types of HTA evidence most useful for decision making. The users were identified as: ministries of health; other government departments; public health insurance bodies; providers and health professionals; universities and research institutes; donor organisations; and pharmaceutical companies. The evidence considered most useful was: safety; economic issues; information on technology effectiveness; and accounting for social/ethical concerns. The safety concerns were explicitly linked to the availability and use of generic medicines where quality could not be guaranteed. The economics issues related to the tension between growing claims on, but limited availability of, health resources.

III. Supply of HTA
All respondents were associated with organisations that generated or facilitated health services research. Research institutes highlighted their ability to provide expertise and skills for HTA research but some respondents noted a lack of human capacity for HTA. Political support was regarded as essential but could be impeded by politicised decision-making, internal politics in the leadership of the HTA process, cultural barriers in data and information sharing, and lack of funding for HTA activities.
The availability and accessibility of local data varied; generally, medicine prices were available but the costs of health services were neither easily known nor available. There are often data on the burden of disease but its application may be limited due to incomplete or unreliable documentation 10 . The organisations which either generate or supply evidence mostly included government agencies, university and affiliated research institutes and donors plus development partners.
Respondents were asked to consider issues related to the wider 'HTA infrastructure' including the availability of existing process and methods manuals, or processes for involving the public in decision making. Respondents focused on the role of the public and civil society, and three themes emerged: 1) the extent of public involvement in consultation processes; 2) the role of advocacy; and 3) the absence of any public role in priority-setting decisions. Many respondents stressed the importance of consulting the public, but noted that in practice there was no involvement. Some respondents posited that such groups adopt an advocacy role by holding decision makers accountable, and creating pressure through media campaigns and other means to highlight dysfunctions in the health system.
Four main areas of training needs for HTA generators and users were identified: 1) research methods in HTA and data gathering (and economic evaluations); 2) identifying and implementing evidence and using it to inform policy; 3) conducting economic evaluations; and 4) developing capacity and building awareness. Some respondents noted that both HTA generators and health policy-makers and practitioners need training in HTA to facilitate the reliable and efficient interpretation and use of research results, translation into policy, and advocacy.

Discussion
HTA was considered an important and valuable priority-setting tool with a key role in HBP design, clinical guideline development, and service improvement. Medicines and vaccines were the health technologies that would most benefit from HTA. The perceived availability and accessibility of suitable local data varied but was broadly considered inadequate and limited. There was a strong need for training support in research methodology and data gathering for HTA evidence. This is, as far as the authors are aware, the first systematic survey of the HTA landscape in SSA. Our survey was extended in a subsequent study in Nigeria with a more detailed understanding of the local context 11 . It is a preliminary survey; we will refine in future iterations with piloting. Some respondents may have known more about HTA than others (for instance in Ghana where there has been HTA activities in recent years 7,12 ), which may have biased the results. It appears that some questions were not entirely understood; this may have contributed to the low response rate. Despite this, we elicited in-depth responses from many respondents. They responded as individuals and so their views may not reflect those of particular agencies or governments. We note that potential collaborators on this study were likely respondents.
Our results largely align with the WHO Global Survey on Health Technology Assessment by National Authorities 13 with regards to capacity needs, especially in the African region. The main barriers were a lack of qualified human resources, funding, and information 13 . The themes are echoed in a recent literature review of HTA in sub-Saharan Africa 14 . The interest in HTA as a priority-setting tool in the context of UHC is evident in the Asian region where HTA is more established than in SSA but there are ongoing challenges in the inconsistent use of HTA as means of updating benefits packages, concerns over transparency, and barriers related to data sharing 15 .
The interest in using HTA to support priority-setting decisions for medicines and vaccines is welcome as these represent a high proportion of healthcare expenditure. Access to such technologies, particularly in the poorest countries, has been facilitated by international development partners but as the national income of aid-recipient countries increases, so do co-financing obligations. At some point countries will 'graduate' from aid -they will need mechanisms to effectively manage healthcare resources. Ghana has begun its journey to reduce reliance on donor assistance for health 7,16 . The Government of Ghana has recently launched the Ghana Health Technology Strategy (1st Edition) 17 .
Although HTA remains relatively under-developed in SSA, there is growing political commitment and policy interest 11,12,[18][19][20] . Priority setting is inevitable: the question is not whether, but rather how, to set them. The HTA challenges outlined here could be mitigated by building HTA systems through pooling resources across countries and harmonising policies in health (e.g. medicines regulatory harmonisation, upstream of HTA 21,22 ). The benefits of such harmonisation can be further enhanced through coordinated action on HTA policies which will help secure innovation uptake subsequent to regulation, at value-based prices reflecting local conditions.

Centre for Health Economics Research and Evaluation, University of Technology Sydney, Broadway, NSW, Australia
This is a great article that used an online survey to explore the perception and views of SSA respondents in the need for HTA, demand for HTA and supply of HTA. Survey response rate was low, however responses were from participants across 14 countries in SSA thus fairly representative of the HTA landscape in SSA. Almost half of the respondents were from Ghana, where progress has been made towards the use of HTA in the health system. Respondents considered HTA was most importantly needed for informing the design of basic health benefits package (for national health insurance schemes) and development of clinical guidelines and disease management pathways. Stakeholders perceived as potential users of HTA were those from the ministry of health, public health insurance bodies. Whilst some respondents were of the view that they could provide expertise and skills for HTA, the majority were of the view that there were limited local human capacity, data and other infrastructures to support HTA.
Specific comments to improving the research paper are summarised below: The title of the article is suggestive of assessing capacity for HTA in terms of data, human and other infrastructure for HTA. However, the results are indicative of assessing what respondents perceived as the potential use, potential users and production of and barriers to producing HTA.

1.
The abstract mentions 51 respondents completing the survey but the results note that only 30 fully completed. The authors should note whether partially completed surveys were included in the analysis or otherwise and provide justification for the choice. 2.
It will be informative to provide a list of the countries that responded to the survey and also a list of where they worked.

3.
Given that the majority of respondents were from Ghana, perhaps authors should provide some reasons/speculations as to why this was the case. Could it be as a result of Ghanaian 4.
respondents being more aware of HTA due to the recent involvement with iDSi projects for HTA and progress made towards using HTA for formal decision making in the country?
The authors should comment on whether the responses are likely to be biased by the general knowledge and awareness of respondents about HTA. Members of AfHEA and the majority of African health economists and policy analysts are mainly involved in health financing and not economic evaluation and HTA.

5.
It will be interesting to know the differences in responses between respondents from Ghana countries that may have some ongoing work towards the use of HTA for health decision making and those that do not. Another area to consider is the differences in responses between respondents according to their area of work.

6.
There are some studies from SSA that have explored the perceptions of decision-makers towards the use economic evaluations and HTA for decision making that covered the themes presented in this paper that the authors could have cited to support their findings. 7.
The discussion mentions Ghana transitioning from reliance upon donor assistance for health. Perhaps the authors should add more context and how that may contribute to the current landscape of HTA in Ghana.

8.
Lastly, the authors mention refining the survey tool in future iterations as it appears some questions were not entirely understood. Was this limitation considered in the data analysis? This point should be noted as a limitation of the study as it may have affected the responses and subsequent results presented in this paper. Also the authors should note what 'refinement' they intend to make to the survey tool -is it content wise, or rewording, etc. 9.

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate? majority of African health economists and policy analysts are mainly involved in health financing and not economic evaluation and HTA.
R: Text edited to "Some respondents may have known more about HTA than others which may have biased the results." 6. It will be interesting to know the differences in responses between respondents from Ghana countries that may have some ongoing work towards the use of HTA for health decision making and those that do not. Another area to consider is the differences in responses between respondents according to their area of work. The authors are experts on HTA and have a wealth of experience in priority-setting, health systems policy, and global health.
In this interesting and important article, they have painted the current HTA landscape in SSA and highlighted areas in need of further investigations. Capacity-building efforts for HTA in SSA can now be refined based on the critical areas highlighted in this initial survey.
The limitations of this survey have been acknowledged. I think the low response rate and incomplete answers are a reflection of the stage of development of HTA in SSA.
Specific comments are provided below: The authorship of the article could have included African researchers currently on the continent on a full-time basis to help with local capacity development and international collaboration.

Are sufficient details of methods and analysis provided to allow replication by others? Yes
If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable
Are all the source data underlying the results available to ensure full reproducibility?