Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world

Dylan Collins; Joseph Lee; Niklas Bobrovitz; Constantinos Koshiaris; Alison Ward; Carl J. Heneghan

doi:10.12688/f1000research.9742.1

Home Browse Simple and adaptable R implementation of WHO/ISH cardiovascular risk...

ALL Metrics

Views

Downloads

Get PDF

Get XML

Export

▬

✚

Software Tool Article

Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world

[version 1; peer review: 3 approved with reservations]

Dylan Collins ¹, Joseph Lee¹, Niklas Bobrovitz¹, Constantinos Koshiaris¹, Alison Ward¹, Carl J. Heneghan¹

Dylan Collins ¹, Joseph Lee¹, [...] Niklas Bobrovitz¹, Constantinos Koshiaris¹, Alison Ward¹, Carl J. Heneghan¹

PUBLISHED 14 Oct 2016

Author details Author details

¹ Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

OPEN PEER REVIEW

REVIEWER STATUS

This article is included in the RPackage gateway.

Abstract

The World Health Organisation and International Society of Hypertension (WHO/ISH) cardiovascular disease (CVD) risk assessment charts have been implemented in many low- and middle-income countries as part of the WHO Package of Essential Non-Communicable Disease (PEN) Interventions for Primary Health Care in Low-Resource settings. Evaluation of the WHO/ISH cardiovascular risk charts and their use is a key priority and since they only exist in paper or PDF formats, we developed a simple R implementation of the charts for all epidemiological subregions of the world. The main strengths of this implementation are that it is built in a free, open-source, coding language with simple syntax, can be modified by the user, and can be used with a standard computer.

Keywords

WHO/ISH, Cardiovascular Risk Charts, Risk Score, R

Corresponding author: Dylan Collins

Competing interests: DC has received payment from the WHO for consulting work. AW and CH have received expenses and grant income from the WHO for projects related to CVD and Self Care in NCDs, and direct a WHO Collaborating Centre. CH also receives funding form the National Institute of Health Research (NIHR) School of Primary Care Research. JL, CK, and NB declare no competing interests.

Grant information: The WHO Collaborating Centre for Self Care paid for the open access publishing fees. No other funding was provided for this work.

Copyright: © 2016 Collins D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

How to cite: Collins D, Lee J, Bobrovitz N et al. Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world [version 1; peer review: 3 approved with reservations]. F1000Research 2016, 5:2522 (https://doi.org/10.12688/f1000research.9742.1) First published: 14 Oct 2016, 5:2522 (https://doi.org/10.12688/f1000research.9742.1) Latest published: 08 Mar 2017, 5:2522 (https://doi.org/10.12688/f1000research.9742.2)

Introduction

Cardiovascular disease (CVD) is the leading cause of death worldwide, including in many low-and-middle income countries (LMIC)^1,2. Preventing CVD is therefore a worldwide priority and the World Health Organisation (WHO) is coordinating a global strategy for LMIC to systematically prevent CVD in primary care³.

In 2007 the WHO and the International Society of Hypertension (ISH) published the WHO/ISH CVD risk charts for all WHO epidemiological subregions of the world⁴. These charts are to be used as part of the WHO’s Package of Essential NCD (PEN) Interventions for Primary Health Care in Low-Resource Settings in jurisdictions that do not have their own population-derived risk assessment algorithms. While these charts are a good resource for many health systems, little is known about their validity⁵. Therefore, it is important that jurisdictions that implement these charts conduct operational research and attempt to validate and optimise them for their setting.

Two paper-based versions of WHO/ISH charts are available for each subregion: one that requires measured total cholesterol and one that does not. The latter was made available for use in settings with limited access to laboratory testing or where the cost of cholesterol testing is prohibitive. Both charts require information on age, gender, diabetes status, smoking status, and systolic blood pressure to stratify people into one of five risk categories of 10-year risk of a fatal or non-fatal CVD event. Further instructions for their use have been published³.

Through our experience collaborating with LMIC with the implementation of WHO PEN, we identified a common need for an open-source tool to facilitate the implementation of WHO/ISH risk charts and operational research of WHO PEN at a population level. We therefore developed an open source tool in R (https://www.r-project.org/), which we describe here and make available to researchers in LMIC.

Methods

Extraction of WHO/ISH cardiovascular risk charts

We extracted all versions of the paper-based WHO/ISH CVD risk charts by hand into a standardized Microsoft Excel template, independently and in duplicate. We used RStudio (version 0.99.489) to compare the duplicate extractions and to calculate Cohen’s kappa coefficient for inter-rater reliability, using the irr package (version 0.84). Discrepancies were reviewed by the same two extractors and resolved by referring to the original paper chart.

Development of the WHO/ISH risk function

One author wrote the initial code for the WHO/ISH risk function in R (DC). This was reviewed and adapted by a second author experienced in the R language (CK). Two additional authors (JL, NB), new to the R language, reviewed the code to ensure the syntax was simple and comprehensible.

Validation

A MatLab implementation of WHO/ISH risk charts for epidemiological subregion SEAR D had been previously reported⁶. We used Octave (www.gnu.org/software/octave/) version 8.3.2 to calculate the SEAR D WHO/ISH risk score for every possible combination of risk factors using the previously reported MatLab implementation, and compared the percent agreement to the risk scores generated by our R implementation.

Results

Extraction of WHO/ISH cardiovascular risk charts

All WHO/ISH risk charts were extracted by hand into a single comma delimited file (Dataset 1). Our function is dependent on this file. Cohen’s kappa for initial agreement between the independent extractors was 0.97, indicating excellent agreement. All remaining discrepancies were resolved by consensus.

	X	dm	gdr	smk	age	sbp	chl	refv	AFR_D	AFR_E	AMR_A	AMR_B	AMR_D	EMR_B	EMR_D	EUR_A	EUR_B	EUR_C	SEAR_B	SEAR_D	WPR_B	WPR_A
1	1	1	0	1	40	120	4	400111204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
2	2	1	0	1	40	120	5	400111205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
3	3	1	0	1	40	120	6	400111206	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
4	4	1	0	1	40	120	7	400111207	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
5	5	1	0	1	40	120	8	400111208	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	<10%	<10%	<10%	10% to <20%	<10%	20% to <30%	30% to <40%	<10%	<10%
6	6	1	0	1	40	140	4	400111404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
7	7	1	0	1	40	140	5	400111405	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%	<10%
8	8	1	0	1	40	140	6	400111406	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
9	9	1	0	1	40	140	7	400111407	10% to <20%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	<10%	10% to <20%	<10%	10% to <20%	<10%	30% to <40%	30% to <40%	10% to <20%	10% to <20%
10	10	1	0	1	40	140	8	400111408	20% to <30%	20% to <30%	>=40%	30% to <40%	30% to <40%	10% to <20%	20% to <30%	10% to <20%	30% to <40%	20% to <30%	>=40%	>=40%	30% to <40%	10% to <20%
11	11	1	0	1	40	160	4	400111604	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	10% to <20%	20% to <30%	<10%	<10%
12	12	1	0	1	40	160	5	400111605	10% to <20%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	<10%	10% to <20%	<10%	30% to <40%	20% to <30%	20% to <30%	10% to <20%
13	13	1	0	1	40	160	6	400111606	20% to <30%	20% to <30%	30% to <40%	>=40%	30% to <40%	20% to <30%	20% to <30%	<10%	20% to <30%	10% to <20%	>=40%	>=40%	20% to <30%	20% to <30%
14	14	1	0	1	40	160	7	400111607	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	10% to <20%	>=40%	20% to <30%	>=40%	>=40%	>=40%	30% to <40%
15	15	1	0	1	40	160	8	400111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
16	16	1	0	1	40	180	4	400111804	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	10% to <20%	30% to <40%	20% to <30%	>=40%	>=40%	>=40%	30% to <40%
17	17	1	0	1	40	180	5	400111805	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%
18	18	1	0	1	40	180	6	400111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
19	19	1	0	1	40	180	7	400111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
20	20	1	0	1	40	180	8	400111808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
21	21	1	0	1	50	120	4	500111204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
22	22	1	0	1	50	120	5	500111205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
23	23	1	0	1	50	120	6	500111206	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
24	24	1	0	1	50	120	7	500111207	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	10% to <20%	20% to <30%	<10%	<10%
25	25	1	0	1	50	120	8	500111208	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	10% to <20%	20% to <30%	>=40%	10% to <20%	<10%
26	26	1	0	1	50	140	4	500111404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
27	27	1	0	1	50	140	5	500111405	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
28	28	1	0	1	50	140	6	500111406	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	<10%	20% to <30%	30% to <40%	10% to <20%	<10%
29	29	1	0	1	50	140	7	500111407	20% to <30%	20% to <30%	30% to <40%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	<10%	20% to <30%	10% to <20%	30% to <40%	>=40%	20% to <30%	10% to <20%
30	30	1	0	1	50	140	8	500111408	20% to <30%	20% to <30%	>=40%	30% to <40%	>=40%	30% to <40%	30% to <40%	10% to <20%	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%	20% to <30%
31	31	1	0	1	50	160	4	500111604	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	<10%	20% to <30%	20% to <30%	10% to <20%	10% to <20%
32	32	1	0	1	50	160	5	500111605	10% to <20%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	20% to <30%	20% to <30%	<10%	10% to <20%	10% to <20%	30% to <40%	30% to <40%	20% to <30%	10% to <20%
33	33	1	0	1	50	160	6	500111606	20% to <30%	20% to <30%	>=40%	>=40%	30% to <40%	30% to <40%	30% to <40%	10% to <20%	20% to <30%	10% to <20%	>=40%	>=40%	30% to <40%	20% to <30%
34	34	1	0	1	50	160	7	500111607	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
35	35	1	0	1	50	160	8	500111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
36	36	1	0	1	50	180	4	500111804	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
37	37	1	0	1	50	180	5	500111805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
38	38	1	0	1	50	180	6	500111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
39	39	1	0	1	50	180	7	500111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
40	40	1	0	1	50	180	8	500111808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
41	41	1	0	1	60	120	4	600111204	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
42	42	1	0	1	60	120	5	600111205	10% to <20%	<10%	<10%	<10%	<10%	20% to <30%	10% to <20%	<10%	<10%	<10%	10% to <20%	20% to <30%	<10%	<10%
43	43	1	0	1	60	120	6	600111206	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	20% to <30%	20% to <30%	<10%	10% to <20%	<10%	20% to <30%	30% to <40%	10% to <20%	<10%
44	44	1	0	1	60	120	7	600111207	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	30% to <40%	30% to <40%	<10%	10% to <20%	10% to <20%	30% to <40%	>=40%	10% to <20%	10% to <20%
45	45	1	0	1	60	120	8	600111208	30% to <40%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	>=40%	>=40%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	>=40%	20% to <30%	10% to <20%
46	46	1	0	1	60	140	4	600111404	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	<10%	10% to <20%	<10%	10% to <20%	30% to <40%	10% to <20%	<10%
47	47	1	0	1	60	140	5	600111405	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	30% to <40%	20% to <30%	<10%	10% to <20%	10% to <20%	20% to <30%	>=40%	10% to <20%	10% to <20%
48	48	1	0	1	60	140	6	600111406	20% to <30%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	>=40%	30% to <40%	10% to <20%	20% to <30%	10% to <20%	30% to <40%	>=40%	20% to <30%	10% to <20%
49	49	1	0	1	60	140	7	600111407	30% to <40%	20% to <30%	30% to <40%	20% to <30%	30% to <40%	>=40%	>=40%	10% to <20%	20% to <30%	20% to <30%	>=40%	>=40%	20% to <30%	10% to <20%
50	50	1	0	1	60	140	8	600111408	30% to <40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	20% to <30%
51	51	1	0	1	60	160	4	600111604	20% to <30%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	>=40%	30% to <40%	10% to <20%	10% to <20%	10% to <20%	30% to <40%	>=40%	20% to <30%	10% to <20%
52	52	1	0	1	60	160	5	600111605	30% to <40%	20% to <30%	30% to <40%	20% to <30%	30% to <40%	>=40%	>=40%	10% to <20%	20% to <30%	20% to <30%	>=40%	>=40%	30% to <40%	20% to <30%
53	53	1	0	1	60	160	6	600111606	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	30% to <40%	20% to <30%	>=40%	>=40%	>=40%	20% to <30%
54	54	1	0	1	60	160	7	600111607	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
55	55	1	0	1	60	160	8	600111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
56	56	1	0	1	60	180	4	600111804	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
57	57	1	0	1	60	180	5	600111805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
58	58	1	0	1	60	180	6	600111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
59	59	1	0	1	60	180	7	600111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
60	60	1	0	1	60	180	8	600111808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
61	61	1	0	1	70	120	4	700111204	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%
62	62	1	0	1	70	120	5	700111205	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	30% to <40%	10% to <20%	10% to <20%
63	63	1	0	1	70	120	6	700111206	20% to <30%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	>=40%	20% to <30%	10% to <20%
64	64	1	0	1	70	120	7	700111207	20% to <30%	10% to <20%	30% to <40%	20% to <30%	30% to <40%	>=40%	>=40%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	>=40%	20% to <30%	20% to <30%
65	65	1	0	1	70	120	8	700111208	30% to <40%	30% to <40%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	30% to <40%	30% to <40%	30% to <40%	>=40%	20% to <30%	20% to <30%
66	66	1	0	1	70	140	4	700111404	20% to <30%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	>=40%	20% to <30%	10% to <20%
67	67	1	0	1	70	140	5	700111405	20% to <30%	20% to <30%	30% to <40%	20% to <30%	30% to <40%	>=40%	30% to <40%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	>=40%	20% to <30%	20% to <30%
68	68	1	0	1	70	140	6	700111406	30% to <40%	20% to <30%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	30% to <40%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%
69	69	1	0	1	70	140	7	700111407	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
70	70	1	0	1	70	140	8	700111408	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
71	71	1	0	1	70	160	4	700111604	30% to <40%	20% to <30%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%	20% to <30%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%
72	72	1	0	1	70	160	5	700111605	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
73	73	1	0	1	70	160	6	700111606	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
74	74	1	0	1	70	160	7	700111607	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
75	75	1	0	1	70	160	8	700111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
76	76	1	0	1	70	180	4	700111804	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
77	77	1	0	1	70	180	5	700111805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
78	78	1	0	1	70	180	6	700111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
79	79	1	0	1	70	180	7	700111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
80	80	1	0	1	70	180	8	700111808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
81	81	1	0	0	40	120	4	400101204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
82	82	1	0	0	40	120	5	400101205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
83	83	1	0	0	40	120	6	400101206	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
84	84	1	0	0	40	120	7	400101207	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
85	85	1	0	0	40	120	8	400101208	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	20% to <30%	20% to <30%	<10%	<10%
86	86	1	0	0	40	140	4	400101404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
87	87	1	0	0	40	140	5	400101405	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
88	88	1	0	0	40	140	6	400101406	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%	<10%
89	89	1	0	0	40	140	7	400101407	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	20% to <30%	10% to <20%	<10%	<10%
90	90	1	0	0	40	140	8	400101408	20% to <30%	10% to <20%	30% to <40%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	<10%	20% to <30%	10% to <20%	20% to <30%	>=40%	20% to <30%	10% to <20%
91	91	1	0	0	40	160	4	400101604	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
92	92	1	0	0	40	160	5	400101605	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
93	93	1	0	0	40	160	6	400101606	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	<10%	10% to <20%	<10%	10% to <20%	<10%	30% to <40%	20% to <30%	10% to <20%	<10%
94	94	1	0	0	40	160	7	400101607	20% to <30%	20% to <30%	30% to <40%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	<10%	20% to <30%	10% to <20%	>=40%	>=40%	20% to <30%	10% to <20%
95	95	1	0	0	40	160	8	400101608	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	30% to <40%	10% to <20%	>=40%	20% to <30%	>=40%	>=40%	>=40%	30% to <40%
96	96	1	0	0	40	180	4	400101804	10% to <20%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%	20% to <30%	<10%	20% to <30%	10% to <20%	30% to <40%	30% to <40%	20% to <30%	10% to <20%
97	97	1	0	0	40	180	5	400101805	20% to <30%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%	30% to <40%	10% to <20%	30% to <40%	20% to <30%	>=40%	>=40%	>=40%	20% to <30%
98	98	1	0	0	40	180	6	400101806	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	10% to <20%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
99	99	1	0	0	40	180	7	400101807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
100	100	1	0	0	40	180	8	400101808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
101	101	1	0	0	50	120	4	500101204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
102	102	1	0	0	50	120	5	500101205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
103	103	1	0	0	50	120	6	500101206	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
104	104	1	0	0	50	120	7	500101207	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
105	105	1	0	0	50	120	8	500101208	<10%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	10% to <20%	<10%	20% to <30%	20% to <30%	<10%	<10%
106	106	1	0	0	50	140	4	500101404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
107	107	1	0	0	50	140	5	500101405	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
108	108	1	0	0	50	140	6	500101406	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
109	109	1	0	0	50	140	7	500101407	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	<10%	20% to <30%	20% to <30%	10% to <20%	<10%
110	110	1	0	0	50	140	8	500101408	20% to <30%	20% to <30%	30% to <40%	30% to <40%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	10% to <20%	30% to <40%	>=40%	20% to <30%	10% to <20%
111	111	1	0	0	50	160	4	500101604	<10%	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%
112	112	1	0	0	50	160	5	500101605	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	<10%	10% to <20%	20% to <30%	10% to <20%	<10%
113	113	1	0	0	50	160	6	500101606	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	10% to <20%	30% to <40%	30% to <40%	10% to <20%	10% to <20%
114	114	1	0	0	50	160	7	500101607	20% to <30%	20% to <30%	30% to <40%	30% to <40%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	10% to <20%	>=40%	>=40%	20% to <30%	10% to <20%
115	115	1	0	0	50	160	8	500101608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
116	116	1	0	0	50	180	4	500101804	20% to <30%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	30% to <40%	20% to <30%
117	117	1	0	0	50	180	5	500101805	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%	>=40%	10% to <20%	30% to <40%	20% to <30%	>=40%	>=40%	>=40%	20% to <30%
118	118	1	0	0	50	180	6	500101806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
119	119	1	0	0	50	180	7	500101807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
120	120	1	0	0	50	180	8	500101808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
121	121	1	0	0	60	120	4	600101204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
122	122	1	0	0	60	120	5	600101205	<10%	<10%	10% to <20%	<10%	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
123	123	1	0	0	60	120	6	600101206	<10%	<10%	10% to <20%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%	10% to <20%	<10%	10% to <20%	<10%	<10%
124	124	1	0	0	60	120	7	600101207	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	<10%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	<10%
125	125	1	0	0	60	120	8	600101208	20% to <30%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	30% to <40%	30% to <40%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	10% to <20%	10% to <20%
126	126	1	0	0	60	140	4	600101404	<10%	<10%	10% to <20%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%	10% to <20%	<10%	10% to <20%	10% to <20%	<10%
127	127	1	0	0	60	140	5	600101405	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	<10%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%
128	128	1	0	0	60	140	6	600101406	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	30% to <40%	10% to <20%	10% to <20%
129	129	1	0	0	60	140	7	600101407	20% to <30%	20% to <30%	30% to <40%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	>=40%	20% to <30%	10% to <20%
130	130	1	0	0	60	140	8	600101408	30% to <40%	20% to <30%	>=40%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	30% to <40%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%
131	131	1	0	0	60	160	4	600101604	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	20% to <30%	20% to <30%	10% to <20%
132	132	1	0	0	60	160	5	600101605	20% to <30%	10% to <20%	30% to <40%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	20% to <30%
133	133	1	0	0	60	160	6	600101606	20% to <30%	20% to <30%	>=40%	30% to <40%	20% to <30%	>=40%	>=40%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%
134	134	1	0	0	60	160	7	600101607	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	20% to <30%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
135	135	1	0	0	60	160	8	600101608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
136	136	1	0	0	60	180	4	600101804	>=40%	30% to <40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	20% to <30%	>=40%	>=40%	30% to <40%	>=40%	>=40%	30% to <40%
137	137	1	0	0	60	180	5	600101805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
138	138	1	0	0	60	180	6	600101806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
139	139	1	0	0	60	180	7	600101807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
140	140	1	0	0	60	180	8	600101808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
141	141	1	0	0	70	120	4	700101204	<10%	<10%	10% to <20%	<10%	<10%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
142	142	1	0	0	70	120	5	700101205	10% to <20%	<10%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	10% to <20%	20% to <30%	<10%	<10%
143	143	1	0	0	70	120	6	700101206	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%
144	144	1	0	0	70	120	7	700101207	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	30% to <40%	10% to <20%	10% to <20%
145	145	1	0	0	70	120	8	700101208	20% to <30%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	>=40%	30% to <40%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	>=40%	10% to <20%	10% to <20%
146	146	1	0	0	70	140	4	700101404	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%
147	147	1	0	0	70	140	5	700101405	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	30% to <40%	10% to <20%	10% to <20%
148	148	1	0	0	70	140	6	700101406	20% to <30%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%
149	149	1	0	0	70	140	7	700101407	20% to <30%	20% to <30%	30% to <40%	20% to <30%	30% to <40%	>=40%	30% to <40%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	>=40%	20% to <30%	20% to <30%
150	150	1	0	0	70	140	8	700101408	30% to <40%	30% to <40%	>=40%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%	20% to <30%
151	151	1	0	0	70	160	4	700101604	20% to <30%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	30% to <40%	20% to <30%	10% to <20%
152	152	1	0	0	70	160	5	700101605	20% to <30%	20% to <30%	30% to <40%	20% to <30%	30% to <40%	>=40%	30% to <40%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	>=40%	20% to <30%	20% to <30%
153	153	1	0	0	70	160	6	700101606	30% to <40%	20% to <30%	>=40%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	>=40%	30% to <40%	20% to <30%
154	154	1	0	0	70	160	7	700101607	>=40%	30% to <40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
155	155	1	0	0	70	160	8	700101608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
156	156	1	0	0	70	180	4	700101804	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%
157	157	1	0	0	70	180	5	700101805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
158	158	1	0	0	70	180	6	700101806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
159	159	1	0	0	70	180	7	700101807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
160	160	1	0	0	70	180	8	700101808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
161	161	1	1	1	40	120	4	401111204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
162	162	1	1	1	40	120	5	401111205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
163	163	1	1	1	40	120	6	401111206	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
164	164	1	1	1	40	120	7	401111207	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
165	165	1	1	1	40	120	8	401111208	10% to <20%	10% to <20%	30% to <40%	20% to <30%	<10%	<10%	10% to <20%	<10%	10% to <20%	10% to <20%	20% to <30%	30% to <40%	10% to <20%	10% to <20%
166	166	1	1	1	40	140	4	401111404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
167	167	1	1	1	40	140	5	401111405	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
168	168	1	1	1	40	140	6	401111406	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
169	169	1	1	1	40	140	7	401111407	10% to <20%	20% to <30%	30% to <40%	10% to <20%	<10%	<10%	10% to <20%	<10%	10% to <20%	10% to <20%	10% to <20%	30% to <40%	10% to <20%	10% to <20%
170	170	1	1	1	40	140	8	401111408	20% to <30%	>=40%	>=40%	>=40%	20% to <30%	10% to <20%	20% to <30%	10% to <20%	30% to <40%	30% to <40%	>=40%	>=40%	20% to <30%	20% to <30%
171	171	1	1	1	40	160	4	401111604	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	10% to <20%	<10%	<10%	<10%	10% to <20%	10% to <20%	10% to <20%	<10%
172	172	1	1	1	40	160	5	401111605	10% to <20%	20% to <30%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%
173	173	1	1	1	40	160	6	401111606	30% to <40%	>=40%	30% to <40%	30% to <40%	20% to <30%	10% to <20%	20% to <30%	<10%	20% to <30%	10% to <20%	20% to <30%	>=40%	30% to <40%	20% to <30%
174	174	1	1	1	40	160	7	401111607	>=40%	>=40%	>=40%	>=40%	30% to <40%	20% to <30%	30% to <40%	10% to <20%	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%	30% to <40%
175	175	1	1	1	40	160	8	401111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
176	176	1	1	1	40	180	4	401111804	>=40%	>=40%	30% to <40%	>=40%	>=40%	30% to <40%	>=40%	10% to <20%	30% to <40%	30% to <40%	>=40%	>=40%	>=40%	30% to <40%
177	177	1	1	1	40	180	5	401111805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	10% to <20%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
178	178	1	1	1	40	180	6	401111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
179	179	1	1	1	40	180	7	401111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
180	180	1	1	1	40	180	8	401111808	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
181	181	1	1	1	50	120	4	501111204	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
182	182	1	1	1	50	120	5	501111205	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
183	183	1	1	1	50	120	6	501111206	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	10% to <20%	<10%	<10%
184	184	1	1	1	50	120	7	501111207	10% to <20%	10% to <20%	20% to <30%	10% to <20%	<10%	<10%	10% to <20%	<10%	<10%	<10%	10% to <20%	20% to <30%	<10%	<10%
185	185	1	1	1	50	120	8	501111208	20% to <30%	10% to <20%	>=40%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	<10%	20% to <30%	20% to <30%	20% to <30%	>=40%	10% to <20%	10% to <20%
186	186	1	1	1	50	140	4	501111404	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%	<10%
187	187	1	1	1	50	140	5	501111405	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	<10%	10% to <20%	<10%	<10%	<10%	<10%	10% to <20%	10% to <20%	<10%
188	188	1	1	1	50	140	6	501111406	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	10% to <20%	10% to <20%	<10%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	10% to <20%	10% to <20%
189	189	1	1	1	50	140	7	501111407	20% to <30%	30% to <40%	>=40%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	<10%	20% to <30%	20% to <30%	20% to <30%	>=40%	20% to <30%	10% to <20%
190	190	1	1	1	50	140	8	501111408	30% to <40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	>=40%	20% to <30%	>=40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%
191	191	1	1	1	50	160	4	501111604	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%	10% to <20%	20% to <30%	<10%	10% to <20%	10% to <20%	10% to <20%	20% to <30%	20% to <30%	10% to <20%
192	192	1	1	1	50	160	5	501111605	20% to <30%	30% to <40%	30% to <40%	30% to <40%	20% to <30%	20% to <30%	30% to <40%	10% to <20%	20% to <30%	20% to <30%	20% to <30%	30% to <40%	30% to <40%	20% to <30%
193	193	1	1	1	50	160	6	501111606	30% to <40%	>=40%	>=40%	>=40%	30% to <40%	30% to <40%	>=40%	10% to <20%	30% to <40%	30% to <40%	30% to <40%	>=40%	>=40%	30% to <40%
194	194	1	1	1	50	160	7	501111607	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
195	195	1	1	1	50	160	8	501111608	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
196	196	1	1	1	50	180	4	501111804	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	20% to <30%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
197	197	1	1	1	50	180	5	501111805	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	30% to <40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
198	198	1	1	1	50	180	6	501111806	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
199	199	1	1	1	50	180	7	501111807	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%	>=40%
This is a portion of the data; to view all the data, please download the file.

Dataset 1.CSV file required for function (file name= “WHO_ISH_Scores.csv”).

Development of the WHO/ISH risk function

We developed a simple function, named WHO_ISH_Risk(), that, when loaded in the R workspace, will calculate the WHO/ISH risk score for any epidemiological subregion (Dataset 2) (Figure 1). We intentionally used simple syntax such that users with a beginner’s level of experience with R can adapt the code as needed.

Dataset 2.Code for the WHO_ISH_Risk() function and worked example (file name = “Worked_Example.rtf”).

Figure 1. The WHO_ISH_Risk() function code in R.

The WHO_ISH_Risk function requires seven parameters: age, gender, smoking status, diabetes status, systolic blood pressure, total cholesterol, and the appropriate WHO epidemiological subregion. These parameters and their codes are summarised in Table 1. The function format in the workspace is: WHO_ISH_Risk(age, gdr, smk, sbp, dm, chl, subregion). No default values are specified for any parameter.

Table 1. Description of the WHO_ISH_Risk() function parameters.

Function parameter	Full parameter name	Parameter class	Parameter values
age	Age	numeric	Continuous (years)
gdr	Gender	numeric	Dichotomous (0=female; 1=male)
smk	Smoking	numeric	Dichotomous (0=not smoker; 1=smoker)
sbp	Systolic blood pressure	numeric	Continuous (mmHg)
dm	Diabetes mellitus status	numberic	Dichotomous (0=non-diabetic; 1=diabetic)
chl	Total cholesterol	numeric	Continuous (mmol/ L); 0=unknown cholesterol
subregion	WHO epidemiological subregion	character	“AFR_D”, “AFR_E”, “AMR_A”, “AMR_B”, “AMR_D”,“EMR_B”, “EMR_D”, “EUR_A”, “EUR_B”, EUR_C”, “SEAR_B”, “SEAR_ D”, “WPR_A”, “WPR_B”

WHO_ISH_Risk() function uses the base package in R and requires no package dependencies. Once the function is loaded in the workspace, it requires access to the comma delimited file named “WHO_ISH_Scores.csv” (Dataset 1). The user needs to ensure that this file is accessible in the working directory of R before running the WHO_ISH_Risk() function. We have included a worked example of how to use the function in Dataset 2.

Internally, the WHO_ISH_Risk() function creates a data frame of the risk factor values passed to it (Figure 1). It then categorises the continuous parameters age, systolic blood pressure, and total cholesterol. Age and systolic blood pressure were categorised according to WHO guidance⁷. Total cholesterol was categorised according to common clinical practice, rounding up from 0.5 to the nearest integer. The categorisation boundaries can be adapted by the user as needed.

A unique identification code is generated corresponding to the combinations of risk factors for each individual. This code is matched to a reference code from the “WHO_ISH_Scores.csv” file, which the function automatically calls into the workspace (Dataset 1). Internally, the function stores the risk scores in a data frame that includes the risk factors, and ultimately returns a vector containing the risk scores.

Validation

Comparison with the published MatLab implementation of the SEAR D risk charts⁶ showed 100% agreement with our R implementation, for all possible combinations of risk factors.

Discussion

To our knowledge, this is the first publically available R implementation of WHO/ISH CVD risk charts for all WHO epidemiological subregions of the world. Our implementation may be used for analysis of cardiovascular risk when electronic patient data is available. The code will automatically apply WHO/ISH risk scores to patients based on age, gender, systolic blood pressure, smoking status, diabetes status, total cholesterol, and epidemiological subregion. This code could be used, for example, during a pilot implementation of WHO PEN to audit the accuracy of risk assessment by comparing documented risk scores to actual risk scores calculated using this tool. We have provided a complete worked example in the data files. While more sophisticated implementations are possible, we intentionally sought to use simple syntax in the base package to allow for easy interpretation and use by novice R users on standard computers.

Although we modelled the function based on WHO PEN guidance for risk assessment, we recognise that some users may wish to change the boundaries of certain risk factor parameters. While WHO PEN guidance specifies the range of systolic blood pressure values for each systolic blood pressure category, it provides no such guidance for categorising total cholesterol. Based on our opinion and clinical experience, and on a previously published implementation in MatLab⁶, we chose to categorise total cholesterol by rounding up at 0.5 to the next integer. These boundaries could be changed by users to adapt to local practice. We caution changing the boundaries beyond recommended guidance.

The “WHO_ISH_Scores.csv” file can be adapted by the user if desired. Each row of the file represents one unique combination of risk factors. The first six columns specify the risk factor values, and the last 14 columns specify the corresponding risk category for a given subregion. These risk categories can be changed by the user, but in their current state they represent the WHO/ISH risk charts as published.

Conclusion

We created a simple R implementation of WHO/ISH CVD risk charts for all WHO epidemiological subregions of the world, requiring only the base R package. It has one dependency file from which it draws the WHO/ISH risk scores based on a combination of seven parameters: age, gender, systolic blood pressure, smoking status, diabetes status, total cholesterol, and epidemiological subregion. This tool can be used, and adapted, by policy-makers and researchers involved in the implementation and evaluation of WHO/ISH CVD risk charts.

Data and software availability

F1000Research: Dataset 1. CSV file required for function (file name= “WHO_ISH_Scores.csv”), 10.5256/f1000research.9742.d138309⁸

F1000Research: Dataset 2. Code for the WHO_ISH_Risk() function and worked example (file name = “Worked_Example.rtf”), 10.5256/f1000research.9742.d138310⁹

Author contributions

DC conceived of the idea, extracted data, wrote the initial code, and manuscript. JL and NB extracted data and contributed to writing the manuscript. CK reviewed and adapted the code and contributed to writing the manuscript. AW and CH reviewed and contributed to writing the manuscript.

Competing interests

DC has received payment from the WHO for consulting work. AW and CH have received expenses and grant income from the WHO for projects related to CVD and Self Care in NCDs, and direct a WHO Collaborating Centre. CH also receives funding form the National Institute of Health Research (NIHR) School of Primary Care Research. JL, CK, and NB declare no competing interests.

Grant information

The WHO Collaborating Centre for Self Care paid for the open access publishing fees. No other funding was provided for this work.

Faculty Opinions recommended

References

1. Mathers CD, Loncar D: Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006; 3(11): e442. PubMed Abstract | Publisher Full Text | Free Full Text
2. WHO: Noncommunicable Diseases, Fact Sheet. 2015. Reference Source
3. WHO: Package of essential noncommunicable (PEN) disease interventions for primary health care in low-resource settings. Geneva, Switzerland; 2010. Reference Source
4. Mendis S, Lindholm LH, Mancia G, et al.: World Health Organization (WHO) and International Society of Hypertension (ISH) risk prediction charts: assessment of cardiovascular risk for prevention and control of cardiovascular disease in low and middle-income countries. J Hypertens. 2007; 25(8): 1578–82. PubMed Abstract | Publisher Full Text
5. Cooney MT, Dudina A, D'Agostino R, et al.: Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation. 2010; 122(3): 300–10. PubMed Abstract | Publisher Full Text
6. Raghu A, Praveen D, Peiris D, et al.: Implications of Cardiovascular Disease Risk Assessment Using the WHO/ISH Risk Prediction Charts in Rural India. PLoS One. 2015; 10(8): e0133618. PubMed Abstract | Publisher Full Text | Free Full Text
7. WHO/ISH: World Health organization/International Society of Hypertension (WHO/ISH) risk prediction charts. 2007. Reference Source
8. Collins D, Lee J, Bobrovitz N, et al.: Dataset 1 in: Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world. F1000Research. 2016. Data Source
9. Collins D, Lee J, Bobrovitz N, et al.: Dataset 2 in: Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world. F1000Research. 2016. Data Source

Comments on this article Comments (2)

Version 2

VERSION 2 PUBLISHED 08 Mar 2017

Revised

Comment

Version 1

VERSION 1 PUBLISHED 14 Oct 2016

Discussion is closed on this version, please comment on the latest version above.

Author Response 20 Oct 2016

Dylan Collins, University of British Columbia, Canada

20 Oct 2016

Author Response

Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and ... Continue reading Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and worked example.

It can be found on my github repository here. Should you have any difficulty downloading from github or any further questions please let me know.

Every Best,

Dylan
Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and worked example.

It can be found on my github repository here. Should you have any difficulty downloading from github or any further questions please let me know.

Every Best,

Dylan
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reader Comment 18 Oct 2016

Rajarshi Guha, National Institutes of Health, USA

18 Oct 2016

Reader Comment

Please make the R code available in plain text format.
Competing Interests: No competing interests were disclosed.
Please make the R code available in plain text format.
Please make the R code available in plain text format.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Discussion is closed on this version, please comment on the latest version above.

Author details Author details

¹ Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Competing interests

Grant information

The WHO Collaborating Centre for Self Care paid for the open access publishing fees. No other funding was provided for this work.

Article Versions (2)

version 2

Revised

Published: 08 Mar 2017, 5:2522

https://doi.org/10.12688/f1000research.9742.2

version 1

Published: 14 Oct 2016, 5:2522

https://doi.org/10.12688/f1000research.9742.1

© 2016 Collins D et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Data associated with the article are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Download

Export To

metrics

	Views	Downloads
F1000Research	-	-
PubMed Central Data from PMC are received and updated monthly.	-	-

Citations

SEE MORE DETAILS

CITE

how to cite this article

Collins D, Lee J, Bobrovitz N et al. Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world [version 1; peer review: 3 approved with reservations]. F1000Research 2016, 5:2522 (https://doi.org/10.12688/f1000research.9742.1)

NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.

track

receive updates on this article

Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?

Key to Reviewer Statuses VIEW HIDE

ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions

Version 1

VERSION 1

PUBLISHED 14 Oct 2016

Views

Reviewer Report 28 Oct 2016

Maria Suarez-Diez, Laboratory of Systems and Synthetic Biology, Wageningen University and Research (WUR), Wageningen, The Netherlands

Approved with Reservations

https://doi.org/10.5256/f1000research.10503.r17004

The authors present an R script and an additional datafile to calculate cardiovascular risk using the Who/ISH risk assessment charts. Previous implementations required the use of MatLab and I believe an R implementation can be an useful addition to the field.

The authors intended to design a code requiring little R expertise. I think the authors have only partly succeeded in accomplishing this goal and the code still needs to be improved.

As mentioned by the other reviewers, RTF is far from an optimal format to distribute the code. The authors should present their code as a package hosted in some repository. Currently the RTF file contains the function definition, data loading and working example in a single view. When creating the package the authors should clearly separate the code from the documentation and provide a working example. If the authors intent this tool to be used and possibly modified by users with little R expertise then a separate file with detailed instruction should be provided.

The authors state that users can modify the WHO_ISH_Scores.csv file to represent other risk categories. That would require explanation of the content of the file. Although the used abbreviations gdr for gender might seem obvious, they should be explained. Also when downloading the file from F1000, the file name is changed and appears as “5ae9107XXX..._WHO_ISH_Scores.csv” so that the reading fails. This could be fixed by including the file as a dataset in the corresponding package.

When data from multiple patients are provided (as in the example) the output is a factor. I think it would help the users match input and output values is some tabular format or some patient identifier is provided in the output.

If any of the input parameters are missing (NA values) the code outputs “NA” but no information regarding the missing values is provided. It might be helpful if the code were to report which value was actually missing.

Fig. 1 shows the code and contains mainly information on how some continuous variables (age, cholesterol and systolic blood pressure are discretized). I think a more efficient way of conveying this information is by including it in Table 1.

Competing Interests: No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

CITE

Report a concern

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:
- We created an R package including all documentation files which
... Continue reading
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:

We created an R package including all documentation files which can be downloaded from github

We no longer suggest users to change the “WHO_ISH_Scores.csv” files and we have updated the file name hosted by F1000 to be “WHO_ISH_Scores.csv”. This file, as suggested, is internal to the package and while we report it here is not necessary to download.

We have added a series of warning messages to help explain outputs of “NA” and to help catch errors in the input values (e.g. out of range parameters)

We have described how continuous variables are discretized in the main text

Every Best,

Dylan Collins
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:

We created an R package including all documentation files which can be downloaded from github

We no longer suggest users to change the “WHO_ISH_Scores.csv” files and we have updated the file name hosted by F1000 to be “WHO_ISH_Scores.csv”. This file, as suggested, is internal to the package and while we report it here is not necessary to download.

We have added a series of warning messages to help explain outputs of “NA” and to help catch errors in the input values (e.g. out of range parameters)

We have described how continuous variables are discretized in the main text

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:
- We created an R package including all documentation files which
... Continue reading
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:

We created an R package including all documentation files which can be downloaded from github

We no longer suggest users to change the “WHO_ISH_Scores.csv” files and we have updated the file name hosted by F1000 to be “WHO_ISH_Scores.csv”. This file, as suggested, is internal to the package and while we report it here is not necessary to download.

We have added a series of warning messages to help explain outputs of “NA” and to help catch errors in the input values (e.g. out of range parameters)

We have described how continuous variables are discretized in the main text

Every Best,

Dylan Collins
Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:

We created an R package including all documentation files which can be downloaded from github

We no longer suggest users to change the “WHO_ISH_Scores.csv” files and we have updated the file name hosted by F1000 to be “WHO_ISH_Scores.csv”. This file, as suggested, is internal to the package and while we report it here is not necessary to download.

We have added a series of warning messages to help explain outputs of “NA” and to help catch errors in the input values (e.g. out of range parameters)

We have described how continuous variables are discretized in the main text

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 25 Oct 2016

Scott A. Chamberlain, rOpenSci project, University of California, Berkeley, Berkeley, CA, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.10503.r17006

The authors describe some R code for helping to calculate cardiovascular risk scores. I think the code needs significant work.

The code is in an .rtf file. This is very bad software practice. The authors

The authors describe some R code for helping to calculate cardiovascular risk scores. I think the code needs significant work.

The code is in an .rtf file. This is very bad software practice. The authors should at the very very least put the code in a file with .R extension.
Ideally the code should be put in an R package. It is relatively straight-forward to make an R package these days. See http://r-pkgs.had.co.nz/ for help. This makes it easy to add documentation (of which there is currently essentially none for the current function included in the manuscript), tests, etc. Looks like the authors have a Github repository (https://github.com/DylanRJCollins/WHO_ISH_R_Implementation) - this could be made into an R package.
Another benefit of making a package, is that R packages have a way of including datasets in them. The csv file is small enough that you can include it in the package if that's appropriate for the use case here. If the `WHO_ISH_Scores.csv` dataset is not likely to change, or not likely to change very often, they can include it in the package.
The dataset `WHO_ISH_Scores.csv` has two columns that are probably row names that should be removed. If they aren't data, remove them.
The dataset `WHO_ISH_Scores.csv`: the data is not in a tidy format that is easy to work with. There shouldn't be numeric data combined with symbols (e.g. >=40%). If possible, I urge authors to find a way to make these into numeric columns while retaining the same information. However, it may be that it's too difficult to separate numeric values from the greater than / less than symbols etc.
R packages should be cited by reference. e.g. "using the irr package (version 0.84)". It is good they cite the version, but put a reference in in your references. Run `citation("irr")` in an R session to get a reference for it.
"We used RStudio (version 0.99.489) to compare the duplicate extractions ...": Rstudio is just an IDE. Say that you used R, not RStudio. It's fine to cite RStudio, but also cite R. Run `citation()` in an R session to get the citation for R.
Octave is an open source language, and thus the authors should share the Octave code.
"[...] we intentionally sought to use simple syntax in the base package" - the authors script is in fact not a package, but if they do make a package this wording is good.
In the Conclusion: "We created a simple R implementation of WHO/ISH CVD risk charts for all WHO epidemiological subregions of the world, requiring only the base R package." - refer to this as "base R", not "the base R package".
If the dataset "WHO_ISH_Scores.csv" can be modified, thus different versions of it may be used by the user, it makes more sense to pass in the dataset as a parameter, instead of hard coding reading in the data in the function.
I urge authors to include a license with their R package (to be created) - and to submit the package to CRAN so it's easy for all to install.

Code comments:

I'm not sure the authors meant to do this, but the use of single ampersand in the `ifelse` statements (e.g., `ifelse(df$age > 17 & df$age < 50, 40, df$age)`) means that they get a vector of logicals (e..g., `df$age > 17 & df$age < 50` evaluates to `TRUE FALSE FALSE FALSE FALSE`) using their example, when I think what they want is a single logical return value. If so, use double ampersand instead: `&&` instead of `&`.
I'm pretty sure, but have not tested thoroughly, that the long series of if statements in the section "Match the look up value with the reference value" can be replaced with just this: `ref[[subregion]][match(df$luv, ref$refv)]`.

Competing Interests: No competing interests were disclosed.

CITE

Report a concern

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.
- We have created an R
... Continue reading
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.

We have created an R package which can be downloaded from github

We provide the code from the R package as a .R file

We have included the “WHO_ISH_Scores.csv” data file in the R package

We tidied the ”WHO_ISH_Scores.csv” files but have kept the risk scores as character strings to retain a literal implementation of the risk charts, as described in the updated manuscript

We have cited R packages and R as suggested

The MatLab code used in Octave has previously been published as cited in the manuscript and is freely available

We no longer suggest users to change the “WHO_ISH_Scores.csv” files

We have included a GPL-3 license for the R Package

Every Best,

Dylan Collins
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.

We have created an R package which can be downloaded from github

We provide the code from the R package as a .R file

We have included the “WHO_ISH_Scores.csv” data file in the R package

We tidied the ”WHO_ISH_Scores.csv” files but have kept the risk scores as character strings to retain a literal implementation of the risk charts, as described in the updated manuscript

We have cited R packages and R as suggested

The MatLab code used in Octave has previously been published as cited in the manuscript and is freely available

We no longer suggest users to change the “WHO_ISH_Scores.csv” files

We have included a GPL-3 license for the R Package

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.
- We have created an R
... Continue reading
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.

We have created an R package which can be downloaded from github

We provide the code from the R package as a .R file

We have included the “WHO_ISH_Scores.csv” data file in the R package

We tidied the ”WHO_ISH_Scores.csv” files but have kept the risk scores as character strings to retain a literal implementation of the risk charts, as described in the updated manuscript

We have cited R packages and R as suggested

The MatLab code used in Octave has previously been published as cited in the manuscript and is freely available

We no longer suggest users to change the “WHO_ISH_Scores.csv” files

We have included a GPL-3 license for the R Package

Every Best,

Dylan Collins
Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.

We have created an R package which can be downloaded from github

We provide the code from the R package as a .R file

We have included the “WHO_ISH_Scores.csv” data file in the R package

We tidied the ”WHO_ISH_Scores.csv” files but have kept the risk scores as character strings to retain a literal implementation of the risk charts, as described in the updated manuscript

We have cited R packages and R as suggested

The MatLab code used in Octave has previously been published as cited in the manuscript and is freely available

We no longer suggest users to change the “WHO_ISH_Scores.csv” files

We have included a GPL-3 license for the R Package

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern

Views

Reviewer Report 20 Oct 2016

Raivo Kolde, Philips Research North America, Cambridge, MA, USA

Approved with Reservations

https://doi.org/10.5256/f1000research.10503.r17007

The paper describes a software tool for calculation of WHO/ISH cardiovascular risk scores for different epidemiological subregions of the world. This could be a useful piece of software for researchers working with cardiovascular epidemiological data and make the practices for calculating such scores more reproducible over multiple studies. However, in the present form I found the paper to be severely lacking in both implementation and substance.

First, the code should not be distributed as an RTF file. R packages are well accepted standard for reproducibly distributing R code that also forces to provide some rudimentary documentation and examples. As an additional benefit, hosting the package in a repository like CRAN, Bioconductor or even Github, makes it easy to install and update the program. There is no reason to not follow this convention for this particular project, thus, the code should be converted into package format and hosted in one of the above-mentioned services to be any use to the community.

The argument of the code in the present form being simply adjustable is not valid. One has to spend quite a bit of time understanding what is happening in the code, especially when not an R expert. For the code to fulfill its purpose it has to be rewritten in a way that adjustable parts are presented to the user as function parameters (with reasonable defaults). This way users have to know even less R to use the code effectively.

Also, having a figure depicting the code is uncommon to say the least and not useful in any way. It would make much more sense to include a worked out example to the text, where you would describe a common use case of your R package.

To enhance usefulness of the package and make it more universally applicable it would be good to see some more risk prediction models like Framingham and SCORE added to the software. This would make it easy to compare the validity of different scores over a study population.

Competing Interests: No competing interests were disclosed.

CITE

Report a concern

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:
- As suggested, we created an R package which
... Continue reading
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:

As suggested, we created an R package which can be downloaded directly from github

We no longer advocate for the code to be adjustable and through the development of a package have made it simpler to use

We removed the figure of the code, and have added a worked example in the text

The main difference between scores like Framingham, SCORE, and QRISK2 is that they have underlying cox model equations which can be implemented in R, whereas WHO/ISH risk charts do not – hence the rationale for the development of this package. While the additional of further risk scores was outside the scope of this work, we welcome collaborators who might want to contribute in this respect.

Every Best,

Dylan Collins
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:

As suggested, we created an R package which can be downloaded directly from github

We no longer advocate for the code to be adjustable and through the development of a package have made it simpler to use

We removed the figure of the code, and have added a worked example in the text

The main difference between scores like Framingham, SCORE, and QRISK2 is that they have underlying cox model equations which can be implemented in R, whereas WHO/ISH risk charts do not – hence the rationale for the development of this package. While the additional of further risk scores was outside the scope of this work, we welcome collaborators who might want to contribute in this respect.

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern
Respond or Comment

COMMENTS ON THIS REPORT

Author Response 08 Mar 2017

Dylan Collins, University of British Columbia, Canada

08 Mar 2017

Author Response
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:
- As suggested, we created an R package which
... Continue reading
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:

As suggested, we created an R package which can be downloaded directly from github

We no longer advocate for the code to be adjustable and through the development of a package have made it simpler to use

We removed the figure of the code, and have added a worked example in the text

The main difference between scores like Framingham, SCORE, and QRISK2 is that they have underlying cox model equations which can be implemented in R, whereas WHO/ISH risk charts do not – hence the rationale for the development of this package. While the additional of further risk scores was outside the scope of this work, we welcome collaborators who might want to contribute in this respect.

Every Best,

Dylan Collins
Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:

As suggested, we created an R package which can be downloaded directly from github

We no longer advocate for the code to be adjustable and through the development of a package have made it simpler to use

We removed the figure of the code, and have added a worked example in the text

The main difference between scores like Framingham, SCORE, and QRISK2 is that they have underlying cox model equations which can be implemented in R, whereas WHO/ISH risk charts do not – hence the rationale for the development of this package. While the additional of further risk scores was outside the scope of this work, we welcome collaborators who might want to contribute in this respect.

Every Best,

Dylan Collins
Competing Interests: No competing interests were disclosed. Close
Report a concern

Comments on this article Comments (2)

Version 2

VERSION 2 PUBLISHED 08 Mar 2017

Revised

Comment

Version 1

VERSION 1 PUBLISHED 14 Oct 2016

Discussion is closed on this version, please comment on the latest version above.

Author Response 20 Oct 2016

Dylan Collins, University of British Columbia, Canada

20 Oct 2016

Author Response

Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and ... Continue reading Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and worked example.

It can be found on my github repository here. Should you have any difficulty downloading from github or any further questions please let me know.

Every Best,

Dylan
Dear Rajarshi Guha,

Thank you for reading our work. I've created a plain text format version of Dataset 2 which is the dataset that contains the R code and worked example.

It can be found on my github repository here. Should you have any difficulty downloading from github or any further questions please let me know.

Every Best,

Dylan
Competing Interests: No competing interests were disclosed. Close
Report a concern
Reader Comment 18 Oct 2016

Rajarshi Guha, National Institutes of Health, USA

18 Oct 2016

Reader Comment

Please make the R code available in plain text format.
Competing Interests: No competing interests were disclosed.
Please make the R code available in plain text format.
Please make the R code available in plain text format.
Competing Interests: No competing interests were disclosed. Close
Report a concern
Discussion is closed on this version, please comment on the latest version above.

Open Peer Review

Reviewer Status

Reviewer Reports

	Invited Reviewers
	1	2	3
Version 2 (revision) 08 Mar 17	read	read	read
Version 1 14 Oct 16	read	read	read

Raivo Kolde, Philips Research North America, Cambridge, USA
Scott A. Chamberlain, University of California, Berkeley, Berkeley, USA
Maria Suarez-Diez, Wageningen University and Research (WUR), Wageningen, The Netherlands

Comments on this article

All Comments(2)

Add a comment

Browse by related subjects

Back to all reports

Reviewer Report

23 Views

14 Mar 2017 | for Version 2

Scott A. Chamberlain, rOpenSci project, University of California, Berkeley, Berkeley, CA, USA

23 Views Cite this report Responses(0)

Approved

I'm mostly satisfied with the author's changes, with a few minor changes.

Thank you for making an R package. I downloaded, and checked the package, with a few observations:

In the package use utils::read.csv instead of just read.csv - you probably saw the warning if you've run R CMD CHECK or devtools::check()
The package has no examples. I'd strongly suggest including examples.
The package has no tests. I'd strongly suggest including tests (though if time is limited, at least include examples).
The authors note that the package now gives warnings when input values are out of range - I'd strongly suggest to document what the correct ranges are for each parameter so users don't have to guess
In addition, if there are sensible default values I'd use those - if applicable - may not be applicable.
Urge authors to document what type of value is expected for each parameter, e.g., age (numeric), subregion (character)

Sincerely,
Scott Chamberlain

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

19 Views

10 Mar 2017 | for Version 2

Maria Suarez-Diez, Laboratory of Systems and Synthetic Biology, Wageningen University and Research (WUR), Wageningen, The Netherlands

19 Views Cite this report Responses(0)

Approved

My main concerns have been addressed in this new version of the manuscript.

I think the developed package represents a useful addition to the field. I was able to install and run the package and indeed the package reports the absence of data points (NA).
Only a minor concern: within R, getting the citation 'citation("whoishRisk")' returns the names of the name of the package, the names of the authors and the R version. I recommend they include a reference to this work.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

23 Views

09 Mar 2017 | for Version 2

Raivo Kolde, Philips Research North America, Cambridge, MA, USA

23 Views Cite this report Responses(0)

Approved

All my main concerns were satisfied with the update. I was able to install and run the package. I guess from visibility standpoint it would be nice to include it to the CRAN database, but otherwise I approve.

Competing Interests

No competing interests were disclosed.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Respond to this report

Responses (0)

Back to all reports

Reviewer Report

37 Views

28 Oct 2016 | for Version 1

Maria Suarez-Diez, Laboratory of Systems and Synthetic Biology, Wageningen University and Research (WUR), Wageningen, The Netherlands

37 Views Cite this report Responses(1)

Approved With Reservations

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

08 Mar 2017

Dylan Collins, University of British Columbia, Canada

Dear Maria Suarez-Diez,

Thank you for your valuable comments. In response we have made the following changes:

We created an R package including all documentation files which can be downloaded from github
We no longer suggest users to change the “WHO_ISH_Scores.csv” files and we have updated the file name hosted by F1000 to be “WHO_ISH_Scores.csv”. This file, as suggested, is internal to the package and while we report it here is not necessary to download.
We have added a series of warning messages to help explain outputs of “NA” and to help catch errors in the input values (e.g. out of range parameters)
We have described how continuous variables are discretized in the main text

Every Best,

Dylan Collins

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

34 Views

25 Oct 2016 | for Version 1

Scott A. Chamberlain, rOpenSci project, University of California, Berkeley, Berkeley, CA, USA

34 Views Cite this report Responses(1)

Approved With Reservations

The authors describe some R code for helping to calculate cardiovascular risk scores. I think the code needs significant work.

The code is in an .rtf file. This is very bad software practice. The authors should at the very very least put the code in a file with .R extension.
Ideally the code should be put in an R package. It is relatively straight-forward to make an R package these days. See http://r-pkgs.had.co.nz/ for help. This makes it easy to add documentation (of which there is currently essentially none for the current function included in the manuscript), tests, etc. Looks like the authors have a Github repository (https://github.com/DylanRJCollins/WHO_ISH_R_Implementation) - this could be made into an R package.
Another benefit of making a package, is that R packages have a way of including datasets in them. The csv file is small enough that you can include it in the package if that's appropriate for the use case here. If the `WHO_ISH_Scores.csv` dataset is not likely to change, or not likely to change very often, they can include it in the package.
The dataset `WHO_ISH_Scores.csv` has two columns that are probably row names that should be removed. If they aren't data, remove them.
The dataset `WHO_ISH_Scores.csv`: the data is not in a tidy format that is easy to work with. There shouldn't be numeric data combined with symbols (e.g. >=40%). If possible, I urge authors to find a way to make these into numeric columns while retaining the same information. However, it may be that it's too difficult to separate numeric values from the greater than / less than symbols etc.
R packages should be cited by reference. e.g. "using the irr package (version 0.84)". It is good they cite the version, but put a reference in in your references. Run `citation("irr")` in an R session to get a reference for it.
"We used RStudio (version 0.99.489) to compare the duplicate extractions ...": Rstudio is just an IDE. Say that you used R, not RStudio. It's fine to cite RStudio, but also cite R. Run `citation()` in an R session to get the citation for R.
Octave is an open source language, and thus the authors should share the Octave code.
"[...] we intentionally sought to use simple syntax in the base package" - the authors script is in fact not a package, but if they do make a package this wording is good.
In the Conclusion: "We created a simple R implementation of WHO/ISH CVD risk charts for all WHO epidemiological subregions of the world, requiring only the base R package." - refer to this as "base R", not "the base R package".
If the dataset "WHO_ISH_Scores.csv" can be modified, thus different versions of it may be used by the user, it makes more sense to pass in the dataset as a parameter, instead of hard coding reading in the data in the function.
I urge authors to include a license with their R package (to be created) - and to submit the package to CRAN so it's easy for all to install.

Code comments:

I'm not sure the authors meant to do this, but the use of single ampersand in the `ifelse` statements (e.g., `ifelse(df$age > 17 & df$age < 50, 40, df$age)`) means that they get a vector of logicals (e..g., `df$age > 17 & df$age < 50` evaluates to `TRUE FALSE FALSE FALSE FALSE`) using their example, when I think what they want is a single logical return value. If so, use double ampersand instead: `&&` instead of `&`.
I'm pretty sure, but have not tested thoroughly, that the long series of if statements in the section "Match the look up value with the reference value" can be replaced with just this: `ref[[subregion]][match(df$luv, ref$refv)]`.

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

08 Mar 2017

Dylan Collins, University of British Columbia, Canada

Dear Scott A. Chamberlain,

Thank you for your thoughtful and considered feedback. We have responded in the updated version as detailed below.

We have created an R package which can be downloaded from github
We provide the code from the R package as a .R file
We have included the “WHO_ISH_Scores.csv” data file in the R package
We tidied the ”WHO_ISH_Scores.csv” files but have kept the risk scores as character strings to retain a literal implementation of the risk charts, as described in the updated manuscript
We have cited R packages and R as suggested
The MatLab code used in Octave has previously been published as cited in the manuscript and is freely available
We no longer suggest users to change the “WHO_ISH_Scores.csv” files
We have included a GPL-3 license for the R Package

Every Best,

Dylan Collins

View more View less

Competing Interests

No competing interests were disclosed.

Back to all reports

Reviewer Report

41 Views

20 Oct 2016 | for Version 1

Raivo Kolde, Philips Research North America, Cambridge, MA, USA

41 Views Cite this report Responses(1)

Approved With Reservations

Competing Interests

No competing interests were disclosed.

Respond to this report

Responses (1)

Author Response

08 Mar 2017

Dylan Collins, University of British Columbia, Canada

Dear Raivo Kolde,

Thank you for your insightful review and comments. In response we have made the following changes:

As suggested, we created an R package which can be downloaded directly from github
We no longer advocate for the code to be adjustable and through the development of a package have made it simpler to use
We removed the figure of the code, and have added a worked example in the text
The main difference between scores like Framingham, SCORE, and QRISK2 is that they have underlying cox model equations which can be implemented in R, whereas WHO/ISH risk charts do not – hence the rationale for the development of this package. While the additional of further risk scores was outside the scope of this work, we welcome collaborators who might want to contribute in this respect.

Every Best,

Dylan Collins

View more View less

Competing Interests

No competing interests were disclosed.

Alongside their report, reviewers assign a status to the article:

Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested

Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.

Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions

Click here to access the data.

Downloaded data do not display as expected? Download the data (131.81KB)

Click here to access the data.

Downloaded data do not display as expected? Download the data

[1] 1. Mathers CD, Loncar D: Projections of global mortality and burden of disease from 2002 to 2030. PLoS Med. 2006; 3(11): e442. PubMed Abstract | Publisher Full Text | Free Full Text

[2] 2. WHO: Noncommunicable Diseases, Fact Sheet. 2015. Reference Source

[3] 3. WHO: Package of essential noncommunicable (PEN) disease interventions for primary health care in low-resource settings. Geneva, Switzerland; 2010. Reference Source

[4] 4. Mendis S, Lindholm LH, Mancia G, et al.: World Health Organization (WHO) and International Society of Hypertension (ISH) risk prediction charts: assessment of cardiovascular risk for prevention and control of cardiovascular disease in low and middle-income countries. J Hypertens. 2007; 25(8): 1578–82. PubMed Abstract | Publisher Full Text

[5] 5. Cooney MT, Dudina A, D'Agostino R, et al.: Cardiovascular risk-estimation systems in primary prevention: do they differ? Do they make a difference? Can we see the future? Circulation. 2010; 122(3): 300–10. PubMed Abstract | Publisher Full Text

[6] 6. Raghu A, Praveen D, Peiris D, et al.: Implications of Cardiovascular Disease Risk Assessment Using the WHO/ISH Risk Prediction Charts in Rural India. PLoS One. 2015; 10(8): e0133618. PubMed Abstract | Publisher Full Text | Free Full Text

[7] 7. WHO/ISH: World Health organization/International Society of Hypertension (WHO/ISH) risk prediction charts. 2007. Reference Source

[8] 8. Collins D, Lee J, Bobrovitz N, et al.: Dataset 1 in: Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world. F1000Research. 2016. Data Source

[9] 9. Collins D, Lee J, Bobrovitz N, et al.: Dataset 2 in: Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world. F1000Research. 2016. Data Source

Simple and adaptable R implementation of WHO/ISH cardiovascular risk charts for all epidemiological subregions of the world

Abstract

Keywords

Introduction

Methods

Extraction of WHO/ISH cardiovascular risk charts

Development of the WHO/ISH risk function

Validation

Results

Extraction of WHO/ISH cardiovascular risk charts

Development of the WHO/ISH risk function

Figure 1. The WHO_ISH_Risk() function code in R.

Table 1. Description of the WHO_ISH_Risk() function parameters.

Validation

Discussion

Conclusion

Data and software availability

Author contributions

Competing interests

Grant information

References

Comments on this article Comments (2)

Open Peer Review

Comments on this article Comments (2)

Open Peer Review

Reviewer Status

Reviewer Reports

Comments on this article

Browse by related subjects

The problem

How to fix it

The problem

How to fix it

Competing Interests Policy

Stay Updated