[version 1; peer review: 1 not approved]
No competing interests were disclosed.
Cardiovascular disease has been the leading cause of death for decades. Risk prediction models are used to identify high risk patients; the most common model used in the UK is ClinRisk’s QRISK3. In this paper we describe the implementation of the QRISK3 algorithm into an R package. The package was successfully validated by the open sourced QRISK3 algorithm and QRISK3 SAS program. We provide detailed examples of the use of the package, including assigning QRISK3 scores for a large cohort of patients. This R package could help the research community to better understand risk prediction scores and improve future risk prediction models. The package is available from CRAN:
Cardiovascular disease (CVD) was responsible for 17.9 million deaths in 2016, which represents 31% of overall global deaths, and over 75% of these deaths happened in low/middle-income countries
The implementation of QRISK3 into R would not only benefit researchers to improve future risk prediction but also enable them to use QRISK3 scores to identify patients at certain risk levels, e.g. for clinical trial recruitment. There is also scope to improve these risk predictions; it has been found that QRISK3 has uncertainty on individual risk prediction
The original QRISK3 algorithm was written in C by ClinRisk under a GNU Lesser General Public License
The QRISK3 algorithm was written in both R (3.4.2) and SAS (9.4)
The QRISK3 package can be directly installed from CRAN
Variables used by the QRISK3 package were summarised and compared to the original algorithm in
| Parameters in QRISK3
|
Meaning of variables | Variables in original
|
|---|---|---|
| age | Specify the age of the patient in year (e.g. 64 years-old) | age |
| atrial_fibrillation | Atrial fibrillation? (0: No, 1: Yes) | b_AF |
| atypical_antipsy | On atypical antipsychotic medication? (0: No, 1: Yes) | b_atypicalantipsy |
| regular_steroid_tablets | On regular steroid tablets? (0: No, 1: Yes) | b_corticosteroids |
| erectile_disfunction | A diagnosis of or treatment for erectile disfunction?
|
b_impotence2
|
| migraine | Do patients have migraines? (0: No, 1: Yes) | b_migraine |
| rheumatoid_arthritis | Rheumatoid arthritis? (0: No, 1: Yes) | b_ra |
| chronic_kidney_disease | Chronic kidney disease (stage 3, 4 or 5)? (0: No, 1: Yes) | b_renal |
| severe_mental_illness | Severe mental illness? (0: No, 1: Yes) | b_semi |
| systemic_lupus_erythematosis | Systemic lupus erythematosis (SLE)? (0: No, 1: Yes) | b_sle |
| blood_pressure_treatment | On blood pressure treatment? (0: No, 1: Yes) | b_treatedhyp |
| diabetes1 | Diabetes status: type 1? (0: No, 1: Yes) | b_type1 |
| diabetes2 | Diabetes status: type 2? (0: No, 1: Yes) | b_type2 |
| Weight (kg) | Weight | Not available |
| Height (cm) | Height | Not available |
| Weight (m) / (Height (cm) /100) 2 | Body mass index (BMI) | bmi |
| ethnicity | 1 White or not stated
|
ethrisk:
|
| heart_attack_relative | Angina or heart attack in a 1st degree relative < 60?
|
fh_cvd |
| cholesterol_HDL_ratio | Cholesterol/HDL ratio? (range from 1 to 11, e.g. 4) | rati |
| systolic_blood_pressure | Systolic blood pressure (mmHg, e.g. 180 mmHg) | sbp |
| std_systolic_blood_pressure | Standard deviation of at least two most recent systolic
|
sbps5 |
| smoke | 1 non-smoker
|
smoke_cat:
|
| townsend | Townsend deprivation scores | town |
The two datasets QRISK3_2017_test and QRISK3_2019_test were used for validation. Risk scores calculated from this QRISK3 package, the original algorithm and the SAS version on the same group of patients was exactly the same. The external validation of this QRISK3 package in a big CPRD cohorts with 3.6 million patients shows a good and similar discrimination (C statistic: 0.85) and calibration to a previous study
A patient cohort with anonymous patient identifiers and CVD risk factors should first be extracted and coded similarly to QRISK3 by the user. Missing values in the dataset should be handled (e.g. multiple imputation) before using this package. Column names of CVD risk factors (e.g. “age”) should then be specified correctly to the QRISK3_2017 function. The function returns calculated risk scores through a dataset with three columns, including patient identifier, calculated QRISK3 score and calculated QRISK3 score with one digit. It also reminds users to double check whether the definition of their variables was the same as the definition of QRISK3. The package also automatically detects whether all variables were coded as numeric and whether age of patients was ranged between 28 and 84, if not an error message returns (explained in
| Error message | Conditions | Explanation |
|---|---|---|
| “Variables including XXX, XXX must
|
When at least one of variables in
|
QRISK3 algorithm needs numeric
|
| “Age of patients must be between
|
When at least one patient in the dataset
|
QRISK3 algorithm was developed
|
| “Variables including XXX, XXX has
|
When at least one of variables in
|
Missing values must be handled
|
Users first need to create a statistical analysis dataset similar to the provided test dataset (e.g. QRISK3_2019_test) which contains information of patients’ identifier and QRISK3 risk factors and mimics QRISK3’s training cohort
This R package successfully implements the QRISK3 algorithm into R, which allows researchers to calculate CVD risk of patients in the next 10 years. The R package was validated by the original algorithm and a SAS version. This is also the first R implementation of the QRISK3 algorithm at the date of writing.
Though QRISK3 was already published and released from the online website, it is time consuming for researchers to calculate QRISK3 risk score, as the online calculator cannot be used as a service to obtain QRISK3 scores for a large cohort, and the original algorithm is written in C rather than a well-established data science language such as R. This package bridges this gap. It allows researchers to obtain QRISK3 scores for large cohorts, which could help to improve model accuracy of QRISK3 and help with any more applied tasks that require knowing CVD risk at a patient level.
Although it is easy to use this R function to calculate a risk score, researchers should carefully check whether their variables are coded the same as the original QRISK3 cohort, otherwise the calculated score might not be the correct risk of the patient in the cohort. For example, a patient who is a smoker is coded as “1” in the variable “smoking” would be in conflict with the definition of the QRISK3 algorithm (“smoking” equals 1 in this R package means non-smoker). Since QRISK is updated annually every spring, researchers who are interested in the latest work should refer to their website
In conclusion, we developed this R package to allow researchers to obtain QRISK3 scores for large cohorts. It allows the research community to better understand and apply a currently used risk prediction model for CVD risk.
Original QRISK3 algorithm:
Package available from CRAN:
Source code available from:
Archived source code as at time of publication:
License: GPL-3
C source code, SAS version and QRISK3_2017_test and QRISK3_2019_test datasets used for validation available from:
Archived C code, SAS version and test datasets as at time of publication:
License: GPL-3
The authors provided an R package for the QRISK3 algorithm to predict the risk of cardiovascular disease. My main comment is related to the size of this package, which has only one simple function with two simple datasets. I am not sure that this tiny package can be presented as an F1000Research article. There are not any new findings represented in this article compared to the author's previous works. I suggest enriching this package with more visualization and data manipulation functions and datasets. Besides, I do not think it needs to highlight validation of R performance with SAS or C. This is part of your quality control, not an outcome. Also, I have some minor comments:
Why didn't you use the R 3.6.2 instead of 3.4.2? Why you did not make the categorical variables as ordered factor R object instead of a simple numeric object? (in this case, you do not need to think about the conflict of the meaning of "1" and "0"). The range of patient ages is not consistent between text and table. "Height", "Weight" and "Weight/Height" in table 1 should be lowercase the same as the package.
Are the conclusions about the tool and its performance adequately supported by the findings presented in the article?
Yes
Is the rationale for developing the new software tool clearly explained?
Yes
Is the description of the software tool technically sound?
Yes
Are sufficient details of the code, methods and analysis (if applicable) provided to allow replication of the software development and its use by others?
Yes
Is sufficient information provided to allow interpretation of the expected output datasets and any results generated using the tool?
Yes
Reviewer Expertise:
Computational biologist
We confirm that we have read this submission and believe that we have an appropriate level of expertise to state that we do not consider it to be of an acceptable scientific standard, for reasons outlined above.
We appreciate reviewers’ time and effort in review this package paper, we would like to make following reply to these comments.
We believe the novelty of this package and this paper is:
QRISK3 is different from other model such as Framingham model (one other popular American CVD risk prediction model). For Framingham model, the model formula was simple and clearly presented in its original paper, and user could easily implement it into R. While for QRISK3, this is different and difficult. The main challenge implementing QRISK into R is the original algorithm written in low level language C and there is no document to explain the meaning of these variables in original algorithm. Users also need to validate their implemented version is the same as the original version of QRISK3, this requires users to have programming knowledge of both R and C while R is the dominated language in data science. Therefore, researchers don’t have table 1 of this paper if they start to implement QRISK3 from the original algorithm. This creates challenges that it is hard to acquire a similar statistic model formula as presented in Framingham, and it is hard to identify which variable represents what risk factors especially considering they were written in C. For example, how to know variable such as “b_impotence2”, “ethrisk” and “fh_cvd” represents “erectile disfunction”, “ethnicity” and “relative of CVD”. QRISK was used to predict 10 years risk, why there are 11 values in a vector which represents baseline risk from the original algorithm, and which variable represents this baseline risk? There is a long distance between input value of predictors to output a correct predicted risk. This package solved all the challenges mentioned above and bridges the gaps.
This R package acts like a bridge which enables researchers to easily access and use QRISK3 model in R like using Framingham model.
Without this R package and this paper described how this R package was developed and validated, researchers may spend months to re-implement QRISK into R and they also need to spend a lot of effort on validating whether their implemented version of QRISK3 does the similar thing as the original algorithm. Overall, our package and this paper solved these, and saved time and energy for researchers who only require calculated QRISK3 score as part of their new research.
It would be easier for user to quickly obtain QRISK3 score. Rather than searching a list of redundant functions, user could instantly test and use this package. There are mature R packages which support reviewer suggested functions. Due to the nature of this R package (i.e. only write with base R language without dependent packages), it is well compatible to all the other R packages. For visualization, users could easily use R package like “ggplot2” with this R package. For data manipulation, R package like “dplyr” is well compatible to our package. The main function of this R package is to provide QRISK3 calculated risk score. What visualization and datasets user might require were purely dependent on their research question. In the current version, we have provided a dataset with patient identifier and calculated QRISK score, which could be easily used by researchers to draw their own visualization.
Though there are only two datasets present in this package, it contains all the key information of what the datasets would look like and what variables were needed. They contain enough information for user to understand and use this package easily. Overall, simplistic is a part of design of this R package and there are well established R packages which could be used with this simple R package. We do not find any necessary reasons to add more functions in this R package, as the aim of this R package is to help users to obtain a QRISK3 score from large dataset quickly and current version fulfilled this purpose. We are happy to add more functions or tools if there are more specific requirements from users in future.
We also believe it is essential to show user how this R package was validated with the original algorithm, as this R package aims to provide a correct QRISK3 predicted score. Users who decide to use our package may wish to replicate our validation process.
1. I need SAS, or SPSS or Stata or R code for :Two cardiac risk scores: QRISK and SCORE . May you share? https://www.researchgate.net/post/I_need_SAS_or_SPSS_or_Stata_or_R_code_for_Two_cardiac_risk_scores_QRISK_and_SCORE_May_you_share. Accessed February 12, 2020.