Using Odih-Score (Modified Klotz Score) for quality of defecation after anorectal malformation following posterior sagittal anorectoplasty (PSARP): Study of validation

Introduction

Anorectal malformation is a congenital disorder in the form of an imperfect anus or without an anus.^1–3 The incidence of anorectal malformations is about 1 per 4000 population annually.^1–3 The prevalence of anorectal malformations in Riau, Indonesia is estimated at 34 cases annually.^4–6 The management of anorectal malformations depends on the classification and severity of the disorder.^7,8 The definitive surgical that is widely used up to this point is the method introduced by De Vries and Pena the Posterosagittal Anorectoplasy (PSARP) approach.^7,8 This surgical method is widely used because it is considered more dynamic and obtains better surgical results in bowel function and fecal continence. The success of the PSARP Technique can be measured by the Klotz Score.^4,9

The Klotz score is one of the parameters that can be used for the evaluation of bowel function or defecation. Klotz scores are classified into excellent, good, sufficient, and insufficient to assess the quality of postoperative defecation.⁹ Previous studies have shown that Klotz scores have good predictive values for the evaluation of postoperative defecation quality function.⁶ This assessment and classification of score results has several assessment items that are sometimes difficult to apply to communicate to patients, so researchers aim to modify the Klotz score to make it easier for researchers and other practicing doctors, especially in Indonesia.

Methods

Results

The study collected data on 95 post-PSARP anorectal malformation patients who met the inclusion criteria out of 282 patients listed in the medical records. The researchers analyzed the participants in a structured manner with five items, and the participants felt that their clinical condition represented well all the symptoms included in the Odih-Score questionnaire. The data was then analyzed for reproducibility, internal consistency, concurrent, construct, and clinical validity in 95 patients. The study reported that 52.63% of the participants were male (with a male-female ratio of 1.2:1), 64.51% had a high lesion type, and 66.31% underwent the full-PSARP technique (as shown in Table 2).

Table 3 presents the results of both the validation and reproducibility tests. The validation test is conducted using the Pearson test and measures the correlation between two sets of data. In our case, the correlation coefficient (r) is calculated, and the value ranges from 0.573 to 0.790. To determine if this is a valid correlation, we compare the calculated r value to the r-table value at a 5% significance level in a static r-table distribution (n=95). The table r value for this distribution is 0.202. If the r-count value is greater than the table r value, then we can consider the correlation coefficient as valid.^10–12 To ensure the validity of the study, we analyzed five items and found that their significance value was less than 0.05, indicating valid results. After that, we conducted a reliability test using the Cronbach alpha test. The results of the validation and reliability tests help us ensure the accuracy and credibility of our findings.¹³ Cronbach’s alpha coefficient of internal consistency for 5 items was 0.814 in 95 patients, indicating excellent reliability (Table 4).

The Odih-Score research data will be analyzed for sensitivity and specificity using Receiver Operating Characteristic (ROC) and Area Under the Curve (AUC) as a diagnostic tool.^14,15 The ROC curve analysis in this study (shown in Figure 1) resulted in an AUC Odih-Score value of 0.798 (95%CI 0.544–0.882) (as shown in Table 5). The optimal value of sensitivity and specificity cut off between the Odih-Score and post-Anorectal Malformation defecation quality with PSARP was found to be 10.89 (rounded to 11). This means that a total Odih-score of at least 11 indicates poor defecation quality in patients after Anorectal Malformation with PSARP (as depicted in Figure 2).

Figure 1. ROC curve.

Figure 2. Cut off point Odih-Score.

Discussion

Anorectal malformation is a congenital disorder where the anus is either absent or imperfect, and surgery is required for definitive management.^1–3 De Vries and Pena introduced the Posterosagittal Anorectoplasty (PSARP) technique which is widely used to improve bowel function and postoperative fecal continence for patients with anorectal malformations.^6–8 Defecation quality evaluation scoring is still commonly used today. The Klotz score is a widely used scale based on previous studies.^16,17 Odih-Score is a modified version of the Klotz score, which is known for its ease of use among clinicians, particularly at Arifin Achmad Hospital. The Odih-Score consists of 5 items, 4 of which are borrowed from the Klotz score, and 1 new item added. Each of these grades is scored using a modified 3-point Likert scale with answer point choices.¹⁸ The results of data analysis using the Pearson validation test showed that Odih-Score has a significance value of Sig.<0.05, with r-count value being greater than the r-table value.

Developing instruments for assessment requires the use of reliable methods, such as questionnaires designed with scoring or scaling systems. It is crucial to ensure that the chosen measurement model is consistent with the assessment methods used. Scoring tools are a vital element of instrument validity.¹⁹ Messick defines validity as a characteristic not of the test, but of the meaning, interpretation, and consequences of its scores.²⁰ Researchers developing measurement tools should consider different perspectives on measurement models and their impact on assessment systems to determine which approach is most appropriate for their research objectives.^21,22

When researchers use a questionnaire to collect data, it is important that the questionnaire is of high quality. A questionnaire is considered to be of high quality if it has been validated and proven to be reliable. The validation test is conducted to determine if the questionnaire is suitable for measuring and obtaining research data from respondents. To conduct the validation test, researchers use the Pearson correlation coefficient to determine the correlation between each item on the questionnaire and the total answers provided by the respondent.^10,11 The research instrument is said to be valid if the value of the r-count coefficient is greater than the value of the r-table or the significance value (Sig.) <0.05.

The reliability test aims to determine if a research instrument produces consistent results when the questionnaire is administered repeatedly.¹³ Before conducting reliability tests, it is important to confirm the validity of the research instrument. In this study, the reliability test analysis indicates that the Odih-Score research instrument is reliable, as the Cronbach alpha value for its five items is 0.814, which is above the accepted threshold of 0.6.¹³

The sensitivity and specificity of Odih-Score as a diagnostic test was tested, and it obtained an AUC value of 0.798. A good test ability is declared if AUC is greater than or equal to 0.7. This shows that Odih-Score has a high diagnostic parameter value strength when evaluating the quality of defecation. The optimal cut-off value for Odih-Score sensitivity and specificity is 11. These analysis results can be used as a diagnostic assessment parameter to determine the quality of post-Anorectal Malformation defecation with PSARP.

Conclusion

Odih-Score (Modified Klotz’s score) can be used as a diagnostic test to evaluate postoperative defecation quality in anorectal malformation patients treated with PSARP. The score is reliable and valid.