Assessing hard and loose “endpoints”: comparison of patient and expert Bristol Stool Scale scoring of 2280 fecal samples

Introduction

Stool consistency is a central component in the description of bowel habits and an important outcome measure to evaluate in the investigation of several gastrointestinal (GI) diseases.¹ Altered bowel habits are often seen as a consequence of diseases in the GI tract, and the water content in the stools affecting the consistency can reflect the intestinal transit time. Rapid intestinal transit time limits the absorption of water and cause loose or liquid stools (diarrhea), whereas slow intestinal transit causes extensive water absorption and harder stools (constipation).² The exact evaluation of stool consistency can be made by measurement of viscosity and stool water content, however this method requires cumbersome laboratory analyses and is not well suited for routine purposes. The form of the stool can serve as a proxy measure for the stool consistency, referring to the visually assessed shape and texture.³

The Bristol Stool Scale (BSS) is one of the most commonly used tools for evaluation of stool consistency.¹ The BSS was developed in the early 1990s as a surrogate marker for whole-gut transit time, but is now a commonly recommended tool in clinical and research settings for the evaluation of stool consistency, rather than transit time.^4,5 BSS is designed to standardize the reporting of stool consistency by a 7-point ranking system, ranging from hard lumps (Type 1 on the scale) to liquid stool (Type 7 on the scale). Types 1 and 2 are considered hard stools corresponding to constipation, whereas types 6 and 7 are considered abnormally loose and watery stools corresponding to diarrhea when evaluated together with other symptoms. Types 3, 4 and 5 are generally considered “normal” stool forms.

The generalizability of BSS as a tool in research is unclear, and fecal water content or dry weight have been suggested as better objective measures.⁵ Reporting can be inaccurate since subjects are asked to report their “average” stool form for the day, but bowel movements may vary in form throughout the day. Moreover, subjects’ interpretation of the BSS scale may vary, leading to inaccurate reporting of stool form. It is suggested that the BSS score is more precise when evaluated by a trained expert than by the patient.⁶ Here, we aimed to evaluate the reliability of the BSS in a cohort of patients with inflammatory bowel disease (IBD), by comparing the patients’ subjective scores with scores assigned by the experienced bioengineers who received the fecal samples.

Methods

Results

Patients versus expert BBS scores

The agreement between patient and expert scores for all assessed time points was good to substantial when assessed with Cohen’s weighted kappa, ranging from 0.66 to 0.72 ( Table 1). Overall, the patients were likely to score lower than the experts. When the scores from the first sample per patient were compared between the patient and the expert ratings, the experts were more likely to assign higher scores compared to the patient’s self-assessment. The proportion of the lowest assigned scores (score 1-2) was 12.1% for patients and 8.1% for the experts. In Table 2, the scores given by the patients and experts are presented for the first assessment time point. The numbers in green represent perfect agreement between the experts and the patients. The numbers in yellow above the diagonal represent the situation when the sample is given a higher score by an expert (over-estimation) and the red numbers below the diagonal represent an under-estimation (experts score lower compared to the patients). The distribution of BSS scores reported by the patient for each value of score given by the expert is presented graphically in Figure 1.

Table 1. BSS score agreement between patients and experts assessed with Cohen’s weighted kappa.

Assessment time point	Original BSS scores (1-7)		Simplified BSS scores (1-3)
	Weighted kappa	95%CI	Weighted kappa	95%CI
1 (n = 992)	0.72	0.69-0.75	0.78	0.74-0.81
2 (n = 513)	0.66	0.61-0.71	0.75	0.69-0.81
3 (n = 363)	0.71	0.66-0.76	0.73	0.66-0.80
4 (n = 290)	0.72	0.66-0.78	0.74	0.66-0.82
5 (n = 122)	0.74	0.66-0.82	0.74	0.63-0.85

Table 2. BSS score agreement between patients and experts at assessment point 1.

		Expert BSS scores
		1	2	3	4	5	6	7
Patient BSS scores	1	13	4	5	0	2	0	0
	2	3	49	18	11	13	2	0
	3	0	3	139	44	24	5	0
	4	0	1	7	275	57	11	0
	5	1	4	7	7	33	5	0
	6	0	2	1	6	25	163	0
	7	0	0	0	1	33	19	29

Figure 1. Distribution of BSS scored by patients for each value of score given by experts.

The figure shows that the patients were likely to score lower than the experts. BSS: Bristol Stool Scale.

BSS score simplified into three categories

When the BSS scores were simplified and all the samples categorized as 1 (scores 1 and 2), 2 (scores 3, 4 and 5) or 3 (scores 6 and 7), patient and expert agreement improved slightly, with Cohen’s weighted kappa ranging from 0.73 to 0.77, indicating substantial agreement for all the assessed time points ( Table 1). The distribution of all scores given by the patients and experts with the simplified scores and assessment point 1 are presented in Table 3. A perfect agreement between the experts and the patients is highlighted in green, the portion of the table highlighted in yellow represents over-estimation by the experts and the red number represents an under-estimation by the experts. The distribution of the BSS scored by the patient for each value of score given by the expert at the first assessment point when the BSS score was simplified into three categories is depicted in Figure 2.

Table 3. BSS score agreement between patients and experts at assessment point 1 using a simplified score.

		Expert BSS scores
		1	2	3
Patient BSS scores	1	69	49	2
	2	9	593	21
	3	2	36	211

Figure 2. BSS scores categorized as 1, 2 and 3.

Score 1 equals original scores 1 and 2 (constipation), scores 2 equals original scores 3, 4 and 5 (normal), and score 3 equals original scores 6 and 7(diarrhea). The figure shows the distribution of the BSS scored by the patient for each score value given by the expert at the first assessment point when the BSS score was simplified into the three categories. BSS: Bristol Stool Scale.

Discussion

BSS is commonly used in both clinical and research settings for the evaluation of stool consistency, however its reliability is unclear as the scores rely on subjective patient reports. Here, we show that there is good to substantial agreement between patient BSS scores and expert BSS scores when evaluating individual scores from 1-7. However, this agreement is even better when scores are compared in simplified categories corresponding to constipation (BSS score 1-2), normal stool (BSS score 3-5) and diarrhea (BSS score 6-7). Our findings highlight that the patient´s exact BSS score is in good agreement with the expert score, and that the patient score is accurate for evaluation of main stool type (constipation, normal or diarrhea). As the main stool type is commonly the one of most interest for the classification of disease, and more subtle distinctions might not be of great clinical importance, we found BSS to be a reliable tool for stool form evaluation.

Several studies have previously reported on the validity and use of BSS. Our findings contradict those of a previous study investigating the rate reproducibility of BSS by comparing BSS scores of thirty-four gastroenterology providers of 35 different stool photographs.¹⁰ Here, they reported high reliability and agreement when BSS scores were used to assess the individual stool type from 1-7. When the experts should categorized the stool type into the three categories constipation, normal and diarrhea (corresponding to the Rome III standard⁸), the reliability and agreement decreased.¹⁰ Of note, in contrast to these prior results, our simplified into three groups was performed based on the original BSS scores and not independently assessed, hence this might explain the different findings. Also, our study compared subjective patient scores to expert scores, which is relevant as the BSS scoring is most often performed by the patients themselves.

A recent study from 2022 comparing subjective IBS patients´ BSS scores with stool water content as an objective measure reported only modest conformity between methods, highlighting that this can affect the classification of IBS-subtype according to BSS score.¹¹ Similarly, a study from 2016 validated the BSS by measuring stool form in 169 healthy adults and comparing it to stool water content and BSS score form 19 patients with diarrhea-predominant IBS.⁶ They reported the BSS to demonstrate adequate validity and reliability, however they detected difficulties around clinical decision points for types 2, 3, 5 and 6.⁶ Although the authors report the BSS to be a valid tool for stool evaluation, it has to be taken into consideration that the uncertainty around scorings affects the classification of stool type into constipation, normal or diarrhea, potentially affecting the clinical outcomes and diagnosis of IBS-subtype. It should be emphasized that self-reported assessments may differ between IBS patients and IBD patients, since psychological factors may have a greater influence of IBS symptoms than IBD symptoms.

According to current literature highly variable results are thus presented for the validity of the BSS when comparing subjective measures with expert measures or fecal water content. BSS is widely used both in clinical and research settings, and is currently used in an increasing number of studies analyzing microbiota to adjust for differences in fecal consistency.⁵ The unclear generalizability of BSS represents a challenge to interpreting such results. Interestingly, a recent study comparing a smartphone application using artificial intelligence (AI) to BSS scores performed by two experts reported high accuracy between the AI and the experts. In addition, they reported the trained AI to be superior to subject self-reported BSS scores, emphasising that AI assessments could provide more objective outcome measures for stool characterization in gastroenterology.¹² This should be taken into consideration in further studies, as it highlights a tool that can potentially improve the lack of reliability related to the subjective measures used in BSS today.

Our study has some limitations. As the follow-up period for the IBSEN III cohort is still not completed, we did not include stool samples from all subjects at all time points, and some patients were included with several samples, whereas others were only included with one sample. Due to the covid-19 pandemic, the 1-year follow-up was delayed for some of the subjects, hence the proportion of included study subjects with more than three stool samples is small. As the BSS is a subjective measure, the uneven distribution of the number of samples per participant might be a weakness affecting the results. When performing a stool sample, only a small part of the whole stool delivery is sampled for analysis. It is likely that parts of the discharge may have different consistencies and appearance, hence the part sampled for analysis might not be representative of the whole stool. This might be of particular importance when evaluating patients with GI disorders such as IBD and IBS, as the stool might be a mix ranging from hard lumps at first followed by looser stools. It would have been interesting to know the participants stool frequency, since the frequency may affect the consistency. Our study also only included two experts. It must be considered that a larger number of experts, or a mean of several expert scores, could have strengthened the results. Similarly, including an assessment of interobserver and intraobserver variabilities, both in the patient group and among the experts, would have strenghtened our study substantially.

Conclusion

Taken together, our findings show that the agreement between patient and expert BSS scores is good, especially when divided into three main stool categories. We found the BSS to be a reliable tool for the categorization of stool type when the BSS scores are categorized into three categories corresponding to the clinically relevant concepts of constipation, normal stool or diarrhea. Indeed, our finding that expert scoring of only a small fecal sample is in good agreement with the patient scoring of the entire stool delivery is a novel observation that should be validated in future studies where photographs of the entire stool delivery should also be included.

Author contributions

HFD, JV, GHM and JTF planned and designed the study. MLH and VK included patients in the IBSEN III trial. GHM and JTF analysed the fecal samples and performed the expert BSS scoring. MCS performed the statistical analyses. HFD wrote the manuscript. JV, MCS, GHM and JTF contributed to data interpretation and critical revision of the manuscript. All authors reviewed and approved the final manuscript.