ALL Metrics
-
Views
-
Downloads
Get PDF
Get XML
Cite
Export
Track
Research Article
Revised

Analysis of internet educational websites on tobacco cessation: A content analysis

[version 2; peer review: 2 approved]
PUBLISHED 22 Nov 2024
Author details Author details
OPEN PEER REVIEW
REVIEWER STATUS

This article is included in the Manipal Academy of Higher Education gateway.

Abstract

Background

There have been significant changes in the lifestyles of individuals in the past few decades, which has led to increased morbidity and mortality worldwide. Both smoking and chewing forms of tobacco are highly prevalent, especially in India, and are implicated as causes of diseases, including oropharyngeal carcinomas. Effective tobacco cessation techniques and sources can help overcome addiction and reduce the disease burden in society. The aim was to evaluate the quality and readability of contents of various sources on an internet website about tobacco cessation.

Objectives

i) To evaluate the readability of internet content (Google) regarding tobacco cessation using Flesch–Kincaid readability tests and the quality of internet content (Google) by using the JAMA benchmark, HONcode and DISCERN questionnaire.

Methods

A content review was employed to screen the content of the Google search engine for educational tobacco cessation websites, and the top 50 websites were selected according to criteria and reviewed by two reviewers. The readability of the internet content (Google) regarding tobacco cessation was evaluated using Flesch–Kincaid readability tests. The quality of the screened sites was evaluated by using the JAMA (Journal of the American Medical Association) benchmark, HONcode (The Health on the Net Code of Conduct) and DISCERN (Discerning the Quality of Information for Choosing Treatments) questionnaire, and the readability and quality of the screened websites were correlated using the above instruments.

Results

FK readability ease was found to be 49% standard and 30% easy. The FK grade test found that 33% of the content could be easily understood by < 5th grade. All 4 JAMA benchmarks were met by 23% of websites, and authorship was the least fulfilled criterion. Correlation analysis revealed a significant association between FK ease score and FK grade score.

Conclusions

The Read-ability Ease and Read-ability Grade Levels of the websites related to tobacco cessation were not standard, and few websites fulfilled the JAMA benchmarks and had HONcode certification.

Keywords

Oral cancer, Oral health, Online resources, readability tests, Smokeless tobacco, Tobacco cessation

Revised Amendments from Version 1

1. The extended data for all tools added as tables
2. The tools of research explained in detail
3. The corrections are made according to the suggestions of the reviewers

To read any peer review reports and author responses for this article, follow the "read" links in the Open Peer Review table.

Introduction

There have been significant changes in the lifestyles of individuals in the past few decades, which has led to increased morbidity and mortality worldwide.1 Habits such as tobacco use, alcohol, diet and other lifestyle factors have been implicated as cause of diseases such as cancers of the body, including oropharyngeal cancer.24 The disease burden is higher in underdeveloped and developing countries than in their economically stable counterparts. The hallmark of these diseases is their preventability. Among adolescents and adults, the burden seen is staggering. According to Global Youth Tobacco Survey (GYTS) and Global Adult Tobacco Survey (GATS), tobacco usage is seen to be 14.6% and 28.6% respectively. With effective motivation and implementation of programs aimed at tobacco control, the burden of such deadly diseases can significantly be decreased. Effective tobacco cessation techniques and internet web contents like websites, applications, and e-educational content can help overcome addiction and reduce the disease burden in society.5,6

Recent decades have also witnessed a substantial rise in the number of individuals using mobile and smartphones and, with it, the internet. Everyone with a smartphone now has easy access to information on the internet, which can have a significant impact on an individual’s lifestyle.7 Regardless of the background of individuals, they can now browse the internet for information related to various health-related issues and conditions.8,9

Various web resources like websites, applications, e-educational contents on tobacco etc., can provide information about tobacco cessation to general population.7 This raises concerns about the quality of information on tobacco cessation and the contents of the same information on the internet, which may or may not be optimal, credible and standardized. Very few investigations reported in the literature have explored the content, quality and readability (comprehensibility) of internet resources on tobacco cessation. They can provide valuable information paving the way for standardizing this tobacco cessation content on various websites specifically tailored to reach common men.

Hence, the present study aimed to evaluate the quality and readability of the contents of various resources on internet websites about tobacco cessation. Objectives: i) To evaluate the readability of internet content (Google) regarding tobacco cessation using Flesch–Kincaid readability tests. ii) Evaluate the quality of the internet content (Google) regarding tobacco cessation by using the JAMA (Journal of the American Medical Association) benchmark, HONcode (The Health on the Net Code of Conduct) and DISCERN (Discerning the Quality of Information for Choosing Treatments) questionnaire.

Methods

The present study was performed using a review of the internet content on tobacco cessation. The study design employed was Cross Sectional for Health Technology Assessment. Google, a common search site, was deployed to search for content on tobacco cessation. The search term used was “How to quit Tobacco?”. After discussion with two Institutional Tobacco Cessation Cell subject experts, the search term was arrived at. Fifty websites that appeared first on the specific day were reviewed for their content, readability & quality. Two experts ABR and KC scored the parameters to do away with the subjectivity bias. In case of any discrepancy in the score between two experts, final decision will be based on the analysis of expert R. Reliability was checked using kappa statistics, whose value was 1 indicating total agreement. Inclusion criteria were websites that are patient focused and websites that do not involve any cost or subscription. Websites that were password-protected and contained only videos and journals were excluded.

Selected websites after screening were evaluated for their quality, content and readability. Flesch–Kincaid readability tests10,11 evaluated the readability of the internet source. Flesch–Kincaid readability tests include Flesch readability ease and Flesch–Kincaid grade level. The JAMA benchmark,12,13 HONcode,13,14 and DISCERN questionnaire were used to assess website quality.15

Ethics

Ethical approval was obtained before the study process from Institutional Ethics Committee. (Protocol ref no. 19051).

Data management and statistical analysis

The parameters were entered in excel sheet and Statistical Package for Social Sciences (SPSS), version 16.0 (SPSS Inc, Chicago IL).16 P value <0.05 was significant. Mean was used for descriptive data. Percentage was used to interpret FK readability tests. Mean and median was taken for DISCERN-16 questionnaire. Student’s t test was employed to evaluate HONcode Scores with DISCERN tool and FK readability scores. Pearson’s correlation was done to assess the relationship between DISCERN, FK scores and HONcode values.

Results

Forty-three websites were reviewed. Seven websites were not included because they did not fulfil the inclusion criteria; hence A total of 21 (49%) websites were of standard Readability with an Ease Score of the Flesch–Kincaid readability test. Overall, 13 (30%) websites had a fairly easy readability ease score. Readability Grade Level analysis of the Flesch–Kincaid readability test indicates that a majority of 14 (33%) websites could be read by individuals lower than 5th grade, while 13 (30%) websites could be read by individuals belonging to 6th grade. A total of 5 (12%) websites could be read by individuals belonging to 8th grade ( Table 1).

Table 1. Website content assessed with Flesch–Kincaid readability tests.

Flesch–Kincaid readability testsWebsites
n %
Readability Ease ScoreVery confusing37
Fairly difficult49
Difficult25
Standard2149
Fairly easy1330
Easy00
Readability Grade Levelcollege level graduate0 0
12th grade1 2
11th grade0 0
10th grade2 5
9th grade1 2
8th grade5 12
7th grade4 9
6th grade13 30
5th grade3 7
<5th grade14 33

The findings of the current study indicate that a total of 10 (23%) of the websites satisfied all 4 JAMA benchmarks for quality and reliability of the content. Overall, the number of websites satisfying 1, 2 and 3 of the JAMA benchmarks was 6 (14%), 7 (16%) and 19 (44%), respectively. A total of 41 (95%) websites satisfied the JAMA benchmark of disclosure, while a total of 15 (35%) websites satisfied authorship criteria. It was also observed that a total of 8 (19%) websites did not have HONcode certification, while a total of 35 (81%) websites had HONcode certification.

The median and mean scores for questions in DISCERN-16. It can be observed that the mean score was maximum for item number 1 and was minimum of 2.91 for item numbers 5, 6 and 8. Item number 5,6 and 8 speaks on clarity, bias and areas of uncertainty. A majority of 21 (49%) of the websites as assessed by DISCERN-16 belonged to the fair category, while none of the websites belonged to the very poor and excellent categories.

The results of the correlation analysis indicate that there were significant associations between HonCODE certification and DISCERN total scores (p=0.021), DISCERN S1 (p=0.033) and DISCERN S2 (p=0.021). It can also be observed that there were statistically significant associations between DISCERN-S1 and DISCERN-S2, DISCERN-S1 and DISCERN-S3, DISCERN-S1 and DISCERN-16 total, DISCERN-S2 and DISCERN-S3, DISCERN-S2 and DISCERN-16 total, DISCERN-S3 and DISCERN-16 total. There were also statistically significant associations between HonCODE certification and DISCERN S1, DISCERN-S2 and DISCERN-16 total scores. Additionally, significant associations were found between FK ease and FK grade scores ( Table 2).

Table 2. Correlation analysis between various evaluation scores.

DISCERN S1DISCERN S2DISCERN S3DISCERN TotalFK EaseFK GradeHonCODE
r-value p-value r-value p-value r-value p-value r-value p-value r-value p-value r-value p-value r-value p-value
DISCERN S1--
DISCERN S20.846**0.000--
DISCERN S30.809**0.0000.789**0.000--
DISCERN Total0.964**0.0000.955**0.0000.857**0.000--
FK Ease-0.1580.312-0.1290.408-0.1300.407-0.1500.336--
FK Grade0.1780.2530.0910.5630.0770.6230.1380.376-0.935**0.000--
HonCODE0.326*0.0330.351*0.0210.2750.0750.350*0.0210.0060.9670.0500.748--

* Significant at 5 % level of significance.

** Significant at 1% level of significance.

Discussion

There have been significant advancements in the field of information technology and mobile phone services. These developments are very dynamic and are witnessing tremendous growth every passing day.9

The present study employed F–K readability, consisting of Flesch ease and grade level in place of a similar index such as the SMOG index. One of the issues with the use of the SMOG (Simple Measure of Gobbledygook) index has been its tendency to forecast readability 2 grades higher than the other estimates. The SMOG index depends upon the strict criteria of 100% accurate answers by participants. The current study's findings indicate that the Ease Score was standard and fairly easy for most of the content included in the study. The current study contrasts with those reported by Wiriyakijja et al.,17 for websites on oral leukoplakia and Diniz-Freitas et al.,18 for tobacco cessation net-contents.

Material in the current study was primarily readable by individuals lower than 5th grades and in agreement with the study conducted by Joury et al.19 The National Institutes of Health recommended readability age is between 7th and 8th grades. However, reduced reading grades lead to dilution of contents.

The results of the present study indicate that 23% of the websites fulfilled all the JAMA (Journal of the American Medical Association) benchmarks. In a similar study conducted by Wiriyakijja et al., it was noted that 17% of websites on oral leucoplakia fulfilled all four JAMA benchmarks,17 and Ni Riordain et al., observed that 45% of websites on head and neck cancer fulfilled all four JAMA benchmarks.20 Diniz-Freitas et al.,18 who observed that none of the websites related to tobacco cessation services by dentists fulfilled all 4 JAMA benchmarks. A majority of the JAMA benchmarks fulfilled in the present study include disclosure (95%) and attribution (79%). The findings are in concordance with those reported by Wiriyakijja et al.,17 for websites on oral leucoplakia but in disagreement with those observed by Diniz-Freitas et al.,18 for websites on tobacco cessation services by dentists. Authorship and currency were the frequently fulfilled JAMA benchmark in websites on adult orthodontics, as reported by McMorrow et al.21

The Health on the Net Code of Conduct (HONcode) is a set of guidelines for medical and health websites that aims to improve the quality of health information on the internet. Its usually seen at the end of the internet content. It was observed in study that 81% of the sites lacked HONcode certification, which is in agreement with those observed by Wiriyakijja et al., Diniz-Freitas et al.,18 and Joury et al.19 and in disagreement with the reports of McMorrow et al.21 It has been reported by Ni Riordain et al., that 39% of websites had HON code seals.20 HON code assesses ethics and reputation rather than quality and information.

The number of subjects browsing internet sources for health information has considerably increased in the recent decade.22 This finding shed light on the importance of the ease and readability of available content.17 Reliability of information in the field of oral cancer in different languages is also not standardized.23

Reading readily available internet content pertaining to tobacco increases anxiety due to the realization of unmet health needs.24 Increased levels of anxiety may also be related to poor health outcomes, and low compliance with treatment regimens and instructions. Therefore, there is a definite need for accurate information related to tobacco cessation on the internet.

Varela-Centelles et al., observed that the overall quality for oral cancer in health care professionals (HCP) addressed websites were of good standards.25 There needs to be a consensus on what constitutes good information for patients who are seeking information on the internet on tobacco cessation.

On the other hand, it can be observed that there is information overload on the internet, and there can be information that is conflicting in nature.26 The subjects need to be sensitized about the issues related to information overload, such as dubious information sources and the availability of too much information for one’s perusal. Patients may specifically be interested in the treatment outcomes associated with tobacco cessation services in particular. This may shape the patients’ expectations to be more realistic in nature, and the patients may be more receptive of various treatment outcomes.27

Various factors might influence the diverse nature of information available on the internet. Various websites are fabricated by a wide variety of individuals hailing from diverse philosophical backgrounds. The purpose of website preparation may also be very different. Moreover, health literacy levels, and access to the internet may vary among patients.

Limitations of the present study will have to be borne in mind. The cross-sectional design of the study and the inclusion of a limited number of websites are its limitations. Internet content is very dynamic and is constantly changing by day. Further studies that consider a greater number of websites need to be conducted. The assessment of the content of the websites was performed by using a checklist prepared specifically for the present study. All the issues related to tobacco cessation may not be encompassed by this checklist. The websites excluded in the present study might be viewed by the patients. The present study also excluded scientific articles and books, which might be good sources of accurate information. The websites were evaluated by a dentist who was qualified and specifically trained for the purpose of the study. This type of dentist might have greater knowledge than those who check the websites. The trained dentist may also be more critical in analysing the contents of the websites than the common person.

Conclusions

It can be observed that the read-ability ease and read-ability grade levels of the websites related to tobacco cessation were not easily read and understandable by the common man. Very few websites fulfilled all the JAMA benchmarks and had HONcode certification. There is a definite need for stricter rules and regulations for websites providing information related to tobacco cessation. There is a need for policy change to reinforce stricter regulation.

Ethics and consent

Ethical clearance was obtained from the Institutional Ethics Committee of Manipal College of Dental Sciences, Mangalore (Ref No. 19051).

Informed consent: Ethical committee permission was taken and was exempted (Ref No. 19051).

Comments on this article Comments (0)

Version 2
VERSION 2 PUBLISHED 23 Jul 2024
Comment
Author details Author details
Competing interests
Grant information
Copyright
Download
 
Export To
metrics
Views Downloads
F1000Research - -
PubMed Central
Data from PMC are received and updated monthly.
- -
Citations
CITE
how to cite this article
Avinash BR, Chengappa K, Shenoy R et al. Analysis of internet educational websites on tobacco cessation: A content analysis [version 2; peer review: 2 approved]. F1000Research 2024, 13:822 (https://doi.org/10.12688/f1000research.146843.2)
NOTE: If applicable, it is important to ensure the information in square brackets after the title is included in all citations of this article.
track
receive updates on this article
Track an article to receive email alerts on any updates to this article.

Open Peer Review

Current Reviewer Status: ?
Key to Reviewer Statuses VIEW
ApprovedThe paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approvedFundamental flaws in the paper seriously undermine the findings and conclusions
Version 2
VERSION 2
PUBLISHED 22 Nov 2024
Revised
Views
4
Cite
Reviewer Report 29 Nov 2024
Venkitachalam Ramanarayanan, Department of Public Health Dentistry, Amrita School of Dentistry, Amrita Vishwa Vidyapeetham, Coimbatore, Tamil Nadu, India 
Approved
VIEWS 4
No ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Ramanarayanan V. Reviewer Report For: Analysis of internet educational websites on tobacco cessation: A content analysis [version 2; peer review: 2 approved]. F1000Research 2024, 13:822 (https://doi.org/10.5256/f1000research.174934.r342717)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Version 1
VERSION 1
PUBLISHED 23 Jul 2024
Views
10
Cite
Reviewer Report 26 Aug 2024
Tarakant Bhagat, BP Koirala Institute of Health Sciences, Dharan, Nepal 
Santosh Kumari Agrawal, Public Health Dentistry, B. P Koirala Institute of Health Sciences, Dharan, Koshi, Nepal 
Approved
VIEWS 10
Abstract and Introduction
Whenever using an abbreviation, please mention it along with the full form for the first time; e.g: Flesch–Kincaid (FK), JAMA, HONcode, DISCERN etc.
Methods
i. Who all were involved in the screening process? Incase multiple ... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Bhagat T and Agrawal SK. Reviewer Report For: Analysis of internet educational websites on tobacco cessation: A content analysis [version 2; peer review: 2 approved]. F1000Research 2024, 13:822 (https://doi.org/10.5256/f1000research.160966.r312908)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.
Views
21
Cite
Reviewer Report 16 Aug 2024
Venkitachalam Ramanarayanan, Department of Public Health Dentistry, Amrita School of Dentistry, Amrita Vishwa Vidyapeetham, Coimbatore, Tamil Nadu, India 
Approved with Reservations
VIEWS 21
Introduction
Effective tobacco cessation techniques and sources: Kindly use a different terminology for sources. 

Some information on the prevalence or burden of tobacco use especially among adolescents or adults could add more context to the introduction. 
... Continue reading
CITE
CITE
HOW TO CITE THIS REPORT
Ramanarayanan V. Reviewer Report For: Analysis of internet educational websites on tobacco cessation: A content analysis [version 2; peer review: 2 approved]. F1000Research 2024, 13:822 (https://doi.org/10.5256/f1000research.160966.r306334)
NOTE: it is important to ensure the information in square brackets after the title is included in all citations of this article.

Comments on this article Comments (0)

Version 2
VERSION 2 PUBLISHED 23 Jul 2024
Comment
Alongside their report, reviewers assign a status to the article:
Approved - the paper is scientifically sound in its current form and only minor, if any, improvements are suggested
Approved with reservations - A number of small changes, sometimes more significant revisions are required to address specific details and improve the papers academic merit.
Not approved - fundamental flaws in the paper seriously undermine the findings and conclusions
Sign In
If you've forgotten your password, please enter your email address below and we'll send you instructions on how to reset your password.

The email address should be the one you originally registered with F1000.

Email address not valid, please try again

You registered with F1000 via Google, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Google account password, please click here.

You registered with F1000 via Facebook, so we cannot reset your password.

To sign in, please click here.

If you still need help with your Facebook account password, please click here.

Code not correct, please try again
Email us for further assistance.
Server error, please try again.