Development and Validation of a Needs Assessment Guide for mHealth-Based Self-Management among Patients with Hypertension

Ethics approval

This study was conducted in accordance with the principles of the Declaration of Helsinki. Ethical approval was obtained from the Institutional Ethics Committee of MAHE, Manipal on 8^th January 2022 (IEC:587/2021) and the study was registered prospectively under the clinical trial Registry of India on 31^st March 2022 (CTRI/2022/03/041544), prior to commencement of the study. The study commenced on 2^nd April 2022. Written informed consent was obtained from all expert prior to participation. Participation was voluntary, and experts were informed of their right to withdraw at any time without consequence.

Study design

The guide was developed through literature review, item generation, face validity assessment, expert content validation, and iterative refinement to understand the needs and expectations of hypertensive patients on self-management through mobile based applications. The process was conducted in four sequential steps.

Step 1: Literature search

Literature review on needs and expectations of hypertensive patients in using mHealth applications for self-management of their disease was done using the standard search engines. The Focus Group Discussion Guide was developed by synthesizing the information gathered through a comprehensive literature review, with the aim of aligning the content with the objectives and purpose of the study.^23–25

Step 2: FGD guide development

The FGD guide was separated into five components which had a welcome content and an introduction to the research objectives The questions were of three types -probe, follow up and exit. The probe had the questions which will help in making them feel them more comfortable and they will be able to share their opinion along with the group. Follow up, which will delve into the main discussion topics and will involve the opinion of the participants. And Exit, which includes the questions to ensure that the moderator did not miss anything in gathering the insights of the participants. Finally, the researcher’s conclusion aided in closing the discussion loop.

Step 3: Face validity

Face validity was assessed prior to the formal content validation process to ensure the clarity and appropriateness of items before expert rating. Five experts – Clinicians experienced in hypertension management, qualitative researchers, and digital health experts reviewed the guides for language, flow and contextual suitability. Feedback on structure, phrasing was received to align with the study’s aims. Minor revisions were made to enhance the clarity and to ensure relevance.

Step 4: Content validation and expert assessment

The content validation of the developed Focus Group Discussion (FGD) guide was conducted in two sequential phases: expert assessment and target population validation.

Expert assessment

A total of ten subject-matter experts were involved in evaluating the initial draft of the FGD guide. These experts included physicians from the Departments of General Medicine and Cardiology, as well as health information management professionals. Experts were selected based on their professional experience and familiarity with hypertension care and qualitative research. Each expert received a participant information sheet, an informed consent form, and an evaluation package containing the FGD guide and an expert evaluation form.

Experts were instructed to review each item in the guide and rate its appropriateness, clarity, comprehensibility, and essentiality. A three-point rating scale was used: 1 = Essential, 2 = Useful but not essential, and 3 = Not necessary. They were also invited to provide written comments and suggestions for improvement. Based on their feedback, redundant or unclear items were revised or removed, and the overall structure of the guide was refined for better alignment with the study objectives. To ensure linguistic and cultural accuracy, the guide was translated into Kannada (the local language) and back-translated into English for verification.

Step 5: Pilot Test: A pilot FGD was conducted using the same guidelines, setting, and participant criteria planned for the main study. This served to both train the moderator and test the appropriateness, clarity, and flow of the interview questions in relation to the study objectives.

A pilot session was held with five hypertension individuals who were selected purposively. The moderator followed the draft FGD guide, while a note-taker documented observations. The discussion lasted approximately 60 minutes and was audio-recorded.

Following the discussion, the recording was transcribed and preliminarily analysed using qualitative data analysis software to identify emerging themes and assess question effectiveness. The debrief between moderator and note-taker highlighted areas for improvement, particularly in the sequencing of questions and the need for additional probing prompts to elicit deeper responses on self-management barriers, their needs and requirement for mHealth application development.

Step 6: Refinement and finalization: Feedback from both healthcare professionals and hypertensive patients resulted in improvements to wording, clarity, and contextual relevance, thereby enhancing the guide’s comprehensibility and cultural alignment. The resulting draft included 19 items, carefully reviewed and finalized in consultation with the experts.

Statistical analysis

Content validity:

In content validity evaluations, quantitative analysis is utilized to ascertain how well things match or reflect a specific domain. This study employed empirical methods to calculate the content validity index and content validity ratio. The domain experts’ opinions were evaluated using the CVR, Kappa statistic, and CVI.

Assessment of content validity:

CV-Index:

The Content Validity Index (CVI) was used to assess the guide’s relevance. Item-level CVI (I-CVI) was calculated as the proportion of experts rating an item as essential. Items with I-CVI >0.79 were retained, those scoring 0.70–0.79 were revised, and those below 0.70 were removed. Scale-level CVI was evaluated using both S-CVI/Ave (average of all I-CVIs) and S-CVI/UA (proportion of items achieving unanimous expert agreement).^18,19

CV-Ratio:

To assess item essentiality, the Content Validity Ratio (CVR) was calculated. Experts rated each item as “essential,” “useful,” or “not necessary.” The CVR score, ranging from −1 to 1, was computed as: CVR = (Ne - N/2) / (N/2), where *Ne* is the number of experts rating the item as essential and “N” is the total number of experts. Higher scores indicate stronger expert consensus on the item’s necessity.^18,19

Kappa statistic coefficient:

To assess content validity beyond the Content Validity Index (CVI) and mitigate chance agreement, the Kappa coefficient was calculated for each item. For computing kappa statistics, probability of random agreement is determined first, or Pc = [N! /A! (N – A)!] × 0.5 N; N = number of panel experts, and A = the number of panel members agreeing that the item is important in this formula. Next, the kappa statistic was computed using the formula K = (I-CVI – Pc) / (1 – Pc). Kappa values were interpreted as excellent (>0.74), acceptable (0.60–0.74), or fair (0.40–0.59), ensuring that expert consensus was significantly above random agreement.²⁰

Face validity

Face validity was assessed by a panel of five experts to ensure that the developed guides were appropriate, comprehensible, and aligned with the study’s scope and objectives. The panel comprised clinicians, qualitative researchers, and digital health professionals who reviewed each item for clarity, relevance, and contextual appropriateness. Based on their feedback, minor modifications were made to improve phrasing, refine terminology, reorder select items, and enhance section instructions to better reflect a patient-centered approach.

Specifically, four questions were reworded, and the overall flow of the guide was modified to more effectively capture the challenges faced by both patients and healthcare professionals. Although face validity is inherently subjective, this step provided valuable insights into the guide’s usability and acceptability prior to undertaking formal content validity and reliability testing.

Content - validity

I-CVI (Pertinency of single/each item):

The results presented in Table 1 indicate that the items in the developed FGD guide were found to be highly relevant, with thirteen out of nineteen items receiving an I-CVI score of 1.00. Additionally, six items received a score of 0.9, suggesting that most of the items were perceived as necessary and useful by the evaluators.

S-CVI (Pertinency of the whole guide):

The S-CVI/Ave, which represents the sum of all item-level content validity indices (17.8) by the total number of items in the scale,¹⁹ was calculated to be 0.93, indicating a high level of content validity. In contrast, the S-CVI/UA, which represents the sum of all ICVIs equal to 1.00¹³ by the total number of items,¹⁹ was calculated to be 0.68, suggesting a moderate level of content validity. Overall, the developed FGD Guide’s content validity is shown to be moderate based on the Universal Agreement technique and high via the Average approach.

CVR:

For every item, a CVR was generated. Non-essential items had a CVR value of less than 0.9. It is possible to remove non-essential components, however in this instance it was not. The content validity ratio scores ranged from 0.8⁶ to 1.00,¹³ with an average of 0.9. The CVR results for the focus group discussion interview guide are presented in Table 2.

Kappa coefficient

All 19 items were considered to have excellent kappa statistics. In which, thirteen of the items are score of 1.00, and six of the items score is 0.899. Kappa calculations are shown in Table 3.

Pilot study

A pilot FGD was conducted one month prior to the main data collection to assess the clarity, structure, and feasibility of the interview guide. The pilot was carried out at the Rural Maternity and Child Welfare (RMCW) a Community Health Center, Manipal, and involved five participants. The session lasted approximately 1.5–2 hours. Prior to the pilot, the researcher refined the structure of the guide and rephrased selected questions to improve clarity and flow. During the pilot, participants’ responses to each topic were reviewed to identify issues related to question comprehension, sequencing, and relevance. Commonly used phrases and recurring expressions were noted, which informed the development of additional probes and minor refinements to the guide. Feedback from the pilot study, along with expert comments, was incorporated to finalize the interview guide.

Limitations

However, there are certain drawbacks to this study:

The study does not develop a new validation framework; instead, it methodically utilizes well-established content validation techniques.

Future studies involving larger and more diverse expert panels are recommended to enhance the robustness and generalizability of these findings.