Evaluating the Malay version of NIOSH WellBQ: A study on reliability and construct validity for enhanced workplace well-being assessment

Study design and sample size

This cross-sectional study used a Google Form to validate the NIOSH WellBQ among civil servants. The inclusion criteria encompass permanent staff, while the exclusion criteria pertain to temporary and contract-basis staff. The survey was widely distributed to various ministries and departments located in Kuala Lumpur, Putrajaya, and Selangor. Before participating in the study, all respondents provided written informed consent. In October 2022, a self-administered Malay version of the NIOSH WellBQ was distributed to 100 employees from various ministries and departments. The decision to include 100 respondents was driven by the aim to achieve a sufficiently diverse and representative sample, allowing for a robust analysis of psychosocial working conditions and employee well-being across different organizational settings and roles. This sample size is considered adequate for statistical significance, ensuring that the findings from the survey can be generalized to a broader population of employees within the targeted ministries and departments. Additionally, the practical consideration of participant burden and time constraints led to the selection of a manageable sample size, enabling a more feasible administration of the questionnaire, and ensuring a reasonable completion time of approximately 30 minutes per participant.

Questionnaire

The NIOSH WellBQ, comprising a total of 68 items, was translated into the Malay language. It encompasses 16 scales, five indices, and 31 individual items that span across five domains: (1) work evaluation and experience; (2) workplace policies and culture; (3) workplace physical environment and safety climate; (4) health status; and (5) experiences and activities outside of work related to home, community, and society.

The NIOSH WellBQ was translated using a back-to-back method from English to Malay and then back to English by a certified translator from Universiti Pendidikan Sultan Idris (UPSI). This rigorous translation process ensured the accuracy and quality of the Malay version of the NIOSH WellBQ. This translation version has undergone content validity assessment by four experts prior to the distribution of this questionnaire.

Procedure

The study’s protocol received approval from the Ethics Committee for Research Involving Human Subjects at Universiti Putra Malaysia on 2^nd February 2023 (Reference No: JKEUPM-2022-986). The questionnaire distribution in October 2022 occurred prior to obtaining ethical approval on 2nd February 2023, due to unforeseen administrative delays, which arose because at that time, the University was in a transitional phase between the COVID-19 control order and transitioning to a hybrid work phase. Delays were encountered in obtaining official letters, compounded by difficulties in accessing Government premises during the distribution of questionnaires at that time. However, ethical considerations were strictly adhered to throughout the data collection process to safeguard participant welfare and maintain research integrity.

Data collection was facilitated through the utilisation of a specific questionnaire. Prior to commencing the study, authorisation was sought and obtained from the pertinent Human Resources Departments within the involved ministries. The respondents received the questionnaire via their official WhatsApp group. Accompanying the questionnaire distribution, written informed consent was meticulously provided to and obtained from each respondent. Prior to their participation, each participant in the study was required to provide written informed consent. This process involved presenting participants with a paper-based consent form that outlined the purpose, procedures, and potential risks and benefits of the study. Participants were given ample time to review the information provided in the consent form and had the opportunity to ask any questions they may have had. Subsequently, those who agreed to participate in the study signed the consent form, signifying their voluntary agreement to take part in the research.

Statistical analysis

Data entry and analysis were performed using SPSS version 25, a widely-used statistical software package for social sciences. Descriptive statistics were calculated to summarise the data, including means and standard deviations for continuous variables and frequencies with percentages for categorical variables. The internal consistency of the measurement scales was evaluated using Cronbach’s alpha coefficients, a commonly used measure of reliability.

An exploratory factor analysis with principal components and varimax rotation was conducted to assess the construct validity of the measurement instrument. This technique helps to identify the underlying factors or dimensions captured by the items in the instrument and to assess how well they align with the theoretical framework.

However, it should be noted that SPSS cannot directly compute more advanced model fit indices such as TLI, RMSEA, and CFI, commonly used in Structural Equation Modeling (SEM). Therefore, the software AMOS was utilised to obtain pertinent findings regarding the goodness-of-fit of the structural models. AMOS is specifically designed for SEM analysis and provides robust tools to evaluate the model fit and test complex relationships among latent variables.

Socio-demographic characteristics

This study encompasses civil servants from diverse grades and service groups in Kuala Lumpur, Selangor, and Putrajaya. Initially, 100 questionnaires were distributed, out of which 95 participants responded. Among the final 95 respondents, 44 were classified as male, and 51 were female. It is important to note that these classifications refer to biological sex, not gender identity. Additionally, the majority of respondents were 37 years old and held a Bachelor’s Degree (30.5%).

Domain 1: Work evaluation and experience

The Malay version of the work evaluation and experience domain exhibited remarkable conformity with the data, as indicated by the following statistical indicators: RMSEA, TLI, and CFI values of 0.075, 0.99, and 0.97, respectively (Table 1). Most subdomains within this specific domain demonstrate consistently good internal consistency, as evidenced by Cronbach’s coefficients exceeding 0.70. However, it is essential to highlight that the assessment of work conditions indicates poor internal consistency, with a Cronbach’s alpha coefficient of 0.159. This discovery is similar to the study conducted while validating the NIOSH Work Well-being Questionnaire in Italian.¹⁴

Domain 2: Workplace policies and culture

Based on the findings, the Supportive Work Culture scale demonstrates good reliability, with a Cronbach’s alpha coefficient of 0.895. Meanwhile, the Health Culture at Work scale shows an acceptable level of internal consistency, with a Cronbach’s alpha coefficient of 0.631. The subdomain assessing benefits exhibits a Cronbach’s alpha coefficient of 0.727, indicating good internal consistency. Additionally, the Organization of Work and Life subdomain achieves a Cronbach’s alpha coefficient of 0.780, signifying a satisfactory level of internal consistency. Unfortunately, the RMSEA, CFI, and TLI values cannot be computed for this domain, as it contains only single items or fewer than three observed variables per latent construct. These fit indices necessitate an adequate number of indicators to evaluate the model’s goodness of fit effectively. Moreover, it is worth noting that the original developers of this questionnaire did not report the RMSEA, CFI, and TLI values.⁶

Domain 3: Workplace physical environment and safety climate

The findings revealed that all scales demonstrated good internal consistency. The Cronbach’s alpha coefficients were 0.931, 0.823, and 0.688 for workplace safety climate, physical work environment satisfaction, and interpersonal conflict and incivility, respectively. Additionally, the validation of the third domain yielded excellent results, indicating a high level of fit for the model, with an RMSEA of 0.086, while the TLI and CFI values were 0.989 and 0.981, respectively.

Domain 4: Health status

Statistical findings on domain 4 indicate a very good model fit with RMSEA = 0.079, TLI = 0.922, and CFI = 0.977. However, it is essential to note that during our validation analyses, certain subdomains (“general health” and “injury”) and constructs (“physical activity,” “tobacco use,” “alcohol consumption,” “work-related injury,” and “injury consequences”) were taken into consideration. Expressly, the “injury” subdomain and its constructs were excluded from the validation analyses due to the extremely low variability in responses provided by the respondents. Respondents found it challenging to accurately assess the effects of work-related injuries in the last 12 months. Similarly, the construct “alcohol consumption” was removed because all responses were coded as 0, indicating no variability. Likewise, the “tobacco use” construct was excluded due to some modalities’ low frequency of affirmative responses.

Regarding the construct “physical activity,” statistical analysis revealed a weak correlation between the two items of the scale. This result can likely be attributed to the diverse and varying nature of work activities performed by the population enrolled in this study. The participants in this study engaged in various and distinct tasks, which may have influenced their responses to the two items on this scale. As for the subdomain “general health,” its inclusion in the validation analyses led to decreased CFI and TLI values; hence, it was excluded from the analyses. Only the subdomains “overall stress (health, finance, relationships, work)” and “cognitive functioning limitations” under mental health exhibited acceptable internal consistency with Cronbach’s alpha values of 0.821 and 0.782, respectively.

Overall, the findings indicate a robust model fit for domain 4. However, excluding certain subdomains and constructs was necessary to ensure the validity and reliability of the validation analyses. These exclusions were made based on the limited variability, low frequency of responses, and weak correlation observed for certain items in the assessment, as well as considerations of the unique characteristics of the study population and their work activities.

Domain 5: Home, community, and society

While Fontana et al.’s study¹⁴ did not include factor analysis for this domain, we chose to incorporate it into our research. This domain consists of two single items, one 2-item index, and one 7-item index. Chari et al.,⁶ the developers of NIOSH WellBQ, reported a low Cronbach’s alpha value of 0.62 for this domain in their original study. However, in our current investigation, both subdomains, namely “Financial security” and “Activities outside of work,” exhibited higher Cronbach’s alpha values of 0.811 and 0.810, respectively, indicating stronger internal consistency. It is important to acknowledge the potential limitations of using a combination of single and multiple items in this domain, as it may impact the interpretation of results and the underlying constructs being measured. Further research and validation studies may be warranted to enhance the measurement of this domain within the context of well-being assessments.

Correlations among selected scales and items

In this study, similar to the research conducted by Chari et al.⁶ and Fontana et al.,¹⁴ an analysis of correlations among selected scales and items was performed to verify concurrent, convergent, and discriminant validity (results presented in Table 2). The primary distinctions when comparing this study with the research conducted by Chari et al.¹ were observed for the following items and scales: “time paucity/work overload,” “meaningful work,” “availability of health programs at work,” “availability of job benefits,” “physical activity,” “tobacco use,” “risky drinking,” “healthy diet,” “support outside of work,” and “activities outside of work.”

Regarding this matter, it is reasonable to expect that the differences in domains 4 (health status) and 5 (home, community, and society) are predominantly correlated to social and cultural differences between the population enrolled in the study. Furthermore, based on the findings, the differences observed in domains 1(work evaluation and experience) and 2 (workplace policies and culture) mainly result from the uniqueness and variations in the work environments of the respondents in the public service, who come from various types of services and positions.