Evaluation of mobile phone precision push service perception

A. Background and rationale

With the rapid advancement of information and communication technology (ICT), mobile devices have increasingly become the primary means for users worldwide to access digital services, particularly via smartphones (Statista, 2023). Precision push notifications, which utilize algorithms and user data modeling to deliver personalized content, have become a central feature of mobile internet ecosystems DeGe and Sang (2024). Despite advancements accelerated by the integration of 5G networks and IoT technology (Zhang et al., 2021), concerns such as notification overload, privacy risks, and gender differences in usability persist, highlighting the need for deeper research into user experiences (Fornes-Leal et al., 2021; Smith and Jones (2022).

B. Uniqueness of this study

While previous studies have largely concentrated on technical optimizations, such as enhancing push speed through edge computing (Hsu et al., 2019) or targeting accuracy via big data (Wang, Chen, & Zhao, 2023), this research adopts a broader approach by comprehensively assessing multidimensional user perceptions. Instead of examining isolated elements like message validity or operability independently, this study incorporates five key dimensions: effectiveness, non-interference, operability, user choice, and information transparency. Additionally, the study uniquely investigates the moderating effects of gender and geographic location, thus addressing current gaps related to demographic-specific analysis (Johnson & Lee, 2023) and providing insights to support ethically sustainable precision push services.

C. Research gaps

Current literature reveals several key unresolved issues regarding precision push services:

D. Research objectives and contributions

This study aims to systematically examine how the five critical dimensions—effectiveness, non-interference, operability, user choice, and transparency—interact to shape user perceptions of mobile precision push services. Additionally, it analyzes demographic influences, specifically gender and geographic factors, to reveal more nuanced patterns that previous studies may have overlooked. The research also investigates the paradox whereby highly precise push notifications can unintentionally reduce user satisfaction due to perceived intrusiveness.

The contributions of this study include theoretical advancement through developing a comprehensive framework that aligns technical design principles with user experience dimensions, thus bridging fragmented analyses from existing literature. Methodologically, the study incorporates demographic variables into its analytical framework, using structural equation modeling (SEM) and Cronbach’s α reliability tests to enhance empirical rigor in human-computer interaction research. Practically, this research provides actionable insights for interface design and content personalization strategies, assisting developers in reducing notification fatigue while improving user engagement within the context of emerging 5G-driven mobile applications.

A. Research model determination

Through research and analysis of the perception of precision push services, a conceptual framework for evaluating precision push perceptions has been developed, as shown in Figure 2 below.

Figure 1. Precision push service working process.

Figure 2. Conceptual framework for evaluating precision push perceptions.

The evaluation of user perception regarding precision push notifications is organized into five hypothetical dimensions. These dimensions were developed based on a comprehensive review of existing literature and empirical studies that focus on targeted messaging, personalized notifications, and user experience factors influencing mobile communication services. The formulation of hypotheses considered multiple critical factors, including accuracy, timing, quality of information, user autonomy, notification frequency, customization options, and transparency regarding data usage and notification mechanisms. Detailed criteria for each dimension were established through an iterative process involving literature analysis and expert consultations to minimize potential bias and variability.

Specifically, the factors hypothesized to influence user perception are as follows:

Data collection employed structured questionnaires distributed digitally, designed with rigorous consideration to avoid potential bias, including randomized question order, clear phrasing, and controlled respondent demographics. Reliability and validity testing, including Cronbach's alpha and KMO tests, were performed to ensure robustness and accuracy of findings.

B. Data collection

This pilot study targeted a clearly defined finite population comprising 180 third-year undergraduate students enrolled in a Communication Technology program. Its primary goal was to validate the research framework and assess the reliability and validity of measurement instruments ahead of larger-scale studies. Employing methodological rigor, the study strategically selected a modest sample size of 64 respondents, balancing statistical validity with feasibility. Using a 95% confidence level and ±10% margin of error, along with a conservative population proportion of 50%, the final adjusted sample size of 64 respondents was deemed sufficient and appropriate, effectively representing 35.6% of the overall population.

C. Instrument development

Quantitative research methods, specifically questionnaire surveys, were employed to analyze and evaluate precision push services. Before questionnaire design, the research objectives and dimensions—encompassing five hypothesized factors and two perceptual dimensions (effectiveness and impact)—were clearly established. The questionnaire was structured across seven dimensions, each containing 4 to 9 quantitative, single-choice questions distributed via Tencent Questionnaire, optimizing data collection efficiency and reducing costs. Given the popularity of smartphones and digital services among young users, university students constituted the primary research sample.

To ensure scientific rigor and reliability, the Likert Scale was utilized, providing respondents with a clear, structured format for expressing agreement or disagreement across statements. Originally developed by psychologist Rensis Likert in 1932, this scale typically features a 5-point rating system, enabling precise differentiation of respondents' attitudes and improving interpretability of the results. The simplicity and clarity of the Likert Scale make it the most widely adopted standard method for contemporary survey researchLikert (1932).

A. Analysis of reliability and validity

Cronbach's alpha coefficient is a method used to assess the internal consistency reliability of a questionnaire or test. This coefficient reflects the average correlation between the items in the questionnaire and is used to measure whether different items collectively assess the same concept. The value of Cronbach's alpha ranges between 0 and 1, with higher values indicating better internal consistency, meaning a stronger correlation between the items. The reliability statistics for each dimension are shown in Table 2.

Based on the analysis results, the overall Cronbach's alpha coefficient for this test questionnaire is 0.932, which meets the expected reliability threshold, and the results can be generally considered reliable. This result in line and supported by the statement of Ahmad et al. (2024) that state the advanced research or research required the high precision, a higher alpha value with 0.8 and above is preferred for ensure the greater reliability. However, the dimensions Non-Interference with User Experience and Impact of Precise Push Service did not meet the reliability requirement of 0.6, indicating the need for further testing and question revision in subsequent assessments. Despite this, the sampling results for these two dimensions follow a normal distribution, meeting the randomness requirement. Therefore, the results can still be used for further analysis, but with cautious evaluation (Taber, 2018).

In the subsequent analysis, we will reduce the weight of these dimensions, relying more on other dimensions with higher reliability. When using data from these dimensions, we will clearly state their limitations to inform users of potential risks. These dimensions will be used primarily for exploratory analysis, rather than as a basis for critical decision-making.

The questionnaire results were also subjected to a validity analysis using the KMO (Kaiser-Meyer-Olkin) test and Bartlett's Test. The closer the KMO value is to 1, the higher the commonality between variables, indicating stronger suitability for factor analysis. The evaluation criteria are as follows: KMO > 0.9: excellent; 0.8-0.9: good; 0.7-0.8: acceptable; 0.6-0.7: marginal; 0.5-0.6: barely acceptable; below 0.5: unsuitable. It is generally accepted that a KMO value greater than 0.6 is appropriate for factor analysis. The results of the validity analysis are shown in Figure 3.

Figure 3. Validity analysis results.

Figure 4. Performance bar charts and radar charts for each dimension.

From the test results, it can be observed that the result of Bartlett's sphericity test is very close to 0, meeting the requirement of being below the usual significance level (e.g., 0.05). This result supported by Rossoni et al. (2016) that state that a statistically significant Bartlett test (p < 0.05) indicates the sufficient correlate with the existing variable for continue the analysis. This allows us to reject the null hypothesis, indicating that there is a correlation between the variables, making them suitable for factor analysis. The KMO value is 0.638, suggesting that the validity results support the appropriateness of conducting factor analysis, and that the overall questionnaire is usable (Hinton, McMurray, & Brownlow, 2021).

B. Analysis of differences

A difference test was conducted across different sample groups for various test dimensions. The analysis results show that only the Operability dimension had a significance value of 0.045, which is less than the threshold of 0.05, indicating a gender difference in this dimension. The test results for other dimensions were much greater than 0.05, showing no significant gender differences in the remaining questionnaire dimensions (Smith & Jones, 2022). In the Operability dimension, the average score for males was higher than for females. Additionally, for specific questions, men and women provided markedly different responses regarding their assessment of operability, which is the source of this difference (Nurqamarani, Fadilla, & Juliana, 2024).

Men and women may interact with applications differently. For example, women might be more inclined to use social media apps, while men may favor gaming or tech-related apps. Men might also be more open and confident in using new technologies, while women may be more cautious and take more time to adapt to new devices. When encountering technical issues, female users may be more likely to seek assistance, while male users may prefer to solve problems independently. These differences lead to varied experiences in terms of operability between men and women in precision push services, and such differences are reasonable and acceptable.

When choosing personalized push content, app developers should consider these gender differences in operability and provide different interfaces and operational methods to achieve better push outcomes and higher push effectiveness (Taylor, 2021).

There were no significant differences in any of the dimensions based on geographic location, which aligns with the expected statistical results.

Correlation analysis is a method used to assess whether there is a statistical association between two or more variables. This type of analysis helps researchers understand the strength and direction of the relationship between variables, though it is important to note that correlation does not imply causation. In this sample, the Pearson correlation coefficient is employed to evaluate the correlations among the measured variables.

C. Correlation analysis

Results marked with two asterisks (**) indicate that the variables show significant correlation at the 99% significance level. The correlation between Service Impact and Information Transparency is marked with a single asterisk (*), indicating a significant correlation at the 95% significance level. The absence of asterisks represents a weak correlation. In this test, Service Impact showed weak correlations with the other dimensions, with some negative correlations also observed (Chen et al., 2022). This indicates that as the effectiveness of precision push services increases, the impact of the service decreases.