Technological innovation for workload allocation in nursing care management: an integrative review

Background Technology reduces the nursing workload, improve the quality care processes, patient’s safety, and avoid staff burnout. Innovative technologies are disrupting healthcare systems by improving the efficiency of processes and management. There is a discussion on the benefits, challenges, and barriers of these technologies and considering human factors of nursing management. The aim was to analyze the influence of technologies on the distribution of workload for nursing care management. Methods An integrative literature review was performed. Four databases were searched: Scopus, Scielo, PUBMED, and CINALH following PRISMA guidelines. Articles published from January 2016 to December 2020, published in English, Spanish and Portuguese were included. Studies were excluded when they were not original research, did not met the quality criteria or they did not answer the research questions. Quality appraisal was performed using the Crowe Critical Appraisal Tool version 1.4 (CCAT). Two reviewers independently examined the title and abstract for eligibility according to the inclusion and exclusion criteria. Results 2818 potentially relevant articles were found, but once the inclusion and exclusion criteria in the abstracts were analyzed, 177 remained for evaluation. After following the PRISMA Guidelines, 35 studies were included in the review. Three categories were identified: Nursing workload; Information technologies and technological means for management; Technology acceptance. Conclusions Technology has the potential to improve care management by estimating nurse workload in ICUs and non-critical units, but scientific evidence is more detailed in the former type of services. The literature provides insights about the factors that factors and the barriers that promote the technology acceptance and usability. We did not find studies comparing technologies and no scientific evidence proving improvements in care.

and abstract for eligibility according to the inclusion and exclusion criteria.

Results
2818 potentially relevant articles were found, but once the inclusion and exclusion criteria in the abstracts were analyzed, 177 remained for evaluation.After following the PRISMA Guidelines, 35 studies were included in the review.Three categories were identified: Nursing workload; Information technologies and technological means for management; Technology acceptance.

Conclusions
Technology has the potential to improve care management by estimating nurse workload in ICUs and non-critical units, but scientific evidence is more detailed in the former type of services.The literature provides insights about the factors that factors and the barriers that promote the technology acceptance and usability.We did not find studies comparing technologies and no scientific evidence proving improvements in care.

Introduction
The nursing staff contribution is essential to guarantee quality and access to health services, as well as the outcomes of care for patients and their families.New technologies should support nursing management and decrease workload to meet the needs of the patients.This aspect is relevant since nursing care is essential for the patient recovery and safety (Moreno-Monsiváis et al. 2015).
According to the World Health Organization, there is a relationship between the nursing workload, the morbidity, and mortality of hospitalized patients (Ball et al., 2018) and after discharged (Kim et al., 2020).Nevertheless, the evidence is not strong to determine the best nurse-to-patient ratio and the effects on the nursing workload and patient outcomes (Coster et al., 2018).
The nursing workload is the amount of time, physical and cognitive effort needed to provide nursing care, in addition to activities related to service management and professional development (Pedroso et al., 2020).Excessive nursing workload goes against the values of humanization of care, patient outcomes, patient safety (Arango et al., 2015), quality of care (Romero-Massa et al., 2011;Chang et al., 2019), omitted care (Moreno-Monsiváis et al., 2015) and nurses' health (Harvey et al., 2020).There is interest in creating technology to integrate scientific evidence and nursing expertise to reveal the relationship between nurse workload, burnout, and care quality (Farid et al., 2020).Thus, an interdisciplinary research team, including nurses and engineers, working together is required to understand this relationship.
The aim was to analyze the use of technologies on the distribution of workload for nursing care management.The following research questions (RQ) were addressed: • What methods are used to calculate nursing workload?
• What factors influence nursing care management technology acceptance by nursing professionals?Methods An integrative literature review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines.Studies were retrieved from SCOPUS, Scielo, PUBMED, and CINAHL in January 2021.The search strategy was: ("analytics" OR "operations research" OR digital* OR "IT" OR technolog*) AND ("nursing workload" OR "nurse-patient ratio" OR "nursing workforce planning" OR "nursing algorithm" OR "nursing personnel staffing and scheduling" OR "nursing staff")).The authors were involved in the identification, screening, and analysis stages.Inclusion criteria considered studies that address the integration of technology on nursing workload, burnout and nursing acceptance, published in full text in English, Spanish or Portuguese classified as scientific articles or reviews, in the areas of nursing, medicine, social sciences, engineering, published between January 2016 and December 2020.This date range was considered to focus on the recent technologies.Articles related with technology that didn't analyze the impact on nursing workload or didn't present technologies were excluded.
For all research, this must include a final section including details of ethical approval, informed consent and, where relevant registration.
Initially 2,818 articles were potentially relevant, of these 21 that were duplicates were eliminated and 129 were discarded by automation tools because they didn't meet the inclusion criteria.In the second phase, the titles of 2668 REVISED Amendments from Version 2   We have taken consideration of the reviewers' comments.We clarify that the purpose of the study is to analyze the influence of new technologies on the distribution of workload for nursing care management.The suggested modification of the first research question is included, the second RQ is eliminated.
The concept of workload is defined in the document on page 2 "The nursing workload is the amount of time, physical and cognitive effort needed to provide nursing care, in addition to activities related to service management and professional development (Pedroso et al., 2020)."We reviewed the table and organized it according with the main categories identified: Nursing workload, Information technologies and technological means for management, technology acceptance, and also taking into account the date of publication.
articles were screened and 2486 were excluded since they didn't meet the inclusion criteria.Later, the abstracts of 182 articles were analyzed.In the third phase, two independent reviewers read the full texts and discarded 96 articles because they didn't contribute to the RQ and 46 because they obtained a low score in the CCAT.Ultimately, 35 studies were included (Figure 1).Quality review of each article was independently reviewed by two members of the research team/authors using the CCAT, that has shown to have high reliability to rate research papers (Crowe et al., 2011).This tool is divided into eight categories and 22 items; each category receives a score on a 6-point scale from 0 (the lowest score) to 5. The total score is expressed as a percentage, and we arbitrarily considered acceptable a score above 60%.When there were differences in the quality assessment of an article, team consensus was achieved by comparing the arguments and evidence.
Data extraction was performed identifying the purpose, the design, participants and context of the study, the type of information collected, and the results.The interdisciplinary work between engineers and nurses made it possible to resolve doubts about technological and nursing concepts.For data synthesis, we performed an inductive content analysis.The information was collected in our database, and then grouped into categories that responded the RQ.

Results
Characteristics of the studies 35 studies from 18 countries were included, 18 observational studies, two quasi-experimental, five qualitative, three mixed methods, four literature reviews and three project evaluations.57.1% were assessed as high quality, 42.9% as medium quality (Table 1).

Elements of nursing management
The studies were classified into three categories: 1. Nursing workload; 2. Information technologies and technological means for management; 3. Technology acceptance. Figure 2 presents a concept map of the categories discussed in the results.
-Nursing workload This category includes the methodologies for estimating nursing care requirements, factors, and approaches for calculating nursing workload and the predictors of nursing burnout and outcomes.
The challenge for workload estimation and prediction systems is having adequate information about the time and effort needed for nursing care activities, considering the interactions and iterations of the day-to-day care process, in physical, cognitive, organizational, and emotional dimensions (Redley et al., 2020).The variability in the patient care process usually has significant skewness.There are methodologies for estimating nursing care requirements, which are classified into those based on expert opinion, benchmarking, nurse-patient relationship, patient prototyping, multifactorial indicators, and task time-based approaches (Griffiths et al., 2020).
There is literature about nurse-to-patient ratios, but the validity and reliability of the studies is weak (Griffiths et al., 2020).In practice, legislation on minimum nurse staffing levels varies.In California (USA), law 394 of 1999 stipulates a 1:6 ratio.In Australia, the Safe Patient Care Act 2016 stipulates a 1:4 ratio.In Queensland, the daytime ratio is 1:4, and nighttime 1:7.In Wales (United Kingdom) the law stipulates a daytime ratio 1:5, and nighttime 1:7.Other countries, such as Colombia (South America), do not have legislation.The methodological proposals on how to calculate the nursing care requirements, nurse staffing, and skill mix, are based on local observations, which don't allow generalization.
Low staffing levels negatively affect safety indicators and staff performance (Simonetti et al., 2020), but estimating the nursing workload, considering emergent care, for decision making, is still an unsolved challenge (Griffiths et al., 2020).There are no systems that estimate workload beyond ad hoc methodologies, based on average measurements that ignore the variability.The most currently used systems are: • GRASP (Grace-Reynolds Application and Study of PETO) (Clark & Poland, 1976).
• TOSS (Time Oriented Score System dedicated to intensive care patients) (Italian Multicenter Group of ICU Research, 1991).
• RAFAELA Patient Classification System including the Oulu Patient Classification instrument and the Professional Assessment of Optimal Nursing Care Intensity Level (PAONCIL) (Aschan et al., 2009).
Comparisons of these systems show significant differences in the results (Griffiths et al., 2020), and there is no evidence that one outperforms the others.In addition, the professional judgement-based approaches are popular and relatively reliable.
Commercial software computes staffing levels, coordinates nursing shifts, and plans nursing activities in case of unexpected staff absences.Tuominen et al. (2020) show that such tools reduce operating costs, the time required by nurse managers during the rescheduling process, the number of understaffed shifts, and unexpected changes in shift assignments.Determinants for the acceptance of the system by the nursing staff: the ease of use of the mobile station, the device, the nursing application and its usefulness, and high expectations about the new process.Lack of reliability and interruptions impeded its use.The impact of technology for predicting nurse workload and deciding optimal staffing levels includes factors such as quality of care, patient safety, staff turnover and satisfaction, and process efficiency.Understanding nurses work is essential to assessing how technologies contribute to quality of care (Redley et al., 2020).There has been insufficient discussion of how technologies can be used to correlate operational cost with quality of care (Griffiths et al., 2020).
In relation to the predictors of nursing burnout and outcomes, patient and nursing staff outcomes are affected by nursing burnout and workload, which is significantly higher in ICUs than in other units (Youn et al., 2020).Burnout is a problem that originates in individuals and organizations (Maslach & Leiter, 2017).High nursing workload and poor skill mix are correlated with higher levels of burnout (Assaye et al., 2020;Simonetti et al., 2020), as does a poor work environment (Pekince & Aslan, 2020).In turn, burnout is associated with lower patient safety and omitted patient care (Dall'Ora et al., 2020).
High nurse workload is associated with higher rates of mortality (Assaye et al., 2020;Aiken et al., 2021) The most studied technologies in ICUs (intensive care unit) were electronic medical records (EMR) and electronic medication management (Redley et al., 2020).Nursing-specific information systems, general computer, communication technologies and electronic data analysis being less often studied.Time savings are proven with the use of technology, but the impact on clinical decision making, interpersonal relationships, and work stress that underpin nursing care quality is unproven with respect to direct benefits.These systems could fail to record legitimate reasons for missing data, which can reduce patient safety (Hope et al., 2019). -

Information technologies and technological means for management
Human and compassionate relationship between patients and nurses requires a meticulous synchronization between patients, healthcare staff, suppliers, processes, technologies, and information.The studies show that information technologies (IT) and technological means can support nursing care (Vorakulpipat et al., 2019;Zhou et al., 2019).IT include EMR and systems for: online input, tracking patients, detect changes in the patient condition and workload reallocation, trace patient's calls, hands-free communication between nurses, detecting falls, barcode technologies for medicine management, data mining on nursing flows, among others (Massarweh et al., 2017;Respicio et al., 2018;Vorakulpipat et al., 2019;Yu et al., 2019;Zhou et al., 2019;Moon, 2019).
For effective IT implementation in nursing, change management and communication are determinants (Giraldo et al., 2018) as well as technology skills training (Farzandipour, 2020).It's also recommended to follow workflows, multiplicity of task and patient care plans (Vorakulpipat et al., 2019).
Current technology to assess the nursing workload doesn't capture the complexity or quality of care, and doesn't recognize the concurrent, iterative, and interrelated nature of cognitive processes and activities in nursing.Therefore, there is a gap in the literature on how technology affects the quality of nursing care and how to measure it (Redley et al., 2020).
The patient's condition has a significant effect on the workload (Sato et al., 2016).The development of early warning systems for patient deterioration (systolic blood pressure, pulse, respiratory rate, and oxygen saturation), using analytics, shows a significant decrease in nursing care time (Mau et al., 2019).These findings could be used to build a model for staffing (Leary et al., 2016). -

Technology acceptance
The factors that influence technology acceptance are the perceive usefulness, the ease-of-use (Berg et al., 2017;Holden et al., 2016), and the participation of nurses in its design (Redley et al., 2019;Pendon et al., 2020).The human and organizational dimensions influence acceptance more than the technological dimensions (Handayani et al., 2018).The use of technology is beneficial when it promotes productivity and nurses' empowerment, patient safety (Beaney et al., 2019), and quality of care (Higgins et al., 2017).Also, there is a positive correlation between the perception of usefulness and ease-of-use of EMR, the scanner to record the medication administration, and the use of lifts to mobilize patients and nurses' technology acceptance (Tubaishat, 2018).These technologies contribute to reducing nursing staff fatigue (Steege & Dykstra, 2016) and nurses' stress, while increasing their ability to focus on patient care (Ozan & Duman, 2020).
Tubaishat ( 2018) and Zhou et al. (2019) found that nurses age, sex, experience level and willingness to use the technology, are moderating variables of technology usage behavior, and predictors of perceived usability of EMR.On the other hand, the interface design of software functions plays a crucial role in technology implementation (Tu et al., 2018).
In contrast, the lack of confidence by professionals, past experiences, heavy workloads, and the time needed to learn are barriers to use technology (Wood et al., 2019).Additionally, nursing staff tend to use "informal workarounds" to deal with exceptional situations in administrative processes that have technology.These practices are a risk for patient safety (van der Veen et al., 2020) and information reliability.

Discussion
Technology in health services has benefits for streamlining processes, improving the quality of care, improving communication by supplying real-time information related to patient management and having data for decision making, process evaluation and results.The implementation success depends on the acceptance of technology.
Nurses have significant interaction with technology to monitor and manage patient care.Most studies look at the interaction of ICU nurses, but the technology requirement should extend to all health services, at all levels of complexity.
Hence, an essential aspect in the implementation of technology is to enhance its acceptance (Phillips, 2019).
IT and care management systems in nursing should allow effective planning of nursing workload.These tools should promote patient safety, quality of care, and staff satisfaction.This is achieved when technology allows staff to spend more time with the patient and less time doing administrative processes.Although the usefulness of technology for nursing practice and clinical decision making has been demonstrated, the study on the impact of these is incipient and there is no evidence on the best way to plan workload allocation in nursing (Griffiths et al., 2020).
There are opportunities for research in this topic because the conclusions in the literature are contradictory.While some studies conclude that technology can increase staff productivity, empowerment, and improve nursing care outcomes, other's show that technology can increase nursing staff stress and burnout.In addition, it unknown how these technologies contribute to promote compassionate nursing care.
Further, the literature recognizes that studies analyzing the effect of technology use in nursing care is not free of bias (Redley et al., 2020).In these cases, although observational practices do not allow us to see cause-effect relationships, it is feasible to think of methodologies that resolve these questions (Griffiths et al., 2020).
Most of the software is limited because they don't use real-time information.Instead, they work under the assumption that a sample is sufficient to generalize the needs of the patient population in the future.Furthermore, these solutions assume that once the contracted staffing level has been found, a fixed nurse-to-patient ratio and fixed number of hours will satisfy nursing care requirements (Griffiths et al., 2020).The strongest assumption relies on average care requirement times and allocating the load assuming that individual variations can be accommodated (Griffiths et al., 2020), ignoring the staff experience levels.Further, information systems should consider the scientific evidence showing that understaffing, low levels of skilled staffing, burnout, and high levels of intent to quit negatively affect clinical outcomes and increase turnover costs (Simonetti et al., 2020).Simultaneously, user-friendly systems are needed to allow nurse managers to make better decisions quickly and efficiently without creating staff burnout and stress (Harris et al., 2018).
The discipline of operational research techniques has proposed techniques to optimize staffing decisions using mathematical models or computer simulations, known as the "nurse rostering problem" and how to reassign staff to units with high demand.These algorithms have been able to prove that allocation based on averages may not be best in the face of variability in patient requirements and that this variability usually means more staffing to meet peak demand.However, how to calculate workload has not been discussed in depth (Griffiths et al., 2020), and how to use IT to estimate and predict workload including all the physical, cognitive, organizational, and emotional dimensions still is a major unsolved question.
Finally, in this research we have not taken into consideration sex and gender differences in the results since the reviewed papers do not clearly conclude on how the associated factors and barriers that promote the technology acceptance and usability is different for different sex and gender.

Limitations
Despite having a rigorous protocol for identifying scientific articles, there are few studies performed in non-critical services.The studies found have heterogeneous methodologies, which made it difficult to compare the evidence in the studies (see Table 1).Further, the literature review did not focus on studying the concept of innovation in nursing workload management as it is analysed in Rylee and Cavanagh (2022) for nursing practice.

Conclusions
Based on the literature analysis, we conclude that IT has the potential to improve care management.There are several technological platforms that propose solutions to manage nurse workload and shift management.Further, the existing models use approaches based on expert opinion, benchmarking, nurse-patient relationship, patient prototyping, multifactorial indicators, and approaches based on task-based time requirements.These approaches differ importantly in how workload is assigned and the variables that are considered.It is equally important to analyze the technology and nursing workload in ICUs and non-critical units, but scientific evidence is more detailed in the former type of services.
The factors that promote the technology acceptance and usability of these models include the satisfaction for an increased time for direct care, a perception of improved patient safety, quality of care, and reduction of workload and staff burnout.
The identified barriers for these technologies are the lack of knowledge of the technological tools, long times needed for their use, and low perceived ease of use.Thus, it is important to involve nursing professionals in the technological design process.
Our study reveals that most of the models and technologies are focused on intensive care units, and few studies for other types of care units.Also, all the considered studies are performed in developed economies, none of them are performed in economies in transition or developing economies.In addition, there is no scientific evidence to compare the technologies between each other and we found no scientific evidence showing improvements in care processes provided after using them.Further, with the proliferation of nursing workforce management systems, a review to evaluate efficacy is warranted.The main research gap that we identify is the lack of studies showing evidence of the impact of technology measured as the correlation between the nursing outcomes and staff well-being with the use of workload estimation and the strategies for planning workload allocation.One key contribution of this study is that the conclusions are obtained from the analysis of a multidisciplinary research team, including nurses with expertise in management and engineers with expertise in healthcare systems.

Claudio-Alberto Rodríguez-Suárez
University of Las Palmas de Gran Canaria, Las Palmas, Spain Dear Authors, In my opinion some methodological aspects in the manuscript should be improved.
Since it is an integrative review, you should include scientific literature and gray literature.In the inclusion and exclusion criteria section, the study designs included and excluded should be detailed.
Figure 1 (flowchart) includes the search results, which in a review is the first result; therefore, Figure 1 should be transferred to the results section and the process of identification and screening of records and inclusion of studies should be described in results.
I believe that the rest of the manuscript is correct after the authors' modifications to the reviewers' comments.

Nancy Purdy
Nursing, Toronto Metropolitan University, Toronto, Ontario, Canada The authors conducted an integrated review of the literature to analyze technology-based tools for their use and impact on nursing workload models, burnout predictors and outcomes as well as factors that influence technology acceptance by nursing professionals.PRSIMA guidelines were used to inform the methods for the literature review.The flowchart of the screening process was accurately and clearly presented.All selected papers were summarized in Table 1 and elaborated upon in the results section.The discussion that followed included a summary of key findings to address the study aims.The conclusions aligned well with the results and discussion sections.
While the overall structure of the paper is adequate, the paper could be improved significantly with a few revisions.The main issue is the need to clearly and consistently articulate the study aims and align these aims with the study title.In the abstract, the aims include an analysis of nursing workload models, predictors of nursing burnout and outcomes and the acceptance of new 1.
technologies for nursing care management.Each of these aims seem divergent and each has a large body of literature.Considering the title of the paper, I think something is missing in the explanation of the study aims.I have assumed that aim is in fact the one that I have noted in the first paragraph i.e. the use and impact of technological innovations or tools for nursing care management which includes those used for nursing workload models, the impact of technological innovations on predictors and outcomes of burnout and overall nursing acceptance of these technological tools.In effect, I am assuming that the authors are intending to report on the integration of all of these concepts -technology and nursing workload, burnout and user acceptance.Therefore, the research questions on page 3 need to be more explicit.For RQ1, a suggested revision would be, " What methods are used to calculate nursing workload?".The concept of nursing workload should also be defined i.e. are you referring to nursing workload measurement?RQ2 is unclear and seems to overlap with RQ1.RQ3 is stated clearly.Any changes made to improve clarity should be reflected in the abstract, methods and remaining sections of the paper.
The search strategy was described but the inclusion and exclusion criteria for the selection of papers should be elaborated upon with sufficient detail that someone else could replicate the study.The authors stated "Articles related with technology that didn't analyze the impact of nursing workload or didn't present technologies were excluded."This is helpful and lead me to conclude that articles needed to address the integration of technology on nursing workload, burnout and nursing acceptance.This needs to be stated more explicitly.

2.
The organization and content of Table 1 needs to be revised to reflect the study aims.Some papers only address one of the concepts of interest but not necessarily the integration of technology and nursing workload, burnout and acceptance.I think that this might be due to the content provided in the 'outcomes' column.Perhaps the content selected for this column could be revised to reflect this aim more clearly.Also, the order in which papers are presented does not seem to follow any rules (not listed in chronological or alphabetical order) nor listed by the concept of interest.For example, the column header for 'Predictors of nursing burnout' lists a paper by Hope et al. but the paper summary does not mention burnout.On the next page (p.7), the column header is 'Nursing Workload Models' but there is a paper by Dall'Ora on burnout.It would be useful to have clear subheadings the reflect the order of the concepts as presented in the aim statement.Also, it is not clear why some papers were selected as there is no mention of technology in the summaries.Perhaps more clarity in the aim statement, inclusion/exclusion criteria and revisions to the 'outcomes' column would help to justify the papers selected and reported in this table.

3.
The concept map is very useful and helps the reader to understand the results of the content analysis.I have two suggestions for your consideration.
In the middle column, I would consider nursing workload models and information technologies to be the main elements of nursing care management.I think that burnout predictors and outcomes would better fall within the third column as they are not a type of management tool.Similarly, I think technology acceptance would better fit under the third column as well as it is not an element of nursing care management but instead is a factor related to technologies.

1.
Suggest adding a 4 th column to list the authors for the papers where related evidence for the theme was found.This will help the reader with understanding the results and discussion sections as well. 2.

4.
Other recommended edits -There are places that require further editing to improve the sentence structure but I have not listed them here (available if requested).The abstract requires the most work (primarily the background and methods sections) as there are duplicate phrases or unclear sentences.Also, human factors is mentioned in the background section but is not discussed anywhere in the paper.

5.
Overall, I think this is a useful research report that adds to our understanding of the impact of technology on nursing workload.I think the proposed revisions will add to the uptake and use of this review.

Claudio-Alberto Rodríguez-Suárez
University of Las Palmas de Gran Canaria, Las Palmas, Spain Dear authors, thank you very much for the opportunity to review this manuscript.This systematic review addresses an interesting topic on nurses' workloads.Since this is a systematic review study, the following comments are based on the PRISMA guidelines.
As general comments, a quality criterion for a systematic review is the registration of the review protocol (for example in PROSPERO).The registration number should be provided in the method and (according to PRISMA also in the abstract).
Abstract: is structured and informative.In the methods section it is possible to improve the information on the following aspects: Specify the inclusion and exclusion criteria."Cinahl" is badly written.In the summary you duplicate the explanation of the Critical Appraisal Tool version 1.4 (CCAT) acronym.The acronyms ICUs and PRISMA are not explained.
Keywords: For proper indexing of the article once it is published, it is appropriate to use keyword descriptor terms (MesH)."Nursing Care Management" is not a MesH.("Nursing Care" is a MesH; there are several terms that include Management that can also be used.Regarding the two terms "Personnel Staffing", "Scheduling Information Systems": there is a single descriptor that can include both "Personnel Staffing and Scheduling Information Systems".
Introduction: The introduction is very sparse; content on Technological innovation and nurses workload could be included.
Methods: It is not understood that you have established among the inclusion criteria searches until the year 2020.Three years have passed since the date of the searches and in that period new studies of interest could have been published.It is a quality criterion to update the searches when at least one year has passed since they were performed if the manuscript has not been published in that period.
Results: The results should be limited to the extraction of data from the studies without interpretation to be made in the discussion.The quality criteria of the studies should be explained in detail in the methods section.When looking at the results in Table 1, it is not possible to interpret the criterion in the last column: Excellent, some limitations... Characteristics of the studies: in the description of the number of studies included, you must unify criteria to express them in numbers or in letters (example: four vs. 4).The acronym ICUs is explained in results (heading: Predictors of nursing burnout and outcomes; last paragraph of this section), but the acronym has been used previously in the same paragraph.Idem EMRs acronym.

Angela María Salazar Maya
Universidad de Antioquia, Medellín, Colombia It is a relevant issue for nursing as a discipline and profession, since evidence is required on how the workload influences the results of the patients cared for.
The search was only carried out until the year 2020, it would be pertinent to extend the search until at least 2022 or the first half of 2023 so that the integrative review is more up-to-date.
The paper presents the 4 categories that emerge from the analysis: workload models, burnout predictors, and outcomes; information technology and the meaning of technology for management; technology acceptance.develop each category.
I consider the discussion and conclusions pertinent.
In a brief search carried out in PubMed, I find articles on the subject.

Strengths
Among the strengths of the review is that it contributes to a need for summative evidence on aspects of nursing care management that had not been identified as influential, such as nursing workload.It recognizes current elements such as the use of technologies for nursing care and concludes that nurses need to work collaboratively with software programmers for the development of new technologies.
Another important aspect is the identification of at least some existing systems, especially in critical care, which could be taken up again to compare their use in future studies.
The review also validates with more evidence that excessive nursing workload has negative impacts on the health of the people cared for, which is a useful manuscript for nurse managers to justify institutional changes.
In nursing, real-time patient assessment has always been key, which would allow the development of schemes with technology; however, it highlights that causality is not the only way of nursing care, and qualitative aspects of care that are performed with humanization and compassionate care should be contemplated; this gives a comprehensive paradigm to the reader of the nursing role.

Weaknesses
The article could improve its presentation of aspects that may be justifiable from the methodology used, such as answering the question: Why did not they use the new PRISMA 2020 scheme?For an integrative review, it should not necessarily be used, but in the methodology, this methodology is used, but it is not mentioned what is or is not used and why.It is not possible to identify the elements suggested in the PRISMA 2020 parameters, regarding aspects such as the date on which each resource was last consulted.It is not clear whether the authors were involved in the screening stages in the selection or only in the phase of reading abstracts.
Table 1 has very valuable information for readers; however, one could be more assertive in the presentation by having more heterogeneity in the presentation of the results in the Data Collection Methods and Results column; it may be that having been elaborated by different authors of the article or supporting persons, the presentation of the information varies too much.Now, it may have been the effect of the limitation they presented in the manuscript that it was too heterogeneous, which they could specify there and cite as an example in Table 1, where this can be seen.

Improvements to the article
The manuscript can also be improved with aspects such as updated references.In the introduction, they have a citation from 2011 (12 years' difference), on a topic that could have more recent references.
The discussion could include aspects of recently published discussions (last two years until 2023) since they were limited to the same studies of the review.
I suggest being more precise in stating the research gaps identified, which may guide readers on topics of research interest, lacking this specification.
Finally, from origin of the article, I suggest reviewing if the word "innovation" is pertinent in the article, since it relates it to the topic of technologies that is in the research questions; however, not necessarily when talking about technology it is associated to an innovation, a technology can be an artifact or a technique; while innovation is to create or modify a product, but it is not what was sought in the review nor was it what was found in the studies.

Final conclusion of the concept
The manuscript is very good in its content, richness, rigorous work, concise, and useful for care management.It can be improved with precisions and updates typical of literature reviews.I congratulate the authors for the phenomenon addressed and the possibility of having this type of review.For all research, this must include a final section including details of ethical approval, informed consent and, where relevant registration.

Is
Table 1 has very valuable information for readers; however, one could be more assertive in the presentation by having more heterogeneity in the presentation of the results in the Data Collection Methods and Results column; it may be that having been elaborated by different authors of the article or supporting persons, the presentation of the information varies too much.Now, it may have been the effect of the limitation they presented in the manuscript that it was too heterogeneous, which they could specify there and cite as an example in Table 1, where this can be seen.○ Answer: We conclude that "The main research gap that we identify is the lack of studies showing evidence of the impact of technology measured as the correlation between the nursing outcomes and staff well-being with the use of workload estimation and the strategies for planning workload allocation".This conclusion is presented in the final section of the paper.
I suggest reviewing if the word "innovation" is pertinent in the article, since it relates it to the topic of technologies that is in the research questions; however, not necessarily when talking about technology it is associated to an innovation, a technology can be an artifact or a technique; while innovation is to create or modify a product, but it is not what was sought in the review nor was it what was found in the studies.
○ Answer: We declare as a limitation of the study that the literature review did not focus on studying the concept of innovation in nursing practice or workload management as in Rylee and Cavanagh (2022) for nursing practice.
We thank the reviewer for the valuable comments made on version 1 of the manuscript.We answer the reviewers' comments as follows: Why did not they use the new PRISMA 2020 scheme?For an integrative review, it should not necessarily be used, but in the methodology, this methodology is used, but it is not mentioned what is or is not used and why.It is not possible to identify the elements suggested in the PRISMA 2020 parameters, regarding aspects such as the date on which each resource was last consulted.It is not clear whether the authors were involved in the screening stages in the selection or only in the phase of reading abstracts.
○ Answer: The first paragraph of the Methods section provides the details of the methodology used that corresponds to the PRISMA 2020 guidelines as suggested by the reviewer.The corresponding paragraph is as follows: An integrative literature review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guidelines.Studies were retrieved from SCOPUS, Scielo, PUBMED, and CINAHL in January 2021.The search strategy was: ("analytics" OR "operations research" OR digital* OR "IT" OR technolog*) AND ("nursing workload" OR "nursepatient ratio" OR "nursing workforce planning" OR "nursing algorithm" OR "nursing personnel staffing and scheduling" OR "nursing staff")).The authors were involved in the identification, screening, and analysis stages.Inclusion criteria considered studies published in full text in English, Spanish or Portuguese classified as scientific articles or reviews, in the areas of nursing, medicine, social sciences, engineering, published between January 2016 and December 2020.This date range was considered to focus on the recent technologies.Articles related with technology that didn't analyse the impact on nursing workload or didn't present technologies were excluded.For all research, this must include a final section including details of ethical approval, informed consent and, where relevant registration.
Table 1 has very valuable information for readers; however, one could be more assertive in the presentation by having more heterogeneity in the presentation of the results in the Data Collection Methods and Results column; it may be that having been elaborated by different authors of the article or supporting persons, the presentation of the information varies too much.Now, it may have been the effect of the limitation they presented in the manuscript that it was too heterogeneous, which they could specify there and cite as an example in Table 1, where this can be seen.Handayani et al. (2018), De Oliveira, Garcia, Nogueira (2016).The records on this new table are more homogeneous.Nevertheless, we still acknowledge the study limitation about the heterogeneous methodologies in the retrieved studies and we cite the example in the limitations.
In the introduction, they have a citation from 2011 (12 years' difference), on a topic that could have more recent references.The discussion could include aspects of recently published discussions (last two years until 2023) since they were limited to the same studies of the review.

○
Answer: It is true that our discussion section is built upon the studies that were retrieved by the proposed systematic review.In fact, we aimed to draw conclusions appropriate in the context of the current research literature and minimize bias and lack of rigor (Hopia et al., 2016).In fact, we aimed to avoid common pitfalls in integrative literature reviews that are described by Toronto and Remington ( 2020 I suggest being more precise in stating the research gaps identified, which guide readers on topics of research interest, lacking this specification.
○ Answer: We conclude that "The main research gap that we identify is the lack of studies showing evidence of the impact of technology measured as the correlation between the nursing outcomes and staff well-being with the use of workload estimation and the strategies for planning workload allocation".This conclusion is presented in the final section of the paper.
I suggest reviewing if the word "innovation" is pertinent in the article, since it relates it to the topic of technologies that is in the research questions; however, not necessarily when talking about technology it is associated to an innovation, a technology can be an artifact or a technique; while innovation is to create or modify a product, but it is not what was sought in the review nor was it what was found in the studies.
○ Answer: We declare as a limitation of the study that the literature review did not focus on studying the concept of innovation in nursing practice or workload management as in Rylee and Cavanagh (2022) for nursing practice.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com

Figure 1 .
Figure 1.Flow chart of the study screening process.

Figure 2 .
Figure 2. Concept map of the studies' categories and results.

Discussion: 1 Reviewer
The discussion is poor, only citing 5 studies.As I have pointed out in the previous comment, you must transfer content from the results to the discussion.The discussion should follow the structure established in the results.Is the topic of the review discussed comprehensively in the context of the current literature?Partly Are all factual statements correct and adequately supported by citations?Partly Is the review written in accessible language?Partly Are the conclusions drawn appropriate in the context of the current research literature?Yes Competing Interests: No competing interests were disclosed.Reviewer Expertise: Nursing I confirm that I have read this submission and believe that I have an appropriate level of expertise to state that I do not consider it to be of an acceptable scientific standard, for reasons outlined above.Version Report 10 August 2023 https://doi.org/10.5256/f1000research.137724.r178450© 2023 Salazar Maya A. This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Table 1 .
Purpose, design, participants and context, data collection method and outcome of each of the 42 studies included in the review.

the topic of the review discussed comprehensively in the context of the current literature? Yes Are all factual statements correct and adequately supported by citations? Yes Is the review written in accessible language? Yes Are the conclusions drawn appropriate in the context of the current research literature? Yes Competing Interests:
No competing interests were disclosed.

Is the topic of the review discussed comprehensively in the context of the current literature? Yes Are all factual statements correct and adequately supported by citations? Yes Is the review written in accessible language? Yes Are the conclusions drawn appropriate in the context of the current research literature? Yes Competing Interests:
No competing interests were disclosed.

have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.
https://doi.org/10.5256/f1000research.137724.r178447

the topic of the review discussed comprehensively in the context of the current literature? Yes Are all factual statements correct and adequately supported by citations? Yes Is the review written in accessible language? Yes Are the conclusions drawn appropriate in the context of the current research literature? Yes Competing Interests:
No competing interests were disclosed.

confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Answer: The first paragraph of the Methods section provides the details of the methodology used that corresponds to the PRISMA 2020 guidelines as suggested by the reviewer.The corresponding paragraph is as follows: Spanish or Portuguese classified as scientific articles or reviews, in the areas of nursing, medicine, social sciences, engineering, published between January 2016 and December 2020.This date range was considered to focus on the recent technologies.Articles related with technology that didn't analyse the impact on nursing workload or didn't present technologies were excluded.
The records on this new table are more homogeneous.Nevertheless, we still acknowledge the study limitation about the heterogeneous methodologies in the retrieved studies and we cite the example in the limitations.It is true that our discussion section is built upon the studies that were retrieved by the proposed systematic review.In fact, we aimed to draw conclusions appropriate in the context of the current research literature and minimize bias and lack of rigor(Hopia et al.,  2016).In fact, we aimed to avoid common pitfalls in integrative literature reviews that are described byToronto and Remington (2020)the document entitled: "A Step-by-Step Guide to Conducting an Integrative Review".The pitfalls are: Drawing conclusions or formulating implications that cannot be supported by the literature, Hopia, H., Latvala, E., & Liimatainen, L. (2016).Reviewing the methodology of an integrative review.Scandinavian journal of caring sciences, 30(4), 662-669.Toronto, C. E., & Remington, R. (Eds.).(2020).A step-by-step guide to conducting an integrative review (pp.1-9).Cham: Springer International Publishing.I suggest being more precise in stating the research gaps identified, which may guide readers on topics of research interest, lacking this specification.