Healthcare stakeholders' perceptions and experiences of factors affecting the implementation of critical care telemedicine (CCT): qualitative evidence synthesis.

BACKGROUND
Critical care telemedicine (CCT) has long been advocated for enabling access to scarce critical care expertise in geographically-distant areas. Additional advantages of CCT include the potential for reduced variability in treatment and care through clinical decision support enabled by the analysis of large data sets and the use of predictive tools. Evidence points to health systems investing in telemedicine appearing better prepared to respond to sudden increases in demand, such as during pandemics. However, challenges with how new technologies such as CCT are implemented still remain, and must be carefully considered.


OBJECTIVES
This synthesis links to and complements another Cochrane Review assessing the effects of interactive telemedicine in healthcare, by examining the implementation of telemedicine specifically in critical care. Our aim was to identify, appraise and synthesise qualitative research evidence on healthcare stakeholders' perceptions and experiences of factors affecting the implementation of CCT, and to identify factors that are more likely to ensure successful implementation of CCT for subsequent consideration and assessment in telemedicine effectiveness reviews.


SEARCH METHODS
We searched MEDLINE, Embase, CINAHL, and Web of Science for eligible studies from inception to 14 October 2019; alongside 'grey' and other literature searches. There were no language, date or geographic restrictions.


SELECTION CRITERIA
We included studies that used qualitative methods for data collection and analysis. Studies included views from healthcare stakeholders including bedside and CCT hub critical care personnel, as well as administrative, technical, information technology, and managerial staff, and family members.


DATA COLLECTION AND ANALYSIS
We extracted data using a predetermined extraction sheet. We used the Critical Appraisal Skills Programme (CASP) qualitative checklist to assess the methodological rigour of individual studies. We followed the Best-fit framework approach using the Consolidated Framework for Implementation Research (CFIR) to inform our data synthesis.  We classified additional themes not captured by CFIR under a separate theme. We used the GRADE CERQual approach to assess confidence in the findings.


MAIN RESULTS
We found 13 relevant studies. Twelve were from the USA and one was from Canada. Where we judged the North American focus of the studies to be a concern for a finding's relevance, we have reflected this in our assessment of confidence in the finding. The studies explored the views and experiences of bedside and hub critical care personnel; administrative, technical, information technology, and managerial staff; and family members. The intensive care units (ICUs) were from tertiary hospitals in urban and rural areas. We identified several factors that could influence the implementation of CCT. We had high confidence in the following findings: Hospital staff and family members described several advantages of CCT. Bedside and hub staff strongly believed that the main advantage of CCT was having access to experts when bedside physicians were not available. Families also valued having access to critical care experts. In addition, hospital staff described how CCT could support clinical decision-making and mentoring of junior staff.  Hospital staff greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and effective communication between the two teams. Interactions between some bedside and CCT hub staff were featured with tension, frustration and conflict. Staff on both sides commonly described disrespect of their expertise, resistance and animosity. Hospital staff thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering staff opportunities to ask questions and disseminating learning resources. Some also complained that experienced staff were taken away from bedside care and re-allocated to the CCT hub team. Hospital staff's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Staff were positive towards CCT because of its several advantages. But some were concerned that the CCT hub staff were not able to understand the patient's situation through the camera. Some were also concerned about confidentiality of patient data. We also identified other factors that could influence the implementation of CCT, although our confidence in these findings is moderate or low. These factors included the extent to which telemedicine software was adaptable to local needs, and hub staff were aware of local norms; concerns about additional administrative work and cost; patients' and families' desire to stay close to their local community; the type of hospital setting; the extent to which there was support from senior leadership; staff access to information about policies and procedures; individuals' stage of change; staff motivation, competence and values; clear strategies for staff engagement; feedback about progress; and the impact of CCT on staffing levels.


AUTHORS' CONCLUSIONS
Our review identified several factors that could influence the acceptance and use of telemedicine in critical care. These include the value that hospital staff and family members place on having access to critical care experts, staff access to sufficient training, and the extent to which healthcare providers at the bedside and the critical care experts supporting them from a distance acknowledge and respect each other's expertise. Further research, especially in contexts other than North America, with different cultures, norms and practices will strengthen the evidence base for the implementation of CCT internationally and our confidence in these findings. Implementation of CCT appears to be growing in importance in the context of global pandemic management, especially in countries with wide geographical dispersion and limited access to critical care expertise. For successful implementation, policymakers and other stakeholders should consider pre-empting and addressing factors that may affect implementation, including strengthening teamness between bedside and hub teams; engaging and supporting frontline staff; training ICU clinicians on the use of CCT prior to its implementation; and ensuring staff have access to information and knowledge about when, why and how to use CCT for maximum benefit.


Moderate confidence
Downgraded to moderate confidence because of minor concerns about methodological limitations, coherence, and adequacy; and moderate concerns about relevance Finding 2: Hospital sta and family members described several advantages of CCT. Bedside and hub sta strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to critical care experts. In addition, hospital sta described how CCT could support clinical decision-making and mentoring of junior sta

High confidence
Graded as high confidence because of no or very minor concerns about methodological limitations, relevance, coherence, and adequacy Low confidence Downgraded to low confidence because of minor concerns about coherence; moderate concerns about relevance; and serious concerns about adequacy Finding 4: Both bedside and hub clinicians expressed difficulties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more co-ordination work, and concern around the presence of cameras

Moderate confidence
Downgraded to moderate confidence because of moderate concerns about methodological limitations, relevance, and adequacy Finding 9: Bedside clinicians were reluctant to use CCT because they lacked clarity about its purpose, were concerned that their decision-making skills would be weakened through remote supervision, and did not consider hub clinicians an equal counterpart in patient management. Hub clinicians were disengaged due to lack of role clarity and limited integration with patient care Kahn 2019; Moeckli 2013; Shahpori 2011a; Sta ord 2008a

Moderate confidence
Downgraded to moderate confidence because of minor concerns about methodological limitations, and adequacy; and moderate concerns about relevance.
Finding 10: Hospital locale shaped prioritisation of CCT, with sta in rural centres noting that CCT was of greater benefit to them considering their sta shortage and lack of critical care resources Kahn 2019; Shahpori 2011a; Ward 2015; Wilkes 2016

Low confidence
Downgraded to low confidence because of moderate concerns about methodological limitations, relevance, and coherence; and serious concerns about adequacy Finding 11: Bedside and hub clinicians perceived the absence of support from, and lack of engagement in dialogue with leaders and senior administrators during the implementation of CCT as major barriers. Listening to sta needs, and creating groundwork connections with them from the outset were perceived as facilitating factors to implementation

Kahn 2019; Wilkes 2016
Low confidence Downgraded to low confidence because of minor concerns about methodological limitations; moderate concerns about relevance; and serious concerns about adequacy Finding 12: Hospital sta expressed it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering sta opportunities to ask questions and disseminating learning resources. Some also complained that experienced sta were taken away from bedside care and re-allocated to the CCT hub team

Moderate confidence
Downgraded to moderate confidence of minor concerns about coherence, and adequacy; and moderate concerns about relevance

Other factors affecting implementation
Finding 19: Hospital sta highlighted that CCT can support ICUs to overcome challenges associated with sta shortages especially during nights and weekends, and in rural hospitals where ICU nurses are assigned to different departments; and with retaining physicians and nurses. Some concerns over the potential negative impact of CCT on overall sta ing levels were also expressed Goedken 2017; Hoonakker 2013; Kahn 2019; Shahpori 2011a

Moderate confidence
Downgraded to moderate confidence because of minor concerns about relevance; moderate concerns about methodological limitations, and adequacy Finding 20: Interactions between some bedside and CCT hub sta were featured with tension, frustration and conflict. Sta on both sides commonly described disrespect of expertise, resistance and animosity High confidence Graded as high confidence because of no or very minor concerns about coherence and adequacy; minor concerns about relevance; and moderate concerns about methodological limitations

B A C K G R O U N D
International interest in the benefits and implementation of telemedicine in a variety of settings and for di erent conditions is growing fast, as evidenced by the recently published Cochrane intervention review (Flodgren 2015) and Cochrane qualitative evidence synthesis protocol (Odendaal 2020). This is especially the case in the care of critically-ill people. The burden of critical illness is higher than is generally appreciated, and is expected to increase as a result of global population ageing (Adhikari 2010; Vincent 2014). Consequently, critical care services in major hospitals are stretched, while smaller hospitals and rural areas have limited access to relevant expertise (Wunsch 2008). In addition, critical care is challenged by inconsistent application of evidence-based guidelines, variation in sta ing levels and clinical outcomes, higher rates of medication errors and adverse drug events (Pronovost 2004;Rothchild 2005), all of which are aggravated by the unpredictable nature of patient conditions, the urgent nature of many admissions to critical care and the need for out-of-hours decision-making. For the purposes of this review, we define critical care as the concentration of healthcare sta and equipment in a distinct area of the hospital in order to care for people whose conditions are life-threatening and who need constant and close monitoring and support.

Description of the topic
Telemedicine has been broadly defined by the World Health Organization (WHO) as: "the delivery of health care services, where distance is a critical factor, by all health care professionals using information and communication technologies for the exchange of valid information for the diagnosis, treatment and prevention of disease and injuries, research and evaluation, and for the continuing education of health care providers, all in the interests of advancing the health of individuals and their communities" (WHO 2010). Critical care telemedicine (CCT) in particular enables a team of critical care doctors and nurses to provide 24-hour remote support to clinicians using audio-visual communication and computer systems. In 2014, it was estimated that 8% of total intensive care unit (ICU) beds in the USA were covered by CCT, with an average growth rate of 8% a year (Khan 2014).
CCT o ers minute-by-minute monitoring and recording of vital organ function, making use of electronic records and remote surveillance in order to facilitate early detection and response to physiological deterioration. In addition, the integration of decisionsupport tools and early-warning systems supports adherence to clinical guidelines, which can level out variations in quality of care. Further advantages of CCT for stakeholders may include additional support for junior sta , with patients and families feeling looked a er. Consequently, CCT has potential to improve clinical outcomes beyond the confines of the ICU for people who may benefit from critical care expertise but are not based in specialist units; for example, they may be in an emergency department, generic ICU or medical/surgical ward. This is possible by extending the availability and reach of critical care expertise through a huband-spoke model, adding a safety net to ward-based and nonspecialist bedside providers.
The hub-and-spoke model of CCT is used in the context of multilocation delivery of critical care services. A remotely-based team of senior and experienced critical care clinicians -called the hub -is networked through audio-visual communication and telemonitoring systems with a number of bedside terminals, clinicians and patients. The hub acts as a single point of contact for critical care advice and support, while through seamless extensions -called spokes -hands-on patient care is provided across multiple locations. In a wider role, the hub can also take on co-ordinating responsibilities, including patient flow through ICUs, brokering admission and discharge of patients, as well as quality, risk and performance management through early-warning capabilities, rounding tools to monitor at-risk patients, inbuilt clinical decision support and prompts for adherence to best practice. In summary, CCT includes the following functionality: synchronous, interactive client-to-provider telemedicine; telemonitoring; client health records; provider-to-provider telemedicine; provider-based decision support; laboratory and diagnostics management; data collection, management and use.
CCT is designed as a continuous form of clinical support to bedside practice, enabling clinical oversight and interactions between providers. In this way, it is distinct from other telemedicine models that mainly o er an interface for sporadic consultation between providers and patients in remote locations, or between generalist and specialist clinicians. Critical care patients' condition can be unstable, can deteriorate unexpectedly and quite rapidly, requiring close monitoring and prompt reaction by a multidisciplinary team of expert clinicians, there and then. As a consequence, critical care services tend to have increased organisational autonomy, resources and sta ing levels compared to other areas of the hospital. These unique features of critical care practice can influence professionals' perceptions, experience and use of CCT, all of which can a ect successful implementation.

How the intervention might work
The implementation of new technologies in healthcare settings is beset with multiple challenges. Reports on the failure of widelyaccepted and seemingly di used health technologies to become embedded in daily practise are commonplace in the literature, even where these have support by both clinicians and politicians (May 2000). To understand where the implementation of such technologies fail, a strong theoretical foundation is needed to guide the evaluation of such programmes. Use of implementation theory can help generate explanatory models and hypotheses about factors influencing implementation of health technologies, leading to the identification of approaches more likely to result in successful implementation.
For the purpose of this review, we will use the Consolidated Framework for Implementation Research (CFIR) (Damschroder 2009) to theoretically conceptualise data from the included studies and to guide the data analysis. CFIR is a 'meta-theoretical' model, made up of constructs generated out of a synthesis of existing theories; one of its strengths and unique features is that it does not depict rigid interrelationships, specific ecological levels, or specific hypotheses. This allows for theory development guided by exploratory questions such as what works, where and why across di erent contexts. The CFIR has been used successfully in reviews of eHealth and is found to o er great theoretical and explanatory capabilities (Ross 2016).
CFIR is composed of five key constructs, each made up of di erent factors that a ect the implementation of innovations into practice (see Appendix 1). In summary, the five key constructs of the CFIR are: The first construct, Intervention characteristics , refers to features of the intervention including its source, evidence base, advantage over other interventions, the extent of its adaptability, 'trialability' and complexity, as well as its quality and cost. The second and third constructs, Inner and Outer Settings , relate to the internal and external environment in which implementation occurs. For example, the inner setting is about features of the structural, political and cultural organisation contexts through which the implementation process takes place; while the outer setting relates to the economic, political and social context within which the organisation resides. The fourth construct refers to the Characteristics of the Individuals who engage with the intervention or the implementation process. Individuals' knowledge and beliefs about the intervention, their self-e icacy, personal attributes and identification with the organisation play a key part in the success or failure of the implementation process. The final construct relates to the Implementation process itself, which includes elements of planning, engaging with leaders, champions and change agents, carrying out the implementation plan and evaluating the process and experience.
Operationalising the CFIR as an organising framework in the context of this qualitative evidence synthesis allows for a theoretically informed approach to data extraction, analysis and synthesis; helps with the interpretation of results; and strengthens the theoretical transferability and comparability of conclusions. At the same time, it allows for testing of the CFIR and consequent elaboration in the context of telemedicine in general, and CCT in particular.

Why it is important to do this review
Cochrane Reviews (Flodgren 2015) on the use of telemedicine indicate that answering questions about its e icacy requires attention to the contextual features of its application, including participants and settings. E ectiveness reviews of CCT in particular report a great degree of variability in e ectiveness (e.g. Young 2011a), likely related to challenges with successful implementation (Thomas 2009). For example, Wilcox 2012 concluded that "the impact of telemedicine likely depends on characteristics of the environment in which it is deployed, including ICU organisation"; however, existing quantitative studies report limited contextual details. Currently, adoption of CCT appears haphazard and unplanned, and decision-making about this lies hidden; this risks patient safety, quality of care and resource waste. Before such complex interventions are to be further developed and implemented, a more complete understanding of the factors that influence successful implementation is necessary (Glenton 2013). These include the perceptions, experiences and values of relevant stakeholders, as well as usability and applicability in di erent contexts.
It is therefore important to complement existing e ectiveness reviews on CCT with a qualitative evidence synthesis that enables understanding of the factors a ecting successful implementation, and illuminates the unintended consequences, acceptability and feasibility of CCT. This is especially important given that, despite a lack of conclusive evidence, there has been a rapid uptake of CCT in North America; and considering that the 24/7 hub-and-spoke model of CCT may have reach beyond critical care -Critical Care Outreach and Emergency Departments, for example -and in this way has great potential to transform the provision, quality and safety of acute care across hospital settings in the future.

How this review might inform or supplement what is already known in this area
This qualitative evidence synthesis addresses a subset of the Flodgren 2015 e ectiveness review on interactive telemedicine. By looking at CCT in particular; it will complement Flodgren 2015 by providing an added layer of knowledge that can enable a more nuanced understanding of the factors influencing implementation of CCT. It also complements the Cochrane qualitative evidence synthesis of experiences of mHealth technologies in primary health care (Odendaal 2020), since critical care represents the acute far end of the health system and the opposite pole to primary care. In addition, CCT is distinct as an application from the traditional models of mHealth, which rely on mobile technology, used in primary care, since it uses a hub-and-spoke model to provide a 24/7 continuous form of clinical support to bedside practice rather than just being an interface for sporadic communication between patients and providers.

O B J E C T I V E S
To identify, appraise and synthesise qualitative research evidence on healthcare stakeholders' perceptions and experiences of factors a ecting the implementation of CCT, and to identify factors that are more likely to ensure successful implementation of CCT for subsequent consideration and assessment in telemedicine e ectiveness reviews.

Type of studies
We included empirical studies that used qualitative designs and methods for data collection and analysis. These included ethnographic studies using participant observation and phenomenological studies using interviews. We considered studies using mixed designs where the qualitative component and findings could be discerned; we also considered qualitative process evaluations as well as formative studies used to inform the design of CCT where the previous statement applied. We included studies regardless of whether these were linked to e ectiveness studies of CCT. We excluded studies that used qualitative data-collection methods but performed quantitative data analysis (e.g. using descriptive statistics). We considered both published and unpublished studies and studies published in any language. We did not exclude studies based on our assessment of methodological limitations, but used this information to assess our confidence in the review findings.

Study participants
We considered all relevant stakeholders with a part to play in the implementation of CCT, including: • All kinds of critical care workers (i.e. professionals, paraprofessionals and lay health workers) who make use of telemedicine to support or provide care to patients or family members, or both. Critical care workers are the main users of CCT and/or are the ones whose daily work is influenced to various degrees by the introduction of CCT. Their views about acceptance, resistance to or rejection of CCT are likely to be a contributing factor to implementation success or failure. • Any other individuals or groups involved in the commissioning, evaluation, design and implementation of CCT. These individuals or groups can include administrative sta , information technology sta , managerial and supervisory sta , and industry partners who may or may not be based in a critical care facility, but must be involved in the use or implementation of CCT. We also considered participants identified as the technical sta who develop and maintain the CCT architecture used, since it is their logic and understanding of critical care services that underpin the final product at the point of use. • Critical care patients and family members who have been the consumers or been involved in the development of CCT. As the recipients of care mediated by CCT, their views are likely to hold insight into factors influencing successful implementation.

Study settings
We included studies of telemedicine programmes implemented in critical care services, irrespective of specialisation (e.g. general, cardiothoracic, liver), or country. For the purposes of this review, we define critical care as the concentration of healthcare sta and equipment in a distinct area of the hospital in order to care for people whose conditions are life-threatening and who need constant and close monitoring and support. Critical care services provide intensive 24-hour monitoring and support of threatened or failing vital functions in people who have illnesses with the potential to endanger life.

CCT interventions
This review focuses on healthcare stakeholders' perceptions and experiences of factors a ecting the implementation of CCT; we considered studies that looked at either the initiation or ongoing delivery of CCT. For the purposes of this review, CCT consists of the following combination: • laboratory and diagnostics management, and patient health records including the continuous electronic recording of patients' vital signs at the bedside linked to a computer system enabling display of real-time data; • provider-based decision support, in the form of clinical decisionmaking algorithms and electronic alerts; and • synchronous, interactive provider/client to provider telemedicine, using a remotely-located team of critical care specialists, including doctors and nurses, who monitor the patients.
We required the presence of all three features to identify an intervention as CCT. We did not consider CCT applications that excluded clinical decision-making as in some forms of plain remote screening.

Search methods for the identification of studies Electronic searches
The EPOC Information specialist helped develop the MEDLINE search strategy in consultation with the review authors. We used the following databases to identify primary research studies for inclusion. •

Searching other resources
We sought related reviews through PDQ-Evidence (www.pdqevidence.org, searched 14 October 2019), the reference lists of which we scanned for relevant studies. We also searched the reference lists of all included studies.

Library
Trusted evidence. Informed decisions. Better health.
Cochrane Database of Systematic Reviews 2019. However, this was not necessary, since we did not find studies in languages other than English.

Sampling of studies
We acknowledge that qualitative evidence synthesis aims for variation in concepts rather than an exhaustive sample, and large amounts of study data can impair the quality of the analysis. Therefore, once we identified all the studies eligible for inclusion, we assessed whether their number or data richness were likely to represent a problem for the analysis, and considered selecting a sample of studies. For the purposes of this review, and given the limited literature on the topic of CCT, we decided against sampling and instead included all the eligible articles.

Data extraction
At least two review authors extracted key features of the included papers independently, using a predetermined table to include: author(s), year, country, hospital type, ICU model and sta ing, CCT system and vendor, study design, data collection and participants. We also extracted data on stakeholders' perceptions and experiences of factors a ecting the implementation of CCT; this included authors' interpretations as well as actual data in the form of quotes or field-note extracts. We considered data presented in either the Results or Discussion sections of the articles.

Appraisal of the methodological limitations of included studies
Two review authors (AX, KI) independently applied a predetermined set of quality criteria to each of the included studies, based on the Critical Appraisal Skills Programme (CASP) quality assessment tool for qualitative studies (CASP 2013). We considered all eligible studies, irrespective of quality. In cases of disagreement between the two review authors, a third member of the team (JP) was invited to adjudicate. We assessed methodological limitations according to the following questions: • Was there a clear statement of the aims of the research?
• Is a qualitative methodology appropriate?
• Was the research design appropriate to address the aims of the research? • Was the recruitment strategy appropriate to the aims of the research? • Were the data collected in a way that addressed the research issue? • Has the relationship between the researcher and participants been adequately considered? • Have ethical issues been taken into consideration?
• Was the data analysis su iciently rigorous?
• Is there a clear statement of findings?
We report our assessment in a Methodological Limitations

Data management, analysis and synthesis
We imported all the included papers into the NVivo qualitative data analysis so ware (QSR International). Data synthesis drew from the CFIR framework (Appendix 1) to examine the available evidence on factors a ecting the implementation of CCT. As noted in the Background, the CFIR is a 'meta-theoretical' model, made up of five constructs: I. Intervention characteristics, II. Inner and III. Outer Settings, IV. Characteristics of individuals, and V. the Process of implementation. CFIR informed but did not restrict data synthesis, with additional themes not captured by CFIR used to challenge and add to previously-held assumptions. This approach led to a more refined understanding of implementation in the context of CCT, building on and extending the propositions of CFIR, thus strengthening the theoretical generalisability of the review findings.
We followed the Best-fit framework approach (Carroll 2013), since this allows examination of the alignment of identified themes with an existing framework, as well as conceptual revisions as necessary. Our approach consisted of four main analysis stages completed by two review authors (AX, KI): First, we developed a coding tree in NVivo based on the CFIR framework and coded data from the included studies against this. Second, themes not accounted for by CFIR were noted, coded and classified under separate constructs. Third, following a consensus approach, we used additional constructs to supplement CFIR; had the framework changed substantially, the papers would be re-coded based on the new framework, but this was not required. Fourth, we revisited the data to explore relationships between themes and constructs in order to develop concise review findings statements that capture the coded data.

Assessing our confidence in the review findings
Two review authors (AX, KI) used the GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research) approach to assess our confidence in each finding (Lewin 2018). CERQual assesses confidence in the evidence, based on the following four key components.
• Methodological limitations of included studies: the extent to which there are concerns about the design or conduct of the primary studies that contributed evidence to an individual review finding • Coherence of the review finding: an assessment of how clear and cogent the fit is between the data from the primary studies and a review finding that synthesises those data. By cogent, we mean well-supported or compelling • Adequacy of the data contributing to a review finding: an overall determination of the degree of richness and quantity of data supporting a review finding • Relevance of the included studies to the review question: the extent to which the body of evidence from the primary studies supporting a review finding is applicable to the context (perspective or population, phenomenon of interest, setting) specified in the review question A er assessing each of the four components, we made a judgement about the overall confidence in the evidence supporting the review finding. We judge confidence as high, moderate, low, or very low. The final assessment was based on consensus among the review authors. All findings started as high confidence and were then downgraded if there were important concerns about any of the CERQual components. The starting point of high confidence reflected a view that each review finding should be seen as a reasonable representation of the phenomenon of interest, unless there were factors that weakened this assumption. Cochrane Database of Systematic Reviews

Summary of qualitative findings table and Evidence Profiles
We present summaries of the findings and our assessments of confidence in these findings in Summary of findings 1. We present detailed descriptions of our confidence assessment in a twopart Evidence Profile (Table 2; Table 3, Additional Tables).

Integrating the review findings with Cochrane intervention review
Findings are used to complement and contextualise the conclusions of the Flodgren 2015 Cochrane intervention review on interactive telemedicine by looking at implementation of CCT in particular. Using a narrative approach, we explored how the CCT implementation factors identified in our findings could explain or influence the outcomes of interactive telemedicine as identified by Flodgren 2015. To achieve this, we initially listed our findings alongside outcomes by Flodgren 2015 and considered how these might interact. Then, informed by standard implementation and improvement science approaches (IHI 2012), we grouped our implementation factors under process, structure and balancing measures that could have a role to play in influencing CCT outcomes. Finally, we presented these in a model with a view to informing hypothesis-testing in future CCT research. At least two review authors worked together on this (AX, KI). The resulting model helps to partly explain the variability seen in the e ectiveness of telemedicine in general, and CCT in particular, identifying factors that need to be considered in future trials and informing the development of future CCT interventions and evaluations.

Review author reflexivity
We maintained a reflexive stance throughout the stages of the review process, from study selection to data synthesis. The review authors discussed progress and made decisions critically. As a review team, we all have clinical backgrounds: in nursing (AX, NM, SB, JP, KI), medicine (MT) and midwifery (JS). In addition, four review authors have received advanced training in implementation science (NM, SB, JS, KI) and are well-versed in relevant theory. NM, AX, MT and JS have been part of a project examining the implementation of CCT at a UK site, but SB, JP and KI are independent of that research. Based on our collective and individual experiences (as clinicians, academics and researchers), we anticipated the findings of our review would reveal a combination of organisational, professional and individual factors influencing the implementation of CCT. Moreover, while the evidence of e ectiveness for CCT is uncertain, we believe that CCT could be e ective if properly implemented. We have remained mindful of our presuppositions, and supported each other to minimise the risk of these skewing our analysis or the interpretation of our findings. As the lead author, AX kept a reflexive journal throughout the review process in which he documented and reflected on progress and decisions made.

Results of the search
We included 13 studies in this synthesis. These were published between December 2008 and April 2019, and were the only studies that met our inclusion criteria ( Figure 1). Cochrane Database of Systematic Reviews

Type of hospitals
Twelve studies ( Wilkes 2016) were conducted in the USA. One study was conducted in Canada (Shahpori 2011a).
One study was conducted in a metropolitan hospital (Sta ord 2008a). One study was conducted in a tertiary hospital (Shahpori 2011a). Three studies were conducted in tertiary-care medical centres, small urban medical centres and rural hospitals (Goedken 2017; Moeckli 2013; Thomas 2017).
One study was conducted in one academic tertiary referral hospital, and two community hospitals (Mullen-Fortino 2012). One study was conducted in 10 prospective payment system (PPS) hospitals, and 18 critical access hospitals (CAHs), 13 of which were rural hospitals (Ward 2015). One study was conducted in three rural a iliated hospitals with a mix of critical access and prospective payment system hospitals (Wilkes 2016). Five studies did not specify the type of hospital (Hoonakker 2013; Hoonakker 2018; Jahrsdoerfer 2013; Kahn 2019; Khunlertkit 2013).

CCT system and vendor
Eight studies specified the CCT system and vendor they had in place ( , which included open-ended questions through which to collect qualitative data; these studies generally reported poorly on the contribution of the qualitative data, lacked a detailed description of their thematic analysis process and did not fully consider qualitative research rigour. Across all the included studies, we identified poor reporting on researcher reflexivity. Overall, most studies reported adequately about their recruitment strategy, data collection and analysis process. The methodological limitations of the included studies are presented in detail in Table 1 (Additional  Tables).

Confidence in the review findings
Out of 20 review findings, we graded five as high confidence, seven as moderate confidence, and the remaining findings as low confidence, using the CERQual approach (See Summary of findings 1). All of the studies were from North America, which was an issue we judged to be of concern for the relevance of several of the findings and have therefore downgraded our CERQual assessment to reflect this concern. Even though ICU practice is a highly standardised field, and CCT is rigidly defined, we could Library Trusted evidence. Informed decisions. Better health.
Cochrane Database of Systematic Reviews not discount the possibility of variations across world regions. The North American focus of the evidence meant we could not assess any of our findings as having no or very minor concerns for relevance. We judged there to be serious concerns for relevance in those findings more likely to be influenced by social norms and local culture, such as those concerned with sta personal attributes or patient and family attitudes towards health care. The issue led to less serious (minor or moderate) concerns for relevance in those findings concerned with standard aspects of ICU organisation and practice governed by internationally-agreed guidelines, such as ICU equipment use, team composition and sta ing levels, as well as standard features of CCT technology. Our explanation of the CERQual assessment for each review finding is available in the Evidence profiles Table 2; Table 3.

Review findings
We mapped CFIR (Damschroder 2009) against the evidence from the included studies leading to the identification of 20 review findings, under six overall domains, reporting on factors a ecting the implementation of CCT. We explore each review finding under these domains in greater depth in the following sections. We point to 'hub' teams to refer to telemedicine sta based remotely in the CCT hub, and 'bedside' teams to refer to those sta based in ICUs providing bedside patient care.

Finding 1: Hospital sta 's personal experience, and anecdotes from colleagues, supported their belief that CCT has positive e ects on patient care. Specifically, these e ects were about patient safety and quality of care, support at the bedside by critical care experts, and standardisation of practice (moderate confidence).
Hub nurses shared positive experiences, and recalled anecdotes from colleagues, of timely support at the bedside by critical care experts made possible through CCT (Khunlertkit 2013; Shahpori 2011a). Based on such experiences, and even in the absence of other evidence (e.g. from research), administrators and hub clinicians felt confident that CCT could impact positively on their patients' quality of care (Moeckli 2013; Sta ord 2008a). Some administrators and hub clinicians perceived that CCT ensured standardised and up-to-date care, which could positively impact patient safety and outcomes (Moeckli 2013; Sta ord 2008a).
["Standardized care is going to be huge, just because we are a smaller facility. … Previously we've shared sta among the [ICUs], but having a set way of how we're treating our veterans and caring, I think will help us ensure that we are up-to-date on the best practice in how we manage things." ICU administrator] (Moeckli 2013).

Finding 2: Hospital sta and family members described several advantages of CCT. Bedside and hub sta strongly believed that the main advantage of CCT was having access to experts when bedside doctors were not available. Families also valued having access to experts. In addition, hospital sta described how CCT could support clinical decision-making and mentoring of junior sta (high confidence).
Bedside and hub sta strongly believed that the key advantage of CCT was a timely intervention by CCT experts, when bedside physicians were not available ( . Some respiratory therapists saw CCT's greatest advantage in its ability to provide access to critical care expertise, when this was not available locally (Thomas 2017); and an opportunity for junior physicians to receive mentoring via virtual means (Moeckli 2013). The availability of critical care expertise locally was also valued by patients' family members (Jahrsdoerfer 2013).

Finding 3: Bedside sta valued the potential adaptability of CCT to speak to local needs and practices. However, this was not always evident, with reported examples being mainly around developing camera usage etiquette and integration with local protocols (low confidence).
Some bedside nurses viewed CCT as having the potential to be implemented e ectively within various healthcare contexts, but only if local practices and user needs were taken adequately into consideration (Moeckli 2013). Although the adaptability of CCT was not a feature mentioned by all stakeholders, some bedside nurses did report it facilitated and enabled the integration of local protocols (Thomas 2017).
["some degree of tailoring is important to speak to perceived needs at di erent ICUs and among di erent populations of users." Researcher interpretation] (Moeckli 2013). Moreover, even though the presence of the camera caused feelings of discomfort for some bedside teams, hub teams were able to adapt their camera usage etiquette to overcome this (Sta ord 2008a).

Finding 4: Both bedside and hub clinicians reported di iculties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more coordination work, and concern around the use of cameras (Moderate confidence).
Bedside nurses complained that CCT charged them with additional workload, mainly administrative work, on top of their already busy schedule (

Finding 5: Cost considerations featured as an influencing factor in a limited way, with only a few examples noting the high cost of implementing CCT, especially compared to the cost of recruiting additional ICU sta (low confidence).
Some bedside nurses and respiratory therapists were concerned that CCT would be used by organisations to make savings on sta costs by reducing the number of ICU sta (Shahpori 2011a Cochrane Database of Systematic Reviews be used as a tool to reduce the number of bedside sta were strongly raised in particular by RNs and RTs." Researcher interpretation] (Shahpori 2011a). Some hub physicians believed that the use of CCT could be a cost-e ective option, although this view was not based on evidence (Sta ord 2008a).

CFIR Domain II: Factors related to the outer setting
Finding 6: Hospital sta as well as family members perceived CCT to be providing a community benefit, specifically relating to patients and families' desire to stay close to their local community without requiring transfer to specialist centres to access critical care expertise (moderate confidence).
Bedside clinicians and administrators shared experiences of CCT allowing patients in rural hospitals to receive expert critical care attention in their community, avoiding the need for patient transfer to regional centres (

Finding 7: Hospital sta greatly valued the nature and quality of social networks between the bedside and CCT hub teams. Key issues for them were trust, acceptance, teamness, familiarity and e ective communication between the two teams (high confidence).
Hub

Finding 8: Hospital bedside sta were concerned over the hub team not being aware of local unit norms, values, and culture. This led local bedside teams to feel that CCT intruded on their practice (moderate confidence).
Bedside physicians and nurses, especially in rural hospitals, felt that hub physicians were not aware of their local culture and practices (

Finding 10: Hospital locale shaped prioritisation of CCT, with sta in rural centres noting that CCT was of greater benefit to them considering their sta shortage and lack of critical care resources (low confidence).
Some bedside and hub physicians in urban hospitals perceived CCT to be of less utility and value to them because they already had in-house expertise (Kahn 2019

Finding 11: Bedside and hub clinicians perceived the absence of support from, and lack of engagement in dialogue with, leaders and senior administrators during the implementation of CCT as major barriers. Listening to sta needs, and creating groundwork connections with them from the outset were perceived as facilitating factors to implementation (low confidence).
Bedside and hub physicians and nurses believed that their senior leaders and administrators should have explained every step of the CCT implementation process more thoroughly, supported and engaged with all the relevant stakeholders early on in the process (

Finding 13: Hospital sta reported the lack of access to information about how CCT sta , policies and procedures can be incorporated into the bedside workflow as a barrier to implementation (low confidence).
Bedside and hub physicians, administrators and CCT coordinators felt inadequately informed about how CCT hub sta , policies and procedures can be incorporated into the bedside workflow. ["You know, that was never really expressed to us, who and what actually we should be doing, for the formal training (of sta )." CCT coordinator. "I just kind of kept expecting to see something … saying, 'This is how we're gonna operate, here's how our policies and procedures are going to work.'" Administrator] (Moeckli 2013). Cochrane Database of Systematic Reviews

facility, and timely co-ordination for CCT implementation (low confidence).
Lacking an adequate engagement strategy was a source of great frustration for some hospital sta (

Finding 18: Hospital sta were encouraged by the visibility of the intended benefits of CCT. They valued both quantitative feedback through auditing, as well as qualitative feedback through reflective accounts (moderate confidence).
Some bedside nurses and hub sta reported that over the course of CCT implementation they noticed improvements in patient outcomes, reduction in rates of medical errors, improved compliance with guidelines, and reduction in the number of healthcare-associated infections; these were mainly evident through clinical audits (

Results of integrating the review findings with the Cochrane intervention review
We explored the extent to which our review findings help explain the conclusions of the Flodgren 2015 review on interactive telemedicine, with a view to identifying avenues for future research and review updates. We concluded that the interventions Cochrane Database of Systematic Reviews considered within Flodgren 2015 were highly variable and too dissimilar from the focus of this synthesis. CCT is distinct as an application from the traditional models of telemedicine since it uses a hub-and-spoke model to provide a continuous form of clinical support to bedside practice 24/7; rather than just being an interface for sporadic communication between patients and providers. Interventions considered in Flodgren 2015 mostly used just one of the CCT features (e.g. laboratory and diagnostics management, or continuous electronic recording of vital signs, or display of real-time data, or provider-based decision support, or provider-to-provider video conferencing), but not a combination of all of the features. Most interventions in Flodgren 2015 used provider-to-patient video conferencing, with some supplementing of this with remote access to patient data. Despite di erences in the nature of telemedicine interventions, the geographic location of included studies in both this review and Flodgren 2015 originated mainly in high-income countries. This points to a significant gap in knowledge of implementation, use and e ectiveness of telemedicine in lower-and middle-income settings.
The main outcomes examined in Flodgren 2015 were: patient mortality, adverse events, healthcare resource use and cost. While certainty in the evidence for these outcomes varied, a common feature was heterogeneity in the meta-analysis, with variability across study results. We argue that a key issue hindering the investigation of this variability lies in the limited consideration and description of the implementation stage, as well as the di erent nature of the various telemedicine interventions. Furthermore, the opportunity for sensitivity analysis is limited because a consistent programme theory about the implementation and e ectiveness of telemedicine is missing from previous reviews. Following a narrative approach, as described earlier (Methods), we grouped our findings under key process, structure and balancing measures of CCT implementation which could be examined in combination with patient outcomes in future telemedicine research and review updates. While outcome measures reflect the impact of CCT on patients, process measures reflect the way CCT works to deliver such impact. In addition, structure measures are important in identifying significant attributes of the wider service in which CCT is implemented; and balancing measures reflect unanticipated consequences of CCT that can influence outcomes either positively or negatively. With a view to informing future development of, and research into, telemedicine, we propose the following model based on our review findings: • Outcome measures: we propose service use, as well as patient and family satisfaction with care as key outcome measures for consideration in future telemedicine research and reviews, in addition to conventional outcomes such as mortality and adverse events. We also recommend more systematic documentation of costs incurred to include recruitment or redeployment of sta , as well as sta training costs (Findings 1, 2, 5, 6); • Process measures: we identify the provision of sta training and information/education resources as important process measures for e ective implementation. Sta engagement through consultation meetings prior to implementation and regular feedback from audits on outcomes and adherence to evidence-based guidelines should also be considered. Failure-to-rescue indicators, including early identification of deterioration, escalation of care, time to consultation and remedial action may act as moderators to patient outcomes (Findings 4,12,13,14,17,18); • Structure measures: organisational and unit/clinic culture, especially for supportive leadership and collaborative approaches are further suggested as important structure measures to consider. Evidence of adjustments to local practices, protocols and workflows to accommodate and integrate telemedicine in daily practice could also be sought. Investment in human resources, by numbers as well as skill-mix, also appears critical (Findings 3,8,10,11,12,16,19); • Balancing measures: aspects of teamness, including communication, professional respect, autonomy and role clarity may be inadvertently a ected through the introduction of telemedicine, and should not be overlooked in future studies (Findings 7,9,15,20).

Review author reflexivity
We describe our initial positioning earlier (see Methods, Review author reflexivity). While our views did not shi significantly in the process of developing the synthesis, the final stages were completed during the COVID-19 pandemic which inevitably coloured our experience. This relates specifically to the two review authors who worked closely on developing the initial review conclusions and implications for practice (AX, KI). The pandemic shone a bright light on the added value of telemedicine in health care generally, and critical care in particular, with reports of CCT enabling clinical teams to respond better at the height of the pandemic. While our review findings were already decided by this time, showing both advantages and pitfalls with implementing CCT, it would be remiss of us to not acknowledge a sense of pressure to deliver not only evidence-based but also practical Cochrane Database of Systematic Reviews implications for policymakers and other stakeholders. During this stage, the two review authors held regular meetings (online) to discuss progress and reflect on the extent to which the pandemic may have influenced our views about CCT, and in turn our review implications and conclusions. A third review author (JP) checked our conclusions alongside the evidence to confirm a close link, before consulting with other members of the wider review team. By remaining close to the evidence from our included studies, and reflecting continuously throughout the process, we remain confident our conclusions and implications present an honest account of the state of the evidence for factors a ecting the implementation of CCT.

Summary of the main findings
For a summary of the main findings, please see the Plain Language Summary.

Comparison with other reviews and implications for the field
Past reviews on the e ectiveness of CCT reported great variability and uncertainty, with their included studies providing a limited understanding of the contextual factors contributing to this (Wilcox 2012). Attention to contextual features has also been highlighted as important in two other Cochrane Reviews of telemedicine and eHealth (Flodgren 2015; Ross 2016). Findings from this qualitative evidence synthesis provide a more complete understanding of the contextual and process factors a ecting implementation, potentially moderating the e ectiveness of CCT. Evidence for this review was derived from di erent stakeholder (providers, administrators, service users) experiences, perceptions and values about the factors enabling or hindering successful implementation.
Our synthesis drew from the CFIR framework to theoretically inform data synthesis, but not to restrict it, thus adding to the literature about the implementation of telemedicine. We also identified two additional themes (see Findings 19 and 20) not classified under the CFIR domains. Consequently, factors including structure, process and balancing measures have been identified and synthesised in a model (see section Results of integrating the review findings) to aid future research and review updates on telemedicine in di erent contexts.
Our work complements the Flodgren 2015 e ectiveness review on interactive telemedicine, which considered the acceptability of telemedicine by patients and healthcare professionals. Evidence of acceptability in that review was limited, mainly identifying costs and di iculties with operating the technology. The included studies in Flodgren 2015 recruited heterogeneous groups and interventions mainly in primary-care settings, which hindered issuing of specific guidance. Our review extends this work by identifying insights on perceived factors that influence implementation (including acceptance, adoption and use) of an advanced form of telemedicine by a range of stakeholders and within a well-defined patient population.
Finally, our findings complement and reinforce findings of the qualitative synthesis by Odendaal 2020, which looked into health workers' perceptions and experiences of using mobile health (mHealth) technologies to deliver primary healthcare services. In both syntheses, CCT and mHealth were perceived by healthcare workers to influence traditional ways of working, either positively or negatively. Negative perceptions in both syntheses related to features of the technology that increased the workload, while positive perceptions related to the potential to improve quality of care. The impact of technology on provider-to-provider collaboration was another shared finding. Collaboration was strengthened through improved and faster peer feedback and expert advice, but hindered when technology was seen as interfering with professional autonomy. While Odendaal 2020 focused on ongoing use of mHealth, our synthesis additionally captured factors that influenced acceptance and adoption of CCT, such as sta training and engagement, leadership and teamness. Twelve out of the 13 studies included in this qualitative synthesis were completed in the USA, and one was conducted in Canada. Research in middle-and lower-income countries is urgently needed. Especially in the context of the COVID-19 outbreak, declared as a pandemic in more than 203 countries, telemedicine implementation and integration within health systems across the higher-, middle-and lower-income countries is quickly becoming a global imperative. Further qualitative research to understand the experiences and perspectives of relevant stakeholders in di erent contexts, with di erent norms, practices and values is needed to inform future implementation, use and development of CCT.

Overall completeness and applicability of the evidence
Issues of health-system infrastructure, workforce capacity, and costs are not fully addressed in the available literature, but are arguably important for those considering investment in CCT. Telemedicine in general has been identified as influenced by such issues as payment and regulatory structures, state licensing and credentialising across hospitals (Hollander 2020). Based on our included studies we are unable to confirm the extent to which such issues also hinder implementation of CCT in particular, which limits our ability to make suggestions about overcoming these.
The included studies did not distinguish results between pre-, peri-and post-implementation of CCT. Even though some studies did include data collection at di erent implementation stages, their analysis and reporting conflated their results. While it is conceivable that di erent factors may have more relevance at di erent stages of implementation, we are currently unable to distinguish these from the available body of evidence.

Cochrane Database of Systematic Reviews
Lastly, we have low confidence in the evidence contributing to eight out of the 20 review findings. We downgraded our confidence in these findings mainly for a lack of adequate data (limited richness within studies and breadth across studies) supporting each review finding, as well as for concerns over relevance, given the North American focus of the dataset.

Limitations of the review
We only included studies published in the English language, although no language limit was imposed during the electronic database search. Nevertheless, we cannot discount the possibility that we may have missed relevant studies written and published in non-English-language journals. The included studies were all from North America (the USA and Canada), which is an issue dealt with in the GRADE-CERQual assessment. We opted for the CFIR (Damschroder 2009) as our implementation framework to guide our data synthesis, which provided structure and transparency to our approach, although limiting the potential for the chance discovery of themes arguably amplified when using a purely inductive approach. We are confident we have not missed any significant themes, given that two additional factors influencing implementation of the CCT, not adequately captured by CFIR, were identified and included in our findings. Moreover, a limitation of using CIFR is that it is a rather linear and reductionist framework, which hinders pulling out relationships and trade-o s between its components. Our approach adopted an implementation lens, which shaped the way in which we approached the literature. Adopting a di erent lens and theoretical stance, arguably, may have led us to emphasise di erent findings.

Implications for practice
Below is a set of questions drawn from the findings of this synthesis that may be helpful to health system, programme managers and other ICU stakeholders when planning and managing the implementation of CCT. These questions were drawn from the review findings in which we had high or moderate confidence.

Patient safety, quality of care and confidentiality
• Have you considered whether it might be useful to monitor the progress and impact of CCT, for instance through both quantitative (e.g. audits) and qualitative (e.g. reflective accounts) approaches? • Can you assure patient privacy and confidentiality in the context of CCT? • Have you thought about how to integrate CCT tools (e.g. decision support) in ICU clinicians' daily practice? • Can you tailor CCT to the needs of di erent ICUs, specifically in relation to local protocols and practices?

Training and mentoring of users
• Is there a detailed, all-steps-included, hands-on training programme on CCT use for all relevant stakeholders? • Have you thought about how you will raise awareness and encourage uptake of training resources among sta ? • Can you include a mentoring component for junior ICU bedside sta , linking them with experienced sta in the CCT hub?
• Do the ICUs actively encourage bedside teams to seek and share feedback from and with their hub colleagues?

Raising awareness
• What strategies are in place to raise awareness among clinical sta about the strengths and challenges of using CCT, before it is implemented? • Have you informed sta about the potential advantages of CCT, for patient safety, quality of care and family satisfaction? • How are family members informed about the strengths and challenges of CCT, for example, the potential it o ers for patients in rural communities to avoid transfer to regional centres?

Building teamness
• Have you considered how you can encourage teamness, trust, communication, familiarisation and collaboration between hub teams and local bedside teams? • Have you clarified the purpose of CCT to both bedside and hub teams? • Have you identified the distinct roles and workflows of bedside and hub teams, and have you communicated these to them? • Have you considered how hub clinicians can participate equally and engage with their bedside colleagues during ICU ward rounds?

Camera usage etiquette
• Have you consulted with bedside and hub teams to develop an acceptable camera usage etiquette? • Have you discussed with the bedside team the presence of the camera, and how it can be used to help them in their daily work (e.g. by being a second pair of eyes, watching restless patients)?

Sustainability and ongoing usage
• Have you ensured ongoing maintenance of the equipment?
• Can you ensure 24/7 availability of IT support to bedside and hub teams.

Resource allocation
• Have you identified optimal and safe sta ing levels for the CCT hub facilities? • Bedside nurses may feel concerned that their experienced colleagues are taken to sta the CCT hub. Have you considered what measures can be taken to ensure sta ing the CCT hub does not negatively influence sta ing levels at the bedside? • Have you considered ways of minimising the potential for the additional workload on bedside teams? • Have you considered o ering ICU nurses and physicians the opportunity to work across bedside and hub teams, to strengthen knowledge-sharing and skill development?

Implications for future research
Future research on the implementation and impact of CCT should also consider, and be designed to examine the questions noted in Implications for practice. Additional implications for research have been identified based on the overview of 13 studies included in this qualitative synthesis, and our GRADE-CERQual assessments of 20 review findings.

Cochrane Database of Systematic Reviews
More detailed reporting is needed in qualitative studies of CCT implementation, especially around researcher reflexivity, sampling and data analysis methods. Future qualitative studies on this topic should transparently report their research methods, including on the researchers' roles and how these may influence the conduct and results of the study.
Further research is needed on implementing CCT outside North America, especially in lower-and middle-income countries, to better understand how di erent norms, cultural practices and health infrastructure may foster or hinder acceptance of CCT.
Research on CCT that includes the perspectives and experiences of family members, and significant others, is missing and urgently needed. Understanding how families perceive the role and usefulness of CCT towards the care of their loved ones, especially in times of pandemics and hospital-visiting restrictions, would provide valuable evidence about the value, contribution and acceptance of CCT among all relevant stakeholders towards meeting the goal of family-centred care.
Further qualitative research is needed on this review topic to strengthen the body of evidence contributing to the review findings, since lack of adequate data was a key reason for downgrading our confidence in many of the review findings.

A C K N O W L E D G E M E N T S
When preparing this review, we consulted EPOC's Protocol and Review Template for Qualitative Evidence Synthesis (Glenton 2020).
The Norwegian Satellite of the E ective Practice and Organisation of Care (EPOC) Group receives funding from the Norwegian Agency for Development Co-operation (Norad), via the Norwegian Institute of Public Health to support review authors in the production of their reviews.  The researchers did not report any inclusion and exclusion criteria for sampling the Tele-ICU nurses. Nor did they explain how the 10 Tele-ICU nurses interviewed from each unit differed from those who were not interviewed The data collection method was clearly described, although the choice of method was not justified. The setting where interviews were conducted was not reported. Although an interview guide was used, the link given to this was inactive. Saturation of data was not discussed, but the number of responses corresponding to each category is reasonable

A D D I T I O N A L T A B L E S
The role of the researchers (bias or influence) during data collection, choice of location, and sample recruitment were lacking The study was granted approval by the institutional research committees. Transcripts were kept anonymous. But no information was provided about participants' explanations about the purpose, benefits, and harms from the study. It was not clear if interviewees were asked to return a signed consent form The data analysis process was described sufficiently. The researchers explained how data were selected from the original sample. Enough data extracts were used to support the study's findings. But the researchers did not discuss their own role and potential bias during analysis and selection of data for presentation The findings were presented clearly. The research team read the transcripts and the interviews were coded by 2 researchers, thus enhancing rigour in the analysis. The findings were discussed in relation to the study's original aim   The researchers clearly described the process of ICU site selection in detail, based on set characteristics and eligibility criteria The researchers justified their data collection choices. Interview and focus group guides were used, which were digitally audio-recorded. Saturation of data was discussed. Details of the site visits were not provided. It is not clear when and for how long observations were held in each unit The paper includes an online supplement which reports on the study methods. The researchers' relationship with participants was reported in a previously published protocol Written consent was provided by the participants and the study has been approved by a University Review Board. The study protocol discussed ethical considerations in detail A rigorous data analysis process was described in full details in the study. A constant comparative approach was used. Interpretation of data were cross-checked with participants. A thematic codebook was developed. The researchers' role during data analysis has been discussed A detailed description of the data analysis process was included. 2 analysts were involved in data analysis and theme identification. Sufficient data were presented to support the study's findings. Both positive and negative outcomes were taken into consideration in the presentation of findings The findings were explicitly described. Extracts from the interviews provided rich insights into the identified themes. A sufficient discussion of the findings about the original research aim was included Given the focus of the study to evaluate the impact of a Tele-ICU programme, the choice of qualitative methodology, without a quantitative component in the study, was not sufficiently justified The researchers employed interviews and observation to address the study's aims. Although a qualitative approach was appropriate, the researchers did not discuss or justify their choice of methods Participant selection was not explained. No information was provided on why some participants were interviewed in the pre-and others in the post-implementation phase Data collection methods were not adequately justified. Not enough information was provided about the observational data. The rationale for using observation was not explained, and it was not clear how observational data corroborated interview data. Saturation of data was not discussed There was no critical examination of the researchers' role during data collection, sample recruitment and choice of research site. The role of observersfor example, participant or non-participant observation, establishment of rapport, maintenance of roleboundaries and Hawthorn effectwas not discussed Approval was granted by the national and local institutional review boards. Informed consent was obtained by participants, but little is reported about how the study was provided to participants, whether participants' identification was concealed Limited information was provided about the codebook development. Thematic analysis was performed, but reporting of the process through which the themes were identified was lacking. Sufficient data were presented to support the findings, while contradictory data were also considered Development of the findings was not adequately explained. The trustworthiness of the findings was enhanced with a consensus coding by 3 researchers, but this process was only applied to 10% of the data  were provided about the non-respondents to the survey formed the basis of the qualitative data in the study. Saturation of qualitative data was not discussed reported in the study vided prior to the survey. The survey was administered anonymously, but issues around maintaining confidentiality of qualitative data were not reported in the study to demonstrate the analysis process The credibility of the qualitative findings was not discussed. An adequate discussion of the findings about the study's aims was provided The setting was clearly described. The researchers clearly explained and justified their choice of data collection methods. But no information was provided about the interview topic guide and observation schedule. Saturation of data was not discussed The researchers' role during the formulation of the research aims, data collection methods, sample recruitment and choice of the research setting was not adequately discussed The study was approved by the participating institutions' review boards. Participants provided informed consent, while identifiers or names in the transcriptions were avoided. But there was insufficient information about how the research was explained to participants The analysis process was described sufficiently. Coding, memos and typologies were used as part of the analysis. An audit trail was used to assure confirmability. Credibility, transferability, and dependability were also considered The findings were explicitly presented, and adequate discussion of these was provided. The number of data analysts was not reported. Credibility of the findings was established through data triangulation  The researchers justified their choice of method. Interview guides were developed. Saturation of data was not discussed. Handwritten notes were taken during the interviews, increasing the risk of inaccuracies. The researchers discussed their efforts to assure accuracy of notes, but whether this was achieved remains questionable The study did not report the researchers' role, potential bias and influence during data collection, recruitment or choice of research sites The study was sufficiently explained to the participants. An informed consent form was signed by the participants, and their anonymity was ensured. Approval was sought by the research company review board. Approval from a university review board, or from the hos-Inter-analyst agreement was mentioned, but a clear indicator for this was not reported. Much of the findings were from the researchers' interpretation, not always supported by participant quotes. The researchers' role, potential bias and influence were not reported The study's findings were not reported in a clear and explicit way, resulting in difficulty in tracking the identified themes. Some discussion about the existing literature was included. Credibility of the study was enhanced by including more than 1 analyst  Finding 3: Bedside sta valued the potential adaptability of CCT to speak to local needs and practices. However, this was not always evident, with reported examples being mainly around developing camera usage etiquette and integration with local protocols.

Methodological limitations
3 studies contributed data to this finding. 1 study discussed researcher reflexivity. 1 study was assessed as having methodological limitations related to recruitment. Finding 4: Both bedside and hub clinicians reported difficulties with the implementation of CCT. Key barriers related to implementation were perceptions of additional workload, need for more coordination work, and concern around the use of cameras

Assessment for each GRADE-CERQual component
Methodological limitations 5 studies contributed data to this finding. None of the studies discussed researcher reflexivity. 3 studies were assessed as having methodological limitations related to data analysis and collection, of which 2 were also assessed as having methodological limitations related to research design; and 1 of the 3 was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations

No or very minor concerns about coherence
Relevance Minor concerns about relevance, because the studies covered different ICU settings from different countries and even though these were all North American the focus of the finding is on standard features of CCT technology that are unlikely to differ significantly across world regions

Adequacy
Moderate concerns about adequacy, because the 5 contributing studies together offer only moderately thin data.
Overall GRADE-CERQual assessment and explanation

Relevance
Serious concerns about relevance, because the studies only covered a small range of settings from only 1 region; these were conducted several years ago and it is likely the health resource allocation model used then is no longer current Adequacy Serious concerns about adequacy, because the 2 contributing studies together only offer seriously thin data.

Overall GRADE-CERQual assessment and explanation
Low confidence Downgraded to low confIdence because we had no or very minor concerns about coherence, moderate concerns about methodological limitations, and serious concerns about relevance, and adequacy

Shahpori 2011a; Sta ord 2008a
Finding 6: Hospital sta as well as family members perceived CCT to be providing a community benefit, specifically relating to patients' and families' desire to stay close to their local community without requiring transfer to specialist centres to access critical care expertise

Assessment for each GRADE-CERQual component
Methodological limitations 5 studies contributed data to this finding. None of the studies discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data collection and analysis, of which 1 study was assessed as having methodological limitations related to research design; and the other was assessed as having methodological limitations related to recruitment. A third study was also assessed as having methodological limitations related to recruitment.   Finding 10: Hospital locale shaped prioritisation of CCT, with sta in rural centres noting that CCT was of greater benefit to them considering their sta shortage and lack of critical care resources

Assessment for each GRADE-CERQual component
Methodological limitations 4 studies contributed data to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and data collection, of which 1 study was also assessed as having methodological limitations related to research design; and the other was assessed as having methodological limitations related to recruitment strat-  Finding 12: Hospital sta thought it was important to promote and offer training in the use of CCT before its implementation. This included rehearsing every step in the process, offering sta opportunities to ask questions and disseminating learning resources. Some also complained that experienced sta were taken away from bedside care and re-allocated to the CCT hub team Relevance Serious concerns about relevance, because the study was from a single setting in 1 country; and local policies, procedures and workflows are likely to differ across settings and countries Adequacy Serious concerns about adequacy, because of only 1 study offering only thin data

Overall GRADE-CERQual assessment and explanation
Low confidence Downgraded to low confidence because we had no or very minor concerns about coherence, minor concerns about methodological limitations, and serious concerns about relevance, and adequacy

Moeckli 2013
Finding 14: Hospital sta 's attitudes towards, knowledge about and value placed on CCT influenced acceptance of CCT. Sta were positive towards CCT because of its several advantages. But some were concerned that the hub sta were not able to understand the patient's situation through the camera. Some were also concerned about confidentiality of patient data Assessment for each GRADE-CERQual component

Cochrane Database of Systematic Reviews
Methodological limitations 1 study contributed data to this finding. It was assessed as having methodological limitations related to research design, recruitment strategy, and researcher reflexivity. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations

No or very minor concerns about coherence
Relevance Serious concerns about relevance, because the data came from a single setting in 1 country; and the issue of nurses' personal attributes and motivation is highly likely to vary significantly across settings and countries

Adequacy
Moderate concerns about adequacy, because the 1 study offered only moderately rich data

Overall GRADE-CERQual assessment and explanation
Low confidence Downgraded to low confidence because we had no or very minor concerns about coherence, minor concerns about methodological limitations, moderate concerns about adequacy, and serious concerns about relevance

Hoonakker 2013
Finding 17: Hospital sta were frustrated due to lacking a clear strategy for engagement; specifically lack of consistent training, the orientation of new and resistant sta to the hub facility, and timely co-ordination for CCT implementation

Assessment for each GRADE-CERQual component
Methodological limitations 2 studies contributed data to this finding. 1 study assessed researcher reflexivity. 1 study was assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having minor concerns about methodological limitations

Coherence
Minor concerns about coherence, because the data were only reasonably consistent within studies

Relevance
Moderate concerns about relevance, because the studies were from a limited range of settings in a single country, and the issue of sta engagement is likely to differ across settings and countries Adequacy Serious concerns about adequacy, because the 2 studies together offer only thin data

Overall GRADE-CERQual assessment and explanation
Low confidence Downgraded to low confidence because we had minor concerns about methodological limitations and coherence, moderate concerns about relevance, and serious concerns about adequacy

Kahn 2019; Moeckli 2013
Finding 18: Hospital sta were encouraged by the visibility of the intended benefits of CCT. They valued both quantitative feedback through auditing, as well as qualitative feedback through reflective accounts

Assessment for each GRADE-CERQual component
Methodological limitations 3 studies contributed data to this finding. 2 studies discussed researcher reflexivity. The body of evidence contributing to this review finding was assessed as having no or very minor concerns about methodological limitations

Cochrane Database of Systematic Reviews
Coherence Minor concerns about coherence, because the data were only reasonably consistent within studies

Relevance
Moderate concerns about relevance, because the data came from a limited range of setting in a single country, and the value sta place on different kinds of feedback, and on reflection in particular, is likely to differ across countries and world regions

Adequacy
Minor concerns about adequacy, because the 3 studies together offer only reasonably rich data

Overall GRADE-CERQual assessment and explanation
Moderate confidence Downgraded to moderate confidence because we had no or very minor concerns about methodological limitations, minor concerns about coherence, and adequacy; and moderate concerns about relevance

Contributing studies
Kahn 2019; Khunlertkit 2013; Thomas 2017 Finding 19: Hospital sta highlighted that CCT can support ICUs to overcome challenges associated with sta shortages especially during nights and weekends, and in rural hospitals where ICU nurses are assigned to different departments, and with retaining physicians and nurses. Some concerns over the potential negative impact of CCT on overall sta ing levels were also expressed

Assessment for each GRADE-CERQual component
Methodological limitations 5 studies contributed data to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to data analysis and collection, of which 1 was also assessed as having methodological limitations related to recruitment; and the other was assessed as having methodological limitations related to research design. Another study was also assessed as having methodological limitations related to research design and recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations

Relevance
Minor concerns about relevance, because while issues concerning retention of physicians and nurses are unlikely to differ significantly across world regions, the studies supporting this finding were only from North America

Adequacy
Moderate concerns about adequacy, because the 5 contributing studies together offered only moderately thin data

Overall GRADE-CERQual assessment and explanation
Moderate confidence Downgraded to moderate confidence because we had no or very minor concerns about coherence, minor concerns about relevance, and moderate concerns about methodological limitations and adequacy

Contributing studies
Goedken 2017; Hoonakker 2013; Kahn 2019; Shahpori 2011a Finding 20: Interactions between some bedside and CCT hub sta were featured with tension, frustration and conflict. Sta on both sides commonly described disrespect of expertise, resistance and animosity Assessment for each GRADE-CERQual component

Cochrane Database of Systematic Reviews
Methodological limitations 7 studies contributed to this finding. 1 study discussed researcher reflexivity. 2 studies were assessed as having methodological limitations related to research design, of which 1 was also assessed as having methodological limitations related to data analysis and collection; and the other was assessed as having methodological limitations related to recruitment. Another study was also assessed as having methodological limitations related to recruitment. The body of evidence contributing to this review finding was assessed as having moderate concerns about methodological limitations

Coherence
No or very minor concerns about coherence

Relevance
Minor concerns about relevance, because while issues concerning friction across different teams of health professionals have been noted in research worldwide, the studies supporting the current finding were only from North America

Adequacy
No or very minor concerns about adequacy

Overall GRADE-CERQual assessment and explanation
High confidence Graded as high confidence because we had no or very minor concerns about coherence and adequacy, only minor concerns about relevance, and moderate concerns about methodological limitations.

A -Intervention Source
Perception of key stakeholders about whether the intervention is externally or internally developed.

B -Evidence Strength & Quality
Stakeholders' perceptions of the quality and validity of evidence supporting the belief that the intervention will have desired outcomes. C -Relative advantage Stakeholders' perception of the advantage of implementing the intervention versus an alternative solution.

D -Adaptability
The degree to which an intervention can be adapted, tailored, refined, or reinvented to meet local needs.

E -Trialability
The ability to test the intervention on a small scale in the organization, and to be able to reverse course (undo implementation) if warranted.

F -Complexity
Perceived difficulty of implementation, reflected by duration, scope, radicalness, disruptiveness, centrality, and intricacy and number of steps required to implement.

Library
Trusted evidence. Informed decisions. Better health.

S21
(remote support or remote surveillance or remote monitoring or remote counseling or remote counselling)