Several authors also endorsed flexible approaches for involving patients [45, 49, 53, 54]. For example, Gibson et al.  used peer reporter interviews (where patient pairs interviewed each other), headline generation (where phrases were created to capture important issues), group discussion (using a Who, Why, When, What, How structure), a written exercise, and questionnaires for non-attendees to find out what youth would like from their follow-up pediatric oncology services. Other techniques identified in studies were the inclusion of higher proportions of patients compared to providers or staff to give patients a stronger voice in the discussion and process  and building in debriefing to provide feedback on how suggestions were acted upon to increase the accuracy of the findings and offer an opportunity for additional input. These techniques proved useful in engaging patients to prioritize stroke service issues and document the process of change of a mental health organization [62, 63]. Others built in regular updates to patient support group to elicit more views, thereby broadening the reach and involvement of patients and providing opportunities to raise and discuss issues of concern in informal settings [48, 54]. One creative technique was a buddy system for users/families to ensure their participation at meetings and throughout implementation/evaluation of a quality improvement project in mental health services .
Engaging patients can also change the culture of staff and care settings. The experiences reported in these articles included shifts in organizational culture promoting further patient participation in service design and delivery, [40, 63, 75] achieving collaboration and mutual learning, [42, 47, 76, 77] and sharing or neutralizing power among patients and providers or staff,  as well as developing new competencies and negotiating for service changes [39, 59] (Table 4). Interestingly, these outcomes tended to arise in mental health settings and from co-design engagements (Table 5). Further analysis of the methods used in these studies revealed key enabling factors including creating deliberative spaces to share experiences, including external facilitation; broadening power and control to include users, values, and beliefs exercises; conducting user/staff/provider training; and implementing a top-down approach from the local authority (Table 5).
Our review builds upon previous reviews in this field by providing insight into the associations between quality improvement methods and the varying system-level outcomes they yield. Indeed, our review echoes previous research indicating that patient engagement can lead to a multiplicity of health services outcomes with sufficient role definition, training, and alignment of patient-provider expectations but that the quality of the reporting has been poor and the full impact of patient engagement is not fully understood [87,88,89]. Previous reviews have been limited to specific countries , care settings (e.g., mental health ), hospitals , or study design (e.g., qualitative studies ). In this way, our review provides a comprehensive perspective of optimal strategies used internationally, across care settings and using multiple methodologies to engage patients, caregivers, and relatives in quality of care improvement initiatives. Our review also provides novel insights into how the level of engagement influences the outcomes, namely, discrete products (e.g., development of tools and documents) largely derived from low-level engagement (consultative unidirectional feedback), whereas care process or structural outcomes (e.g., improved governance, care or services) mainly derived from high-level engagement (co-design or partnership strategies). If the benefits of engaging patients in the design or delivery of health care are to be realized at an organization or system level, then effective strategies and the contextual factors enabling their outcomes need to be identified so that learning can be generalized. Importantly, our review provides guidance on the effective strategies and contextual factors that enable patient engagement including techniques to enhance the design, recruitment, involvement, and leadership action, and those aimed to create a receptive context.
Several types of planned thematic articles also will strengthen the field. Articles assessing the state of the science, as well as the barriers and challenges researchers face, will facilitate the development of effective solutions, such as new theories, frameworks, and research standards, approaches and methods. Other planned thematic articles will offer practical guidance and insights to implementation researchers. The series labelled "The Practice of Implementation Science" will offer practical, science-based guidance to researchers facing the need to select among available theories, frameworks, research approaches and designs, methods and other options. A planned series of articles written by implementation research users (in the policy and practice communities) will offer implementation researchers insights into the needs, values and preferences of these stakeholders. Enhancing communication between implementation researchers and implementation research users represents an important goal of the journal.
Innovative approaches are needed to maximize fit between the characteristics of evidence-based practices (EBPs), implementation strategies that support EBP use, and contexts in which EBPs are implemented. Standard approaches to implementation offer few ways to address such issues of fit. We characterized the potential for collaboration with experts from a relevant complementary approach, user-centered design (UCD), to increase successful implementation.
Using purposive and snowball sampling, we recruited 56 experts in implementation (n = 34) or UCD (n = 22). Participants had 5+ years of professional experience (M = 10.31), worked across many settings (e.g., healthcare, education, human services), and were mostly female (59%) and white (73%). Each participant completed a web-based concept mapping structured conceptualization task. They sorted strategies from established compilations for implementation (36 strategies) and UCD (30 strategies) into distinct clusters, then rated the importance and feasibility of each strategy.
Despite its potential, it remains unclear how UCD fits within the evolving landscape of implementation research and practice. Implementation is already a highly interdisciplinary field and new collaborations between implementation experts and UCD experts will be essential to capitalize on the promise of UCD for health services. Experts from these two fields have only recently begun joining together to examine the role of design in implementation, and their efforts have been primarily in the form of conceptual frameworks (e.g., [15, 31]). As a step toward better understanding the alignment of implementation and UCD strategies, we used concept mapping  to characterize how experts from each discipline conceptualize the relations among the strategies described in these frameworks. Our study offers a novel empirical understanding of the proposed conceptual relationship between these two disciplines.
Cluster map of implementation and user-centered design (UCD) strategies. The map reflects the product of an expert panel (valid response n = 55) sorting 66 discrete strategies into groupings by similarity. Circles indicate implementation strategies and diamonds indicate UCD strategies. The number accompanying each strategy allows for cross-referencing to the list of strategies in Table 1. Light-colored clusters are comprised entirely of implementation strategies; dark-colored clusters are comprised entirely of UCD strategies; and multi-colored clusters are comprised of strategies from both disciplines. Spatial distances reflect how frequently the strategies were sorted together as similar. These spatial relationships are relative to the sorting data obtained in this study, and distances do not reflect an absolute relationship
Implementation science and UCD offer complementary approaches with several key points of interdisciplinary alignment. It may be ideal for implementation and UCD experts to work side-by-side to execute strategies from trans-discipline clusters, but work sequentially or in parallel for strategies from discipline-specific clusters. Yet such collaboration could encounter challenges for a variety of reasons. Experts tended to modestly favor their own discipline in their importance and feasibility ratings, suggesting that multi-disciplinary teams could disagree about how to prioritize various strategies when resources are limited. It will also be important to develop supports for multidisciplinary implementation-design teams, drawing on the growing science of team science . In the future, our research team plans to investigate UCD-focused team science resources (e.g., mentored development programs) and tools (e.g., shared online workspaces) to complement the limited, but growing, offerings of implementation training initiatives  and our UCD strategy glossary for implementation experts . Our efforts will be informed by continued analysis of additional data collected from participants in this study. For example, participants also provided rank-order and qualitative feedback about challenges and desired supports for cross-discipline collaboration (see the study protocol  for details).
The study identifies a number of key barriers to and drivers of successful service delivery from the perspective of grantees implementing GMH projects. Findings highlight several opportunities to mitigate common challenges, providing recommendations for strengthening systems- and project-level approaches for delivering mental health services. Further, more inclusive research is required to inform guidance around service delivery for successful implementation, better utilization of funding and improving mental health outcomes among vulnerable populations in low-resource settings. 2b1af7f3a8