The purpose of this study was to understand leadership perspectives on DBT implementation and sustainability after training completion to inform agency administrators looking to implement DBT within their community agencies. The study explored both consistencies and changes in leadership perspectives over time [23]. In the prior evaluation, perspectives focused on the fit of the treatment within the existing services provided at the agency (e.g., integrating DBT in to the agencies’ clinic structure and population, providing adequate resources to administer the model, reducing personnel concerns through careful selection of clinicians to be trained [23]). Administrators in the present study communicated barriers related to the intensive training structure of DBT, the model’s demanding requirements, and the challenges related to the target population. Three themes that remained consistent pre- and post-implementation were: administrators’ (generally positive) opinions of DBT, concerns related to agency resources, and staff selection and turnover. Implications and applications of administrative leader perspectives are discussed below.
Evaluate for goodness of fit
Prior interviews indicated some agencies’ concern related to the goodness of fit between DBT and the agency’s current practices [23]. Agencies facing the greatest difficulties with integrating DBT within their programming structure ultimately abandoned the implementation of the EBP, also known as “de-adoption” [30]. Unfortunately, these administrators denied participation in the present interviews, preventing researchers from further exploring barriers to sustainability within their agencies [31]. Although training and implementation efforts by agencies were voluntary, and research staff took strides to adequately prepare agencies for integrating DBT into their current practices, it is apparent that the leap for some agencies was too great. The following suggestions may help further prepare administrators wishing to adopt these new treatments into their agencies.
Try to understand the demands of implementation
Administrators pre- and post-implementation commented on the increased demands required of DBT. For example, the DBT model requires each trained clinician to have a caseload of clients who are appropriate for individual DBT treatment (with weekly or twice-weekly sessions), conduct weekly group DBT sessions (with approximately eight clients) which are recommended to run 2 h at a time, participate in DBT supervision meetings with other DBT clinicians, and be available by phone at all hours and days of the week. It should be noted that while certain demands may be unique to DBT implementation, many EBPs require added duties that can make it difficult to integrate and maintain EBPs within CBBH agencies [3, 20]. These increased demands on clinicians’ time as well as the effect the change of treatment delivery has on the agency’s billing are important components for administrators to think carefully about to adequately assess for model feasibility [32,33,34].
In addition to understanding the increased demands placed on clinicians, administrative leaders need to consider the additional monitoring that may be necessary to ensure treatment fidelity of this EBP over time. While the current DBT intensive training implementation strategy did not include review of digital recordings, and also did not require consistent review of particular cases, therapy notes, or diary cards, integrating these strategies into agency practice should be done to avoid drift from the DBT model; however, these additional steps to monitor clinician adherence to treatment over time reduce clinicians’ billable hours within a clinic and should be considered when preparing for the implementation process. This current model included some consultation where clinical questions about cases were asked and answered to create a structurally adherent program without significant procedures to monitor therapist or session treatment adherence. This was done as a means to enhance feasibility, although it may have been at the cost of model fidelity.
Even with the reduced responsibilities compared to other EBPs that are highlighted above, the high level of accountability and fidelity required of DBT implementation reportedly was difficult for some agencies. Future agency leaders looking to implement DBT may benefit from understanding the significant difficulties associated with establishing strong methods for adhering to EBP protocol and ongoing monitoring of the quality of treatment delivery. Specifically, the process of delivering evidence-based quality care often requires key structural components to be in place [35] such as administrative support in the form of maintaining and training appropriate staff, effective management of agency funds, and providing appropriate time for staff to fulfill all roles and responsibilities [36]. If the structural components are not present, even a motivated clinical staff may not be able to sustain an EBP over time. If the structural components have been appropriately attended to, then key factors to enhance DBT success include interpersonal variables within organizations such as supervision, team cohesion, team communication, and team climate [6] as well as staff interest and expertise [8]. While the involved agencies in the present study communicated their interest in adopting DBT early on [23], some agencies were unable to start or maintain DBT implementation due to various organizational barriers. Additionally, the research team did not adequately assess if agencies had experience being trained in and successfully sustaining any EBPs prior to the start of the present study. Therefore, it is possible that some of the stated difficulties administrators reported about DBT are confounded with the difficulties of implementing an EBP, in general. Either way, it is important for administrators to think critically and problem-solve potential barriers within the agency prior to adopting an EBP (e.g., funding over time) to help improve the likelihood for long-term sustainability.
In addition to barriers with initial or sustained implementation of DBT, some interviewees indicated that their agencies started adapting the treatment model due to concerns about funding or staff responsibilities to address concerns the agencies thought they could not overcome. Adaptation of EBPs has been shown to be a common practice in CBBH agencies [8]. Frequently, agencies in this study reported limiting the availability of clinicians by removing the option for clients to call for skill support after clinic hours. While agencies may not initially plan to change the DBT model, adaptation may occur from the lack of infrastructure to handle the demands of treatment implementation. Adaptations may allow clinicians to devote greater efforts to delivering components that they believe are more likely to be effective in helping their patients and provide greater flexibility in choosing what components they think are likely to be most appropriate for a given patient. While this is appealing to many practicing therapists, this approach should be carefully considered. Past research has shown that adapting DBT may compromise treatment effectiveness and sustained DBT implementation [37]. This is especially significant as clinical practitioners wanting to provide high quality care for their clients may apply the model in ways that mistakenly remove core and influential components, either reducing positive client impact or creating detrimental client outcomes [38].
As recommended by one administrator, advanced training in how to supervise DBT might be helpful, as an appropriately trained supervisor may be better able to monitor fidelity. Administrative leaders also highlighted the importance of agency support (e.g., reduced expectations of clinicians, increased recognition for clinician and agency efforts) considering the increased demands of DBT, which enhanced morale and overall program success [39]. Planning ways to maintain quality implementation of EBPs (e.g., treatment adherence monitoring, built-in time for clinician paperwork, EBP funding) is an important step for long-term success and sustainability.
Attempt to work through negative attitude and prejudice
Clinicians’ negative perspective on working with DBT clients was also a common concern before and after implementation. Overall goals of treatment implementation can be halted, and outcomes of clients can be negatively impacted if administrators and their clinical staff have low opinions of those who would otherwise benefit from EBPs [40]. Trainers and agencies should consider addressing the potential barrier of low therapist commitment to the client population. If administrators are committed to providing DBT to enhance patient care, gathering feedback prior to training on therapist negative attitudes or prejudices toward BPD or clients with BPD can help administrative leaders focus their efforts toward increasing engagement in working with this population. Motivational assessments could be conducted throughout training to determine if additional training time should be dedicated to garnering clinician buy-in. Importantly, clinicians’ opinions have been found to change over time to be more favorable toward clients with BPD once they undergo training [12]. Long-term benefits of DBT implementation as stated by community agencies include increased self-efficacy and compassion for practicing clinicians, a clinic’s ability to address unique symptoms, and greater levels of hope and functioning for their clients [8]. Even still, administrators willing to assess and address clinician engagement and bias toward BPD may benefit from increased clinician motivation, better client services, and a greater likelihood of DBT sustainability [41, 42].
Get ready by harnessing adequate resources for implementation
Administrators noted the importance of sufficient resources not only with respect to training (pre-implementation [23]), but also to sustainable implementation (post-implementation). Many administrators stated that implementation would have been impossible without financial support provided by the behavioral health managed care company and the counties to offset costs associated with personnel attending training and providing DBT. This finding is consistent with the literature on sustainability of other EBPs [39]. Administrative leaders noted that the higher billing rates were beneficial so that their teams could dedicate time and energy to learning and implementing the treatment. This might not have been possible without a one-time infusion of funds from the county and the behavioral health managed care company to the provider organization that was used to offset decreased clinician productivity rates.
Financial considerations are an essential component of EBP sustainability [39] and have been frequently discussed in different EBP literature (e.g., EPIS [32], Getting To Outcomes [33], CFIR [34]). Administrators should spend adequate time researching the gains that may be obtained from investing in an EBP (e.g., better client outcomes, increased billing rate of therapists) as well as the costs (e.g., lost income during training periods). Specifically, administrators should set up meetings with potential trainers to understand the time commitment, training obligations, and details of daily treatment delivery to determine if the EBP is a financially viable and profitable investment. Prior to adopting DBT within an agency, administrators should consider expanding their knowledge of upfront costs, hidden costs, and a long-term funding stream to promote DBT sustainability [43].
Consider preparing clinicians and your agency
Administrators highlighted concerns related to selecting competent clinicians to deliver DBT. Having staff with high levels of interest and expertise may help facilitate successful DBT implementation [8]. Yet, even with administrators’ careful selection, 45% of the original, trained therapists left their agencies during this period [12]. Turnover within agencies has been shown to be a predictor for deterioration of EBP over time [44]. And while DBT’s sustainability compares favorably with other EBPs, all models struggle with staff turnover; staff turnover negatively impacts sustainability [10]. Open communication between researchers, administrators, and clinicians about literature on therapist turnover and factors associated with retention in EBP may be beneficial to clinician selection [44]. For example, administrators believed that only senior staff with extensive clinical experience and high levels of flexibility would be successful, still some clinicians came to training with low opinions of the potential for DBT success. Yet clinicians who held lower opinions of DBT made substantial gains in their opinions throughout the training [12]. Interestingly, while some therapeutic factors such as therapeutic relationship [45] and therapeutic progress [46] have been shown to influence staff turnover, organizational factors such as low organizational support, staff morale, productivity, and organizational effectiveness [10, 47] along with financial burdens [48] may also lead to staff turnover. Moreover, the implementation of an EBP may increase the likelihood that turnover will occur [45]. Making sure the satisfaction of agency staff is high and the structure of the organization is stable prior to adopting an EBP may be just as, if not more important, than which staff are trained to deliver the new EBP [10]. Therefore, administrators may benefit from discussions with clinical staff about interest and commitment to an EBP. Moreover, assessing capacity of the agency and clinicians may be an important first step prior to moving forward with EBP adoption.
Limitations
There were several limitations to the current study. First, including a larger number of agencies and administrators in the research study may have allowed for thematic saturation and reduced the possibility that particular themes remained unexplored in this particular CBBH setting. While administrator perspectives seem to represent a small sample of agencies, this sample size is common in studies utilizing qualitative leadership perspectives in clinical leaders (N = 15) [49], agency directors (N = 7) [50], and administrators (N = 16) [4]. Furthermore, previous research has concluded that as few as six to up to 12 interviews can produce thematic saturation in qualitative samples [51]. Importantly, these perspectives are often unexplored [9, 10] even though they may provide valuable insight into implementation efforts [2, 5].
Administrator response rate was an additional limitation to the outcomes of the present study. Only eight of the original administrators were included in the present evaluation (approximately 61.5%). It is unknown how the inclusion of all the original administrators might have impacted the present study’s findings. Moreover, due to two agencies failing to initiate DBT implementation and agency administrators disinterest in continuing the study, the research staff was unable to gather their commentary about sustainability. Insights on the explicit barriers to implementation and eventual de-adoption of DBT from agency administrators that were no longer implementing DBT would have been valuable.
Generalizability was an additional limitation of the current project. Perspectives were collected from eight out of ten agencies located in eastern Pennsylvania counties which may not reflect other sustainability efforts in more urban areas or other regions facing different challenges or restraints. Moreover, the perspective provided on behalf of each agency is limited by the fact that it represents only one or two people’s opinions.
In addition to adaptations some agencies utilized, limitations seen by administrative leaders in these Pennsylvania counties may have stemmed from possible clinician nonadherence to the model. Moreover, the strain on clinicians implementing a treatment with high risk and difficult clients may have been too great if they were not provided adequate supervisor support to guide and ensure adequate treatment adherence.
It is possible that administrator reports of some of the barriers to sustaining DBT within their agencies may have occurred regardless of the EBP implemented. In fact, more recent work conducted with DBT sustainability suggests that there are factors that make all EBP implementation difficult (e.g., workforce turnover); yet, DBT compares favorably to other EBPs in maintaining treatment fidelity and outcomes over time [52]. The researchers did not collect data to understand if agencies were successful in implementing other EBPs; therefore, outcomes are limited in understanding if all the present barriers were limited to DBT implementation or could be expected in the implementation of another EBP.
Results from this qualitative study reflect administrators’ perspectives, but may not have fully captured the reasons for successful or problematic DBT implementation (e.g., administrator beliefs for clinician termination vs. surveying clinicians about why they exited their agencies). The study may have benefited from collecting data via observational measures or real-time measurements throughout the course of DBT implementation, rather than the retrospective nature of the current administrator interviews. Future research studies may benefit from incorporating quantitative analyses to interpret additional factors that significantly impact implementation efforts.