*Result*: Therapist effects in internet-delivered cognitive behavior therapy for anxiety and depression.
*Further Information*
*Objective: Several clinical studies have found that therapists differ in their client outcomes, supporting the notion of "therapist effects" in psychotherapy. However, only a handful of studies have investigated therapist effects in internet-delivered cognitive behavior therapy. This study aimed to examine therapist effects in internet-delivered cognitive behavior therapy treatment of anxiety and depression in routine care.
Method: Data of 8,145 clients who were treated by 44 therapists were examined. Generalized linear mixed models were performed to identify the presence of therapist effects and the amount of variance attributable to therapists across several outcomes: (a) change in symptoms over time, (b) the occurrence of clinically significant change, (c) treatment completion, and (d) client satisfaction.
Results: Significant therapist effects were observed across all outcomes, indicating that there were differences between therapists in each outcome domain. However, the therapist effects appear relatively modest overall, with therapists explaining 1.5% and 1.4% of variance in change over time in anxiety and depression, respectively, 0.6% and 0.7% of the variance in the occurrence of clinical change in anxiety and depression, respectively, 2.3% of treatment completion, and 1.4% of client satisfaction.
Conclusions: The findings suggest that there are differences among therapists across a range of outcomes. However, these differences account for a modest proportion of the overall variation in client outcomes. Future research is needed to replicate these findings across different contexts before firm conclusions are drawn. (PsycInfo Database Record (c) 2026 APA, all rights reserved).*
Therapist Effects in Internet-Delivered Cognitive Behavior Therapy for Anxiety and Depression
<cn> <bold>By: Ivy F. G. Feliciano</bold>>
>
> <bold>Lauren Staples</bold>
>
>
> <bold>Amelia Scott</bold>
>
> <bold>Michael P. Jones</bold>
>
> <bold>Heather Hadjistavropoulos</bold>
>
> <bold>Nickolai Titov</bold>
>
>
> <bold>Blake F. Dear</bold>
>
>
<bold>Review of: </bold>ccp0000994_RevisedSupplementary_Material.docx
<bold>Acknowledgement: </bold>Jasper Smits served as action editor.Ivy F. G. Feliciano played a lead role in conceptualization, data curation, formal analysis, methodology, project administration, and writing–original draft. Lauren Staples played a supporting role in writing–review and editing and an equal role in data curation. Amelia Scott played a supporting role in conceptualization and writing–review and editing. Michael P. Jones played a supporting role in methodology and writing–review and editing. Heather Hadjistavropoulos played a supporting role in writing–review and editing. Nickolai Titov played a supporting role in data curation, supervision, and writing–review and editing. Blake F. Dear played a lead role in supervision and writing–review and editing, a supporting role in data curation, formal analysis, and methodology, and an equal role in conceptualization.
It is widely accepted that mental health care should be informed by evidence to ensure the provision of safe and effective treatments (Australian Government Department of Health, 2010; Coombs et al., 2011; Nundy et al., 2022). Consistent with this principle, most mental health research has focused on evaluating the effectiveness of different types of psychotherapies (van Agteren et al., 2021). Comparatively, there has been much less research on the people who deliver these interventions, the psychotherapists (Okiishi et al., 2006). Research on the therapist is important because it has implications for training and supervision, service design and delivery, and efforts to improve clinical outcomes.
The notion that different therapists may produce better or worse client outcomes was first explored 50 years ago (Ricks, 1974). In 1974, Ricks examined the long-term outcomes of adolescents and found that those who were treated by a particular therapist were significantly better able to adapt to changes in their adulthood compared to those treated by the average therapist (Ricks, 1974). Since then, several studies have also found that therapists influence client outcomes, suggesting that therapists can make a unique contribution over and above the effects of a specific psychological intervention (Edmondstone et al., 2023; Erekson et al., 2020; Firth et al., 2020; Goldberg et al., 2018). Consistent with this, a recent systematic review (
The degree to which therapist effects explain variance in client outcomes appears to vary greatly across studies. Although the review conducted by Johns et al. (2019) found the variance in outcomes due to therapists was 5%, results across studies were significantly different (i.e., ranging from 0.2% to 29%). Several studies exploring therapist effects have also examined therapist predictors that may influence the magnitude of observed therapist effects. For example, Okiishi et al. (2006) conducted a study (
Internet-delivered cognitive behavior therapy (iCBT) is one type of structured treatment that has attracted significant interest over the last decade and a half. iCBT is increasingly being implemented and offered as routine care in many countries and jurisdictions with encouraging results (Etzelmueller et al., 2020), and numerous studies have found iCBT to be effective for a range of anxiety and depressive disorders (Andrews et al., 2018). iCBT uses structured therapeutic modules delivered by online systems, which aid the client and therapist by providing core therapeutic content and automated tasks such as regular symptom assessment, often reducing the amount of therapist time required (Bennett-Levy et al., 2010). However, despite this, iCBT therapists still have significant clinical responsibilities, including providing general psychotherapeutic support, tailoring treatment to each client, encouraging treatment engagement, and supporting skills practice. Reflecting this, research has consistently found therapist-guided iCBT to be associated with superior clinical outcomes compared to completely self-guided iCBT involving no therapist interaction (Andersson & Titov, 2014). Nevertheless, the structured nature and the routine measurement of client progress in iCBT mean that it provides a valuable opportunity for exploring therapist effects. There is also a key question of whether therapist differences exist in this form of treatment, where significant therapeutic content is provided by modules, and the role of therapists varies from traditional face-to-face therapy.
To date, only two clinical studies have examined therapist effects in iCBT and whether therapists themselves might account for variance in iCBT treatment outcomes. The first study (
This study aimed to extend the available literature by examining the presence and extent of therapist differences using large samples of therapists and clients providing iCBT in routine care. The study focused on therapist effects in iCBT treatment for anxiety and depression, given their high prevalence and comorbidity (Javaid et al., 2023; Kessler & Bromet, 2013; Wu et al., 2025) and the extensive evidence base for iCBT in treating these conditions (Andrews et al., 2018). The present study also sought to examine therapist effects in the context of a range of relevant outcomes, including symptom change over time, the occurrence of clinically meaningful change, overall treatment satisfaction, and treatment completion. We hypothesized that there would be differences among therapists across these outcomes, and that therapists would account for a significant proportion of the variance in each outcome.
Method
> <h31 id="ccp-94-2-88-d3e233">Setting</h31>
This study used data collected from a high-volume national digital mental health service in Australia, the MindSpot Clinic. The MindSpot Clinic (
Ethical approval was granted to use therapist and client data collected by the clinic between 2012 and 2018 by the Macquarie University Human Research Ethics Committee Medical Sciences Committee (Ref 520221198342289).
<bold>Therapists</bold>
Data from 44 therapists who had seen at least 30 clients each were included in this study. Nine therapists were excluded for having seen less than 30 clients (
<bold>Clients</bold>
The initial data included 12,090 clients treated between 2012 and 2018. During this period, the clinic did not administer Week 1 questionnaires (for operational reasons) for approximately 12 months, resulting in 3,945 without baseline data. Clients who did not have outcome scores at Week 1 of treatment were excluded from the analysis. Consequently, a final data set of 8,145 clients was available for analysis. Clients were Australian adult residents (aged 18+) who were eligible for publicly funded health services and who reported primary complaints of anxiety and/or depression. Clients were allocated to therapists by administrative staff based on therapist capacity and availability using a standardized workload model. Thus, no systematic clinical algorithms are employed in the clinic, where, for example, more severe or complex clients are allocated to senior therapists.
<h31 id="ccp-94-2-88-d3e263">Treatment Approach</h31>To limit the potential effects of different treatments and client populations on estimates of therapist effects, this study employed data from one specific suite of internet-delivered transdiagnostic treatment offered by the clinic for anxiety and depression (Dear et al., 2016; Fogliati et al., 2016; Titov, Dear, Staples, Terides, et al., 2015). This suite of treatment consisted of the Wellbeing Course (ages 26–65), the Wellbeing Plus Course (ages 65+), and the Mood Mechanic Course (ages 18–25), which are composed of the same therapeutic content and teach the same cognitive and behavioral skills. The only major difference between the treatments is that they cater to the different ages through age-appropriate case stories and skills-use examples, which are woven throughout all the treatment materials (Dear et al., 2015, 2018; Titov, Dear, Staples, Terides, et al., 2015; Titov et al., 2016). The therapeutic content involves evidence-based cognitive behavior treatment components for managing symptoms of anxiety and depression. This includes psychoeducation, cognitive restructuring, arousal control strategies, graded exposure, behavioral activation, and relapse prevention as core components and involves several other optional components, including sleep training, assertiveness training, and problem solving. The core therapeutic content and skills are delivered via five online lessons with homework assignments, detailed examples and case stories, and several optional resources covering the different issues (e.g., managing sleep, problem solving, assertive communication). These lessons are presented in a slide show format and include information and text-based instructions, real-world examples, and relevant case stories. These examples and case stories aim to assist in showing how the information and skills can be used as well as to normalize the challenges in learning and applying the skills.
Clients are instructed to read the lessons over 8 weeks according to a structured schedule. Each lesson is estimated to take between 10 and 20 min to read. Access to subsequent lessons is dependent on clients completing the preceding lessons. Clients also receive regular automatic emails, which notify them of new course materials and reinforce completion of materials and practice of skills.
<h31 id="ccp-94-2-88-d3e291">Therapists</h31>All clients are allocated a dedicated therapist before starting treatment who introduces themselves via telephone and via a secure message during the first week of treatment. Clients are offered regular weekly contact from their therapist via telephone or the secure messaging system. During weekly contact, therapists aim to support client engagement in treatment, build therapeutic bonds with clients, and support their processes of change by addressing barriers to understanding and applying the skills and reinforcing progress. Therapists monitor client progress throughout treatment and assertively engage clients where clinically indicated; for example, where clients appear to be disengaging from treatment, experiencing severe symptoms, presenting with significant psychosocial complexity, or are at mental health risk. Prior studies in the clinic have found therapists spend an average of 111 min supporting each client through treatment, with considerable variability between clients (range = 40–412 min;
During treatment, clients complete standardized measures of symptoms prior to starting treatment, every week during treatment except Week 5, and at 3-month follow-up. At the end of treatment, they also complete a standardized satisfaction questionnaire, which inquires about their overall treatment satisfaction and feedback about how services could be improved. Clients complete these questionnaires via the clinic’s online platform, which they log in to each week while completing treatment using a unique user account and password.
<bold>Generalized Anxiety Disorder–7</bold>
The Generalized Anxiety Disorder–7 (GAD-7) is a seven-item self-report measure of anxiety symptoms designed to screen for probable cases of generalized anxiety disorder (Spitzer et al., 2006). The GAD-7 has an internal consistency of Cronbach’s α .92 and test–retest reliability of .83 (intraclass correlation coefficient [ICC]; S. U. Johnson et al., 2019; Spitzer et al., 2006). The GAD-7 has a score range of 0–21. Scores are classified as 0–4 (minimal), 5–9 (mild), 10–14 (moderate), and 15–21 (severe). The GAD-7 has been found to be a good screener for a broad range of anxiety disorders (S. U. Johnson et al., 2019; Kroenke et al., 2007).
<bold>Patient Health Questionnaire–9</bold>
The Patient Health Questionnaire–9 (PHQ-9) is a nine-item self-report measure of depression symptoms designed to screen for probable cases of major depressive disorder (Kroenke et al., 2001). The PHQ-9 has an internal consistency of Cronbach’s α .89 and test–retest of .84 (Kroenke et al., 2001). Scores range from 0 to 27, which can be classified as 0–4 (none), 5–9 (mild), 10–14 (moderate), 15–19 (moderately severe), and 20–27 (severe). Scores of 10 or more indicate a diagnosis of depression.
<bold>Treatment Satisfaction</bold>
We operationalized treatment satisfaction as client self-reported satisfaction at the end of treatment. Specifically, we used responses to the question, “Overall, how satisfied were you with the course?” to which clients responded using a 5-point Likert scale ranging from
<bold>Treatment Completion</bold>
The online platform used to deliver the treatment records clients’ online activity and engagement with the treatment, including lesson completion. The present study operationalized treatment completion as having completed the five treatment lessons.
<h31 id="ccp-94-2-88-d3e345">Data Preparation</h31>A robust, multistep blinding and de-identification procedure was developed to protect therapist privacy and confidentiality based on previous therapist effect studies (Anderson et al., 2009; Green et al., 2014). The clinic’s primary data custodian extracted and prepared a data set of therapists and their clients’ outcomes. The custodian then de-identified therapists and clients for the research team while retaining reidentification codes in a separate data set. This de-identified data set was provided to one of the authors, who then further de-identified the therapists and clients while retaining reidentification codes in a separate data set. This “double” de-identification procedure was used to ensure the anonymity of the therapists and their respective client outcomes.
The final data set had missing data (29%) as some clients did not complete questionnaires during some weeks of treatment. Missing responses on both GAD-7 and PHQ-9 measures consisted of 17% at Week 2, 38% at Week 3, 18% at Week 4, 47% at Week 6, 49% at Week 7, and 35% at posttreatment. Additionally, 40.5% of responses on satisfaction were missing at posttreatment. The statistical analysis employed for the study (i.e., multilevel modeling) is able to handle missing data; however, where the missing data pertain to the Level 1 predictors or response variables (as in the case of this study), mixed models are unable to accommodate this and subsequently exclude these variables (van Buuren, 2018). There is now considerable evidence from missing data research in iCBT to suggest that treatment adherence and pretreatment symptoms are nonignorable mechanisms of missing data in iCBT (Karin et al., 2018). Consistent with this research, missing response data were addressed through an adjusted replacement strategy using multiple imputation under the missing at random assumption. The multiple imputation procedure considered treatment adherence, pretreatment symptom severity, time, and therapist as predictors in generating replacement values, consistent with prior research (Karin et al., 2018). Five data sets were generated using the multiple imputation procedure, and the results of analyses using these five data sets were then pooled using Rubin’s rules (Rubin, 1987).
<h31 id="ccp-94-2-88-d3e369">Data Analysis</h31>The analysis employed generalized multilevel modeling (GLMM) using SPSS Version 29 software. Multilevel models are a type of regression analysis with data from various levels of groupings (e.g., clients at Level 1, therapists at Level 2) where the outcome is measured at the lowest group level (Leyland & Groenewegen, 2020b). Importantly, multilevel models use a shrinkage effect to partially pool group-level estimates toward the overall mean (Bell et al., 2019), which helps to account for differences in caseload size and variability—for example, by reducing the influence of therapists with fewer clients on the final model estimates. GLMMs are a type of multilevel model that generalizes the multilevel model to other types of response variables and distributions (e.g., gamma, binary responses; Bono et al., 2021).
Statistical modeling was developed by starting with a null, unconditional model with no predictors, and levels of stratification were added gradually. Estimations (i.e., Bayesian information criterion) were compared to find the best fitted covariance structure for the data. A total of six multilevel models were used. Two models looked at change over time for GAD-7 and PHQ-9 outcomes, two models estimated the probability of a clinically significant change in GAD-7 and PHQ-9, one model estimated the probability of treatment completion, and the last model looked at the probability of client satisfaction with treatment.
For the first two longitudinal models, we wanted to examine whether therapists varied in the effect of time in treatment on client anxiety and depression symptom outcomes. The hierarchical data structure consisted of three levels where repeated outcome measures were nested within clients, and clients nested within therapists. Consistent with previous literature (Edmondstone et al., 2023), we specified our three-level models as expressed in Equation 1.<anchor name="eqn1"></anchor>
where the outcome γ is the client symptom outcome for client 
where μ0
For the remaining four models, we fitted a series of two-level binomial models where client outcomes were nested within therapists. These models examined (a) the probability of a clinically significant (i.e., defined as a ≥50%; yes/no) change in the outcome measures by posttreatment, (b) the probability of treatment completion (i.e., defined as ≥5 lessons completed; yes/no), and (c) the probability of clients’ satisfaction with treatment (i.e., defined as being “very satisfied” or “satisfied”; yes/no). Null, unconditional two-level models were specified as shown in Equation 3.<anchor name="eqn3"></anchor>
where ln(
Consistent with past therapist effects research, we intended to identify the magnitude of therapist effects by using the ICC (Wampold & Owen, 2021). The ICC reflects the proportion of the variance attributable to the grouping structure or second level in a two-level model (Hox, 2010). In this study, the ICC can be interpreted as the amount of variance in the outcomes explained by therapists. However, the ICC can only easily be calculated and interpreted for models with random intercepts only (Kreft & De Leeuw, 1998; Nakagawa & Schielzeth, 2013). As such, we used the ICC for our two-level binomial models with random intercepts only. To calculate therapist effects for clinically significant change, treatment completion, and satisfaction, the ICCs were estimated using the delta method. In random slopes models, the ICC differs at each value of the random slope due to its dependence on each unit of the predictor and is better described as the variance partitioning coefficient (VPC; Goldstein et al., 2002; P. C. D. Johnson, 2014). As such, with our three-level random intercepts and random slopes models, which estimated therapist effects in change in client GAD-7 and PHQ-9 outcomes over time, the VPC was calculated at posttreatment according to the standard approach (Leyland & Groenewegen, 2020a). In the present study, the VPC was calculated for the posttreatment time point and consequently reflects the proportion of unexplained variance in posttreatment symptoms due to therapist differences in intercepts and slopes.
Similar to previous studies, we wanted to identify therapists with outcomes that were higher and lower than the average therapist. As a result, percentage change (from Week 1 to posttreatment) was computed for each therapist on all six outcomes, and the mean, highest, and lowest therapist outcomes were also noted for each outcome.
Given the amount of missing data, sensitivity analyses were also conducted to assess the impact of the missing data on our findings. These analyses repeated all the main analyses without imputed data.
<h31 id="ccp-94-2-88-d3e618">Data Availability</h31>Data can be accessed for validation purposes, subject to appropriate Australian Human Research Ethics Committee approval.
Results
> <h31 id="ccp-94-2-88-d3e624">Preliminary Analyses</h31>
Therapists had seen between 31 and 669 clients, with a mean of 185 and a median of 149 clients per therapist. The demographic and clinical characteristics of the client sample are presented in Table 1. Notably, clients predominantly consisted of females (72.1%) who were between the ages of 18 and 40 (57.1%;
>
><anchor name="tbl1"></anchor>
<bold>Change in Client Outcomes Over Time</bold>
The results of the two-level GLMM for anxiety symptoms are presented in Table 2. Time in treatment had a significant effect on anxiety symptoms (γ10 = 2.393,
>
><anchor name="tbl2"></anchor>
>
><anchor name="tbl3"></anchor>
The results of the two-level GLMM for depression symptoms are presented in Table 4. This GLMM indicated time in treatment had a significant effect on depression symptoms (γ10 = 2.495,
>
><anchor name="tbl4"></anchor>
<bold>Clinically Significant Change</bold>
The results of the models examining clinically significant change (i.e., defined as ≥50% change) are presented in Table 5. The model estimated that there was a 47.4% (95% CI: 45.6, 49.2) and 45.9% (95% CI: 44.1, 47.6) probability of clients achieving clinically significant improvements in anxiety and depression symptoms, respectively, at posttreatment. There was significant unaccounted variance found at the therapist level for both anxiety (μ0j = 0.026,
>
><anchor name="tbl5"></anchor>
<bold>Treatment Completion</bold>
The results of the model for treatment completion can be found in Table 5. This model estimated 59.9% (95% CI: 57.2%, 62.5%) of clients completing treatment lessons within the 8 weeks of treatment. Additionally, there was significant unaccounted variance at the therapist level (μ0j = 0.098,
<bold>Satisfaction</bold>
The results of the unconditional model for treatment satisfaction are presented in Table 5. The model estimated 80.5% (95% CI: 78.7%, 82.2%) of clients as being satisfied with the treatment. There was significant variance found at the therapist level (μ0j = 0.178,
<bold>Change in Client Outcomes Over Time</bold>
The results of the two-level GLMM for anxiety and depression symptoms without imputed data are presented in Supplemental Tables S1 and S2, respectively. The validity of the models for depression was uncertain, with one possible reason being the lack of power to run the specified (random slopes) model. Nonetheless, the results similarly found time in treatment was a significant predictor of anxiety symptoms (γ10 = 2.393,
<bold>Clinically Significant Change</bold>
The results of the models without imputed data examining clinically significant change are presented in Supplemental Table S3. The results found similar estimates where there was a 56.2% (95% CI: 54.5% to 57.9%) and 56.4% (95% CI: 54.9% to 57.8%) probability of clients achieving clinically significant improvements in anxiety and depression symptoms, respectively. Significant unaccounted variance was also found at the therapist level for both anxiety (μ0j = 0.013,
<bold>Treatment Satisfaction</bold>
The results of the model for treatment satisfaction without imputed data are presented in Table 5. This model found a similar estimate of 84.1% (95% CI: 82.7% to 85.4%) of clients being satisfied with the treatment. Similar to the main analysis, significant variance was found at the therapist level (μ0j = 0.033,
Discussion
>
This study examined the presence and magnitude of therapist effects in iCBT for adults with anxiety and depression across a range of outcomes. The study used the data of 44 therapists and 8,145 clients from a large national digital psychology service, with iCBT provided as routine care and data collected over 6 years. Overall, we found significant reductions in symptoms of anxiety and depression over the course of treatment and high client satisfaction with the treatment. As per our hypotheses, we found significant variance in therapists’ clients’ symptom change in anxiety (range: 38.6%–43.4%) and depression (range: 39.4%–42.9%) over time. We also found significant variance in therapists’ proportions of clients achieving clinically significant reductions in anxiety (range: 42.0%–51.9%) and depression (range: 40.2%–51.9%), completing treatment (range: 47.1%–71.1%), and being satisfied with the treatment (range: 71.0%–84.6%). Importantly, therapists appear to account for a relatively small proportion of the total variance across the different outcomes examined (i.e., 0.6%–2.3%). Overall, these findings suggest that significant differences exist between therapists in iCBT for anxiety and depression, but therapists account for a relatively modest amount of the overall variance in outcomes.
The finding that therapists differ in their client outcomes, that is, that there are therapist effects, is consistent with several previous studies examining therapist effects in routine face-to-face care (Edmondstone et al., 2023; Erekson et al., 2020; Goldberg et al., 2018; Janse et al., 2024; Mahon et al., 2023). Many previous studies of traditional face-to-face therapies have found evidence of therapist effects, and while varying between studies, therapists account for an average of 5% (range: 0.2%–29%) of the variance across a range of outcomes and treatment contexts (Johns et al., 2019). Thus, the findings of the present study are broadly in line with the existing literature of face-to-face therapies. However, the present study extends significantly on the few available studies examining therapist effects in iCBT (Almlöv et al., 2009, 2011). Only two such studies have been published to date, and they did not find significant evidence of therapist effects using data from clinical trials of iCBT for depression and anxiety. However, one limitation noted by both those studies was the relatively small number of therapists (i.e., <10 therapists) and clients (i.e., <120) available for analysis, which may have meant they were underpowered to detect small therapist effects (Almlöv et al., 2009, 2011). In contrast, the present study had a much larger sample of therapists and clients and found evidence of therapist effects across a broad range of outcomes, including the magnitude of symptom change in treatment, the occurrence of clinically significant outcomes, treatment completion, and treatment satisfaction. Thus, the findings of the present study are significant in suggesting that, consistent with previous studies of face-to-face therapy, therapist differences are present in structured iCBT for anxiety and depression, and that therapists are potentially an important factor in clinical outcomes. Future studies exploring therapist effects in different digital mental health interventions and settings are vitally needed to both replicate and contextualize the findings of the present study.
One key question is what magnitude of therapist effect, usually examined through the variance explained, is practically meaningful and important. Unfortunately, this is a difficult question to answer as there are no agreed benchmarks or standards for the field. In the present study, we found therapists to account for around 0.6% and 0.7% of the variance in the occurrence of clinically meaningful change (i.e., defined as ≥50% change) in anxiety and depression, respectively. This seems like a very modest effect; however, when looked at in more practical terms, it might still be meaningful. For example, looking at the estimates of proportions improving in depression for each therapist, the therapist with the best outcomes (51.9%; CI: 46.5, 57.2) might be expected to help an additional 4–15 clients per 100 clients seen, compared to the therapist with the lowest outcomes (42.0%; CI: 36.6, 47.6). Thus, while caution is needed in such interpretation, given the uncertainty in estimates, even small effects may be important and highlight opportunities for improving the outcomes of iCBT (Etzelmueller et al., 2020). Given the large numbers of clients often seen by iCBT services, even small therapist effects may have important implications for the value of therapist training and supervision, which may help to attenuate therapist differences and optimize client outcomes. Nevertheless, there is a need for further research to explore and consider benchmarks for the practical importance of therapist effects. Related to this, there is also a need for further research to understand what drives or contributes to the size of therapist effects in different contexts, and how training and supervision may be employed to improve outcomes (Crits-Christoph et al., 1991; Janse et al., 2024; Johns et al., 2019).
The fact that therapist effects were found in the present study is significant, given that therapist roles and work are often somewhat different in iCBT compared with traditional face-to-face therapies (Bennett-Levy et al., 2010). In fact, there are several reasons why therapist effects might not exist or be relatively small in heavily structured and protocolized treatments like iCBT. First, a significant proportion of therapeutic messaging, content, and client skill development is standardized in iCBT, which may reduce the magnitude of any therapist differences. For example, the online lessons in iCBT typically provide much of the therapy content and experience, with regular automatic emails also providing therapeutic support and reinforcing the process of therapy. Therapists also often interact significantly less with clients in iCBT, with therapists often spending 1–2 hr supporting the average client (Etzelmueller et al., 2020), compared with several hours in traditional face-to-face care. Thus, the fact that evidence of therapist effects was found in the present study is important. However, the highly structured nature and differing roles of therapists in iCBT likely explain why the effects observed in the present study are more modest compared to some other studies of face-to-face therapies (Johns et al., 2019). This highlights an interesting avenue for future research, namely, exploring whether larger therapist effects might be observed in “blended-care” treatment approaches, where iCBT is used alongside face-to-face care (Kooistra et al., 2014; Mathiasen et al., 2016) or teletherapy treatments (Giovanetti et al., 2022) where technology is used but the therapist role is more similar to traditional face-to-face therapy (Egede et al., 2015; Stubbings et al., 2013).
There are several potentially important contextual issues that should be considered when interpreting the present study’s findings. One key consideration is that the therapists in the present study were all employed at one clinic and were predominantly nationally registered psychologists. In addition, all therapists were provided with substantial standardized initial training over several weeks and significant, structured weekly individual supervision. These factors may have reduced variation in clinical knowledge, skill, and practice, which might have otherwise contributed to larger therapist effects. It is worthwhile noting that the available literature already suggests that therapist effects are larger outside of clinical trial contexts where there is less standardization and more variation in practice (Crits-Christoph et al., 1991). Thus, it is not clear whether the current findings might generalize to other contexts where therapists do not receive similar or as much training and supervision, and there is more opportunity for variation in practice. For example, it would be interesting to examine therapist effects in iCBT delivered by community clinicians (Hadjistavropoulos et al., 2016) or clinicians using iCBT as a part of their practice but who are not employed in a single clinic (Guliani et al., 2022). Another key consideration is that the present study focused on the treatment of anxiety and depression in clients who had primary difficulties with anxiety and depression. It is possible that larger therapist effects may emerge in contexts where iCBT therapists are working with clients with more diverse or specific clinical presentations (e.g., social anxiety, panic, posttraumatic stress disorder, chronic pain). Future research is needed to explore these issues and to examine the presence and magnitudes of therapist effects in iCBT and other digital interventions in a range of contexts. Nevertheless, the findings of the present study highlight that therapist effects can be present in iCBT and that this area warrants such research.
The findings of the present study should also be considered in light of some key limitations. First, consistent with most routine care settings (Fernandez et al., 2015; Hoxha et al., 2022), there was a meaningful amount of missing data at posttreatment (i.e., 35%), and it is possible that this may have had some effect on the findings. The multiple imputation procedure was employed to address missing data, but it is possible that this procedure may have itself missed important patterns within the missing data. Second, this study only examined therapist effects in outcomes for the duration of treatment and did not include outcomes at long-term follow-up. Third, no data on the therapists were available (e.g., qualifications, experience, characteristics, behaviors), which meant we could not explore therapist factors that may influence clinical outcomes and be related to therapist effects. Thus, future research exploring the relationships between therapist characteristics, therapist effects, and clinical outcomes in iCBT would be valuable. Fourth, although no particular algorithm was used in assigning clients to therapists, clients were not randomly assigned. It is possible that some therapists may have worked (by chance) with disproportionately severe or engaged clients, or that some other features of the clients may have influenced the results. Furthermore, as is common with therapist effects studies (Edmondstone et al., 2023; Erekson et al., 2020; Goldberg et al., 2018; Janse et al., 2024; Mahon et al., 2023), the available data set consisted of therapists who had treated different numbers of clients, which may have influenced the estimation of effects and the study’s findings. Fifth, specific details on the nature of therapists’ engagement with clients (e.g., frequency of therapist and client contact) were not included in the study as this was not extractable for the present study. It is possible that some therapists may have had more contact with their clients than others, which may have influenced results. Despite these limitations, the present study has several important strengths. First, it used the data of a significantly larger number of therapists and clients compared to past research from a large national digital mental health service providing treatment as routine care. Second, it examined potential therapist effects across a broad range of clinically important outcomes; specifically, the rates of change, the occurrence of clinically meaningful change, treatment completion, and treatment satisfaction. Also, while sensitive to accurate specification, it is a strength that the multiple imputation was employed using available missing cases research in an attempt to ensure missing cases were represented in the findings (Karin et al., 2018, 2021).
In conclusion, the present study adds to the existing literature by examining the presence and magnitude of therapist effects in routine care iCBT for anxiety and depression. The findings suggest that there are significant differences among therapists across a range of outcomes, and these differences account for a modest proportion of the overall variation in client outcomes. However, given the large variability in the magnitude of therapist effects across studies and the absence of studies examining therapist effects in iCBT, further research across a range of contexts, including among different client groups and clinical service contexts, could assist in identifying factors that optimize service delivery, inform training and supervision of therapists, and enhance client outcomes.
References
<anchor name="c1"></anchor>Adelson, J. L., & Owen, J. (2012). Bringing the psychotherapist back: Basic concepts for reading articles examining therapist effects using multilevel modeling.
Almlöv, J., Carlbring, P., Berger, T., Cuijpers, P., & Andersson, G. (2009). Therapist factors in Internet-delivered cognitive behavioural therapy for major depressive disorder.
Almlöv, J., Carlbring, P., Kllqvist, K., Paxling, B., Cuijpers, P., & Andersson, G. (2011). Therapist effects in guided internet-delivered CBT for anxiety disorders.
Anderson, T., Ogles, B. M., Patterson, C. L., Lambert, M. J., & Vermeersch, D. A. (2009). Therapist effects: Facilitative interpersonal skills as a predictor of therapist success.
Andersson, G., & Titov, N. (2014). Advantages and limitations of Internet-based interventions for common mental disorders.
Andrews, G., Basu, A., Cuijpers, P., Craske, M. G., McEvoy, P., English, C. L., & Newby, J. M. (2018). Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis.
Australian Government Department of Health. (2010).
Bell, A., Holman, D., & Jones, K. (2019). Using shrinkage in multilevel models to understand intersectionality: A simulation study and a guide for best practice.
Bennett-Levy, J., Richards, D. A., & Farrand, P. (2010). Low intensity CBT interventions: A revolution in mental health care. In J.Bennett-Levy, D. A.Richards, P.Farrand, H.Christensen, K. M.Griffiths, D. J.Kavanaugh, B.Klein, M. A.Lau, J.Proudfoot, L.Ritterband, J.White, & C.Williams (Eds.),
Bono, R., Alarcón, R., & Blanca, M. J. (2021). Report quality of generalized linear mixed models in psychology: A systematic review.
Coombs, T., Stapley, K., & Pirkis, J. (2011). The multiple uses of routine mental health outcome measures in Australia and New Zealand: Experiences from the field.
Crits-Christoph, P., Baranackie, K., Kurcias, J. S., Beck, A. T., Carroll, K., Perry, K., Luborsky, L., McLellan, T., Woody, G. E., Thompson, L., Gallagher, D., & Zitrin, C. (1991). Meta-analysis of therapist effects in psychotherapy outcome studies.
Dear, B. F., Fogliati, V. J., Fogliati, R., Johnson, B., Boyle, O., Karin, E., Gandy, M., Kayrouz, R., Staples, L. G., & Titov, N. (2018). Treating anxiety and depression in young adults: A randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy.
Dear, B. F., Staples, L. G., Terides, M. D., Fogliati, V. J., Sheehan, J., Johnston, L., Kayrouz, R., Dear, R., McEvoy, P. M., & Titov, N. (2016). Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for social anxiety disorder and comorbid disorders: A randomized controlled trial.
Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Zou, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., McEvoy, P. M., & Titov, N. (2015). Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: A randomized controlled trial.
Edmondstone, C., Pascual-Leone, A., Soucie, K., & Kramer, U. (2023). Therapist effects on outcome: Meaningful differences exist early in training.
Egede, L. E., Acierno, R., Knapp, R. G., Lejuez, C., Hernandez-Tejada, M., Payne, E. H., & Frueh, B. C. (2015). Psychotherapy for depression in older veterans via telemedicine: A randomised, open-label, non-inferiority trial.
Erekson, D. M., Clayson, R., Park, S. Y., & Tass, S. (2020). Therapist effects on early change in psychotherapy in a naturalistic setting.
Etzelmueller, A., Vis, C., Karyotaki, E., Baumeister, H., Titov, N., Berking, M., Cuijpers, P., Riper, H., & Ebert, D. D. (2020). Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: Systematic review and meta-analysis.
Fernandez, E., Salem, D., Swift, J. K., & Ramtahal, N. (2015). Meta-analysis of dropout from cognitive behavioral therapy: Magnitude, timing, and moderators.
Firth, N., Saxon, D., Stiles, W. B., & Barkham, M. (2020). Therapist effects vary significantly across psychological treatment care sectors.
Fogliati, V. J., Dear, B. F., Staples, L. G., Terides, M. D., Sheehan, J., Johnston, L., Kayrouz, R., Dear, R., McEvoy, P. M., & Titov, N. (2016). Disorder-specific versus transdiagnostic and clinician-guided versus self-guided internet-delivered treatment for panic disorder and comorbid disorders: A randomized controlled trial.
Giovanetti, A. K., Punt, S. E. W., Nelson, E. L., & Ilardi, S. S. (2022). Teletherapy versus in-person psychotherapy for depression: A meta-analysis of randomized controlled trials.
Goldberg, S. B., Hoyt, W. T., Nissen-Lie, H. A., Nielsen, S. L., & Wampold, B. E. (2018). Unpacking the therapist effect: Impact of treatment length differs for high- and low-performing therapists.
Goldstein, H., Browne, W., & Rasbash, J. (2002). Partitioning variation in multilevel models.
Green, H., Barkham, M., Kellett, S., & Saxon, D. (2014). Therapist effects and IAPT Psychological Wellbeing Practitioners (PWPs): A multilevel modelling and mixed methods analysis.
Guliani, H., Witt, J., Peynenburg, V., Wilhelms, A., Nugent, M., Dear, B. F., Titov, N., & Hadjistavropoulos, H. D. (2022). Cost-effectiveness of varying degrees and models of therapist-assisted transdiagnostic internet-delivered cognitive behaviour therapy: Evidence from a randomized controlled trial.
Hadjistavropoulos, H. D., Nugent, M. M., Alberts, N. M., Staples, L., Dear, B. F., & Titov, N. (2016). Transdiagnostic Internet-delivered cognitive behaviour therapy in Canada: An open trial comparing results of a specialized online clinic and nonspecialized community clinics.
Hox, J. J. (2010).
Hoxha, K., Hung, Y. W., Irwin, B. R., & Grépin, K. A. (2022). Understanding the challenges associated with the use of data from routine health information systems in low- and middle-income countries: A systematic review.
Janse, P., Geurtzen, N., Scappini, A., & Hutschemaekers, G. (2024). Disentangling the therapist effect: Clustering therapists by using different treatment outcomes.
Javaid, S. F., Hashim, I. J., Hashim, M. J., Stip, E., Samad, M. A., & Al Ahbabi, A. (2023). Epidemiology of anxiety disorders: Global burden and sociodemographic associations.
Johns, R. G., Barkham, M., Kellett, S., & Saxon, D. (2019). A systematic review of therapist effects: A critical narrative update and refinement to Baldwin and Imel’s (2013) review.
Johnson, P. C. D. (2014). Extension of Nakagawa & Schielzeth’s
Johnson, S. U., Ulvenes, P. G., Øktedalen, T., & Hoffart, A. (2019). Psychometric properties of the GAD-7 in a heterogeneous psychiatric sample.
Karin, E., Crane, M. F., Dear, B. F., Nielssen, O., Heller, G. Z., Kayrouz, R., & Titov, N. (2021). Predictors, outcomes, and statistical solutions of missing cases in web-based psychotherapy: Methodological replication and elaboration study.
Karin, E., Dear, B. F., Heller, G. Z., Crane, M. F., & Titov, N. (2018). “Wish You Were Here”: Examining characteristics, outcomes, and statistical solutions for missing cases in web-based psychotherapeutic trials.
Kessler, R. C., & Bromet, E. J. (2013). The epidemiology of depression across cultures.
Kooistra, L. C., Wiersma, J. E., Ruwaard, J., van Oppen, P., Smit, F., Lokkerbol, J., Cuijpers, P., & Riper, H. (2014). Blended vs. face-to-face cognitive behavioural treatment for major depression in specialized mental health care: Study protocol of a randomized controlled cost-effectiveness trial.
Kreft, I. G., & De Leeuw, J. (1998).
Kroenke, K., Spitzer, R. L., Williams, J. B., Monahan, P. O., & Löwe, B. (2007). Anxiety disorders in primary care: Prevalence, impairment, comorbidity, and detection.
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001). The PHQ-9: Validity of a brief depression severity measure.
Lee, E., & Hong, S. (2021). Adequate sample sizes for a three-level growth model.
Leyland, A. H., & Groenewegen, P. P. (2020a). Apportioning variation in multilevel models. In P. P.Groenewegen (Ed.),
Leyland, A. H., & Groenewegen, P. P. (2020b). What is multilevel modelling? In P. P.Groenewegen (Ed.),
Mahon, D., Minami, T., & Brown, J. (2023). The variability of client, therapist and clinic in psychotherapy outcomes: A three-level hierarchical model.
Mathiasen, K., Andersen, T. E., Riper, H., Kleiboer, A. A. M., & Roessler, K. K. (2016). Blended CBT versus face-to-face CBT: A randomised non-inferiority trial.
Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models.
Nielssen, O., Dear, B. F., Staples, L. G., Dear, R., Ryan, K., Purtell, C., & Titov, N. (2015). Procedures for risk management and a review of crisis referrals from the MindSpot clinic, a national service for the remote assessment and treatment of anxiety and depression.
Nundy, S., Cooper, L. A., & Mate, K. S. (2022). The quintuple aim for health care improvement: A new imperative to advance health equity.
Okiishi, J. C., Lambert, M. J., Eggett, D., Nielsen, L., Dayton, D. D., & Vermeersch, D. A. (2006). An analysis of therapist treatment effects: Toward providing feedback to individual therapists on their clients’ psychotherapy outcome.
Richards, D. A. (2014). Clinical case management supervision. In C. E.Watkins & D. L.Milne (Eds.),
Ricks, D. F. (1974). Supershrink: Methods of a therapist judged successful on the basis of adult outcomes of adolescent patients. In D. F.Ricks, A.Thomas, & M.Roff (Eds.),
Rubin, D. B. (1987).
Schiefele, A. K., Lutz, W., Barkham, M., Rubel, J., Böhnke, J., Delgadillo, J., Kopta, M., Schulte, D., Saxon, D., Nielsen, S. L., & Lambert, M. J. (2017). Reliability of therapist effects in practice-based psychotherapy research: A guide for the planning of future studies.
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006). A brief measure for assessing generalized anxiety disorder: The GAD-7.
Stubbings, D. R., Rees, C. S., Roberts, L. D., & Kane, R. T. (2013). Comparing in-person to videoconference-based cognitive behavioral therapy for mood and anxiety disorders: Randomized controlled trial.
Titov, N., Dear, B. F., Nielssen, O., Wootton, B., Kayrouz, R., Karin, E., Genest, B., Bennett-Levy, J., Purtell, C., Bezuidenhout, G., Tan, R., Minissale, C., Thadhani, P., Webb, N., Willcock, S., Andersson, G., Hadjistavropoulos, H. D., Mohr, D. C., Kavanagh, D. J., …Staples, L. G. (2020). User characteristics and outcomes from a national digital mental health service: An observational study of registrants of the Australian MindSpot Clinic.
Titov, N., Dear, B. F., Staples, L. G., Bennett-Levy, J., Klein, B., Rapee, R. M., Shann, C., Richards, D., Andersson, G., Ritterband, L., Purtell, C., Bezuidenhout, G., Johnston, L., & Nielssen, O. B. (2015). MindSpot clinic: An accessible, efficient, and effective online treatment service for anxiety and depression.
Titov, N., Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Sheehan, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., & McEvoy, P. M. (2015). Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: A randomized controlled trial.
Titov, N., Fogliati, V. J., Staples, L. G., Gandy, M., Johnston, L., Wootton, B., Nielssen, O., & Dear, B. F. (2016). Treating anxiety and depression in older adults: Randomised controlled trial comparing guided
van Agteren, J., Iasiello, M., Lo, L., Bartholomaeus, J., Kopsaftis, Z., Carey, M., & Kyrios, M. (2021). A systematic review and meta-analysis of psychological interventions to improve mental wellbeing.
van Buuren, S. (2018).
Wampold, B. E., & Owen, J. (2021). Therapist effects: History, methods, magnitude, and characteristics of effective therapists. In M.Barkham, W.Lutz, & L. G.Castonguay (Eds.),
Wu, Y., Li, X., Ji, X., Ren, W., Zhu, Y., Chen, Z., & Du, X. (2025). Trends in the epidemiology of anxiety disorders from 1990 to 2021: A global, regional, and national analysis with a focus on the sociodemographic index.