Telehealth Versus Face-to-face Psychotherapy for Less Common Mental Health Conditions: Systematic Review and Meta-analysis of Randomized Controlled Trials (2024)

Reviewed by Jan Lindsay, Merel Keijsers, and Kah Peng Wong

Background

Worldwide, mental health disorders are a leading cause of distress and disability, with 1 in every 4 people expected to be personally impacted throughout their lifetime [1]. Some evidence suggests that mental health difficulties may be increasing; a previous systematic review found a small but significant increase in mental illness prevalence rates from 1978 to 2015, although the authors note that this may have been driven by demographic changes across this period [2]. In addition, the emergence of COVID-19 has seen mental health adversely impacted worldwide [3,4]. This seems to indicate that this already debilitating problem may become a further global burden in the future. Thus, it seems crucial for quality mental health support to be widely available to the public in a safe and accessible way.

Although telehealth was available and suggested to be effective for psychotherapy before the COVID-19 pandemic [5], its uptake was somewhat limited within the delivery of psychological services [6]. A study in the United States found that before the COVID-19 pandemic, psychologists were hesitant to use telehealth owing to lack of training, concerns for client safety, and privacy, among other concerns [7]. In addition, a qualitative study of mental health professionals highlighted concerns around the quality of the patient-therapist relationship [8]. Given the health risks posed by face-to-face meetings, especially for older people or otherwise vulnerable, there was a rapid shift to remote delivery in health care services worldwide [9-12].

Although the pandemic was the catalyst that thrust telehealth to the forefront of health care delivery, there are many advantages to telehealth service provision for mental health. Telehealth extends care to patients with limited access to in-person therapy, including those in rural and remote areas. A narrative review examining telehealth access in rural communities in the United States found telehealth to be a convenient and efficient way to treat patients, and participants reported acceptability and satisfaction with telehealth services [13]. Furthermore, telehealth also offers a safe and effective option for those who may have access issues or face stigmatization [14]. For some conditions, such as substance use disorder, access to therapy delivered remotely may increase engagement with treatment services among groups who would not otherwise attend therapy [15]. For patients being treated for substance abuse, video-delivered treatment was preferred to face-to-face treatment, mostly because of convenience and increased confidentiality [16]. Taken together, the availability of telehealth facilitates increased access of care to those unable or unwilling to engage in face-to-face therapy and promotes continued therapeutic engagement owing to its flexibility and privacy.

Telehealth may also enhance care accessibility for those requiring specialized therapies or those with less common mental health conditions that may not be treated by all clinicians. The skills needed to effectively treat those with less common or more complex mental health conditions or to adequately deliver less common therapy types may require additional training, guided supervision, professional development, or years of clinical experience. This is further compounded in rural and remote areas, where health care disparity is well documented [17-19]. Telehealth presents a potentially effective medium to connect patients requiring specialized forms of care with relevant, qualified therapists.

Objectives

Evidence supports the use of telehealth for application in some psychotherapies [5,20,21] and the management of common mental health conditions, including reviews in this series for depression (Scott AM et al, PhD, unpublished data, February 2022), anxiety [22], and posttraumatic stress disorder (PTSD) [23]. It is important to rigorously assess whether its effectiveness is generalizable beyond these groups. The aim of this systematic review is to assess whether there are any differences between telehealth-based psychotherapy and face-to-face psychotherapy across outcomes (patient, process, and cost) for less common mental health conditions (eg, substance use disorder, eating disorders, or childhood disorders) and physical conditions requiring psychological support (eg, cancer or chronic fatigue syndrome).

Overview

We aim to find, appraise, and synthesize studies that compared psychotherapy delivered via telehealth (video, telephone, or both) versus face-to-face for patients of any age in the primary health care setting. This systematic review is reported following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2009 statement [24], and the review protocol was developed prospectively.

Inclusion and Exclusion Criteria

Search Strategy to Identify Studies

Study Selection and Screening

Titles and abstracts were screened independently by author pairs (AMS, RP, MC, JC, NK, HG, and PG) against the inclusion criteria. In addition, 1 author (JC) retrieved full texts, and 2 authors (HG and NK) screened the full texts for inclusion. Any disagreements were resolved by discussion or reference to the third screener. The forward backward citation analysis was conducted by 1 author (JC) and screened by 3 authors (HG, NK, and RP), and full text was obtained by HG. The study selection process for includable studies is reported in the PRISMA flow diagram (Figure 1), and studies excluded at the full-text screening stage are in Multimedia Appendix 3 with reasons for exclusion.

Data Extraction

We used 2 data extraction forms for study characteristics and outcome data, which were piloted on 2 studies in the review. Data from the included studies were extracted independently by 2 authors (NK and HG) into the data extraction forms (Textbox 1), and discrepancies were resolved by discussion or by reference to a third author.

Risk of Bias in Included Studies

A total of 2 review authors (HG and NK) independently assessed the risk of bias for the included studies using the Cochrane Collaboration Risk of Bias Tool 1, as outlined in the Cochrane Handbook [27], and all disagreements were resolved by discussion.

The following domains were assessed for possible bias: (1) random sequence generation, (2) allocation concealment, (3) the blinding of participants and personnel, (4) the blinding of the outcome assessment, (5) incomplete outcome data, (6) selective outcome reporting, and (7) other bias (focusing on potential biases due to funding or conflict of interest).

Each domain was graded as low, high, or unclear, including quote or summary from the relevant trial, which summarized why the grading was applied.

Data Synthesis and Analysis

Review Manager 5.4, the Cochrane Collaboration tool for conducting meta-analyses and creating forest plots, was used to calculate the treatment effect [28]. As all outcome measures were continuous, we used mean difference or standardized mean difference (SMD). We performed meta-analyses only when possible (when ≥2 studies or comparisons reported the same or similar outcome) and where appropriate data were available that allowed us to calculate the SMD. Where these data were not available and thus meta-analysis was not possible, we narratively report the results. We anticipated a considerable heterogeneity between studies and used a random-effects model.

The unit of analysis was the individual, which was available for every study in this review. We did not contact study authors to provide missing data. We used the I² statistic to examine the heterogeneity of the included studies. Subgroup analyses were conducted according to the duration of follow-up: posttreatment and 3, 6, and 12 months.

As <10 trials were included in any data synthesis, we did not create a funnel plot, and sensitivity analyses were not conducted. We planned to conduct a subgroup analysis of gender, setting, age, and sensitivity by including or excluding studies at high risk of bias; however, the low number of included studies did not allow for this.

Search Results

The primary study search found 5423 references, and 1877 additional references were found in the forward and backward citation search and clinical trial registries. After deduplication, 5536 records were screened in title and abstract. A total of 5493 references were excluded, and 43 full texts were assessed for inclusion. Moreover, 12 RCTs (across 14 articles) were included in this systematic review, and 9 were able to be meta-analyzed (Figure 1). We found 2 potentially relevant but still in-progress clinical trials (Multimedia Appendix 4 [29,30]).

Characteristics of Included Studies

Of the 12 included RCTs, 10 (83%) were conducted in the United States, and the other 2 (17%) studies were conducted in the United Kingdom. A total of 931 patients were included in aggregate. Studies have examined psychotherapy delivered for a variety of less common mental health conditions and other conditions requiring psychotherapy. Of the 12 studies, 2 (17%) included patients with type 1 diabetes mellitus, 2 (17%) included patients with addiction disorders, 1 (8%) (reported in 4 articles) treated patients with bulimia nervosa or eating disorder not otherwise specified, 3 (25%) studies included participants with children’s disorders (including disruptive behavior disorder, tic disorders, and attention-deficit hyperactivity disorder), 2 (17%) included patients with chronic illness (chronic fatigue syndrome and chronic multisymptom illness), 1 (8%) study included patients with a range of mental health conditions, and 1 (8%) included patients with cancer who had high psychological needs. The types of therapies varied by target condition: of the 12 studies, 5 (42%) used CBT, 4 (33%) used a family therapy (parent-child interaction therapy, parent training, and behavioral family systems therapy for diabetes), 2 (17%) used addiction therapies (opioid treatment program and acute therapy service), and 1 (8%) used a cognitive behavioral intervention for tics. Finally, of the 12 studies, 3 (25%) used the telephone to deliver telehealth, 7 (58%) used video, and 1 (8%) had included video and telephone groups, and in 1 (8%) study, it was unclear whether video or telephone was used. All studies compared the telehealth intervention to face-to-face intervention (Table 1).

Risk of Bias

Overall, of the 12 studies, 10 (83%) adequately reported on random sequence generation and selective reporting. Declarations of conflicts of interest and funding (reported under other bias) were adequately reported for only 25% (3/12) of the studies, with the remaining 75% (9/12) not reporting this clearly. Allocation concealment was not clearly reported in most studies, with only 8% (1/12) of the studies reporting this satisfactorily. The blinding of the outcome assessment and incomplete outcome data were at high risk of bias for 25% (3/12) of the studies, with the remaining 75% (9/12) of the studies rated at either unclear or low risk of bias. Notably, the blinding of participants and personnel was a high bias risk for all 12/12 (100%) studies, as the telehealth versus face-to-face nature of the interventions was incompatible with blinding (Figure 2).

Primary Outcome: Global or Symptom Severity

A total of 6 scales across 7 studies were used to report outcomes related to symptom severity (see Multimedia Appendix 5 for a summary of scales used).

In addition, 7 studies reported sufficient data for this outcome and were able to be pooled and meta-analyzed (Figure 3). Data were available for four time point subgroups—immediately after treatment and 3- to 4-, 6-, and 12-month follow-ups.

There were no significant differences in severity outcomes between telehealth and face-to-face therapy immediately after treatment (335 participants; mean difference 0.05, 95% CI −0.17 to 0.27; P=.65) or at any of the follow-up time points, including 3 to 4 months (65 participants; SMD −0.08, 95% CI −0.57 to 0.41; P=.75), 6 months (71 participants; SMD 0.19, 95% CI −0.45 to 0.82; P=.57), and 12 months (106 participants; SMD 0.15, 95% CI −0.23 to 0.53; P=.44).

There was moderately high heterogeneity reported for the 6-month follow-up subgroup (I²=43%).

Secondary Outcomes

Tertiary Outcomes

Principal Findings

This systematic review of 12 trials shows insufficient evidence of a difference between psychotherapy delivered via telehealth (telephone or video) and face-to-face therapy, when treating less common mental health conditions or physical conditions requiring psychological support. There were no significant differences between telehealth and face-to-face delivery for patient outcomes (symptom severity, symptom improvement, or global function), immediately after treatment, or at any follow-up time point. For process outcomes (working alliance or therapeutic quality and client satisfaction), there was no significant difference between telehealth and face-to-face care delivery for either clients or therapists, although 1 study reported no difference between groups for therapist satisfaction. Although financial outcome data on costs were not meta-analyzable, patients with substance abuse disorder valued telehealth therapy more highly than face-to-face therapy, treatment costs were lower for telehealth than for face-to-face therapy for patients with bulimia nervosa (especially over large geographical areas), and the cost of therapists’ time was equivalent, regardless of delivery mode, for patients with cancer receiving CBT. This suggests that telehealth is at least as cost-effective as face-to-face care and potentially perceived as more valuable by the client. Overall, the risk of bias of included studies was unclear, owing to unclear reporting, and blinding of participants was not possible because of the nature of the interventions.

Although we found no significant differences between telehealth and face-to-face delivery of psychotherapy across any outcome, to assess equivalence between telehealth and face-to-face psychotherapy, CIs around the effect estimate should be examined to determine whether they exclude the minimally important difference [44]. In the absence of a prespecified minimally important difference, we accept Cohen cutoff for a small effect (0.2), whereby a CI between (−0.20 and 0.20) suggests equivalence between telehealth and face-to-face therapy, and a CI outside these bounds indicates that the minimally important difference cannot be excluded and there is the possibility of a small effect favoring one or the other intervention. For the primary outcome, symptom severity (Figure 3), immediately after treatment, the upper-bound CI is >0.2, so it is possible that the true effect favors face-to-face therapy. For 12 months after treatment, the CI ranges from a possible small effect favoring telehealth to a possible medium effect favoring face-to-face therapy. The same could be applied to all other time points and outcomes to assess the evidence for equivalence. Although we can demonstrate that there is insufficient evidence of a difference between telehealth- and face-to-face–delivered psychotherapy, we cannot conclude whether they are equivalent, given that the CIs around the effect size are rarely narrow enough to exclude the minimally important difference. For common mental health conditions, there is evidence that telehealth is an effective modality in the provision of psychological therapy as face-to-face therapy. There is some evidence of equivalence between videoconferencing and face-to-face care for depression [20], anxiety [21,22], PTSD [23], and psychotherapy broadly [5]. Furthermore, there is evidence of telephone-delivered therapy being effective for depression and anxiety [45]. Although these reviews suggest comparability between telehealth and face-to-face psychotherapy delivery, they all included nonrandomized and noncontrolled studies, which may introduce bias. This review shows no evidence of difference in patient, process, or cost outcomes between telehealth and face-to-face psychotherapy across more diverse patient groups.

Comparison With Prior Work

This review and meta-analysis shows that telehealth psychotherapy may be similar to face-to-face psychotherapy in treating populations with less common mental health disorders or physical conditions that require psychological support. These synthesized findings support previous primary research suggesting that psychotherapy delivered via telehealth for the treatment of mental health conditions may be comparable with conventional face-to-face therapy. A previous review examined the use of video therapy across a range of mental health conditions, including some of the less common conditions reviewed here, and found video-delivered therapy was equivalent to face-to-face care for outcomes of clinical effectiveness, treatment adherence, and patient satisfaction [14]. In line with our narrative findings for financial outcomes, they also found video therapy to be less costly than face-to-face care. This contrasts with a recent scoping review, finding that telehealth service provision across health care in Australia does not routinely reduce the cost of care delivery [46]. Our findings also support the results of a single-arm study conducted in Japan examining video-delivered CBT, which found that this delivery mode is feasible for the treatment of bulimia nervosa and binge-eating disorders. Previous evidence regarding the impact of telehealth on working alliance is mixed. An RCT examining psychologists’ perceptions of therapeutic alliance in videoconferencing found that therapeutic alliance was rated significantly lower for telehealth than for face-to-face care [47]. A survey conducted during the COVID-19 pandemic of psychotherapists’ experience with remote care found that it was “better than expected” but that telehealth care could not be compared with face-to-face care [48]. In contradiction, results from a more recent survey study found that telehealth was widely accepted by primary mental health care providers [49], although this was not specific to delivery of psychotherapy via telehealth. A recent study examining working alliance via telehealth for anxiety disorders found that these clients had a stronger working alliance with their clinician when treated via telehealth [50]. Our findings, using only data from RCTs, support previous research suggesting that working alliance is as strong in telehealth as it is in face-to-face care. However, further research is needed to fully understand how telehealth changes the client and clinician relationship dynamic and how this may change circ*mstantially based on clinician and client perceptions of telehealth and the patient’s specific treatment needs.

Strengths and Limitations

This review has many strengths, which add weight to our findings and conclusions. We applied rigorous methodology to find includable studies by establishing a prospective protocol and following PRISMA guidelines. Clear, strict inclusion and exclusion criteria allowed for studies in a variety of different health conditions to be synthesized and systematically reviewed. Further, we only included RCTs, and bias was reviewed for all included studies.

However, there are some limitations to our findings. First, although includable, there were no eligible randomized studies available for telehealth treatment of some less common mental health conditions, such as schizophrenia, bipolar disorders, and personality disorders. This limits the generalizability of these findings across these serious mental health concerns. To assess whether treatment of these conditions is feasible by telehealth, evidence beyond randomized trials should be examined or further high-quality research primary conducted. Second, we only included studies of therapies delivered verbally via telephone or video, as this is most similar to the face-to-face nature of primary care, and we intentionally excluded chat-based or self-guided internet therapy modalities. There is emerging evidence to support the efficacy of chat for mental health treatment services [51,52]. There is also a growing body of work on internet-based therapies for the treatment of psychological conditions such as addictions [53], eating disorders [54], and depression [55] and for the delivery of specific therapies such as CBT [56]. Therefore, although these therapeutic approaches are outside the scope of this review, the role of chat-based or self-guided internet therapy cannot be discounted for remote management of mental health difficulties. Third, most included trials were conducted in the United States, with 2 from the United Kingdom. These health care systems may not be comparable in other countries or regions [57], which limits the generalizability of our findings across medical systems internationally. Fourth, the risk of bias in included studies was largely unclear. We were unable to conduct prespecified subgroup analysis excluding studies at high risk of bias, owing to the small number of studies eligible for inclusion. The possibility of risk of bias in included studies should be considered when interpreting these results. Fifth, we included both telephone and video modalities as telehealth and did not conduct a sensitivity analysis to test any differences between these modalities owing to the small number of included studies. It is possible that there may be differences between telephone and video telehealth care, and future studies may explore this. Sixth, although we anticipated heterogeneity and used a random-effects model, some measures of heterogeneity are high. In each of these cases, the maximum number of studies available at the time point was 3, and it is thought that even when appropriate, meta-analysis with a small number of included studies can lead to fluctuations in the I² statistic and should be interpreted with caution [58]. The small number of included studies precluded explorations into heterogeneity, so it is unclear whether heterogeneity observed is solely due to variation between included studies or whether instability of the I² statistic due to the small number of included studies inflated the estimate. Regardless, the presence of heterogeneity highlights differences between included studies and reinforces the need for large, high-quality studies exploring psychotherapy delivered via telehealth versus face-to-face care for less common mental illnesses. Seventh, although the outcomes selected are appropriate for the study question and aims, they are all clinician or patient self-report measures, which are subject to measurement and other biases. Finally, the maximum follow-up time for included studies was 12 months, and there was variability in the follow-up periods among studies. The management of mental illness can be chronic or lifelong, so our results do not speak to the effectiveness of telehealth for longer-term management of these conditions.

Clinical and Research Implications

There are some important clinical implications of this research. To date, there has been some reported hesitancy from clinicians to use telehealth in their practice [6]. This appears driven by care providers rather than care receivers; patients report equal satisfaction and experience of therapeutic alliance when receiving individual care via telehealth versus face-to-face [59]. Therapist hesitancy may be due to lack of training, concerns about the quality of the therapeutic alliance including rapport building, ethical concerns around risk management, and technological limitations [7,60-62]. Given the increasing body of evidence demonstrating the similarity of mental health care delivered via telehealth compared with face-to-face, it is critical that therapist barriers toward telehealth modalities be addressed. This may take various potential forms, including the provision of training for the delivery of specific therapies via telehealth, which could be incorporated into professional development or tertiary training. Furthermore, regulatory bodies (eg, Australian Health Practitioner Regulation Agency in Australia) could also provide support and advice for the implementation of telehealth infrastructure such as billing processes or technical logistics.

In addition to these clinical implications, there are several possible directions for further research. Given the diverse range of patient populations, therapies, and psychological conditions that may be treated using telehealth, and the multiple modes of care delivery (ie, telephone, video, or blended), it may be beneficial to investigate how to optimize telehealth therapy for various patient groups. Future considerations could include understanding whether certain conditions are better suited to video or telephone delivery and whether telehealth is as effective when treating complex or comorbid mental illnesses and identifying whether there are any groups for which telehealth is not recommended. Developing specific and structured protocols or guidelines for the delivery of psychotherapy via telehealth to diverse patient groups will help ensure the consistent provision of best-practice telehealth care.

Conclusions

The COVID-19 pandemic has pushed telehealth to the forefront of mental health care out of necessity. This review shows that there is insufficient evidence of difference between psychotherapy delivered via telehealth and psychotherapy delivered via face-to-face care for the management of less common mental and physical health conditions requiring psychological support. There was insufficient evidence of difference between groups across patient, process, and cost outcomes, including symptom severity, improvement, function, therapeutic working alliance, satisfaction, and cost. However, CIs often included the minimally important difference, so we cannot conclude whether psychotherapy delivered via telehealth versus via face-to-face are equivalent. Further research is needed to assess the efficacy of telehealth for some conditions for which this review found no evidence (such as schizophrenia and bipolar disorders) and to optimize the delivery of telehealth interventions across diverse patient groups. The current evidence indicates that psychotherapy delivered via telehealth may be an alternative to face-to-face psychotherapy for the treatment of less common mental health conditions and physical conditions requiring psychological care.

Conflicts of Interest: None declared.

2. Richter D, Wall A, Bruen A, Whittington R. Is the global prevalence rate of adult mental illness increasing? Systematic review and meta-analysis. Acta Psychiatr Scand. 2019 Nov;140(5):393–407. doi:10.1111/acps.13083. doi:10.1111/acps.13083. [PubMed] [CrossRef] [CrossRef] [Google Scholar]

3. Torales J, O'Higgins M, Castaldelli-Maia JM, Ventriglio A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int J Soc Psychiatry. 2020 Jun;66(4):317–20. doi:10.1177/0020764020915212. [PubMed] [CrossRef] [Google Scholar]

5. Backhaus A, Agha Z, Maglione ML, Repp A, Ross B, Zuest D, Rice-Thorp NM, Lohr J, Thorp SR. Videoconferencing psychotherapy: a systematic review. Psychol Serv. 2012 May;9(2):111–31. doi:10.1037/a0027924.2012-14616-001 [PubMed] [CrossRef] [Google Scholar]

7. Pierce BS, Perrin PB, McDonald SD. Pre-COVID-19 deterrents to practicing with videoconferencing telepsychology among psychologists who didn't. Psychol Serv. 2020 Nov 30;:157–66. doi:10.1037/ser0000435.2020-87907-001 [PubMed] [CrossRef] [Google Scholar]

8. Hoffmann M, Wensing M, Peters-Klimm F, Szecsenyi J, Hartmann M, Friederich H, Haun MW. Perspectives of psychotherapists and psychiatrists on mental health care integration within primary care via video consultations: qualitative preimplementation study. J Med Internet Res. 2020 Jun 18;22(6):e17569. doi:10.2196/17569. https://www.jmir.org/2020/6/e17569/v22i6e17569 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

9. Chang JE, Lai AY, Gupta A, Nguyen AM, Berry CA, Shelley DR. Rapid transition to telehealth and the digital divide: implications for primary care access and equity in a post-covid era. Milbank Q. 2021 Jun;99(2):340–68. doi:10.1111/1468-0009.12509. doi:10.1111/1468-0009.12509. [PMC free article] [PubMed] [CrossRef] [CrossRef] [Google Scholar]

11. Punia V, Nasr G, Zagorski V, Lawrence G, Fesler J, Nair D, Najm I. Evidence of a rapid shift in outpatient practice during the COVID-19 pandemic using telemedicine. Telemed J E Health. 2020 Oct;26(10):1301–3. doi:10.1089/tmj.2020.0150. [PubMed] [CrossRef] [Google Scholar]

12. Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D, Curtis S, Roman M, Poon EG, Ferranti J, Katz JN, Tcheng J. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc. 2020 Apr 20;:957–62. doi:10.1093/jamia/ocaa067. http://europepmc.org/abstract/MED/32311034.5822868 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

14. Fletcher TL, Hogan JB, Keegan F, Davis ML, Wassef M, Day S, Lindsay JA. Recent advances in delivering mental health treatment via video to home. Curr Psychiatry Rep. 2018 Jul 21;20(8):56. doi:10.1007/s11920-018-0922-y.10.1007/s11920-018-0922-y [PubMed] [CrossRef] [Google Scholar]

15. Postel MG, de Jong CA, de Haan HA. Does e-therapy for problem drinking reach hidden populations? Am J Psychiatry. 2005 Dec;162(12):2393. doi:10.1176/appi.ajp.162.12.2393.162/12/2393 [PubMed] [CrossRef] [Google Scholar]

16. King VL, Stoller KB, Kidorf M, Kindbom K, Hursh S, Brady T, Brooner RK. Assessing the effectiveness of an Internet-based videoconferencing platform for delivering intensified substance abuse counseling. J Subst Abuse Treat. 2009 Apr;36(3):331–8. doi:10.1016/j.jsat.2008.06.011.S0740-5472(08)00121-9 [PubMed] [CrossRef] [Google Scholar]

18. Pong RW, Desmeules M, Lagacé C. Rural-urban disparities in health: how does Canada fare and how does Canada compare with Australia? Aust J Rural Health. 2009 Feb;17(1):58–64. doi:10.1111/j.1440-1584.2008.01039.x. doi:10.1111/j.1440-1584.2008.01039.x.AJR1039 [PubMed] [CrossRef] [CrossRef] [Google Scholar]

19. Smith KB, Humphreys JS, Wilson MG. Addressing the health disadvantage of rural populations: how does epidemiological evidence inform rural health policies and research? Aust J Rural Health. 2008 Apr;16(2):56–66. doi:10.1111/j.1440-1584.2008.00953.x.AJR953 [PubMed] [CrossRef] [Google Scholar]

20. Berryhill MB, Culmer N, Williams N, Halli-Tierney A, Betancourt A, Roberts H, King M. Videoconferencing Psychotherapy and Depression: A Systematic Review. Telemed J E Health. 2019 Jun;25(6):435–446. doi:10.1089/tmj.2018.0058. [PubMed] [CrossRef] [Google Scholar]

21. Berryhill MB, Halli-Tierney A, Culmer N, Williams N, Betancourt A, King M, Ruggles H. Videoconferencing psychological therapy and anxiety: a systematic review. Fam Pract. 2019 Jan 25;36(1):53–63. doi:10.1093/fampra/cmy072.5090669 [PubMed] [CrossRef] [Google Scholar]

22. Krzyzaniak N, Greenwood H, Scott AM, Peiris R, Cardona M, Clark J, Glasziou P. The effectiveness of telehealth versus face-to face interventions for anxiety disorders: a systematic review and meta-analysis. J Telemed Telecare. 2021 Dec 03;:1357633X211053738. doi:10.1177/1357633X211053738. [PubMed] [CrossRef] [Google Scholar]

23. Scott AM, Bakhit M, Greenwood H, Cardona M, Clark J, Krzyzankiak N. Real-time telehealth versus face-to-face management for patients with PTSD in primary care: a systematic review and meta-analysis (forthcoming) J Clin Psychiatry. 2022;Forthcoming [PubMed] [Google Scholar]

25. Clark JM, Sanders S, Carter M, Honeyman D, Cleo G, Auld Y, Booth D, Condron P, Dalais C, Bateup S, Linthwaite B, May N, Munn J, Ramsay L, Rickett K, Rutter C, Smith A, Sondergeld P, Wallin M, Jones M, Beller E. Improving the translation of search strategies using the Polyglot Search Translator: a randomized controlled trial. J Med Libr Assoc. 2020 Apr;108(2):195–207. doi:10.5195/jmla.2020.834. http://europepmc.org/abstract/MED/32256231.jmla-108-195 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

27. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M. Cochrane handbook for systematic reviews of interventions. Hoboken, New Jersey, United States: John Wiley & Sons; 2019. [PMC free article] [PubMed] [Google Scholar]

29. TIA: a feasibility study of telemedicine in addictions. ISRCTN Registry Identifier: ISRCTN36756455. 2021. [2022-02-21]. https://www.isrctn.com/ISRCTN36756455.

31. Burgess M, Andiappan M, Chalder T. Cognitive behaviour therapy for chronic fatigue syndrome in adults: face to face versus telephone treatment: a randomized controlled trial. Behav Cogn Psychother. 2012 Mar;40(2):175–91. doi:10.1017/S1352465811000543.S1352465811000543 [PubMed] [CrossRef] [Google Scholar]

32. Comer JS, Furr JM, Miguel EM, Cooper-Vince CE, Carpenter AL, Elkins RM, Kerns CE, Cornacchio D, Chou T, Coxe S, DeSerisy M, Sanchez AL, Golik A, Martin J, Myers KM, Chase R. Remotely delivering real-time parent training to the home: an initial randomized trial of Internet-delivered parent-child interaction therapy (I-PCIT) J Consult Clin Psychol. 2017 Sep;85(9):909–17. doi:10.1037/ccp0000230.2017-27639-001 [PubMed] [CrossRef] [Google Scholar]

33. Day SX, Schneider PL. Psychotherapy using distance technology: a comparison of face-to-face, video, and audio treatment. J Counsel Psychol. 2002;49(4):499–503. doi:10.1037/0022-0167.49.4.499. [CrossRef] [Google Scholar]

36. Himle MB, Freitag M, Walther M, Franklin SA, Ely L, Woods DW. A randomized pilot trial comparing videoconference versus face-to-face delivery of behavior therapy for childhood tic disorders. Behav Res Ther. 2012 Sep;50(9):565–70. doi:10.1016/j.brat.2012.05.009.S0005-7967(12)00091-5 [PubMed] [CrossRef] [Google Scholar]

38. McAndrew LM, Greenberg LM, Ciccone DS, Helmer DA, Chandler HK. Telephone-based versus in-person delivery of cognitive behavioral treatment for veterans with chronic multisymptom illness: a controlled, randomized trial. Mil Behav Health. 2018;6(1):56–65. doi:10.1080/21635781.2017.1337594. http://europepmc.org/abstract/MED/31192051. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

39. Crow SJ, Mitchell JE, Crosby RD, Swanson SA, Wonderlich S, Lancanster K. The cost effectiveness of cognitive behavioral therapy for bulimia nervosa delivered via telemedicine versus face-to-face. Behav Res Ther. 2009 Jun;47(6):451–3. doi:10.1016/j.brat.2009.02.006. http://europepmc.org/abstract/MED/19356743.S0005-7967(09)00042-4 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

40. Ertelt TW, Crosby RD, Marino JM, Mitchell JE, Lancaster K, Crow SJ. Therapeutic factors affecting the cognitive behavioral treatment of bulimia nervosa via telemedicine versus face-to-face delivery. Int J Eat Disord. 2011 Dec;44(8):687–91. doi:10.1002/eat.20874. [PMC free article] [PubMed] [CrossRef] [Google Scholar]

41. Mitchell JE, Crosby RD, Wonderlich SA, Crow S, Lancaster K, Simonich H, Swan-Kremeier L, Lysne C, Myers TC. A randomized trial comparing the efficacy of cognitive-behavioral therapy for bulimia nervosa delivered via telemedicine versus face-to-face. Behav Res Ther. 2008 May;46(5):581–92. doi:10.1016/j.brat.2008.02.004. http://europepmc.org/abstract/MED/18374304.S0005-7967(08)00032-6 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

42. Watson M, White C, Lynch A, Mohammed K. Telephone-delivered individual cognitive behavioural therapy for cancer patients: an equivalence randomised trial. Psychooncology. 2017 Mar;26(3):301–8. doi:10.1002/pon.4338. [PubMed] [CrossRef] [Google Scholar]

43. Xie Y, Dixon JF, Yee OM, Zhang J, Chen YA, Deangelo S, Yellowlees P, Hendren R, Schweitzer JB. A study on the effectiveness of videoconferencing on teaching parent training skills to parents of children with ADHD. Telemed J E Health. 2013 Mar;19(3):192–9. doi:10.1089/tmj.2012.0108. [PubMed] [CrossRef] [Google Scholar]

44. Treadwell JR, Uhl S, Tipton K, Shamliyan T, Viswanathan M, Berkman ND, Sun X, Coleman CI, Elshaug AG, Singh S, Wang S, Ramakrishnan R. Assessing equivalence and noninferiority. J Clin Epidemiol. 2012 Nov;65(11):1144–9. doi:10.1016/j.jclinepi.2012.05.001.S0895-4356(12)00115-1 [PubMed] [CrossRef] [Google Scholar]

45. Coughtrey AE, Pistrang N. The effectiveness of telephone-delivered psychological therapies for depression and anxiety: a systematic review. J Telemed Telecare. 2016 Dec 30;24(2):65–74. doi:10.1177/1357633x16686547. [PubMed] [CrossRef] [Google Scholar]

47. Rees CS, Stone S. Therapeutic Alliance in Face-to-Face Versus Videoconferenced Psychotherapy. Professional Psychology: Research and Practice. 2005 Dec;36(6):649–653. doi:10.1037/0735-7028.36.6.649. [CrossRef] [Google Scholar]

48. Humer E, Stippl P, Pieh C, Pryss R, Probst T. Experiences of Psychotherapists With Remote Psychotherapy During the COVID-19 Pandemic: Cross-sectional Web-Based Survey Study. J Med Internet Res. 2020 Nov 27;22(11):e20246. doi:10.2196/20246. https://www.jmir.org/2020/11/e20246/v22i11e20246 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

49. Malas N, Klein E, Tengelitsch E, Kramer A, Marcus S, Quigley J. Exploring the Telepsychiatry Experience: Primary Care Provider Perception of the Michigan Child Collaborative Care (MC3) Program. Psychosomatics. 2019 Mar;60(2):179–189. doi:10.1016/j.psym.2018.06.005.S0033-3182(18)30312-8 [PubMed] [CrossRef] [Google Scholar]

50. Watts S, Marchand A, Bouchard S, Gosselin P, Langlois F, Belleville G, Dugas MJ. Telepsychotherapy for generalized anxiety disorder: impact on the working alliance. J Psychother Integrat. 2020 Jun;30(2):208–25. doi:10.1037/int0000223. [CrossRef] [Google Scholar]

51. Dowling M, Rickwood D. Online counseling and therapy for mental health problems: a systematic review of individual synchronous interventions using chat. J Technol Hum Serv. 2013 Jan;31(1):1–21. doi:10.1080/15228835.2012.728508. [CrossRef] [Google Scholar]

52. ho*rmann S, McCabe KL, Milne DN, Calvo RA. Application of synchronous text-based dialogue systems in mental health interventions: systematic review. J Med Internet Res. 2017 Jul 21;19(8):e267. doi:10.2196/jmir.7023. http://www.jmir.org/2017/8/e267/v19i8e267 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

53. Gainsbury S, Blaszczynski A. A systematic review of internet-based therapy for the treatment of addictions. Clin Psychol Rev. 2011 Apr;31(3):490–8. doi:10.1016/j.cpr.2010.11.007.S0272-7358(10)00177-7 [PubMed] [CrossRef] [Google Scholar]

55. Josephine K, Josefine L, Philipp D, David E, Harald B. Internet- and mobile-based depression interventions for people with diagnosed depression: a systematic review and meta-analysis. J Affect Disord. 2017 Dec 01;223:28–40. doi:10.1016/j.jad.2017.07.021.S0165-0327(17)30725-5 [PubMed] [CrossRef] [Google Scholar]

56. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry. 2014 Oct;13(3):288–95. doi:10.1002/wps.20151. doi:10.1002/wps.20151. [PMC free article] [PubMed] [CrossRef] [CrossRef] [Google Scholar]

57. Schütte S, Acevedo PN, Flahault A. Health systems around the world - a comparison of existing health system rankings. J Glob Health. 2018 Jun;8(1):010407. doi:10.7189/jogh.08.010407. doi:10.7189/jogh.08.010407.jogh-08-010407 [PMC free article] [PubMed] [CrossRef] [CrossRef] [Google Scholar]

58. Thorlund K, Imberger G, Johnston BC, Walsh M, Awad T, Thabane L, Gluud C, Devereaux PJ, Wetterslev J. Evolution of heterogeneity (I2) estimates and their 95% confidence intervals in large meta-analyses. PLoS One. 2012;7(7):e39471. doi:10.1371/journal.pone.0039471. https://dx.plos.org/10.1371/journal.pone.0039471.PONE-D-12-03154 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

59. Jenkins-Guarnieri MA, Pruitt LD, Luxton DD, Johnson K. Patient perceptions of telemental health: systematic review of direct comparisons to in-person psychotherapeutic treatments. Telemed J E Health. 2015 Aug;21(8):652–60. doi:10.1089/tmj.2014.0165. [PubMed] [CrossRef] [Google Scholar]

61. Brooks E, Turvey C, Augusterfer EF. Provider barriers to telemental health: obstacles overcome, obstacles remaining. Telemed J E Health. 2013 Jun;19(6):433–7. doi:10.1089/tmj.2013.0068. [PubMed] [CrossRef] [Google Scholar]

62. Lopez A, Schwenk S, Schneck CD, Griffin RJ, Mishkind MC. Technology-based mental health treatment and the impact on the therapeutic alliance. Curr Psychiatry Rep. 2019 Jul 08;21(8):76. doi:10.1007/s11920-019-1055-7.10.1007/s11920-019-1055-7 [PubMed] [CrossRef] [Google Scholar]