EXECUTIVE SUMMARY

  1.  Psychological therapies are an important part of the delivery of health care within the NHS and the private sector. They are highly valued by patients who increasingly choose counselling and psychotherapy in preference to medication.

  2.  NICE guidelines now exist to support the delivery of psychological therapies across a range of mental health conditions, including depression and anxiety.

  3.  The NICE guideline development process is robust and transparent. NICE guidelines are based on evidence reviews, with systematic review and randomised controlled trial (RCT) evidence given most weight.

  4.  There are two disadvantages of maintaining this rigid hierarchy of evidence:

  5.  The first disadvantage relates to the lack of systematic review and RCT evidence for the psychological therapies:

  6.  Mental health research is seriously under-funded. RCTs are expensive, as are systematic reviews to synthesise RCT data. There is limited systematic review and RCT evidence for the efficacy of psychological therapies, with the exception of CBT for a range of conditions. Many psychological therapies remain unevaluated by RCT. Therefore NICE guidelines are based on a robust but very narrow evidence base.

  7.  BACP has concerns, therefore, about gaps in the evidence and in service recommendations based on a restricted evidence base. Reliance on a limited range of evidence based treatments may disadvantage patients through restricting patient choice for and access to a range of interventions and over-resource standard treatments that are not panaceas and will not suit all patients.

  8.  The second disadvantage relates to the downgrading of other types of research evidence, such as case studies and effectiveness studies, which are needed to assess not only whether a treatment works, but if and how it works in practice.

  9.  Studies that show that a therapy can work in the trial context must be complemented by other methodologies (such as audit and benchmarking) that can assure that their delivery in routine settings (such as the NHS) is still producing positive outcomes. It is important to assess not only whether a treatment works, but how it works in practice.

  10.  BACP recommends that NICE reviews its evidence evaluation process to admit a range of quantitative and qualitative evidence in the evaluation of psychological therapies, including highly controlled studies, case studies and effectiveness studies.

  11.  Besides the ways in which NICE's evaluation may disadvantage certain groups of patients, BACP has concerns about the implementation of NICE guidance:

  12.  There is concern that NICE guidelines for psychological therapies might be used as a basis for new commissioning strategies or for re-designing existing psychological therapies when the evidence underlying their recommendations does not support this.

  13.  Implementation of NICE guidance based on a narrow evidence base will severely limit treatment options for patients at a time when the Government is responding to public concern about lack of access to, and health inequities in, the provision of psychological therapies, and prioritising patient choice.

  14.  The concerns stated here are shared not only by BACP but also by psychological therapists in other professional bodies and by researchers in both academic and practice settings. This lack of confidence in the evaluation process in itself constitutes a challenge to the NICE decision making process.

AREA OF EXPERTISE

  15.  BACP is recognised by legislators, national and international organisations and the public, as the leading professional body and the voice of counselling and psychotherapy in the United Kingdom; with over 26,000 members working to the highest professional standards.

NICE's Evaluation Process, and Whether any Particular Groups are Disadvantaged by the Process

  16.  NICE guidelines are based on evidence reviews, with systematic review and RCT evidence given most weight. The guidelines use predetermined and systematic methods to identify and evaluate evidence relating to the specific condition in question. Where evidence is lacking, the guidelines incorporate statements and recommendations based upon consensus statements developed by the guideline development group.

  17.  NICE acknowledges that clinical guidelines have limitations and that "they are not a substitute for professional knowledge and clinical judgement. They can be limited in their usefulness and applicability by a number of different factors (including) the availability of high quality research evidence ... (and) the generalisability of research findings" (1).

  18.  BACP applauds the transparency and rigour of the NICE evidence review process. However, we believe that the current NICE evaluation process, based on a rigid hierarchy of evidence, disadvantages the psychological therapies (and thus the patients receiving therapy) on several counts:

  19.  Mental health has long been under-researched and under funded. The lack of research funding for the psychological therapies compared with the funding available to evaluate pharmacological and other technologies means that there is limited RCT evidence for the efficacy of psychological therapies (2).

  20.  RCTs are able to indicate whether or not a therapy works and which therapy works best, as well as indicating when therapies are actually doing more harm than good. However, RCTs are expensive and there is limited RCT evidence for psychological therapies which means that many therapies are unevaluated.

  21.  Because NICE guidelines utilise a hierarchy of evidence that places systematic reviews and RCTs at the top, and because there are very few highly controlled trials of psychological therapies, NICE guidelines for psychological therapies make recommendations based on a very narrow evidence base.

  22.  BACP has concerns, therefore, about gaps in the evidence and in service recommendations based on a restricted evidence base. Reliance on a limited range of evidence based treatments may disadvantage patients through restricting patient choice for and access to a range of interventions and over-resource standard treatments that are not panaceas and will not suit all patients.

  23.  Because many therapies are not evaluated by RCT, they tend to be excluded from NICE guidelines, or, if they are included because based on consensus statements, they tend not to be recommended as first line treatments. NICE repeatedly states that "It is important to remember that the absence of empirical evidence for the effectiveness of a particular intervention is not the same as evidence for ineffectiveness" (1) but the current hierarchy of evidence inevitably excludes or downgrades non RCT evidence.

  24.  When seeking evidence of causal relationships, or unbiased comparisons of treatments, RCT methodology is likely to be the method of choice in most circumstances. However, even among those who accept the primacy of RCTs as a method of scientific evaluation, there are a number of criticisms of their applicability to routine service provision:

  25.  Psychological therapy does not lend itself easily to evaluation by RCT. The biomedical paradigm underlying much RCT evidence is reflected in the wide use of manualised treatments for patients with DSM based diagnoses. By contrast, most patients present in the NHS with wide-ranging difficulties such as marital problems, bereavement, problems associated with ill health and so on. Patients may be worried, anxious or depressed, they may have multiple problems; they do not always fit neatly into diagnostic categories.

  26.  Certain types of manualised (or manualisable) interventions, such as Cognitive Behavioural Therapy (CBT) or Interpersonal Therapy (IPT) lend themselves to evaluation by RCT. Other types of intervention (such as the commonly practiced integrative therapy) are less easily evaluated in trial settings. This means that there is a robust evidence base for certain types of interventions such as CBT and IPT, but there is less RCT evidence for other psychological therapies. No evidence of effectiveness is all too often construed as evidence of no effectiveness in a system that gives most weight to systematic review and RCT evidence. At the same time, over-reliance on evidence from RCTs which use strictly manualised treatments can lead to inappropriate assumptions about the effectiveness of a particular "brand name" therapy (3).

  27.  There is evidence to suggest that the quality of the therapist-patient relationship or therapeutic alliance is relevant to treatment efficacy (4). Trial evidence tends to focus on the type of treatment offered (eg, CBT or IPT) rather than the person giving it. By giving most weight to reviews and RCTs, guidelines may promote best technologies over best practitioners.

  28.  Psychological therapies are by definition relational therapies. The therapeutic alliance between therapist and patient is an important influencing variable in terms of outcome; it tends to be best captured by qualitative research and case studies.

  29.  Studies that show a therapy can work in the trial context must be complemented by other methodologies (such as audit and benchmarking) that can assure that their delivery in routine settings is still producing positive outcomes. For example, the "hourglass" model of treatment development has been described (5) in which highly controlled studies are relevant for only one portion of the development cycle, while less controlled methods (such as case studies and effectiveness studies) have crucial roles early and late in the development of the therapy respectively. The relationship between RCT based evidence and systematic data collection from routine settings (audit, benchmarking, quality evaluation) and the role of qualitative research need to be reviewed in order to improve the NICE evaluation process, and to make the ensuing guidelines applicable to the NHS.

Concerns about the Implementation of NICE Guidance and Corresponding Challenges to NICE Decisions

  30.  Current government initiatives have placed considerable emphasis on "patient choice". However, there is an increasing focus on a single model of psychological therapy—cognitive-behavioural therapy (CBT)—because of its robust RCT evidence base. When other forms of psychological intervention have been compared with CBT in DH funded RCTs, findings have shown broad equivalence of outcomes, for example, in depression and anxiety in primary care (6). However, the weight of evidence for CBT has tended to mean that these therapies are overlooked.

  31.  The over reliance on CBT evidence has led to the identification of a shortage of CBT practitioners which requires additional funding to correct. This approach has generated artificial problems regarding resources (ie, practitioners) to deliver psychological therapies.

  32.  There is a growing debate within the area of the psychological therapies as to the contribution (ie effectiveness) of practitioners versus specific therapies. There has been research arguing for both sides of the case (7), although it is becoming clear that the effectiveness of practitioners may be of at least equal importance (8).

  33.  To date, RCTs have investigated technologies, rather than practitioner effects. The contribution and variability of practitioners is an important component which is currently being determined from analyses of large data sets collected from routine NHS mental health settings (9). But because this data has not been collected within an RCT, it is not included in the NICE hierarchy of evidence. Such a strategy places NICE at a distance from everyday practitioners and does not facilitate practitioners adopting and implementing NICE guidance.

CONCLUSION

  34.  BACP considers the instigation of NICE, with its rigorous and transparent hierarchy of evidence, to have been a major step forward in the development of evidence based guidelines for the psychological therapies within the NHS.

  35.  However, public confidence is waning in NICE guidance because its recommendations do not reflect NHS practice. Reliance on robust systematic review and RCT evidence currently leads to an over emphasis on certain brand name therapies (CBT, IPT) with resulting narrow recommendations which the practitioner in the NHS finds hard to equate with the complexity of problems with which patients present in routine NHS settings.

  36.  Given the diversity of human beings, we need to ensure that patient choice is a reality by funding research into psychological approaches other than CBT.

  37.  The relationship between RCT evidence and systematic data collection from routine settings (audit, benchmarking, quality evaluation) and the role of qualitative research need to be reviewed in order to improve the NICE evaluation process (and its hierarchy of evidence) so as to make NICE guidelines relevant and applicable to the NHS.

RECOMMENDATIONS

  38.  We recommend that the Government should set up a review of the evidence hierarchy which NICE relies on for its mental health guidelines, to investigate the impact of current criteria for evaluating research into psychological therapies and consequent clinical guidelines on patient choice, innovative services, and patient care.

  39.  Future guideline development groups set up by NICE for mental health guidelines should have a broader balance and cross-section of professional stake holders and peer reviewers to try to ensure researcher-allegiance bias does not distort the guideline development process. These appointments should be transparent and decided by elected representatives from the stake holder organisations.

  40.  NICE should publish the estimated costs of implementing mental health guidelines in terms of treating unmet need, delivering new psychological treatments, workforce and training implications and service redesign. These monies should be ring-fenced as additional investment provided via Strategic Health Authorities before clinical guidelines are issued.

  41.  Prior to NICE's review of its Depression and Anxiety guidelines in 2008, an evaluation of what impact they have had, and whether they are being implemented, should be undertaken by the Audit Commission. Where implementation is patchy or slow, a commissioning strategy should be included as part of the review process for clinical guidelines.

  42.  The Department of Health should work with NICE, the professional bodies in psychological therapies and the mental health charities, to agree a national research programme, which identifies the gaps in the evidence (across all the mental health guidelines), and priorities for research, and provide funding for these to be undertaken as an important part of the development and implementation programme for NICE guidelines.

  43.  NICE and the Department of Health should work with the professional bodies, with research departments for psychological therapies and with mental health research charities to establish an evaluation and audit infrastructure within NHS services which will enable ongoing improvements in practice, and better monitoring of whether clinical guidelines are having beneficial impacts on patient care.

REFERENCES

1.  NICE (2007). Drug misuse: opiate detoxification for drug misuse. Draft clinical practice guideline.

2.  The Sainsbury Centre for Mental Health (2007). We need to talk; the case for psychological therapy on the NHS.

3.  Bower P and King M (2000). Randomised controlled trials and the evaluation of psychological therapy, in N Rowland and S Goss (eds) Evidence-based counselling and psychological therapies; Research and applications, London: Routledge.

4.  Roth A and Fonagy P (1996). What works for whom? A critical review of psychotherapy research. London: Guilford.

5.  Salkovskis P (1995). Demonstrating specific effects in cognitive and behavioural therapy, in M Aveline and D Shapiro (eds) Research foundations for psychotherapy practice, Chichester: Wiley.

6.  King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, Byford S (2000). Randomised controlled trial of non-directive counselling, cognitive-behavioural therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technology Assessment, 4 (19).

7.  Psychotherapy Research, 2006, 16, 143-187.

8.  Lutz W, Leon SC, Martinovitch Z, Lyons JS, Stiles WB (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counselling Psychology, 54, 32-39.

9.  Mellor Clark J, Barkham M, Mothersole G, McInnes B, Evans R (2006). Reflections on benchmarking NHS primary care psychological therapies and counselling. Counselling and Psychotherapy Research, 6; 1: 81-87.

Nancy Rowland

British Association for Counselling and Psychotherapy

March 2007