EXECUTIVE SUMMARY

  1.  Psychological therapies are a crucial part of the delivery of health care within the NHS and NICE guidelines now exist to support these being commissioned across a range of mental health conditions. Government is also responding to public concern about lack of access and serious health inequities in the provision of psychological therapies, as recommended by NICE, through Department of Health pilot initiatives such as Improving Access to Psychological Therapies. However, there are a range of problems with implementing NICE's guidelines in mental health. Moreover, the systematic failure of implementation results in a reduction in public and professional confidence in NICE as well as a failure to deliver evidence-based health care, at an estimated cost of £64 billion in long-term incapacity and ill-health. [159]The mental health guidelines have questionable utility.

  2.  The main focus for NICE is on new developments in drugs. The conventional gold standard methods of randomised controlled trials (RCTs) in which patients can be "blind" to the competing medication is an appropriate method for scientific evaluation as to whether new drugs are cost-effective but is more problematic when applied to the area of psychological interventions. This creates a gap between treatment recommendations in mental health guidelines which are not being systematically tested in routine practice settings. This results in unrealistic assumptions, therefore, in terms of service redesign, and which ignore the capacity of NHS clinicians and service managers to deliver these new treatment approaches.

  3.  Without a scientific methodology which properly evaluates the evidence base for psychological therapies as they occur in NHS settings, the resulting recommendations based mainly on RCTs and small patient samples can identify a treatment approach which is most effective but misidentify or fail to identify the treatment which is most cost-effective. The risk is that when investment is available to implement NICE guidance, this will not lead to improved mental health outcomes.

  4. There are a range of concerns, therefore, about gaps in the kind of evidence, which NICE's existing guidelines for mental health mainly rely on. These are:

  4.1  A restricted model of science means that NICE guidance might be potentially misleading when generalized to patient populations in routine clinical practice [see 6 to 6.6]

  4.2  Reliance on treatments tested only by RCTs threatens to restrict patient choice for important interventions, or individualized combinations of interventions, and over-resource standard treatments that are not panaceas and will not suit all patients [see 7 to 7.7]

  4.3  The undue weighting given to evidence of treatment efficacy, which assumes it is the treatment model which matters, fails to account for evidence that what often matters more to a patient is the quality and experience of the practitioner, which then leads guidelines towards promoting best technologies over best practitioners [see 8 to 8.3]

  4.4  Our limited evidence base for comparing different psychological therapies with each other in trials can be extrapolated for use in cost-benefit analyses, but probably only cautiously, to add value to existing services. There is concern that NICE guidelines might be misused as a basis for new commissioning strategies or for redesigning existing NHS psychological therapies when the evidence underlying their recommendations does not support this. No national audit has been undertaken for NICE's mental health guidelines, so the beneficial impacts of NICE guidelines on cost-effectiveness, if any, are not yet known [see 9 to 9.3]

AREA OF EXPERTISE

  5.  The signatories of this submission collectively represent an expert panel in the areas of research and delivery of psychological therapies in NHS service settings. We represent a wide range of approaches to the psychological therapies as well as a thorough knowledge base of the research evidence and methodologies associated with them.

WHY NICE'S DECISIONS ARE INCREASINGLY BEING CHALLENGED AND CONFIDENCE WANING

  6.  Because current NICE policy adopts a single/restricted model of science. Much of the evidence gathered from randomised controlled trials (RCTs) is derived from studies that do not correspond with routine NHS practice in the area of the psychological therapies. This is important in that it means that much of the evidence used in NICE guidelines does not bear directly on the indications for, or effectiveness of, NHS psychological therapies.

  6.1  Patients are selected in trials according to very restricted clinical criteria that do not accord with the presenting problems of patients in the NHS. By contrast, most patients present in the NHS with wide-ranging difficulties. Although the former provides a good basis for controlled science, it is too restricting when it comes to "applied science" which is what is required in NHS settings.

  6.2  Randomised allocation of patients to different treatment groups is not what happens in routine clinical practice where patients are treated as individuals with particular needs and in many cases clear preferences for their type of therapy. Further, NHS service delivery is very much a function of costs and resources, both of which are not considered sufficiently within RCTs.

  6.3  The result from any RCT is, in effect, only a single observation. The logic of RCTs requires multiple (not just one or two) replications in order to build a robust knowledge base. Building an evidence-base on a single RCT is unsafe. However, some current NICE guidelines (eg, for depression) cite the highest level of evidence (termed level "A") to include a single RCT. This is a relatively low level of evidence as compared with meta-analytic studies (ie, the study of multiple findings from multiple RCTs) which are also graded as level "A" evidence.

  6.4  There is a widely recognised effect within research termed "researcher allegiance".[160] In effect, this states that researchers research the type of psychological therapy to which they have some form of allegiance. Not surprisingly, most of the findings from RCTs involving psychological therapies arrive at findings which support the therapeutic approach to which the researcher has an allegiance. This is probably one of the most pervasive and powerful effects across all RCTs in the psychological therapies and derives from unwitting biases taking place in some of the 100s of decisions made in the course of a single study. The evidence which needs to be sifted out from the existing studies relates to those RCTs which are (a) led by people without a stated preference for one of the therapies, and/or (b) have a clear proponent/expert of each of the approaches being evaluated in a study, and/or (c) yield evidence contrary to the researcher's therapeutic allegiance.

  6.5  It will likely be impossible to test out all psychological treatment approaches using adequately constructed RCTs because studies of the required size are too costly (>£1 million) and securing the results takes too long (may be four years). Much of the evidence from RCTs in the psychological therapies derives from studies with only 20 patients in the condition being evaluated. It is generally accepted that such studies do not have sufficient numbers of patients to make the claims they do because the small N makes them more vulnerable to biases (ie, an unwitting bias will have a greater impact amongst a smaller sample of patients). Often the effects achieved are because the target measure is so specific.

  6.6  One recent Cochrane Review found that the average number of clients in RCTs as a whole (as opposed to treatment conditions) of Generalised Anxiety Disorder was only 53 and concluded that "it seems highly unlikely, therefore, that any of these studies were adequately powered"—meaning that the number of clients was too small. [161]By contrast, studies using patients from routine NHS settings and totalling multiples of 100s are not considered as legitimate evidence by NICE because they do not use an RCT design. [162]There is a need for both types to evidence.

  7.  Because it restricts patient choice. Current government initiatives have placed considerable emphasis on "patient choice". However, there is an increasing focus on a single model of psychological therapy—cognitive-behavioural therapy (CBT)—in which this model is repeatedly identified in HTA and other government-funded trials as the form of intervention to be researched such that this model yields a disproportionate amount of evidence which could restrict patient choice.

  7.1  When other forms of psychological intervention have been compared with CBT in government funded RCTs, findings have shown broad equivalence of outcomes, for example, in depression and anxiety in primary care. [163]However, such therapies have not been adopted within the NICE guidelines.

  7.2  The over-emphasis on CBT has led to the identification of a shortage of CBT practitioners which would then require additional funding to correct. This approach has generated artificial problems regarding resources (ie, practitioners) to deliver psychological therapies.

  7.3  There are other effective psychological therapies which are being viewed as "second class" because they often do not have RCT evidence associated with them: this is a funding issue and not an effectiveness issue. Absence of evidence of effectiveness is not the same as evidence of ineffectiveness. One example of this mismatch is the fact that the model of therapy most often espoused by practitioners is that of "integrative" therapy and yet this has never been evaluated in an RCT and probably couldn't because it is not manualised and each practitioner is delivering their own form of integrative therapy honed over years of experience.

  7.4  A model of evidence which cannot evaluate the most commonly delivered form of therapy at the scientific level which it deems to be highest (ie, RCTs) exemplifies the gap between the perceived relevance of NICE and routine NHS practice.

  7.5  Such a strategy also restricts professional practice. An electronic petition submitted to 10 Downing Street (closing date 3 March 2007) objecting to the over-emphasis on CBT was signed by 10,025 (presumably) professionals: [164]"We the undersigned petition the Prime Minister to consider other psychotherapy approaches, not only cbt, in the proposed expansion of psychotherapeutic services within the NHS, instead of restricting choice for members of the public to one only model of therapy."

  7.6  An approach which would make far more useful clinical guidelines, starts with what the patient (or service user) needs, and with what practitioners are then able to offer them as an effective treatment, which is supported empirically by evidence from routine practice, thus promoting actual improvements in services.

  8.  Because NICE guidance focuses overly on technologies at the expense of practitioners and common factors. The content of guidelines—and invariably the research—focuses on specific treatment approaches (eg, CBT, problem-solving) when this is only one component within the service delivery framework. Most crucially, psychological therapy approaches are delivered by practitioners (ie, people) and while NICE guidance promotes evidence-based therapies, there is little—if any—attention to evidence-based practitioners. Given that practitioners are the most valuable resource, it would appear sensible to place equal focus on practitioners as is currently placed on technologies.

  8.1  There is a growing debate within the area of the psychological therapies as the contribution (ie effectiveness) of practitioners versus specific therapies. There has been research arguing for both sides of the case. [165]The most logical response to this situation is that the government should invest not only in the "technology" of interventions but also in practitioners themselves when it is becoming clear that the effectiveness of practitioners may be of at least equal importance. [166]

  8.2  Because of the focus in RCTs on technologies in which practitioners are all assumed to be "equal", such studies have not investigated practitioner effects. The contribution and variability of practitioners is an important component which is currently being determined from analyses of large data sets collected from routine NHS mental health settings. But, as above, because this data has not been collected within an RCT, it is not being considered by NICE. Such a strategy places NICE at a distance from everyday practitioners and does not facilitate practitioners adopting and implementing NICE guidance.

  8.3  In addition, current NICE policy over-emphasizes techniques rather than factors which are common across interventions. In RCTs, treatment is administered according to closely monitored protocols, whereas much NHS treatment is not so restricted and is more flexible. NHS psychotherapy for given individuals often draws upon different treatment approaches. In addition, practitioners differ in style and effectiveness, and the critical, evidence-based influence of establishing a "therapeutic alliance" between patient and therapist—a matter at the heart of psychotherapy—is sidelined.

  9.  Because NICE guidelines are being misused in relation to the actual body of evidence. There are several problems in using NICE guidelines to introduce health efficiency savings in mental health, and some evidence to suggest this is not being achieved. Two examples are (a) the primary impact of the guidelines for anxiety and depression on encouraging continued soaring costs for anti-depressants, and (b) the resources on computerised CBT, subsequent to NICE's technology assessment, when there are equally effective alternatives which are free to access. [167]Such outcomes could lead to commissioners seeking to de-invest in psychological therapies. There is a risk that patient care then suffers.

  9.1  It is also questionable as to whether clinical guidelines in and of themselves are an effective way to change professional behaviour, for example, amongst doctors. The development and dissemination of NICE's guidelines represents a huge investment, but there has been very little evaluation of what the impacts of the guidelines on costs and benefits are, and no national audit of whether NICE's mental health guidelines represent value for money. [168]

  9.2  NICE have undertaken an appraisal of the impact and implementation of their guideline for schizophrenia in terms of the cost savings to the health service through less in-patient stays. They have not undertaken similar appraisals of their other mental health guidelines however, for example, whether the guidelines for depression and anxiety are being implemented, and with what impacts on health and other related costs?

  9.3  Much of the evidence derives from the US which has a different funding system to the UK so that findings on cost effectiveness do not translate well. Thus, NICE's guidelines have had little to say about who should deliver interventions, or where, despite the evidence that this has a large bearing on cost-effectiveness. Also, because the guidelines are condition-specific, patients with multiple problems, who can be a heavy burden on services, tend to get overlooked. Moreover, as noted above, there are very few studies where cost-effectiveness has been designed into the trial, which show one treatment approach consistently outperforms another.

CONCLUSION

  10.  Public confidence is waning in NICE guidance because its policies do not reflect NHS practice. Although for some purposes, it is appropriate to classify psychiatric problems in terms of diagnosed "conditions" according to standard medical practice, sometimes this does not map onto practitioners' experiences dealing with people who have mental conflict and distress. By their very nature, RCTs tend to select very specific foci for study. The results lead to an over-simplification which the practitioner in the NHS finds hard to equate with the complexity of problems with which patients present in routine NHS settings.

  10.1  This is especially important when considering the diagnosis and treatment of "personality disorder", because it is widely recognised that with respect to personality disorder, psychiatric diagnoses are conflicting and confusing. Moreover, much "depression" and "anxiety" occurs in the context of patients' wider personality difficulties, and chronically embedded social and relationship problems. Correspondingly, many patients need help in developing through, rather than being "treated for", their difficulties.

RECOMMENDATIONS

  11.  We consider the instigation of NICE and the notion of "guiding principles" to have been a major step forward in building an evidence base for the psychological therapies within the NHS. However, by the very nature of the diversity of human beings, we need to ensure that patient choice is a reality by funding research into psychological approaches other than CBT and by placing an equal emphasis on evidence collected from routine settings to that currently derived from randomised controlled trials. We offer the following recommendations.

  11.1  The Government should set up a review of the evidence hierarchy which NICE relies on for its mental health guidelines, led by CSIP, to investigate the impact of current criteria for evaluating research into psychological therapies and consequent clinical guidelines on patient choice, innovative services, and patient care.

  11.2  Future guideline development groups set up by NICE for mental health guidelines should have a broader balance and cross-section of professional stake holders and peer reviewers to try to ensure researcher-allegiance bias does not distort the guideline development process. These appointments should be transparent and decided by elected representatives from the stake holder organisations.

  11.3  NICE should publish the estimated costs of implementing mental health guidelines in terms of treating unmet need, delivering new psychological treatments, workforce and training implications and service redesign. These monies should be ring-fenced as additional investment provided via Strategic Health Authorities before clinical guidelines are issued.

  11.4  Prior to NICE's review of its Depression and Anxiety guidelines in 2008, an evaluation of what impact they have had, and whether they are being implemented, should be undertaken by the Audit Commission. Where implementation is patchy or slow, a commissioning strategy should be included as part of the review process for clinical guidelines.

  11.5  The Department of Health should work with NICE and the professional bodies in psychological therapies and the mental health charities to agree a national research programme, which identifies the gaps in the evidence (across all the mental health guidelines), and priorities for research, and provide funding for these to be undertaken as an important part of the implementation programme for NICE guidelines.

  11.6  NICE and the Department of Health should work with the professional and research departments for psychological therapies and mental health research charities to establish an evaluation and audit infrastructure within NHS services which will enable ongoing improvements in practice, and better monitoring of whether clinical guidelines are having beneficial impacts on patient care.

Professor Michael Barkham

University of Leeds

March 2007

159   We need to talk: The case for psychological therapy on the NHS (2006). A report from Mental Health Foundation, MIND, rethink, The Sainsbury Centre for Mental Health, Young Minds. Back

160   Jacobson NS (1999) The role of the allegiance effect in psychotherapy research: Controlling and accounting for it. Clinical Psychology: Science and Practice, 6, 116-119. Back

161   Hunot V, Churchill R, Silva de Lima M, Teixeira V. (2007). Psychological therapies for genarlised anxiety disorder. Cochrane Database of Systematic Reviews, Issue 1. Art No. CD001848: DOI: 10.1002/14651858. CD001848.pub4. Back

162   Stiles WB, Barkham M, Twigg E, Mellor-Clark J, Cooper M. (2006). Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies as practiced in UK National Health Service settings. Psychological Medicine, 36, 555-566. Back

163   King M, Sibbald B, Ward E, Bower P, Lloyd M, Gabbay M, Byford S (2000). Randomsied controlled trial of non-directive counselling, cognitive-behavioural therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technology Assessment, 4 (19). Back

164   e-petition: http://petitions.pm.gov.uk/Therapy/ Back

165   Special issue on therapist effects in Psychotherapy Research, 2006, 16, 143-187. Back

166   Lutz W, Leon SC, Martinovitch Z, Lyons JS, Stiles WB (2007). Therapist effects in outpatient psychotherapy: A three-level growth curve approach. Journal of Counseling Psychoogy, 54, 32-39. Back

167   MoodGym: http://moodgym.anu.edu.au/ Back

168   Gilbody S, Whitty P, Grimshaw J (2003). Educaitonal and organisational interventions to improve the management of depression in primary care. Journal American Medical Association, 289, 3145-3151. Back