The Continence Foundation is a small UK-wide charity concerned with bladder and bowel control problems in adults. Our membership includes members of the public as well as a wide range of health professionals (clinicians, GPs, nurses, therapists etc) The involvement of the charity with NICE is substantial:

—  Work has been carried out on two Guidelines in the continence field since 2003: one on urinary incontinence in women was published in October 2006 and another on faecal incontinence is due for launch in June 2007. We have been involved in a long campaign for a guideline concerning male continence problems, and other groups excluded from the urinary incontinence scope.

—  Technology Appraisals in this area have been concerned with surgical procedures. A suggestion was made concerning a TA on a drug but this was not followed through.

—  Interventional Procedures in this clinical area are regularly considered.

—  As current Chair of the Patients Involved in NICE (PIN) Group, the Director of the Continence Foundation, Dr Judith Wardle, is in regular contact with many other organisations representing patients and also with the Patient and Public Involvement Unit of NICE.

—  Our recommendations appear at the end of this submission.

Why NICE's Decisions are Increasingly Being Challenged

  1.  NICE guidance has been challenged since it began work in 1999: some people will remember, for instance, the challenge regarding the guidance on beta interferon. What has changed is the increasing profile of such challenges, which in turn encourages more individuals, as well as patient groups, to challenge decisions. There is also more willingness by clinicians to challenge decisions.

  1.1.  Some of the increase may be attributable to changing attitudes: a more combative approach to official decisions of any kind. This is compounded by the attitude of the media which likes to publicise stories about an individual mounting a challenge, and sometimes does this without any attempt to understand the complexities of the clinical recommendations: which individuals can benefit from a treatment, whether evidence is as yet insufficient to make definite recommendations etc. But challenges are also being made because of loss of confidence that NICE's decisions are unbiased: see comments in the next section.

Whether Public Confidence in the Institute is Waning, and if so Why

  2.  Public confidence in NICE decisions is waning as part of general attitudes to what is happening in the NHS. There is a perception that decisions in the NHS, both nationally and at PCT level are being driven far more by cost than by clinical effectiveness. People see that individual PCTs are indeed cutting services without sufficient consideration for the long-term consequences. Therefore, they see NICE as part of that process, so that NICE is blamed for the refusal of treatment to individual patients, even where it is the PCT Board and not the NICE guidance that is behind that refusal.

  2.1  NICE is regarded as insufficiently independent. Politicians actually had an even greater influence until recently over which topics were considered by NICE, since NICE is now allowed to make its own selections of which topics to propose. However, the decision about which topics go forward and the precise wording of those topics chosen for Guidelines, still lies with "the minister". This means that organisations, whether clinicians or patient groups, are unable to get any information about why a particular topic and the client groups to be covered were chosen, or to challenge those decisions. Indeed, it is impossible to find out whether ministers have ever refused to let a topic go forward. There is a widespread belief that some guidelines have been created because of high profile adverse events that were seen to be politically damaging. However, the lack of transparency about the process of deciding which guidelines should proceed means that it is impossible to know whether this belief is well-founded.

NICE's Evaluation Process, and Whether any Particular Groups are Disadvantaged by the Process

  3.  As stated above, there is a lack of transparency about which topics are chosen for evaluation.

  3.1  Although individuals and organisations may now suggest topics for evaluation, there seems to be no mechanism for providing feedback to the people who made the suggestion regarding whether it has been accepted into the process and its progress through the system. It is essential that this is rectified if trust in NICE is to be restored. See final section of this submission, regarding problems of communication with NICE.

  3.2  It is difficult for lay people from whatever background to participate in NICE evaluation processes. Even where the Interventional Procedures team of the PPI Unit supplies a lay explanation of the procedure, that is little help with understanding the document sent out for consultation. Lay members of a Guideline Development Group may have real difficulty with long tables of evidence presented, unless the staff from the collaborating centre supply verbal explanations of what particular studies claim to prove and any flaws in the study design. While some collaborating centre staff are very conscious of the mixed nature of a guideline development group, others need training in communication skills.

  3.3  Where lay people are members of a Guideline Development Group, they can be made to feel sidelined by the attitude of the clinical members of the committee. In some cases, the clinicians concerned may not even realise how dismissive they are being about "non-expert" comments. The extent to which lay members feel welcomed seems to vary according to which collaborating centre has the lead: this needs to be investigated. The PPI unit offers training to lay participants to enable them to participate in NICE processes, but there is no training offered to health professionals in how to work with lay people.

  3.4  Health economists seem to find it particularly difficult to present their work in a form that is intelligible to the non-specialist. It is, therefore, very difficult for a lay participant to challenge the economic evidence.

  3.5  Regardless of the comments above, the Continence Foundation recognises that NICE's international reputation for producing guidance based on detailed, unbiased consideration of the evidence is richly deserved.

  3.6  Perhaps, NICE's communication team needs to do more to publicise to the general media just how much work goes into each document produced.

  3.7  There should also be greater emphasis on how NICE responds to stakeholder comments: all comments made about a draft guideline (except for a few that are "commercial in confidence") and the responses to those comments from the Guideline Development Group, indicating whether the final version has changed as a result, are made available when the final version of the guideline is published. However, only someone aware that these documents can be found in the "Development History" of a guideline, would think to look. If stakeholder comments were more prominent, the public would have more confidence that NICE is being transparent about how it responds to pressure groups of all kinds, including manufacturers. Also, NICE should discourage manufacturers from marking their comments "in confidence": we have seen comments so designated that would not reveal anything that could affect the manufacturer's business.

  3.8  There is concern that staff of NICE central office edit some of the recommendations made by guideline development groups: this can dilute the clinical value of the message.

The Speed of Publishing Guidance

  4.  Since NICE has only recently changed its timetables for producing guidance—both Guidelines and Technology Appraisals—we feel the shorter timetables should be allow to bed down before any further changes are suggested.

The Appeal System

  5.  We have no experience of this.

Comparison with the Work of SIGN

  6.  Our only experience of the work of SIGN is the guideline on "Management of Urinary Incontinence in Primary Care." SIGN does not publish the evidence for guidelines in the great detail provided by NICE—we assume that the literature review, however, is equally thorough.

  6.1  It seems to have greater acceptance by both health professionals and the public for the guidelines. This may be because they collect evidence at an early stage from patient and carer groups about the issues they would like to see addressed. The recommendations in the guideline on urinary incontinence are presented in clear language, giving both the implications for practice and what the recommendations mean for patients.

  6.2  We also note that the average time line for production of guidelines is shorter than for NICE.

The Implementation of NICE Guidance

  7.  The process of producing guidance for implementation is flawed. During the period when consultation is taking place on a draft guideline, an implementation group is convened. We have no way of knowing how the decision is taken about who to invite to join that group. The implementation group is supposed to help NICE's implementation team to produce documents to aid implementation, but they are doing this without knowing whether there will be significant changes to recommendations as a result of the consultation. This gives the impression that very little is likely to change. The implementation team seek assistance from other people suggested by the implementation group and then produce drafts of slides to aid implementation and members of the group may comment. The next anyone sees of those slides is when they are published with the final guideline, with no attempt made to explain any changes. The slides and the economic report are very selective about which aspects of the full guideline are focussed on. In the case of the guideline on urinary incontinence in women, a recommendation for the use of a non-proprietary form of a drug (which has side-effects which are poorly tolerated) formed part of the initial costing. The recommendation was clearly made on cost grounds, not on clinical effectiveness, but it was omitted from the final version of the costing report on the grounds that it would have "a minimal budget impact nationally". This sent out mixed messages to those who might implement the guidelines regarding whether the recommendations were to be treated seriously or not. NICE currently has no ongoing relationship with the members of the implementation group after the publication of the guideline concerned.

  7.1  Monitoring of implementation is provided by the Healthcare Commission, but Trusts are only asked a general question about whether they are implementing NICE guidance: there is no specific information about particular guidance. Detailed evidence about failure to implement is generally provided by patient organisations, with help from individual clinicians who feel they are not being allowed to give their patients optimal treatment because their local Trust is not prepared to fund it. Patient organisations would prefer to work with NICE to monitor implementation of guidance—especially where they agree with that guidance. However, there is no mechanism for NICE to support that work. We appreciate that NICE does not have the manpower to monitor all individual guidance. However, we believe that a mechanism needs to be found to monitor a selection of individual guidance, perhaps on the basis of suggestions by either patient or professional groups.

  7.2  For monitoring of implementation to work, recommendations in guidelines need, as far as possible, to have auditable outcomes. This needs to be made clear to guideline development groups at an early stage.

Additional Comment on Communication with NICE

  8.  The Continence Foundation has made an official complaint to NICE about repeated failures of communication. The Foundation has not been able to find out why a guideline topic on urinary incontinence was restricted to women, even though a significant number of stakeholders said, during the consultation on the Scope that it should cover both sexes—NICE would not tell us how many of the stakeholders said this. Subsequently the Foundation has worked with other charities and the British Association for Urological Surgeons (BAUS) to persuade NICE to start the process for a guideline on urinary incontinence in men (and also people with neurological conditions, who were also excluded from the first guideline). After our initial discussions, voluntary feedback from NICE was non-existent. Requests to specific individuals for information on progress were met with promises to report back, but the promises were not fulfilled. At one point, the Foundation and BAUS put in a detailed submission to support the topic. We are now in a situation where there are not one but two guidelines somewhere in the system and no-one seems able to explain why the topic was split in two. There are staff and clinical advisors to NICE now working on moving the guidelines forward, including one neurological conditions; but what concerns us most is that no-one at NICE seems to want to identify the lessons to be learnt from what has gone wrong.

  8.1  The Continence Foundation would be willing to share with the Health Committee the text of its complaint to NICE and the inadequate reply received, but would prefer the material to remain confidential, since we are hoping eventually to resolve the matter with NICE.

Recommendations

—  Remove the political influence from topic selection, or if that is not accepted, be open about which decisions on topics have been influenced by ministers.

—  Where an individual or group has proposed a topic, feedback should be provided on whether it has been accepted and if so, on its progress.

—  Training should be given to staff of collaborating centres on how to present evidence to groups that include lay representatives.

—  Training should be given to health professionals on working in groups with lay representatives: SIGN already does this.

—  Stakeholder comments on draft guidelines and the guideline development group replies should be made more prominent on the NICE website. This would give the public more confidence that NICE is operating transparently.

—  NICE should discourage manufacturers from designating their comments "in confidence" when they do not contain any commercially sensitive information.

—  NICE should adopt from SIGN the practice of asking patient and carer groups in advance what issues they would like addressed in a guideline: the groups consulted would be wider than those actually represented on the guideline development group.

—  NICE should clarify how the members of an implementation group for a guideline are chosen. They should work more closely with that group on developing the implementation documents, and continue to liaise with them after publication of the guideline to encourage and monitor implementation.

—  A mechanism should be found to monitor a selection of individual items of guidance, perhaps on the basis of suggestions by patient or professional groups.

Dr Judith Wardle

Director, Continence Foundation

March 2007