80. The procedures used by NICE to evaluate new and existing therapies and interventions for use in the NHS have been developed over time. Some have changed as a result of both internal and external reviews.[43] However, problems remain. The evaluation process was at the heart of many of the criticisms raised over the course of this inquiry. This is discussed below.

81. Witnesses highlighted a range of issues relating to the evaluation process, including:

• The quality and scope of the process;
• The appeals system;
• The speed of guidance.

82. Technology appraisals were the main focus of criticism with some criticisms also applying to clinical guidelines. We received little evidence about the appraisal of interventional procedures or the development of public health guidance, but some of the points made were also relevant to these types of guidance.

The quality and scope of the evaluation process

83. NICE has responded to the changing environment in which it operates by introducing changes of its own, such as its fast-track system for the appraisal of specific medicines. The ongoing development of NICE's methods of working and its open discussion of them were praised by the Department:

It [is] right that NICE's process and methodology is the subject of continued development and debate and [the Department] welcomes the open and consultative approach NICE takes to the development of its work.[44]

Many witnesses also praised the robust nature of the processes used by the Institute to evaluate interventions and treatments. For instance, the British Psychological Society stated:

The methods chosen by NICE are rigorous, transparent and stand scrutiny by any international standards.[45]

84. Nevertheless, concerns about how medicines and other treatments are assessed by NICE were raised by many witnesses. In this section we discuss:

• Topic selection;
• Problems with the use of QALYs, including the range of costs taken into account;
• The evidence on which NICE bases its recommendations;
• Expert involvement in the development of guidance; and
• Consultation.

TOPIC SELECTION

85. As we described earlier,[46] NICE, in collaboration with the Department of Health, uses information provided by the Horizon Scanning Centre at the University of Birmingham to select the topics to be examined by the Institute as health technologies, clinical guidelines and public health guidance. Most of our evidence related to topics selected for health technology appraisal.

86. NICE does not assess every new medicine which enters the market, or every new indication for that medicine.[47] Instead, the NICE technology appraisal programme is limited to a small number of new, often expensive, products chosen to address "the questions that those who are delivering services in the UK healthcare system need answering".[48] This means that most medicines and other technologies which are prescribed in England have not been assessed by NICE. PCTs told us that NICE-evaluated treatments constituted only a "fairly small percentage of [their] work".[49]

Skew towards secondary care

87. Witnesses argued that the selection of only a small proportion of interventions, and more specifically new drugs, skewed NHS spending towards new and expensive medicines for acute illness. We were told that this was likely to continue, with many more cancer medicines, for example, in the pipeline.[50] Some witnesses warned that the narrow selection of topics for technology appraisal caused greater focus on secondary care as opposed to primary care. The Association of the Directors of Public Health (ADPH) stated:

Since these are almost invariably more expensive than what went before and in the acute sector, the net effect is a diversion of resources away from lower tech and more affordable models of care, and from primary and community care into the acute sector…[51]

88. According to Dame Gill Morgan from the NHS Confederation, "There will always be a bias towards curative care".[52] This is in contrast to recent Government policy, which has encouraged a move towards the provision of procedures and consultations in local settings that would previously have taken place in hospital.

Disinvestment

89. In keeping with the criticism outlined above, we heard that more should be done to encourage disinvestment from approaches and technologies that offer poor value for money.[53] This is an area that NICE has pledged to make a larger component of its work[54] but to date few older therapies have been covered by NICE guidance. Professor Appleby stated that more should be done to encourage PCTs in this regard:

The idea that if something is not killing you and may be doing you some good it is probably worth having is not how NICE or the NHS should operate. The area of disinvestment is a much more difficult area to go into, and that is why I believe PCTs need a lot more support. NICE could do more on that front, and I am sure it wants to.[55]

90. Research has suggested that PCTs are keen to improve commissioning to exclude less effective approaches.[56] It appears however that PCTs lack the information necessary to allow them to make decisions about disinvestment. Dr Tim Crayford of the ADPH told us that disinvestment information would be very useful for PCTs.[57] Professor Devlin confirmed:

PCTs are not always well informed of some of the data that could support them in decisions regarding either investment or disinvestment.[58]

91. Others argued that encouraging PCTs to disinvest in ineffective treatments should go hand in hand with a more general focus on cost-effectiveness. Professor Raymond MacAllister from University College Hospital told us:

There are clear areas of investment in health at present where I believe we are pretty wasteful….[59]

92. Questioned about this by the Committee, Mr Dillon denied that there was a "raft" of obsolete therapies being prescribed by clinicians.[60] He told us that NICE had not been able to identify any widespread use of out-of-date technologies:

The fact is that they are just not there in the way that people think they are. The health professionals do not indulge routinely and profitably in things that have absolutely no value whatsoever, to a level that would make it possible for us to say that there is a whole raft of things that should be stopped altogether.[61]

He added, however, that there were some cases of medicines being overused in inappropriate circumstances. In such cases, NICE could issue advice to ensure such treatments were used correctly.

93. NICE's argument is disingenuous. There are numerous interventions used in the NHS which have few benefits. The Cochrane Collaboration is discovering them all the time.[62] Of course, few are as Mr Dillon said "of absolutely no value whatsoever"[63], but this is far too severe a test and certainly not one used when NICE performs its original health technology appraisals.

94. It seems to us appropriate that topics are selected for interventional procedures, clinical guidelines and public health guidance. It is not appropriate, however, to limit technology appraisals to selected, often new and expensive, products. Instead, as we recommend below, all new drugs should be assessed.

95. Witnesses were concerned that NICE's focus on acute treatments, in particular medicines, could skew NHS spending towards selected new and expensive (NICE-approved) drugs for acute illness. We discuss this further in the next chapter.

96. In our previous report we recommended that NICE give more emphasis to examining old technologies to encourage disinvestment. This the organisation has failed to do as fully as we expected. Its statement that few interventions have absolutely no benefit may be true but is irrelevant. Many treatments currently used are not cost-effective as many studies attest. NICE should adopt a similar standard of cost-effectiveness in assessing such treatments as it uses in its technology appraisals. The organisation must now give more emphasis to disinvestment. One approach would be to undertake more MTAs, which would reveal the existing treatments that provide poor value for money.

QUALITY-ADJUSTED LIFE YEARS

97. NICE requires a unit of measurement which enables it to compare different drugs or interventions. The unit it uses is the QALY. A QALY combines information about the benefits of a drug or intervention to provide a measure of both the extension of life and the improvement in quality of life.

98. According to NICE, the use of QALYs assumes that the extension of life and quality of life can be captured, for instance, with EQ5D.[64] This uses a visual analogue score and patient responses on:

• physical mobility;
• ability of patients to look after themselves;
• ability to carry out normal activities of daily living;
• absence of pain and discomfort; and
• absence of anxiety and depression.[65]

99. A single QALY would indicate one year in perfect health. The value of a year in less than perfect health would be a fraction (eg. 0.5) of a QALY. Improvements in length and quality of life are referred to as fractions of a QALY.

100. To assess cost-effectiveness, the QALY score is integrated with the price of treatment using the incremental cost-effectiveness ratio (ICER). This represents the change in costs in relation to the change in health status. The result is a 'cost per QALY' figure, which allows NICE to determine the cost-effectiveness of the treatment.

101. NICE has stated that it uses a "threshold range" to determine whether the cost per QALY of a treatment offers value for money. It provides its advisory bodies with a framework for decision-making as follows:

• Below an ICER of £20,000 per QALY, judgements about the acceptability of a technology as an effective use of NHS resources are based primarily on the cost effectiveness estimate.
• Above an ICER of £20,000 per QALY, judgments about the acceptability of the technology as an effective use of NHS resources are more likely to make more explicit reference to factors including the degree of uncertainty of the ICER, the innovative nature of the technology, the particular features of the condition and population receiving the technology, and (where appropriate) the wider societal costs and benefits.
• Above an ICER of £30,000 per QALY the case for supporting the technology on these factors has to be increasingly strong. Recommendations for interventions costing more than £20-£30,000 per QALY must be explained.[66]

102. The system is an attempt to bring consistency to the measurement of health outcomes across different specialties and for different conditions. The cost per QALY does not indicate the price of annual treatment with the medicine. An expensive treatment may have a low cost per QALY if it brings significant benefit to patients; likewise, a cheaper treatment may have a high cost per QALY if the degree of benefit is relatively low. The table below, taken from an article by Professors Nancy Devlin and David Parkin, shows the authors' estimated cost per QALY for selected health interventions assessed by NICE.[67]

Intervention	Cost per QALY (£)
Smoking cessation	430
Asthma inhalers	5000
Zanamavir (for at risk groups)	20,400
Stents	25,000
Zanamavir (for all suitable patients)	38,000
Orlistat	46,000

Table 2.

Devlin and Parkin, Health Economics 2004

103. Both QALYs themselves and the way in which they are used were widely criticised in the evidence submitted to the inquiry. Some witnesses complained that NICE was too rigid in its assessments of cost-effectiveness and was over-reliant on the use of QALYs.[68] Some claimed QALYs failed to take into account other, equally valid, information and that the use of QALYs led to guidance that was unfair to particular groups of patients.[69]

104. In contrast, other witnesses argued that a method of comparing different treatments in an objective way which took account of longevity and quality of life was essential.[70] The Institute added that QALYs should "only inform, and not determine, NICE guidance".[71]

QALY assumptions

105. Many criticisms centred on the assumptions that are included in the calculation of QALYs. It was argued that assumptions about what exactly constituted the 'quality' for which life years are adjusted involved value judgements, such that the factors included in a QALY and the weight given to them may vary. Mr Steve Winyard from the Royal National Institute for Blind People (RNIB) described the difficulty in making such value judgments in relation to sight, or sight loss:

Whom do you ask? You want to measure the importance of sight to quality of life. Does one ask the sighted public, or people who are in the process of losing their sight, or do you ask those who lost their sight five years ago, for example? You will get different answers from those different groups. Which questions does one ask?[72]

106. Medicines manufacturers felt that the weighting given to different factors included in QALY calculations could be inappropriate. As Dr Richard Barker from the Association of the British Pharmaceutical Industry (ABPI) put it, "A model…is only as strong as the assumptions".[73] Dr Rafiq Hasan from Novartis agreed:

Sometimes one questions the assumptions that go into some of the models. We would welcome a broader debate about some of the assumptions that go into those models.[74]

107. Some professional groups also had little confidence in the use of QALYs. Dr David Anderson from the Old Age Faculty of the Royal College of Psychiatrists told us that experts often did not understand how some QALY-based decisions were made, and believed the process to be somewhat arbitrary:

When clinicians hear about health economic analyses many of them see it as made-up stuff. You just take some data and create an equation that is based on assumption after assumption…You fiddle about with an equation and come out with a number. If you want you can fiddle about with it some more and come out with a different number.[75]

108. Other professionals questioned the consistency of QALYs. Professor Raymond MacAllister from University College Hospital stated:

Depending on how complex the model is one can get an answer that can be inconsistent from one assessment to another.[76]

He warned that clinicians could easily lose faith in QALY-based economic assessments if they had just "one or two examples" of where the process had led to a seemingly inaccurate result. Moreover, he suggested that there was widespread suspicion of the use of the QALY as used by NICE:

The health economic assessment is a dark art for a clinician. It does not take much for someone who is a clinician, or even someone who is interested in drug policy, to be made sceptical about some of the outputs.[77]

Bias

109. Witnesses argued that the use of QALYs could lead to bias against treatments for long-term, chronic conditions. Lower costs per QALY—and therefore greater likelihood of NICE approval—were claimed to be associated with treatments for acute conditions. Mr Winyard told the Committee:

Drugs that extend life will always achieve higher values …

The use of QALY values puts people with long-term conditions at a disadvantage over people with life-threatening conditions.[78]

110. Witnesses contended that the reason for this skew towards acute illness was because quality of life measures were not rated as highly as other factors.[79] Drug manufacturers agreed that QALYs did not attribute sufficient importance to quality of life data. Dr Barker told us that, although QALYs were "a necessary tool", they were insensitive to changes in quality of life that, although small, were important to patients:

The current process makes no distinction between an improvement of 0.2 of a QALY to 0.4 versus 0.8 to 1.0. If you are at the 0.2 level and you have a really poor quality of life, a doubling of your quality of life might be more valuable than a 20% increase or just over for a patient with a less severe condition.[80]

Support for QALYs

111. On the other hand, several witnesses provided strong support for QALYs, pointing out that QALYs were necessary to ensure consistency when evaluating different treatments for different conditions. Professor Stirling Bryan from Birmingham University stated:

For many conditions the use of a QALY works well and allows for a common yard-stick to be used in comparing very different conditions and…Thus, it is possible to ensure some degree of consistency when making decisions on therapies for heart disease and treatments for arthritis.[81]

112. Dr Tim Crayford of the ADPH described the QALY as "a great leveller". He agreed that it should not be the only decision-making tool, but stated:

I think that that methodology, which looks at the potential benefit, the time at which it occurs in someone's life and how long people might benefit, some sense of that going into an equation to help us weigh up whether some things are more cost-efficient than others, is a very good idea…some sort of measure that looks at the length of benefit and the size of benefit is absolutely necessary.[82]

113. While Professor Rawlins agreed that there were "uncertainties" about the details of the use of QALYs, he maintained that there was no better way of doing the work:

The QALY is a measure that allows us to compare the value, the health gain, of one intervention for one condition with another intervention for another condition. It has been researched extensively. There are hundreds and hundreds of articles about it. It has been investigated in Europe, in North America, in South America, even in China, and it is a reasonably robust approach to utility; in other words, the health gain.[83]

Mr Dillon added:

We believe QALYs are the best tool around at the moment to enable you to [understand the costs that will be foregone for other treatments] and it is important that that is applied consistently.[84]

114. NICE officials, and the Department of Health, also stressed that the cost per QALY calculation was not the only basis for NICE guidance, or even for its assessment of cost-effectiveness. Professor Rawlins told us that, "not everything can be expressed necessarily in QALYs".[85]

115. Since the QALY will necessarily remain at the heart of NICE's work, it is important to make the economic models they are used in as accurate and robust as possible. Professor Bryan recommended following up specific guidance after it had been promulgated to test whether the predictions of the model used in the cost-effectiveness analysis were borne out in reality:

If the conclusion was that there was a high degree of correlation between the predicted and actual results that would give us greater confidence in the modelling work.[86]

Wider economic benefits

116. When NICE was established, the Department of Health insisted that the organisation should not take into account the economic benefits of treatment to carers, or savings related to benefits, tax allowances or productivity. Many witnesses argued that the exclusion of societal gains compromised the validity of QALY-based cost-effectiveness calculations.[87]

117. Professor Peter C Smith from York University told us that the system had first been developed as it had for very understandable reasons but thought that changes could now be made to incorporate the wider benefits of treatment:

[NICE] has been wise to keep the definition quite narrow and constrained so that everyone knows what it is trying to achieve with its figures, but I would hope that as we gain experience its methodology could begin to embrace broader benefits, and broader costs, associated with treatments.[88]

Others agreed.[89] The Academy of Medical Sciences stated:

NICE [should] take the overall burden of disease into account, to include societal costs to patient carers, unemployment costs or the expenditure of social services.[90]

118. NICE told us that it could consider personal social service costs, but was specifically precluded from considering costs such as those related to employment or disability by the existing legislation:

In our clinical guidance programmes, we take into account the cost to the NHS in personal social services. That is because our statutory instruments limit us to that perspective. It would be possible, if Parliament wished us to, to take a broader economic perspective, but that, to be honest, is your decision, not ours.[91]

119. The Minister agreed that there was "a real issue and a debate to be had" on the subject of including the broader costs to society in NICE's cost-effectiveness assessments. She warned, however, of the potential bias of such considerations:

We need to be careful that we do not skew decisions away from certain groups within our community towards others…If we required [NICE] to consider the costs of not returning to work, would that skew away from the elderly and the long-term chronic diseases where people have no chance of returning to work because of their conditions? It seems to me that is the essential challenge and it is raised in other issues as well, such as the cost of care. It is difficult.[92]

120. We heard much criticism of the use of QALYs. Some of the criticisms seem to be the special pleading of disappointed parties. It is vital that a method which allows comparison of the benefits and costs of different treatments for different conditions is used in cost-effectiveness evaluations. However, it is also vital that the system is accurate and reflects the real costs to society and the benefits to patients. We recommend that:

• Research is undertaken to follow up specific guidance to see whether the predictions of the cost-effectiveness analysis are borne out in practice;
• Wider benefits and costs, such as costs borne by carers and social care services, be more fully incorporated into NICE's assessment. We were told that this would have to be a decision for Parliament.

A discussion of the changes to the QALY threshold which would be needed to accommodate an earlier evaluation appear later in the report.

EVIDENCE

121. NICE must examine all the available evidence on a therapy or intervention during the development of its guidance. This includes meta-analyses, randomised clinical trials, observational studies, case studies and other types of evidence that are in the public domain.[93]

122. Witnesses referred to a number of problems regarding the evidence used by NICE:

• The lack of suitable evidence;
• The design of clinical trials;
• Poor use of the available evidence; and
• Lack of access to all the evidence.

Lack of suitable evidence

123. Some types of evidence are considered more valuable than others for the assessment of medical treatments or procedures. For instance, a well-designed clinical trial involving large numbers of patients with an appropriate control group is more likely to yield robust results than a smaller scale observational study. NICE 'grades' evidence presented on this basis, as follows:

• Level 1 evidence (the highest quality) includes meta-analyses and systematic reviews of randomised controlled trials;
• Level 2 evidence includes high quality systematic review of case-control or cohort studies with a low risk of bias;
• Level 3 evidence includes non-analytic studies such as case reports;
• Level 4 evidence includes expert opinion or consensus.

In its guidance, NICE indicates the 'strength' for the basis of each recommendation.

124. NICE informed us that it was rare for all the evidence on a particular topic to be of a high standard:

The best available evidence is rarely (if ever) complete. It may be of poor quality, lack critical elements, or both.[94]

Evidence may be lacking in some specific areas because less research is done in the first place. Relatively few studies are undertaken in the field of public health for example. Professor Rawlins described this as an "information gap"[95] that could give other interventions an advantage, simply because more related research existed.

125. Even when the evidence exists, it may not present the relevant facts. Professor Carl Klaxton of York University stated that this was a particular problem for medicines:

There is constant frustration about the relevance and quality of the clinical trial evidence. As to its relevance, there are inappropriate comparators, primary endpoints that are not easily linked to ultimate health outcomes and a shorter follow-up than we would like to see. There are inclusion and exclusion criteria that do not match the NHS.[96]

126. Professor Rawlins stated that there were several difficulties with the evidence provided by medicines manufacturers around or soon after launch. The first concerned a lack of health-related quality of life data. The second was the lack of information on comparator drugs, as most medicines are compared to placebo in clinical trials. The third, related concern was the lack of evidence of effectiveness compared to other classes of drug.[97] Finally, poor cost effectiveness data is a particular problem:

The much greater problem we have actually than all of that is the cost effectiveness, and the problem with cost effectiveness, to be honest with you, is that companies have not had experience of having to do sophisticated cost effective analyses, and they are on a steep learning curve.[98]

Design of clinical trials

127. The type of clinical trial that drug companies undertake in order to gain a licence for their medicine does not necessarily provide the kind of information which is required for a consideration of cost- and clinical effectiveness for the NHS. Mr Ian Beaumont from Bowel Cancer UK recommended that NICE base its recommendations on the type of data more usually supplied by clinical trials:

Instead of finding the QALY and the overall survival rate, it would look at things like progression-free survival and disease-free survival for people after surgery. If NICE's appraisals and reviews were more in line with the way drugs actually worked then their method of evaluating them would be more meaningful.[99]

128. Other witnesses argued that the opposite should happen, ie. that clinical trials should move into the "real world" and aim to provide data that is more relevant for NICE. Dr Kiran Patel, a cardiologist from the South Asian Health Foundation, told the Committee that clinical trials often had design faults that meant they were not applicable to routine practice:

A lot of the trials concerned with myocardial infarction do not include people over the age of 80, but it is the over 80s who come into our coronary unit day in day out. If one applied evidence-based medicine down the line one would conclude that people over 80 are not represented in any clinical trials and therefore they should not be given any treatments within those trials. Clearly, we do not do that.[100]

He added that the patients who are most likely to receive medicines for heart disease have one or more comorbidity. Such patients are not included in clinical trials, again limiting the applicability of the evidence.

129. Moreover, even when drug companies make some effort to provide relevant evidence, the data generated is often difficult to use. For example, inappropriate surrogate (indirect) markers of efficacy are employed rather than direct measurements of patient-reported symptom improvement.[101]

130. Many witnesses argued that closer collaboration between NICE and medicines manufacturers during the drug development process was needed to improve the design of clinical trials.[102]

131. This has been highlighted before. Closer working between NICE and the manufacturer was a key recommendation of the Cooksey report. Dr Barker told us that there had been movement towards greater cooperation already:

In terms of better dialogue earlier in the process…so that the company knows what, in a sense, it needs to prove (in other words, what will be regarded as an economically sound case for this medicine), we are already in dialogue with NICE about how we might do that, how we might introduce it earlier in the process of development—not ten to 12 years before but maybe a year or two before launch—a dialogue on what the company ought to be able to prove.[103]

132. The better design of trials to take account of cost-effectiveness would be particularly effective if it occurred in all countries in which large-scale clinical trials take place.

133. NICE agreed that discussion with pharmaceutical companies during the development of drugs could be beneficial, particularly in reference to the 'conditional licensing' recommendations of the Cooksey report. Professor Rawlins stated:

We are discussing with other interested parties, particularly the MHRA…what sorts of arrangements we might have to put in place to enable us to look at the cost-effectiveness of drugs at an early stage. I do not think this is at all impossible and there are examples of really early drugs which we have looked at in the past where we have found it possible to do this, perhaps being a little bit more imaginative about the approaches we take.[104]

134. In addition, NICE could introduce incentives to encourage drug companies to design appropriate trials. NICE currently publishes a broad grading of the evidence it has received (see paragraph 123), on the basis of number of patients involved, whether a control group is used etc.; an alternative or additional measure would be to publish a detailed assessment of the quality of the research it receives.

Poor use of evidence

135. Some witnesses thought that NICE did not use evidence properly, even when it was available. They claimed that the high value placed on certain types of research meant that the results of many useful studies were discounted. Lifeblood: The Thrombosis Charity blamed NICE's "overly-prescriptive formulaic approaches to data gathering" for ignoring larger studies, with admitted design faults, while considering smaller randomised controlled trials with stricter protocols.

136. Witnesses also claimed that the studies which were ignored were more likely to be independent (ie. funded by an academic or other non-commercial source) than other, large-scale and expensive clinical trials. Given that the results of clinical trials are more likely to be designed and funded by the pharmaceutical industry and to favour the sponsor or manufacturer,[105] witnesses feared that this would lead to bias in guidance in favour of new products over older medicines or other, perhaps non-pharmacological, interventions. As Professor Mike Richards, the National Cancer Director, stated:

It is not just about drugs. There are a lot of other innovations that can save lives. Making sure people are aware of healthy life-styles factors, getting screening programmes introduced and early diagnosis are extremely important. Remember that we have to look at the whole of that.[106]

Access to evidence

137. While NICE has access to published data, it does not have automatic access to the information relating to a particular therapy which is not in the public domain. Unpublished research, which the MHRA may use, is not necessarily available to NICE.

138. This Committee recommended in its first Report on NICE that all information be shared between the MHRA and NICE. This has happened to a limited extent. The Institute itself stated that the sharing of summary information with the MHRA, and increased use by manufacturers of the European Medicines Evaluation Authority (which, when licensing, publishes a discussion of the studies considered online[107]), has improved its access to trial information.

139. Nonetheless, witnesses maintained that more could be done to improve access to information, stating, "the decisions that NICE make are just as important as those of the licensing authority and it should have similar powers".[108]

140. Some witnesses proposed the mandatory registration of all clinical trials to improve NICE's access to relevant material further. The problem of the non-publication of negative trial results has been highlighted many times in the past, not least by this Committee in its inquiry into the Influence of the pharmaceutical industry. As Professor Claxton stated:

There have been a number of examples where trials that have been conducted have been withheld from an appraisal. Those trials are always ones that show the product in a poor light.[109]

Professor Rawlins agreed that more should be done to ensure registration:

There is increasingly registration of clinical trials. I do not think it goes as far as it should go.[110]

Professor Adrian Towse of the Office of Health Economics endorsed this view:

As far as concerns the voluntary registration scheme, it seems to me that it is the last chance saloon. The industry must get its act together and disclose what trials are going on; otherwise, sooner or later somebody will impose some requirements on them.[111]

141. Professor Jon Nicholl from Sheffield University told us that at least trials taking place within the NHS now have to be registered:

The key first step is registration and that is being addressed at the moment. All trials that are started which involve NHS patients or premises need to be registered. Quite rightly, the results of all those trials need to be made publicly available.[112]

142. However, fundamental problems remain. Notably, NICE has to make public the evidence it bases its recommendations on, but some of the unpublished data it might wish to use is confidential for commercial reasons.

143. NICE does not have all the information it needs to assess and compare treatments. First, while access to EMEA documents and other changes have improved NICE's access to information, it still does not have access to all the relevant information which is available. Secondly, clinical trials undertaken by pharmaceutical companies understandably focus on generating data about the drug's efficacy and safety, which is required for the licensing process; such trials are not usually designed to generate the type of data on cost-effectiveness which NICE requires. Third, in some areas, without commercial sponsors, notably public health and many physical and psychological therapies, there is little research about the cost-effectiveness of different interventions.

144. We recommend that NICE be granted the right to see all the evidence the MHRA uses when making its decisions. We appreciate that this would mean that there would be some commercial-in-confidence material that NICE could not make public when it published its guidance.

145. We welcome the fact that both NICE and drug companies are aware that they need to collaborate closely to ensure that clinical trials are undertaken with the needs of NICE appraisal in mind. The Government should encourage all countries in which large-scale clinical trials take place to adopt a similar policy. We support the mandatory registration of all clinical trials so that the results of all negative trials are accessible. We recommend that NICE assesses and reports the quality of the research it receives.

146. More publicly funded research should be undertaken to assist the development of public health guidance and other areas without commercial sponsors.

EXPERT INVOLVEMENT

147. NICE Appraisal Committees, evidence review groups (ERGs), which assess whether there is adequate evidence to allow technology appraisals to take place, and guideline development groups (GDGs) include people from a range of backgrounds. ERGs and GDGs ought to include relevant experts. Appraisal Committees, in contrast, are unlikely to contain those with expertise in the area covered by the particular product or approach under consideration since their membership does not alter with each guideline or appraisal.

Appraisal committees

148. Many witnesses were critical of appraisal committees' lack of relevant expertise.[113] Dr Anderson from the Royal College of Psychiatrists, stated:

Appraisal committees extraordinarily exclude experts deliberately. NICE has made that its policy.[114]

The charity Leukaemia CARE told us:

There are no consultants with a knowledge of haematological cancers sitting on any of the committees who pass judgement on blood cancer products.[115]

The charity added that "use of appropriate expertise on the appraisal committees" would help develop "an equitable and homogenous health service that is not just fair to all who have a need to use it, but is seen to be fair to all".

149. Professor Rawlins explained that Appraisal Committees might not include experts as members, but experts did play a role in their deliberations. Experts were invited to attend committee meetings on specific days, to answer questions and provide clarification to the members:

The appraisal committee always has at its meeting experts who are not members, not voting members, they are there to help the committee understand the issues, but the committee itself makes the decision and it has to be like that really.[116]

150. Witnesses said that NICE did not use experts well, but that affiliating experts with Appraisal Committees for the duration of the consideration of a topic could improve matters:

Charlotte Atkins: …this Committee has advisers who stay with it for the whole period of an inquiry and they advise it and give it the value of their expertise. What I suggest is that [experts] can simply be professional advisers to the committee.

Dr Beverley Hunt[117]: What you describe is an excellent idea. I had not thought of it in that way. You have someone there for most of the process but he or she is not a member of the committee but sits in. It sounds a very logical way forward.[118]

Guideline development groups

151. GDGs present a different problem. Although these groups include experts, some witnesses thought that not all GDGs included the most relevant experts; as a result the guidance produced was flawed. Dr Beverley Hunt of Lifeblood: The Thrombosis Charity [speaking of the GDG which examined venous thromboembolism (VTE)] stated:

The problem is that on that committee there are no experts….Therefore, they will review what the statisticians have said and come out with an answer which is usually there or thereabout, but they do not have an understanding of the nuances or the papers in the area.[119]

Some witnesses claimed that the wrong emphasis was placed in the VTE guideline on mechanical methods compared to chemical methods because of a lack of relevant expertise.[120] Dr Hunt and Professor Roger Atkins of the British Orthopaedic Association (BOA) explained that the guidance was not as relevant to clinical practice because the right experts were not included in the guideline development process.[121]

152. We questioned NICE about the use of experts in GDGs. Professor Rawlins denied that experts were not included in the process. He stated:

We always have experts in the condition but we also have more generalists to give a broader view and we also include two patients or, in the case of children, the parents of patients with the particular condition.[122]

He added that it would not be possible to include more than a certain number of participants in any group, but that additional experts could be called when necessary:

So we have a broad base and, of course, the guideline development group…often does invite other experts in other areas. If you try to get everybody on, then the whole thing becomes unmanageable. [123]

Conflicts of interest

153. Witnesses claimed that the Institute did not include experts on its Appraisal Committees because it "believes that they have a vested interest" in the guidance produced.[124] However, most specialists working in a particular field of medicine have undertaken work for a manufacturer or been linked to the pharmaceutical industry in some other way during their careers.

154. Dr Anderson from the Faculty of Old Age Psychiatry at the Royal College of Psychiatrists admitted that many experts would have links to the industry, but he denied that this would affect the group's decision:

We all live in a world where we face conflicts of interest every day of our lives. There must be a process to manage that and to declare those interests, but it is much better to have an informed group looking at the most complex problems in healthcare than a range of specialists who do not even know what Alzheimer's disease is until they have spent six months in the group.[125]

Other witnesses recommended that as long as any conflicts of interests were declared this should not preclude the inclusion of experts.[126]

155. NICE officials maintained the need for strict rules regarding conflicts of interest, however. Professor Rawlins stated:

we have a very clear statement of conflicts of interest. If they are clearly personal financial specific interests then [members of advisory bodies] must not just declare them but not take part in the proceedings. When it comes to outside experts or organisations—and it might be organisations as well as experts—then we require them to tell us what their interests are and those are recorded.[127]

Mr Dillon added:

In circumstances where a member of an independent advisory body has a conflict in relation to a particular topic, they do not take part in the business of formulating the recommendation.[128]

He stressed that it was not "inevitable" that every expert had received funding from drug companies during their careers.

156. Many witnesses thought that too few experts with the relevant detailed expertise were involved in the process of producing guidance. Since they have a permanent membership, Appraisal Committees are unlikely to have such experts. They do consult experts, but this is unsatisfactory because such experts appear for the day alone. We therefore recommend that Appraisal Committees appoint specialist advisers, without voting rights, to work with the Committee throughout consideration of a technology appraisal or clinical guideline. This will improve guidance and ensure public and patient confidence in the system. Decisions about which experts should be appointed should remain the responsibility of NICE following consultation with the appropriate clinical bodies.

INVOLVEMENT OF THE PHARMACEUTICAL INDUSTRY

157. NICE's Scottish equivalent has a more "collaborative relationship" with the pharmaceutical industry during medicines technology appraisal than NICE. Companies can rework elements of their submissions and resubmit when necessary.[129] Some witnesses suggested that such an SMC-type approach should be adopted by NICE.[130]

158. Medicines manufacturers argued that their representatives should have a greater role in the evaluation process. Dr David Brickwood from Johnson & Johnson thought that at the very least pharmaceutical companies should have a similar role to patient organisations:

Stakeholders such as patient organisations and clinicians are invited into those [Appraisal Committee] sessions, but, in the case, particularly, of single technology appraisals, where the company probably has the greatest knowledge of the development of the product and probably spent ten years developing it, then the company is excluded from that process.[131]

Others recommended that the manufacturer should have more opportunity to consult with NICE. Many claimed that the Institute should provide more feedback to the industry on the assessment of its products.

159. In particular, some pharmaceutical companies want access to the economic models used by NICE. This was the focus of the judicial review that took place in 2007 (see Chapter 2). The manufacturers of the Alzheimer's drug donepezil (Aricept), Eisai and Pfizer, demanded access to a "fully transparent working version of the calculations used in the cost effectiveness model for independent evaluation and comment". NICE refused to let the companies have access to a usable version of the modelling tool; instead a 'read-only' version was released.

160. Eisai stated that without access to the model "full and proper investigation of the conclusions reached" by NICE were impossible, and the manufacturers could not:

check the accuracy of the formulae used in the model…;

assess the overall quality/ validity of the model;

assess the sensitivity of the model to the inputs used; and

test the model using alternative inputs and combinations of inputs, where those used by the Assessment Group are controversial.[132]

The company added that "The economic models used by NICE are highly complex and errors readily occur", making access to the model necessary so that consultees can comment properly and, "maintain confidence in the approach adopted by NICE". Other companies agreed with the stance taken by Eisai and its marketing partner Pfizer.[133]

161. NICE defended its position as follows:

The companies get all the spreadsheets. What they cannot do is to put the numbers in they want to see how that would change the conclusion that they want…[134]

Mr Dillon suggested that if companies so wished, they could reconstruct the model with the figures provided by NICE and change the assumptions themselves.[135]

CONSULTATION

162. There are many opportunities for stakeholders to participate in the development of NICE guidance, for example through consultation on the assessment report, appraisal consultation document and final appraisal determination. However, some witnesses complained that NICE did not take sufficient notice of consultees' point of view once it was submitted.[136] Mr Beaumont from Bowel Cancer UK stated:

NICE appears to pay lip service to patient groups and the views we put forward, including our most recent submission to NICE which was the most comprehensive we had ever provided, did not appear to be taken into consideration. We believe strongly that there should be greater engagement between NICE and patient groups.

Professor Roger Atkins of the BOA and others also argued that submissions were virtually ignored:[137]

When we received the draft document at the turn of 2006 the BOA, specialist societies and the group I chair wrote a 17-page document of commentary, including the latest evidence, and received no reply. We emailed the chairman of [the National Collaborating Centre for Acute Care (NCCAC)] on a number of occasions and received no reply.

163. Some witnesses complained that responding to NICE consultations was time consuming.[138] Small charities found that the number of consultations taking place across the UK was a particular problem; we were told that it was difficult to respond individually to all NICE equivalent organisations in England, Scotland and Wales.[139] The relatively short period of consultation added to the problem.[140]

164. Not all patient groups agreed with this viewpoint, however. Some praised the many opportunities for consultation during the guidance development process. Breakthrough Breast Cancer stated that it:

…welcome[d] the move by NICE to run consultations on many of their processes…This demonstrates a willingness to be efficient and inclusive.[141]

Dr Kiran Patel also told us:

If you look back 10 years, the access which organisations such as ours had in terms of policy-making and developing guidance was more or less zero. NICE has afforded us an opportunity to access the decision-making process as a stakeholder and we have utilised that very well. From our point of view we have had very good access at all levels of the process.

165. NICE is proud of the amount of consultation that takes place during the development of its guidance. Professor Rawlins described consultation as, "a very, very important quality control safeguard mechanism in our processes". He added:

We would be very, very reluctant to stop doing that, it is a very important part of our process.[142]

166. The Institute denied that consultees' submissions were ignored.[143] It told us that it was not common practice to respond individually to every submission made to the consultation process. Instead, consultees and detailed responses to all comments they made were listed on the Institute's website once guidance was published.

167. NICE did accept that it was disappointed about some aspects of its consultation processes, however. In particular, while PCTs have always been able to contribute through the NHS Confederation,[144] since 2002 they have asked for their input during the development of technology appraisals.[145] Unfortunately, they do not often respond to consultation. Mr Dillon stated:

What we have discovered over the years is that the extent to which those PCTs actively get involved is quite low. That is disappointing and it is as much our responsibility as it is the community of PCTs to do something about that. We are concerned.[146]

He added that the NHS Confederation was discussing with PCTs proposals for new arrangements which would allow them to contribute to NICE's work more efficiently:

It is absolutely in our interests and it is the interests of good quality guidance and it is the right thing to do and we are very happy to work with the Confederation to make that work better.[147]

168. The wide consultation which takes place during the development of NICE guidance is greatly valued. While we agree that it is difficult for some organisations to respond within the often brief time limits, we recognise that a long consultation period would slow the guidance production time further. Nevertheless, the situation would be improved if NICE were to give interested stakeholders greater warning of forthcoming consultations, to allow them to organise their resources in time to respond effectively.

169. Some consultees complain that their views are ignored. We understand that NICE does not have the resources to respond individually to each consultee. NICE could, however, issue a standard response to inform every consultee how it will respond and setting out how the system works.

Appeals

170. Appeals against NICE decisions can be made by interested parties when, "the Institute has failed to act fairly, has exceeded its powers or has formulated guidance which cannot reasonably be justified".[148] Appeals may not be made on other grounds, such as the interpretation of the evidence; they are based on process alone. NICE does not permit new information to be introduced during the appeal stage. The process is based on English law.

171. Appeals are held in public by a panel of three non-executive directors of NICE or two non-executive directors plus an NHS clinician. An individual with experience in the relevant industry (chosen following consultation with the relevant trade associations) and a lay representative are also present on the panel.

172. The number of appeals against NICE decisions has not increased significantly since the Institute's establishment. However, a high proportion of recommendations are appealed against each year, perhaps reflecting the dissatisfaction with elements of the evaluation process described above. Most appeals relate to technology appraisals.[149] There was considerable criticism of the appeals system.

173. Appeals are not always made on valid grounds. The South Asian Health Foundation told us that appeals could be considered as "a reasonable and appropriate strategy for the pharmaceutical industry…to challenge unfavourable decisions from NICE". Professor Rawlins agreed that the industry sometimes used appeals in this way:

Sometimes we have what the industry have themselves described as "gaming appeals" where we have done a class appraisal of several different drugs in the same class and where one manufacturer is appealing the fact that we have not picked his out as the best. Then we have to hear the appeal. They have made an appeal and our rules are that we listen to the appeal. In those sorts of appeals, we have always rejected them.

174. Professor Claxton added that the increasing requirement for the NICE and its committees to prove their case during appeals contributed to "a dangerous situation". He stated:

Many of the appeals have been upheld and so [the Appraisal Committee's] ability to say no has been somewhat undermined…That starts to shift the burden of proof back onto the institute and the independent groups but within a process where they do not have the resources and time to do a full independent analysis.[150]

175. Some witnesses argued that changes to aspects of the technology appraisal would reduce the number of appeals made. Dr Anderson of the Royal College of Psychiatrists claimed that changing the make-up of the appraisal committees, for example, would mean that appeals were less likely:

If you had informed appraisal groups they would not be so contentious even though they made tough decisions, and I do not believe you would have as many appeals against them.[151]

176. This view was echoed by manufacturers. They also thought that closer working with NICE during the assessment process would reduce the numbers of appeals. Novartis told us:

We believe the degree of challenge to NICE decisions would be reduced considerably if a more inclusive approach were taken during the assessment process, enabling a dialogue from the beginning on the appropriate patient population in whom treatment may be cost effective, on methods, on assumptions, on data quality and on modelling techniques.

177. There appears to be strong dissatisfaction with the appeal system itself, however. Many witnesses bemoaned its limits, indicating that appeals should be possible on grounds beyond those currently permitted by the legislation. While some appeals are upheld, large-scale changes are very rare.[152] As Mr Gray from GlaxoSmithKline stated, the limits of the process explain why so many appeals meet this fate:

When you look at quite why so many appeals appear to fail, it is because in many ways you have appealed about something that is not being assessed in the appeal—and, big surprise, lots of things do not get through appeal.[153]

178. To improve the appeals system it was proposed that:

• The introduction of new evidence be permitted;
• NICE make more effort to explain the reasons for the initial decision; and
• The appeals system be made independent of NICE.

NEW EVIDENCE

179. Witnesses argued that NICE should permit new evidence to be presented at an appeal as it may give potentially useful information about the treatment in question. Mr Beaumont called the Institute's refusal to accept additional evidence during appeal, "a cop-out".[154] We were informed of many examples of the exclusion of seemingly valuable new evidence from the appeals process. For instance, Merck Serono highlighted a case where new evidence "which validated the use of particular assumptions within its economic modelling" came to light after draft guidance rejecting the drug was published. The new evidence could not be presented during the appeal. The appeal itself delayed guidance publication; the company claimed that this delay in combination with the exclusion of new evidence meant that the final guidance was possibly mistaken:

This [new] information could not be assessed by NICE. Therefore a total of 21 months elapsed with no possibility of any new evidence being considered for nearly 18 months. This new evidence could mean that drugs originally given negative guidance might be given a positive guidance.[155]

180. On the other hand, allowing additional evidence at the appeal stage would extend the process significantly, and might discourage companies from producing high quality trial data at the time of first assessment. It might also risk more "gaming" appeals.

BETTER COMMUNICATION

181. We were informed that better explanations of the reasons for rejecting appeals could improve the process and lead to fewer appeals being brought.[156] NICE has often been accused of communicating poorly with the general public and manufacturers, about this and other areas of its work.[157] As Mr Dillon told us:

I am not sure we have been as successful in engaging the general public.[158]

182. Professor Raymond MacAllister thought that better communication strategies could also improve the public's perception of the appeals process and of Institute more generally:

When one says "no" obviously one will be unpopular unless one says clearly why one is saying that. At times I believe that the reason why it says "no" is not clearly stated. It is very easy for those who do not like the "no" to marshal considerable forces in the press and elsewhere to attack that decision and many times the criticism is very unfair. A more clear description of why "no" has been said would be helpful.[159]

INDEPENDENCE

183. The appeals process is handled internally by NICE. This may add to the dissatisfaction and perceived unfairness experienced by appellants. Some witnesses felt that the lack of independence limited the validity of the appeals procedure[160], as Mr Beaumont from Bowel Cancer UK told us:

It is crazy that the people who made the decision in NICE then sit in judgment on themselves in the appeal. What possible motive would they have to change their minds when they would be regarded as having got it wrong in the first place? The appeals process in NICE should be independent.[161]

Many manufacturers agreed. Dr Brickwood from Johnson & Johnson stated:

We would advocate that the committee and its structure has a total independence from NICE.[162]

184. We note the pressure to change the grounds for appeal, but consider changes might cause more problems than they solved. Allowing additional evidence at the appeal stage would extend the process significantly, and might discourage companies from producing high quality trial data at the time of first assessment. It also might risk more "gaming" appeals. We make recommendations in the next section which we expect will lead to fewer appeals being brought in the first place.

Speed of guidance

185. NICE technology appraisals take between 9-12 months (STA) and two years (MTA). Clinical guidelines and public health guidance take longer. The evidence we received indicated that the speed of technology appraisals was of great concern.[163] Mr Winyard described the appraisal process as "wretchedly slow".[164] The Minister agreed:

Is there an issue about speed? Yes, and everybody recognises that.[165]

NICE BLIGHT: THE CONSEQUENCES

186. Manufacturers, clinicians and patients groups alike highlighted problems related to the delay in access to treatment caused by the relatively long period between licensing and publication of NICE guidance. We heard many complaints about this situation, which is often described as 'NICE blight'.

187. Clinicians are less likely to prescribe a medicine during the period between licensing and the issue of guidance as they may either prefer to wait or may be specifically forbidden to prescribe the medicine by their PCT. Whether or not patients have access to the drug at this point therefore depends on where they live. Mr Winyard from the RNIB pointed out:

In some areas…patients will be treated but in others they will be turned away. For example, in Staffordshire there appears to be a blanket ban on treatment [for macular degeneration] and in many areas patients have to go blind in one eye before the primary care trust will consider treating the second. This is fundamentally wrong.[166]

188. Delayed guidance harms patients who are waiting for treatment. Dr Crayford stated:

The bigger problem with the delay in producing guidance, say from the time which the drug comes to the market, is more the effect it has on patient groups and patients' expectations. That is the bigger and more difficult thing to handle [compared to financial effects on PCTs].[167]

189. On the other hand, it is possible to prescribe licensed medicines before the relevant NICE guidance is published. It is very difficult to take a medicine off the 'available list' once it had already been widely prescribed. This clearly creates problems if the NICE guidance eventually finds that it is not cost-effective. Rapid assessment of a medicine and determination of how it should be used helps to avoid these problems.[168]

190. Delays also bring the risk that the assessment will no longer be relevant by the time clinicians are able to use it. Dr Kiran Patel stated that:

There is a real danger that by the time some NICE guidance is published it is outdated.[169]

He echoed the views of many, stating that, "We need to have a mechanism to speed [guidance development] up".

191. NICE itself has recognised the need for faster guidance. It introduced its STA process for precisely this reason and has clearly stated that it would like to issue guidance earlier than at present. Professor Rawlins told us:

We have the ambition of trying to make sure that the NHS has advice around about the time of launch—it cannot necessarily be at launch, but within a month or two or three.[170]

FASTER GUIDANCE

192. Witnesses argued that faster guidance could be issued if:

• NICE adopted a similar system to that in place in Scotland; or
• Reduced the rigour of its evaluation.

193. The STA has increased the speed of NICE guidance; however, it remains slower than the process used as standard by the SMC. Patient groups told us that patients benefited from the swifter process in place in Scotland.[171] The speed of the Scottish system, and the fact that the SMC evaluates all new medicines immediately, means that 'approved' medicines are available to Scottish patients sooner than to their English counterparts.

194. Professor Rawlins pointed out that the SMC procedure was shorter than that of NICE because it did not involve the same consultation and appeals processes. Mr Dillon also indicated that the guidance documents issued by the SMC are much less detailed that those of NICE:

What NICE does is to identify precisely the circumstances in the form of the clinical advice that we offer that the drug should be used in. So we would identify specific populations, we would identify specific features of the way that the drug should be administered and the circumstances in which it should be used. That just takes longer to do.[172]

The Minister lent her support to the evaluation system used by NICE. While she admitted it was a longer process than that of the SMC, she added:

We, as ministers, prefer the process of NICE, we think it is more robust, transparent, it is being improved with certain considerations moving to public session and it is respected, therefore, internationally as the right way to proceed.[173]

Mr Dillon added, however, that NICE would be willing to consider measures to reduce the delay to guidance release:

I am concerned for any delay… it is our job, along with the Department of Health's and the departments involved in the topic selection process, to minimise this as much as we can. I am sure that there is more that we can do to achieve that.[174]

195. The ABPI told us that there is a period of four months on average between licensing of a medicine by the MHRA and launch of that medicine on to the market.[175] A shorter evaluation, along the lines of that of the SMC, could fit into this gap.

A less rigorous process

196. NICE technology appraisals are widely considered to be thorough. However, some witnesses argued that a shorter, less rigorous process which examined all drugs would bring many benefits. Dr Crayford told us that PCTs would benefit from a faster system that evaluated all medicines, even if it was less stringent than the current procedures:

We obviously need a degree of scientific rigour…[but] in order to broaden [the process] (and I would argue that needs to be broadened considerably), it needs to become less rigorous and provide us swiftly with best quality clinical evidence for a much wider range of clinical interventions.[176]

197. Witnesses pointed out that the uncertainty of cost-effectiveness at this early point could be mitigated through the use of a lower cost-per-QALY threshold, which could be revised later. Professor Smith told us:

one could negotiate a lower price for an interim period until further research was forthcoming, and so on. There are lots of imaginative possibilities to speed up the process and protect the NHS from making adverse decisions in a rush but also to encourage companies to continue to come forward with new treatments.[177]

198. Professor Rawlins denied that a "quick and dirty" evaluation was the solution. He told the Committee that such a move would reduce the value of the appraisal process and risk making recommendations that would wrongly guide the use of resources in healthcare:

We rely and depend on the rigour of our system not just to withstand the law and all that type of thing but to actually make sure that we are fair to everybody who uses the National Health Service, because saying yes to something that is cost ineffective will deprive other people of cost effective care, and saying things that are cost ineffective when they are cost effective really will have the opposite effect and that would be wrong.[178]

199. We agree with the Minister that the time taken to publish guidance is a serious cause for concern. In our view, the delay between medicines licensing and guidance publication does indeed 'blight' healthcare delivery by the NHS. The reduced access to those licensed medicines that are eventually proven to be cost-effective before NICE's assessment takes place is unacceptable. It is also unacceptable that Scottish patients have access to new medicines while English patients do not.

200. A shorter, less in-depth initial evaluation of medicines at an early point would be useful. It is important that clinicians have access to independent information about new therapies as soon as they are available. However, a quick, in-depth, fully consultative evaluation for all new medicines by the time of launch is not possible. We therefore recommend that NICE should examine all new medicines for their indications as set out in the marketing authorisation. Assessment should be carried out during the period between licensing and launch. It should be brief and published prior to, or at the time of, launch. There should be no formal appeal process and only limited consultation. These brief assessments should be followed by a larger scale multiple technology appraisal for selected products (an MTA or STA as appropriate) at a later date, when more evidence is available. The technology appraisal should include current levels of consultation. The guidance issued at this later stage should be definitive, over-riding that issued earlier.

201. Since providing an evaluation of all drugs at launch will be a more rough and ready process, it would be inappropriate to use the same threshold range as the full assessment. One of the aims of the new process is to ensure that treatments which are obviously cost effective are available at an earlier stage than at present. We therefore recommend that a threshold below the current range be used in these early assessments. This could be raised for individual products in special circumstances, for instance where no other treatment exists. At the time of the full assessment, the cost per QALY threshold could increase.

44   Ev 2 Back

45   Ev 192 Back

46   See Chapter 2 Back

47   Unlike the Scottish Medicines Consortium (SMC); see above Back

48   Q 75. Andrew Dillon Back

49   Q 291 Back

50   Q 31 Back

51   NICE 111 Back

52   Q 316 Back

53   Ev 180 Back

54   Ev 4 Back

55   Q 550 Back

56   Q 527 Back

57   Q 340 Back

58   Q 529 Back

59   Q 177 Back

60   Q 655 Back

61   Q 655 Back

62   Cochrane Library: http://www.cochrane.org/index.htm. See also BMJ Clinical Evidence; BMJ 10 November 2007  Back

63   Q 655 Back

64   An assessment tool known as EQ5D or the EUROQOL, accessible at www.euroqol.org. A visual analogue score refers to a situation where patients respond to a question using a visual scale where one end of the spectrum represents, for instance, no mobility at all and other full mobility. Back

65   Ev 8 Back

66   Ev 8 Back

67   Devlin N and Parkin D, Health Economics 2004, 13: 435-452 Back

68   Q 608 Back

69   Ev 164 Back

70   NICE 105, 106 Back

71   Ev 8 Back

72   Q 615 Back

73   Q 387 Back

74   Q 606 Back

75   Q 230 Back

76   Q 187 Back

77   Q 208 Back

78   Ev 212 Back

79   Eg. those more clearly related to mortality or morbidity Back

80   Q 397 Back

81   NICE 105 Back

82   Q 316 Back

83   Q 85 Back

84   Q 116 Back

85   Q 85 Back

86   Q 196 Back

87   Winyard, S: The cost of sight loss in the UK. RNIB campaign report 23 August 2004 Back

88   Q 177 Back

89   Ev 29, 37 Back

90   Ev 23 Back

91   Q 87 Back

92   Q 772 Back

93   Technical terms mentioned here are defined in the glossary Back

94   Ev 7 Back

95   Q 316 Back

96   Q 491 Back

97   Q 667 Back

98   Q 669 Back

99   Q 263 Back

100   Q 273 Back

101   NICE 114. For example, trials of anti-arrhythmic medication might measure the reduced number of heart beats as opposed to patient-reported improvement or reduced mortality.  Back

102   Q 263, 370, Ev 86 Back

103   Q 387 Back

104   Q 641 Back

105   Q 210, 516, Lexchin et al, BMJ 2003;326:1167-1170 Back

106   Q 31 Back

107   Known as the European Public Assessment Report Back

108   Q 494 Back

109   Q 494 Back

110   Q 671 Back

111   Q 495 Back

112   Q 493 Back

113   Q 231, Ev 25 Back

114   Q 231 Back

115   Ev 131 Back

116   Q 686 Back

117   From Lifeblood: The Thrombosis Charity Back

118   Q 595 Back

119   Q 579 Back

120   Q 595, 597 Back

121   Q 590 Back

122   Q 91 Back

123   Q 91 Back

124   Q 231 Back

125   Q 233 Back

126   Ev 25 Back

127   Q 103 Back

128   Q 105 Back

129   Q 374, 371 Back

130   Ev 23, 24 Back

131   Q 370 Back

132   NICE 118 Back

133   Ev 183, 222 Back

134   Q 681 Back

135   Q 684 Back

136   Q 224 Back

137   Q 612, 240 Back

138   Ev 113 Back

139   Ev 135 Back

140   Responses to NICE consultations must be given within 4 weeks. See Annex Back

141   Ev 62 Back

142   Q 170 Back

143   Q 700 Back

144   Q 364 Back

145   Ev 14 Back

146   Q 705 Back

147   Q 706 Back

148   NICE terms of reference: http://www.nice.org.uk/niceMedia/pdf/DirectionFromSecretaryOfState2005.pdf  Back

149   See Chapter 2 for details Back

150   Q 499 Back

151   Q 279. See paragraph 156 Back

152   According to NICE, small changes of wording are often all that is required Back

153   Q 387 Back

154   Q 281 Back

155   Ev 154 Back

156   Q 386 Back

157   Ev 91, 92, 113 Back

158   Q 648 Back

159   Q 221 Back

160   See Ev 30 Back

161   Q 283 Back

162   Q 384 Back

163   See Ev 23, 24 Back

164   Q 638 Back

165   Q 769 Back

166   Q 604 Back

167   Q 308 Back

168   This was highlighted to us during our visit to the SMC Back

169   Q 286-7 Back

170   Q 665 Back

171   Q 638 Back

172   Q 662 Back

173   Q 769 Back

174   Q 663 Back

175   Information supplied by the ABPI (NICE 72A) Back

176   Q 329 Back

177   Q 191 Back

178   Q 665 Back