PROFESSOR TIM HOPE, MR NORMAN GLASS AND MR WILLIAM SOLESBURY

24 MAY 2006

  Chairman: Good morning to our second panel. Professor Tim Hope, Professor of Criminology at the University of Keele; Norman Glass, the Chief Executive of the National Centre for Social Research; and William Solesbury, the Senior Research Fellow at ESRC and the Centre for Evidence-Based Policy and Practice. This is a panel that should have all the answers and I am sure you were fascinated by the comments that were made earlier. I am going to ask my colleague Brian Iddon if he will begin on this session.

  Q987  Dr Iddon: There are those who say that evidence-based policy-making, which the Government is creating a fuss about at the moment as you well know, is incompatible with the realities of the electoral cycle. I know Tim Hope has had something to say about that in the past. Do you agree with the statement that they are incompatible or is there some compatibility feasible?

  Professor Hope: I do not think anyone would demur from the general expectation or hope that politics and policy could be informed by rational debate and rational information. I think the close coupling of politics on the one hand and science on the other, or the utilisation of knowledge on the one hand and the production of knowledge on the other, which has occurred during the development of the idea of evidence-based policy-making, raises certain questions and concerns about protecting and safeguarding both the quality of knowledge production and the quality of knowledge utilisation. So my concerns and my worries are about the close coupling of these two activities, largely because, to address your point, both of course are legitimate activities and have their own forms of public accounting, if I could put it like that, politics through the political process and science through the scientific process, through the process of peer review and so on. I think when one couples them together the, if I might put it, power and influence of politics tends to infect the procedures and processes of knowledge production of science, to its detriment, and I think to the detriment of the public interest.

  Q988  Bob Spink: Could I just interject. Are you really saying that sometimes the Government are selective about what evidence they choose and what evidence they use?

  Professor Hope: Yes.

  Q989  Chairman: Or does not use it at all?

  Professor Hope: Or does not use it at all. The process of selecting evidence is a common one for both politics and for science itself. The discipline of science, the practice of science is all about selection, so it is not that government is any more or less selective than science is selective in terms of how it produces knowledge, but that there are different principles involved.

  Q990  Dr Iddon: Norman Glass, you have had difficulties with this concept of marrying the two together, have you not?

  Mr Glass: Yes. Maybe I just have a sunny disposition but it does not really bother me as a big issue that there is a difference between the facts. In social research facts are always questions of probability as well. I do defer a bit to our physical and biological scientists. I wish I had the chance to do experiments in the way they do experiments. The facts that we gather tend to be more fungible. You believe them through the weight, the repetition, the quantity of things happening much the same way rather than any particular piece of fact that emerges at a particular time. Then the fact is that politicians like the rest of us, find some sorts of facts easier to accept than others when they tend to confirm strongly held beliefs that they have acquired through lots of other ways of acquiring beliefs. So I just tend to think that is life. You produce facts in as rigorous a way as you can and you hand them to people with proper explanations of what you think they mean, but politicians have to make decisions on the basis of a lot of things—past commitments, their own innate prejudices, the views of their colleagues, what they think the public will bear. I think you do a very difficult job and I do not envy you. I do not find this kind of contrast between hard science, which shows things, and then what we do, which is always played around with by politicians, terribly difficult to live with. In the end—and this is what keeps me going—if the facts continue to point in the same way people will change their opinions. It may take a while but it will happen in the end. There is only so long you can go on believing the impossible. It wears you down after a while.

  Q991  Dr Iddon: In our case it is proportional to our majorities. William Solesbury?

  Mr Solesbury: I think part of the problem arises from the term "evidence based". It is also a term that is in the title of the Centre where I work but it was given to us by the ESRC when we were funded initially. I think the concept that policy should be based on evidence is something that I would rail against quite fiercely. It implies first of all that it is the sole thing that you should consider. Secondly, it implies the metaphor `base' and implies a kind of solidity, which I agree with Norman is often not there, certainly in the social sciences although I think to a great degree, bearing in mind the earlier discussion on probability, not always in the natural and biological sciences. If we have a concept of evidence being able to inform policy, that is something that we can all feel much more comfortable about because in that context evidence is not the only thing that informs policy. I do not need to lecture you on the nature of politics. I referred in the memorandum to conceptualisation of the four Is which is policy shaped by information, interests, ideologies and institutions. Certainly in the training and consultancy work that I have done with government departments, I have always found that a very fruitful way of getting them to think about a particular contribution that evidence and information can make and how it can be weighed against other things. I think the problem is also not just in the terms. The Patricia Hewitt example in the previous session is a nice example of the curious way in which politicians and the civil servants who advise them are rather bad at saying we have reached a decision by weighing these things up and have chosen, because of the nature of the evidence, to give less weight to that and more weight to these other considerations. There is a curious desire on the part of politicians and perhaps ministers in particular to have the evidence very firmly on their side. That leads I think to these rather perverse statements like the one that was quoted of Patricia Hewitt that she wanted to have the research that demonstrated it was a good thing to have home births. There are all sorts of other reasons surely why home births would be a good concept without necessarily having to be demonstrated to be scientifically more likely to produce safe babies or reduce mortality.

  Mr Glass: Can I come in on that just to follow up on the point about Patricia Hewitt. I do not know the context in which Patricia Hewitt made her statement, but you can imagine a context in which it would not be absurd or anti-scientific or something like that. For example, if you decided on the basis of evidence or prejudice, or whatever it might be, that you were going to have a policy of encouraging home births, and there has been a lot of stuff for ages about whether mothers prefer home births even if it is slightly riskier (because we do not all try to absolutely minimise risk) so if you have decided on the basis of this kind of evidence that home births were a policy that you were going to promote or encourage or support, then it might be sensible to say, "And we want evidence that will support this policy in the sense if we are going to have this policy we may need to know how best to implement it, how it should be delivered, what kinds of ways of delivering it are less risky than others." I am not defending Patricia Hewitt. I do not know the context in which she made that statement. I only saw what was in the papers and a long history as a civil servant told me not to believe what is in the papers. So I would be careful about saying that what she said was anti-scientific or contrary to the notion that evidence should inform policy. It may be that what she said was indeed that and of course it may have been an inopportune thing to say. I have worked with Patricia Hewitt in the past and I do not think she is the kind of person who just rules out evidence on the basis of her subjective prejudices.

  Q992  Chairman: I want to come on to you Professor Hope. You have been extremely critical of the way in which the Home Office has used your research and the way it has misinterpreted your research and presented it as evidence to support policy. What evidence do you have that this is a systemic problem within the system or is it just a one-off?

  Professor Hope: I have no other contemporary evidence other than my own experience of observing the behaviour, as it were, of the Home Office in this regard, although I did serve as a research officer in the Home Office from 1974 to 1991, so I have my recollections to draw upon in that regard. There are two things I would want to say about my criticism. First of all, the criticisms I have levelled against the Home Office have been in the context of a scientific discourse or a scientific set of criticisms. The circumstances that arose in this case were, in simple form, the Home Office researchers acquired data that we had already analysed on the contract to the Home Office in connection with evaluating the Reducing Burglary Initiative. The Home Office acquired the public data that we had used (that is the crime figures) and re-analysed those figures and produced a set of findings on the basis of their analysis in an official Home Office publication, Findings 204. I had pointed out prior to the Home Office publishing this that I thought their interpretation differed from our own and I had identified where I thought the difference lay in terms of our respective scientific methodologies and the bases on which we had produced our work and the bases on which the Home Office had produced their findings. Despite that, they proceeded to publish their own analysis. The inferences from that analysis were, let us say, rather more congenial and favourable to the political interests in this programme than were my own. I have set that out more fully in an academic paper published in the journal Criminology and Criminal Justice which I have provided to your Clerk.

  Q993  Chairman: We have got that. My question to you is do you feel that that is systemic within the system or was that just a one-off where it was desired to have a political result?

  Professor Hope: I would like to believe that it was a one-off but I have no specific evidence except I suppose hearsay and certainly understandings I have from colleagues within the British Society of Criminology, our professional body. At the British Society of Criminology conference in the University of Bangor in July 2003 there were a number of papers to be given by academics on the basis of contracted work that they were involved in, as I was, for the Home Office. A number of the researchers were advised not to present their papers at the last minute even though they had been advertised in the programme by Home Office officials.

  Q994  Chairman: By the Home Office itself?

  Professor Hope: Yes, so that would suggest to me certainly there was at that time or in those circumstances something rather more systemic about the desire to control the release of information and analysis, certainly about the Crime Reduction Programme.

  Q995  Chairman: Norman, I know you were heavily involved with the Sure Start programme. We had frequent statements by ministers that this was an evidential programme, the evidence had come from the United States where they had got very significant achievements through the programme there, but you claimed that there was no evidence or there was little evidence. The question I ask you is: does it matter if it was a good programme?

  Mr Glass: I am not sure I could have said that since I was the one who assembled the evidence so there may be a misunderstanding here. I pick up on William's point which is "evidence influenced". Like him, I do not like the phrase "evidence based" it is not the way policy gets made. In the case of the Sure Start programme what we did was we followed a fairly open process. We were influenced very heavily by a series of experimental studies in the United States, many of them different but relating to early years programmes, which appeared to show significant improvements on a number of measures. That is the United States, they are experimental programmes, and you cannot just take them over and apply them here. We were influenced by issues of evidence from our own birth cohort studies which showed that many of the influences in people's later lives were present in the first seven years of their lives and that those were the most significant influences affecting people's lives, in so far as you could see what affected people's lives. There was a lot of evidence on the importance of things like parental attachment and so on. There was a lot of stuff around of that kind which did not point to particular programmes but nevertheless pointed in the direction of saying that early years mattered and probably mattered more than interventions you could make later on in people's lives and that there were things that appeared to be effective which were being carried out elsewhere. That is where you get to the point of saying, "Well, at that point, do I let go of the billiard cue and actually hit the ball or do I wait around for more evidence to come in and try and do it?" I think our view at that stage was that there was sufficient evidence, if you like, adjacent to the issue of whether an early years programme could be successful which was convincing to think that we ought to have some sort of early years programme which we did not previously have. I am sorry if I have been reported as saying I did not think the evidence was there because. In Sure Start, because of the way we did the process, we had three public seminars to which we invited academics, we had ministers present at some of these seminars, we had a literature review of what was known about early years which we commissioned from the Thomas Coram Research Unit and so on. In that particular programme there was more of what I would call evidence around than there had been for some other programmes with which I am afraid I have been associated in the past. In that case I think there was a lot of evidence but what there was not—and that gets back to our social sciences—was experimental evidence of randomly allocating people to a programme and not to a programme and then seeing what happens. We have very little of that in this country. There is quite a bit of it in the United States. We tend to have the policies; they tend to have the evidence. It is a rather interesting mix. For various reasons, and it is a point I would try and make to the Committee, ministers and governments in this country, both Conservative and Labour (there is no difference) have been very reluctant to allow social scientists to carry out experimental programmes where people, with their consent of course, are allocated at random to a programme and then to a control group. We have done very little of that. That would be much stronger evidence. It is still not clinching but it would be much stronger evidence.

  Q996  Chairman: Just before I bring Robert in here, could I have a very yes or no answer really. I can understand a policy being based on a fairly small body of evidence in the case of Sure Start but rolling it out on the scale at which Sure Start was rolled out without in fact—

  Mr Glass: Now you are coming more to my point.

  Q997  Chairman: without continuing to monitor and gain the evidence and then say is this worth the investment that we are making, would you saying that is an issue?

   Mr Glass: I certainly believe and I argued strongly at the time that there was sufficient evidence to have a small Sure Start programme. I think we started off with a programme of 200 which is not that small but was still small. My view—and I argued it at the time when I was in the Civil Service and I have argued it subsequently—was that we should have learned much more about the experience from those 200 before we rolled it out on any scale. I do feel that we have rolled it out on the basis of inadequate evidence about how best it should be done as much as whether it has an effect, and being clear about the kinds of impact we wanted this programme to have. If that is what you are saying I said, then I agree with that. We rolled it out too much, too fast and too inadequately reviewed.

  Q998  Chairman: That is what I understand you to be saying.

  Mr Glass: I am sorry I misunderstood before.

  Chairman: Robert?

  Q999  Mr Flello: Before I get into my question I just wanted to pick up on the point about the social science experiments. To what extent do you feel those are hampered by ethics committees, for example, saying "You cannot go off and do that. You have to stop people having that experience because it would be better if they did not have it?"

  Mr Glass: What you have just said is very close to my heart. We are increasingly running into this issue where ethics committees, which are largely of medical origin, are applying the same sorts of criteria to social research that they apply to medical research. We have an ethics process in our own organisation. I think most academic researchers are subject to them. The problem is that in this case I think that we are being subject to a level of ethical scrutiny which is appropriate to a process where people may live or die but is not appropriate to a process about whether they will get £10 a week or £5 a week. These ethical committees consist, I have to say—and I am conscious that Dr Harris and for all I know many others are medical practitioners around here—of medical practitioners, many of whom have very little familiarity with social science or the methods of social science and tend to bring with them the concerns (quite rightly, that is what they are on those ethics committees for) of their experience in the medical sciences. We usually get the through the ethics committee with no great difficulty but it is very time-consuming and it does, I think, hinder many of the kinds of experiments that I would like to see happening. We have had two major policy experiments in recent years, the first ever on a major scale, both of them to do with people with long-term sickness and disability, and I think both of them have been very, very rigorously carried out. Ministers agreed to them in the end but it took a long time to get ministers to agree to them and it certainly presented very serious operational difficulties and therefore cost to put it through the conventional ethical procedure.