Knowledge for Policy

What can we learn from recent attempts to improve the evidence base and its use in policy development and implementation?

In 2005, OPM and Queen Mary University of London were joint hosts for a series of three seminars on ways of improving the usefulness of 'knowledge' at all stages in the life cycle of public service policies.

The three knowledge for policy seminars focused on a) extending, b) using and c) managing the knowledge base respectively, but there were a number of recurring themes. These are outlined below, along with illustrative examples of the specific points that people raised.

At each seminar a 'research' and a 'practice' speaker each made a short presentation; this was followed by an hour of general discussion. We hope to publish a short report of the seminars in the near future

Seminar 1: Extending the knowledge base

Speakers were: William Solesbury (QMUL) and Dan Murphy (Inland Revenue/DWP)
Identifying knowledge gaps, commissioning research or other forms of new evidence, managing knowledge production, achieving usefulness and useability

Seminar 2: Using the knowledge base

Speakers were: Sue Goss (OPM) and Kevin Lloyd (OPDM). Accessing knowledge, judging relevance and rigour, organisational learning

Seminar 3: Managing the knowledge base

Speakers: Annette Boaz (QMUL) and Andrew Maginn (NRU, ODPM). Knowledge management, reconciling different kinds of knowledge, reviewing and synthesising, meta-evaluation

Themes

We have grouped the main themes that came out of the three seminars under common headings.

1. Understanding the policy process

Policy-makers require research for different reasons, e.g. to understand ‘what works’ in a particular area to make the argument for change, or to respond quickly to issues that become high profile.
Policy-making is faster and more ad-hoc than research/evaluation.
The research community needs to become more familiar with the different policy processes and dynamics (e.g. policy making cycles), to help ensure that research is fit for purpose.
There is existing evidence that offers learning about the policy process – a meta analysis of this evidence might be useful.
Research should continue to contribute to our understanding of policy trends, rather than just evaluating programmes. Identifyinggaps, overlaps, synergy and dissonance between programmes would add value to programme evaluations and would facilitate better ‘read across’ on key policy issues; it would also allow better use of the research community.
Forward looking or horizon scanning research should be retained.

2. The policy/research interface

Gaining an understanding of the power relationships at work in the policy process is essential for the research community – for example identifying power brokers
How researchers can identify gateways into a policy process characterised by cross-cutting themes requires further exploration
Competitive relationships also exist between different schools of research (e.g. economists and social researchers), and there is a need to reduce the frustration felt by policy-makers attempting to reconcile seemingly incompatible findings from different schools
More opportunities to share experiences are needed, such as forming multi-disciplinary teams, workshops, learning events, and networking. These could be effective ways to bridge the gap between researchers, research commissioners and policy makers, and help to make the models, frameworks and theories in use more explicit
Researchers need to challenge policy-makers about what they want.

3. Research design issues

’Shallow’ and ‘deep-mined’ case studies serve distinct purposes, but there is a tendency for the former to be misused to serve the purposes of the latter – this risks policy initiatives being rolled out based on an insufficient evidence base
Informal methods (anecdotes, stories) and ‘shallow’ case studies can often be relevant and can have impact as evidence, and can be used to stimulate learning. But they do not tell the whole story: some local experiences may not get heard; short-term ‘noise’ in the system may be confused with policy results; and analysis may be missing
A mix of long-term and short-term research is necessary to reflect the evolutionary nature of policy
Studies could be front-loaded (more literature reviews, for example) given that evidence is particularly useful at the early stages of policy development
Mining (rather than conducting primary research) is often a rewarding exercise, given the amount of research potentially available to policy makers.

4. Communicating research findings

The process of disseminating research findings is just as important as conducting the research itself, and needs to be built in from the start of the process. Research reports should be accessible– often reports are too lengthy and dense
Researchers should share initial or interim findings as early as they can in the research process, given the nature of the constraints acting upon policy makers
Paramount in policy makers’ minds is the need to communicate findings in varied formats to a range of audiences – researchers need to take account of this
Marketing skills are relevant for researchers, alternative ways of presenting and disseminating findings would have more resonance with policy-makers and politicians
However, researchers are not always best placed to develop and present policy recommendations from research, and intermediaries such as technical journalists or government researchers who have knowledge of the political and policy context may be helpful
Evaluators should offer conclusions as well as analysis. Sometimes researchers shy away from drawing out the implications of their findings – too many ‘health warnings’ are not helpful to policy-makers
Academics could themselves help to fill the gap in the market for operationally-focused, policy aware research, by producing more accessible research

5. Research dynamics

Knowledge in the public sector is a resource like any other, and must be maintained and handed on; but this is problematic because so much of the ‘resource’ is currently in people’s heads
The pool of knowledge needs to be maintained. This requires, for example, commissioning research about issues as well as evaluations of initiatives and programmes
The emphasis on finding the ‘holy grail’ of research – the answer to a policy question – can be a barrier to building a ‘learning culture’ amongst policy makers. Policy makers can feel it is their job to ‘spot’ the one piece of research that provides that answer, rather than to accrue a knowledge base
The process of disseminating research is like a business: researchers sell a product to the people who pay for it. Research commissioners at the centre of departments act as ‘wholesalers’– selling research throughout their policy community. Researchers and ‘wholesalers’ should be engaging in a constructive dialogue to ensure that a ‘market’ for the research can be identified
However, can be problematic as researchers may be trying to sell a ‘product’ to people who think they do not need it, or are not sure what they’re looking for
The research community should flag up new areas for policy development, based on their findings.

Find out more about our research work.