This NCRM event was held at the British Academy on 27th October 2015. Key, contemporary questions around the role and value of qualitative research and evidence in policy-making, not least how it is valued relative to other sciences were discussed.
There is a fascinating tension between the widely appreciated understanding of the essential role of social science in policy-making - not least around complex, uncertain and contentious environmental and technological challenges - and the suspicion that evidence based on qualitative methods lacks the robustness of other sciences. Of course, no scientific method (or, indeed mix of methods) can be judged better or worse than another in isolation from the research or evidence question it seeks to inform and address.
The problem that evidence based on analysis of narrative, discussion and commentary can be regarded as inferior to that which provides statistical representativeness and reproducibility is not new1. Issues of quality are endemic across the spectrum of evidence categories: from experimental, through model based, epidemiological, observational and narrative2. It can feel like an ‘uneasy truce’ or even a ‘cold war’3 between the different sciences, sometimes resulting in qualitative research being accepted merely in the context of providing nuance for quantitative studies. Hence, social sciences are often included late in policy-making and research: as communication rather than evidence. To make the best of social sciences their contributions must be fully integrated at the beginning of enquiries.
Despite the Government Chief Scientific Adviser, Sir Mark Walport, reaffirming the ‘central contribution of the social sciences and humanities to informed decision-making by national governments and transnational organisations’ (Times Higher Education, 22/10/15) tensions remain around the value and role of social science evidence. Ian Boyd, Chief Scientific Adviser of the Department for Environment, Food and Rural Affairs, expressed some of these during this event, drawing on examples of policy-making dilemmas and emergencies that the Department has to respond to and the importance of the scientific method in this regard. In the Government’s recent response4 to the House of Commons Science and Technology Committee’s call for a wider public debate on GM there was an insistence that science-based assessment of risks to human health or the environment must be the focus of the decision-making process. But social science methods could help to understand how to effectively frame communications on GM so as to encourage constructive public debate as to the acceptability or tolerability of different policy options. Roland Jackson explored the importance and experience of deliberative public engagement in this regard.
A failure to collate evidence as to the underlying framings of public responses lies at the heart of a number of societal challenges to technological development. Ultimately innovation may not be tolerated for perfectly logical reasons that have far less to do with concerns about potential risks than with disquiet about institutional motives and behaviours and perceived threats to things that people value5. Seeking to drive policy-making purely on an assessment of the physical risk (in itself often the subject of large uncertainties, ambiguities and indeed ignorance) will continue to be counterproductive.
Andy Stirling picked up this tension in observing6: ‘problems of ambiguity arise when experts disagree over the framing of possible options, contexts, outcomes, benefits or harms. They cannot be reduced to risk analysis and demand plural and conditional treatment’. It is the power of Q2 social science that is essential in this regard: i.e. evidence approaches which mix complementary quantitative and qualitative methods - to understand the ‘how’ and ‘why’ of people’s responses.
Qualitative social science must meet common quality criteria around transparency, clear match of method to question, open interpretation of accessible data, and potential for third parties to independently review evidence.
- The aims must be transparent
- The match of method to question has to be clear
- Interpretation of the data must be open in terms of the error and uncertainty
- Third parties should be able to review the evidence and arrive at an independent interpretation of the findings, and
- The data must be accessible
Importantly, social scientists who bring qualitative evidence into ‘interdisciplinary deliberation’7 of the sort essential to policy-making around global challenges must be open to challenge on their evidence. They must be sensitive to, rather than annoyed by, how their science is perceived and valued.
Video from this event is available on www.ncrm.ac.uk/resources/video or on the NCRM YouTube channel.
References
1 Davoudi, S., Harper, G., Petts, J. & Whatmore, S. (2015) Judging research quality to support evidence-informed environmental policy. Environmental Evidence, 4:9
2 Hazardous Substances Advisory Committee (2015) Considering Evidence; The Approach taken to Hazardous Substances in the UK. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/452336/considering-evidence-hsac-approach.pdf
3 Veltri, G.A., Lim., R., Miller, R. (2014) More than meets the eye: the contribution of qualitative research to evidence-based policy-making. Innovation: the European J. of Social Sciences Research, 27(1): 1
4 House of Commons Science and Technology Committee (2015) Advanced genetic techniques for crop improvement: regulation, risk and precaution: Government Response to the Committee’s Fifth Report of Session 2014-15. The Stationery Office, London
5 Petts, J. (2014) Context matters to human perception and response. Chapter 9 in Annual Report of the Government Chief Scientific Adviser, ibid
6 Stirling, A. (2010) Keep it complex. Nature, 468, December, 1029
7 Owens, S. (2012) Experts and the environment: The UK Royal Commission on Environmental Pollution 1970-2011. J of Environmental Law, 24(1), 1-22