A group of colleagues stood around a table

Developing an innovative new method for qualitative researchers

A team of four researchers – Dr Susie Weller, Dr Emma Davidson, Professor Ros Edwards and Professor Lynn Jamieson – pioneered the breadth-and-depth method for analysing large volumes of qualitative data.

The team developed the method over a five-year period, supported by two NCRM grants.

Impact achieved

The breadth-and-depth method provides researchers with a valuable new tool. By combining computational text analysis with conventional qualitative methods, it enables qualitative researchers to ask new questions and discover new insights from existing data.

Using the method, researchers can pool data from different sources and harness underused datasets. The approach can discern patterns, trends and associations, while producing in-depth understandings about our society with the potential for generalisability.

The team has trained more than 500 researchers globally on how to use the breadth-and-depth method, with course participants implementing the method in published work. The team has also created online learning resources and produced multiple publications, including a book and nine journal articles.

"There is a huge appetite amongst researchers, not only for the method, but more generally for innovative ways of exploring and reusing the range of qualitative data available,” the researchers explained in their impact prize application."

"Combining qualitative datasets increases the diversity of sample populations and contexts, enhances the possibility of theoretical generalisability and strengthens claims from qualitative research about understanding how social processes work."

How involvement with NCRM made a difference

The team began their work on the new method after considering the untapped potential of large qualitative datasets. Their goal was to explore how some of the principles of big data analysis could be applied to qualitative data to yield new insights on critical social issues. A major challenge was finding out how to do “big qual” while retaining the rigour of traditional qualitative research.

"Big data is often associated with quantitative datasets which, when analysed computationally, reveal patterns, trends and associations,” the team said in their impact prize application. “Yet there is a wealth of qualitative data available through archives and data sharing. Often underutilised, these present exciting research opportunities."

When the team first presented their plans for large-scale qualitative analysis at NCRM’s 2016 Research Methods Festival, attendees asked “why would you want to do that?”. Although researchers were interested in the possibilities of working across multiple datasets, some were cautious about the ethical and epistemological implications.

However, the team continued to pursue the idea, aware of the increasing expectations on researchers to make data available for reuse as part of the open science agenda, and the lack of use of public datasets. In recent correspondence with the UK Data Archive, they found that of the 1,103 qualitative or mixed-method datasets available via the archive, 71 per cent had been downloaded 20 times or fewer.

Pioneering a new method

NCRM provided the team with funding to create the breadth-and-depth approach over a five-years period. The funding also enabled the team to create learning resources and deliver training on the method.

This pioneering work put the team at the forefront of new wave of big qual development. The method enables researchers to explore large volumes of data through computational text analysis, giving breadth to their research, while retaining the depth, richness and rigour of traditional qualitative approaches.

"The method has been described as field-changing by academics advocating a range of methodological approaches," said the researchers. Professor Bren Neale, of the University of Leeds, explained that big qual represent a "quantum leap" in the interpretation of qualitative research data and a "seismic shift" in the nature of qualitative evidence and findings.

Training researchers

The team has run training on the method for more than 500 researchers in the UK and other countries. Course participants have used the method in published papers and included it in successful applications for new research.

Researchers are also learning how to use and teach the method through an NCRM online tutorial, an archived teaching dataset and the Big Qual Analysis Resource Hub. The team has written the first big qual analysis textbook, which focuses on the breadth-and-depth method, and published nine articles.

"Rather than asking 'why?', researchers are now stating ‘show me how’," said the team. "The uptake of our resources, training and bespoke guidance on the method by scholars globally is evidence of the growing impact and reach of our NCRM work."

International collaborations

Universities in the UK and US have requested training on the method. They include Harvard University, the University of Pennsylvania and the University of Liverpool. The team was also invited to run a workshop at European Congress of Qualitative Inquiry.

They have worked with international researchers in Canada and the European Union, and were invited to contribute to an international paper, for which they used the method to pool and analyse datasets from 10 national contexts. The method has also gathered interest from organisations outside of academia, for example a charity, a not-for-profit research centre and a government department.

"Our work has played a central role in promoting international interest in large-scale qualitative analysis," the team said. "The international reach of our impact on the research methods landscape is evidenced by burgeoning requests for training and customised support for teams in applying the method to their own work."

Closing the borderlines between quantitative and qualitative research

By adding breadth to qualitative analysis, the team has created a method that enhances the possibilities for generalisability of findings. The team says that the method has closed the borderlines between quantitative and qualitative research, and that it generates possibilities for collaboration between researchers in different disciplines, sectors and geographical locations.

They explain this has the potential to change the way that qualitative evidence is understood and used in policy.

"The method does this by ensuring that qualitative data and forms of enquiry are not neglected by big data analytics," said the team. "Rather, it foregrounds the necessity of having analytical processes that can discern breadth through high-level patterns, trends and associations, alongside nuanced, contextualised, in-depth understandings."