NCRM Podcasts

Catherine McDonald  0:00  
Hello and welcome to Methods a podcast from the National Centre for Research Methods. In this series part of the EU Horizon 2020 funded Youth Life project we're looking at how researchers can do better longitudinal research on youth transitions. I'm Catherine McDonald and today I'm talking to JD Carpentieri, Associate Professor of Social Science and Policy in the Department of Education, Practice and Society at University College London, and an Honorary Research Associate at the Centre for Longitudinal Studies. JD engages in cohort study based qualitative research, focusing on identity, ageing, health and physical activity. He is currently co-investigator on a qualitative Longitudinal Study of cohort members experiences of and perspectives on long-COVID. I began by asking JD about the rationale behind his focus on qualitative longitudinal data collection in relation to cohort studies.

JD Carpentieri  1:00  
I think it started with me discovering the UK's birth cohort studies in about 2008 or so. And being absolutely fascinated with what those studies could tell us at say population level about you know, patterns and trends about say health in the UK or wellbeing over time in the UK. But I was a little bit nonplussed or surprised to find out that there wasn't very much qualitative research being done with members of these cohort studies. And I thought that there was a need for much more individual level research with these people. And some people were already doing research like this, like Jane Elliott, for example, who was then the director of the Centre for Longitudinal Studies. And so I wanted to be able to capture the individual level stories that were underpinning these broad scale patterns and trends that the quantitative data in the cohort studies were showing us.

Catherine McDonald  2:08  
Yes, because behind every data, there's a human story isn't there, there's a life going on.

JD Carpentieri  2:13  
Exactly. And I can give an example. So a few years ago, I worked on a project in which we were looking at the retirement expectations of the people born in 1958. So the 1958 birth cohort, and for the women in this cohort, the state pension age changed quite radically, only a handful of years ago, I think it was 60. And then it went up to 65, or 66. And a lot of women from that age group got caught out, they didn't realise that they weren't going to be able to retire on state pension age at the age of 60. But one of the things that the quantitative people did in that study was they were asking people in their late 50s, at this time when they expected to retire. And so they were trying to get policy relevant estimates of people's expectations for when they were going to give up the paid labour and move out of the labour market. And it looked like a lot of people were still expecting to continue working at least till 66, or 67. And quite a fair amount, still expected to continue working after that age. And I worked on a qualitative component of this broader mixed method study. And when we interviewed people and talk to them about how they viewed their lives, when they were say, looking ahead to their mid 60s, and then into their late 60s, a lot of them did talk about wanting to continue working. But they talked about it in a way that wasn't captured by the quantitative data. So for example, they would paint a picture of work being a little part of their life, rather than being the main central part of their sort of daily life. So they might envision themselves working two days a week for five hours a day. And they would envision themselves wanting to do this for the social interaction, and to keep their brain working and stuff like that. But they very often hadn't thought about the likelihood of that actually being able to happen, the likelihood of for example, their employer saying to them, sure, you can go down from 40 hours a week to working 14 hours a week or something like that. So they had the expectation of continuing to work well into their 60s. But their vision of what that work was going to be like didn't really match up with what work is typically like and what their working lives were like right now. So one of the policy messages that that gave us was that if the government does want to keep people in work and wants to be non-punitive in doing so, then it needs to work much harder. harder at making the workplace an area in which people can slowly step down their hours and their commitments of the number of days and things like that.

Catherine McDonald  5:10  
Sure. So sort of moving on how do you see the relationship between theory and qualitative research?

JD Carpentieri  5:19  
Oh, it's a good question that. I see qualitative research, I suppose primarily for me as a way of trying to add nuance to theory or to explore the complexities of how theories are actually played out in real life. So, several years ago, I worked on a project in which we were interviewing people in their late 70s. And again, this was a mixed methods project. And one of the theoretical frameworks that we used in our qualitative work was this theory of successful ageing, which is called selection, optimization and compensation. And this theory posits that in order to adapt to decline, whether physical or cognitive, older people will engage in being more selective about the activities they do, they will try to optimise certain skills or abilities. And they will use compensatory strategies like for example, using a walking stick. So we were able to analyse people's narratives about how they adapted to physical decline. And we were able to look at what they said and see how what they said related to this theory of selection, optimization and compensation. And it's a theory that primarily has been applied only quantitatively with sort of measurements and batteries of instruments that try to capture people's attitudes to these forms of adaptation. But we were able to talk to people about how those attitudes actually were expressed in real life working situations. So I think we're able to contribute a little bit to understanding how the theory plays out in the real world.

Catherine McDonald  7:12  
So that's obviously one example of mixed methods research that you've undertaken. Are there any other examples you'd like to share with us?

JD Carpentieri  7:18  
Yeah, I'll pull up an example. That's not related to cohort studies, because I do think mixed methods has value pretty much everywhere. I think it was a six-country kind of Pan-European study that I did in which we were looking at and attempting to assess various government initiatives aimed at helping what they called Low educated adults to get new qualifications. So these would be adults who were say in their 30s, or 40s, who had not completed secondary school, for example, and governments were trying to incentivize these adults to get the equivalent of secondary qualifications. And we were able to get some very good quantitative data on the effectiveness or lack thereof, in some cases of these programmes across the various countries. But we were also able to do qualitative, semi structured interviews with some of the adults participating in these programmes. And through this combination, we were able to see both when and in what context programmes seemed to be successful. But through the interviews, we were also able to understand why or why not programmes and policies would succeed. So sometimes people would describe one weakness of a policy or programme that would look like maybe just a small weakness from the policymakers perspective, but say, from a single mother of three children, this weakness would be something like the time the programmes were scheduled, and it would completely overwhelm her ability to succeed in the programme. So we were able to provide detailed insights into why and why not the policies and programmes would succeed.

Catherine McDonald  9:15  
And do you have a favoured model or a kind of best plan of research methodology that you tend to adopt and that you like to adopt? And if so, why?

JD Carpentieri  9:24  
I would say that, it always depends on what the research objectives are, and also what the available resources are. So I'll have done mixed methods studies that were sequential. So for example, sometimes with the qualitative coming before the quantitative, but I've also done sequential mixed method studies in the other direction. And I've done parallel mixed method studies. And sometimes you find yourself thinking actually, this particular study should be purely qualitative or should use a couple of different qualitative methods? I think it definitely depends on the particular research aims and questions. And it also definitely depends on resources, every funding proposal I've ever written, has started out significantly more ambitious than the final project.

Catherine McDonald  10:21  
And that leads me on to ask actually, what sorts of research questions or issues do you see as best suited for mixed methods longitudinal research? Is it possible to say?

JD Carpentieri  10:31  
Oh, that's a good question. I would tend to say that it's areas that researchers or policy people would call people use different terms, but may be complex or tricky, or I've heard people use the term wicked problems before to discuss issues that aren't quickly and relatively straightforwardly amenable to policy solutions. So for example, I've not studied obesity. But I could imagine that obesity, it's a quite complex policy problem, or physical inactivity is a quite complex policy problem, in which we need to understand broad scale patterns and trends about people's behaviour. But we also need to understand the complexities of people's lives. So that policy doesn't end up being a blunt instrument, and so that our quantitative research findings don't end up kind of missing really key points. I've spoken, for example, to people who study welfare conditionality. And so this is about the conditions that the state puts on recipients of welfare benefits. And it's only through talking to people who are living on the margins of society, about their lives and about the barriers to, for example, gaining employment, or maintaining employment, or showing up to the welfare office at exactly the time that you're supposed to. Only through understanding those things through a combination of qualitative and quantitative research, can we truly and fully understand how to possibly address the problem. And I suppose the bit that I mentioned earlier about understanding people's attitudes to the state pension age and when they're going to retire, we need the quantitative data in order to give us a picture, a bird's eye view of the patterns and trends in people's attitudes and expectations. But we also need the qualitative material in order to get a clearer picture of what's underlying people's attitudes and perspectives. And only through both of those, are we going to be able to craft effective policies.

Catherine McDonald  11:26  
And do you have an approach to things like comparison and generalizability? And if you do, what is that approach?

JD Carpentieri  13:11
Oh, that's an interesting question. I mean, generalizability is a very tricky one. I'm often because of the types of studies that I do working within the cohort studies. There's a lot of times where I'm the only qualitative person in a room. And the question of generalizability comes up quite a lot. And sometimes someone can be a bit critical and say that, since my findings from interviews with say, 40 cohort members or whatever, aren't statistically generalizable, then they don't necessarily have scientific value or policy value. And I guess my argument there is that I'm not trying to statistically generalise, but I am trying to theoretically, generalise, I am trying to come up with messages and meanings that aren't just specific to a person who I happen to have interviewed, but that have their root and their value in shared human conditions. So again, to go back to being in your 60s and thinking about getting older and continuing to work up to a later state pension age. I couldn't generalise statistically from the stories that people were telling me about their working lives. But we do know that work has it stresses and its demands and its benefits say socially and economically and stuff like that. So I do know that the stories these people are telling me they're not just rooted solely in their own lives, they are growing out of a world that we are familiar with and that we have some understanding of, and then those stories can then feed back into our policies that are developed for the work world or for retirement.

Catherine McDonald  15:11  
So focusing on the people involved in your research, so the lives behind the data. With longitudinal research, obviously, attrition is a problem. How do you keep people involved and onboard with your research? Have you got any top tips?

JD Carpentieri  15:26  
This is one where my top tip is let other people who are better at it, do it for you. And in my particular case, one of the things that I like about doing longitudinal qualitative research within the context of the UK's birth cohort studies is that the research centres who run these cohort studies, they have teams that work very, very hard and very successfully, to keep individuals in these studies over their lifetimes. And because these individuals stay in the studies over their lifetimes, or most of them do, and because these research centres work so hard to establish and maintain good relationships with their cohort members, then what we find when we do qualitative research, is that attrition is much lower across waves than we would find with typical qualitative samples. So I'm currently doing research on individuals who have long COVID. And it's a three wave 18 month study. And we have very low attrition levels across the waves. And I think that that is largely due to the fact that these individuals because they're part of cohort studies, they have a strong conceptualization of the value of research. And I also think that there's an element of commitment from cohort members to being part of the research world and part of these studies. And an element of perhaps you could call it pride or sense that they're contributing to the world's knowledge and to better health for other people, etc. In some of the qualitative studies I've done, we will make one of our questions, we'll ask people how they feel about being part of the cohort studies. And they very often talk about how they feel that by being part of the study, they're making a small but meaningful contribution to the world. And I think that then benefits me as a researcher, because they're more likely to stay in my study.

Catherine McDonald  17:44  
Absolutely. And indeed they are, of course, what about ethical dilemmas? Have you faced any? And if so, would you have any advice to pass on as a result?

JD Carpentieri  17:55  
I think the most salient ethical challenge that I've faced in doing cohort study based qualitative research is that there are different understandings of data protection across different paradigms and methodologies of research. So for example, for people who have spent their lives doing quantitative research, it's just assumed that you never know the real name of the person who appears in your data. Everything is anonymous to the researcher who's doing the analysis, because you're analysing at population level. And there's loads of data protection standards built in, and quite rightly, of course, but then, with qualitative research, you're contacting people directly, and you're interviewing them. And they're telling you very often quite intimate and sensitive details about their personal and private lives, because they're telling you about what matters in their lives. And so sometimes it can be very challenging to work out with the quantitative partners, how you're going to ensure that those sensitive details that me as the researcher is receiving, are going to be treated with the respect and rigour that they deserve to be treated with. And there's also challenges about how you make sure that these personal stories from cohort members can't possibly be linked to their lifetime of quantitative data. So with regard to that last one, we always build in firewalls and safeguards. So if I'm interviewing someone in a qualitative study, I will inevitably know their real name, but I will have no way of connecting that individual who I'm interviewing with the quantitative data that's been collected from that individual over their life course. So there's a firewall built between me as a qualitative researcher and the quantitative data. The main thing I've found, is when I talk to my colleagues who are, I wouldn't say normal qualitative researchers in terms of not working with the cohort studies, is they have to go through many fewer hoops in terms of data protection and data security. But I for very good reasons have to go through a lot more bureaucratic procedures in order to ensure the protection of the data, I always hire someone a professional to go through and remove any potentially identifying details from any of our transcripts before those transcripts are made available to the broader research community.

Catherine McDonald  19:31
Wow, it’s fascinating to me as a non-academic, to hear of things like that firewall, and all of the lengths that you know, are gone to to ensure that anonymity, it’s really fascinating to hear about.

JD Carpentieri  21:07
Just to add, I think it's particularly important with regard to what I was talking about earlier about the cohort members sense of commitment to the cohort studies. And they're quite noble feeling that they're helping contribute to the broader good, even if they didn't feel that we would have a very strong ethical obligation to protect their data. But the fact that they feel committed to the studies, that just to me increases the moral sense that we need to be extraordinarily committed to protecting their data and protecting their identities and security.

Catherine McDonald  21:48  
Yes, I understand. So moving on to analysing the data. What approach do you take, and why do you take that approach?

JD Carpentieri  21:55  
I tend to be a bit of a dilettante in this regard. And I like to try different analytic techniques for different studies. So thinking over studies that I've done in say, the last five or six years, I think I've done thematic analysis, I've definitely done a lot of narrative analysis, which tends to be my favourite, because I love studying the stories that people construct about their lives, whether the big things in their lives, or just stories about small mundane details, I think there's a lot that can be learned about people and the world through studying those stories. One project I worked on a few years ago, we did collect narratives from individuals. But we decided to be a little bit experimental. And we engaged in what's called quantification. And what this involves, is applying quantitative ideas and approaches to qualitative material. And so in this case, we went through the narratives that people were telling us. And we actually counted certain aspects of these narratives. And we tried to see if having more of a certain component of a narrative was associated with some of the outcome variables that we were interested in. And that was quite fun to do, and a nice complement to the narrative analysis we were also doing on that project. I've also worked on a project with some quantitative researchers who specialise in machine learning. And in this project, we had more than 10,000 pieces of qualitative material. And these quantitative researchers did computer analysis of all of this qualitative material to see if they could spot patterns and trends in the material that the human eye or a qualitative analyst couldn't spot. And I really enjoyed doing that as well.

Catherine McDonald  24:04  
And what about the writing up stage? Again, you know, how do you approach it? And have you got any advice that you'd like to share?

JD Carpentieri  24:11  
At this stage of my career my main strategy is try to partner with other people who like writing more than I do. I really enjoy the initial stages of writing where you're getting the ideas on the page. And I really do not enjoy the phase of writing in which you're converting, let's say 12,000 words into the 8000 words that will fit into the specific journal you want to send your article to. So when I have to I do that on my own when I don't have to, I try to take charge of the initial part of the writing and let a colleague, as long as they're happy, to do the second part of the writing. And I suppose career wise, you definitely don't want to accidentally exploit people, but you do have to try to find the parts of the job that you enjoy doing. And try to hurl yourself into doing those. And ideally, I think the best partnerships are where you have two people who like to do different but complementary aspects of the same job. And that is something that I actively look for in research partnerships nowadays.

Catherine McDonald  25:33  
I can well imagine. I'm curious, though, what is it about getting - even as I say the question, I feel it's kind of obvious - but what is it about getting the 12,000 words down to 8,000 that you don't enjoy?

JD Carpentieri  25:44  
Well, what I enjoy about the first 12,000 words is that's where you express your ideas, and you see how those ideas relate to each other. And that's the bit where I feel like I'm actually learning from the research that I've conducted, that's the part of writing where I feel that I'm learning. And then the next phase of cutting things down, is the part where I feel like we're almost getting rid of some of the aspects of what we've learned in order to cut it down into whatever formula it has to fit into for the specific journal that we're sending our article to.

Catherine McDonald  26:28  
Understood. Yeah. So one final question. And that is, what would you say to that early career version of yourself?

JD Carpentieri  26:37  
I was lucky enough to come into academia laterally from another career in my late 30s. So I was able to skip the stage that a lot of young researchers have to suffer through, I didn't have to do multiple successive postdocs, like a lot of younger researchers have to do. And I got very lucky in that regard. I would say, though, for any early career researcher, you're obviously trying to craft a career. And that's very difficult in the current academic environment. But I would say that as much as possible, while trying to craft your career and make your own way, try to work towards having your own autonomy. What I mean by that is that academia is always making excessive demands on us, on our time and on our energy, on our intellectual ability, and also our emotional energy. And while you're trying to craft a career, it's in some ways, the easiest thing to do is to say yes to everything, because you want to keep all of your options open. And to always work as much as is physically possible to work. But I think a good lesson for people is from an early point in your career, to start trying to build little bits into your career where you are saying no to some things, or you are making strategic decisions about how much time to invest in a particular activity or a particular commitment. Because otherwise, it's easy to just feel yourself being pushed around by forces larger than you all the way through the early phases of your career. I guess there's one other thing that I would say is that one of the things I have found is that through meeting people, and basically trying to be nice to people, that can make a massive difference in your career. I noticed very early on that amongst the successful academics that I was working with. Some of them weren't necessarily very nice people. And some of them were some of the nicest people I'd ever met. And the lesson that I took from that is that to rise up in your career, you don't have to sacrifice kindness. You don't have to sacrifice generosity, you can be a generous and gentle and supportive person and still develop a very successful career. Some people develop a successful career through being cutthroat, but you really really don't have to do that and the more nice people rising up the ranks there are the better it is for all of us.

Catherine McDonald  29:40  
My thanks to JD Carpentieri. The Youth Life project is funded by the EU Horizon 2020 Research and Innovation Programme and is a twinning initiative between the Universities of Southampton, Tallinn and Bamberg, and the Netherlands Interdisciplinary Demographic Institute. You can find out more about the project at www.euqualimix.ncrm.ac.uk. This was a Research Podcasts production. Thank you for listening and remember to subscribe wherever you receive your podcasts.