AO: Here the interlocutor describes a project where having the data available to look at (rather than just the final output) would have been particularly helpful. She also describes how it is difficult to ask the same client for funding to go back and conduct the qualitative research again because in theory it has already been done by someone else (even though the data from the initial research is not available to look at). To me this seems to suggest that clients and funders need to also consider datasets as part of normal expected project deliverables... and if that is the case, better guidelines for the care and management of such datasets need to also be in place for such clients and funders to ensure that they are appropriately handling and storing this data.
"Angela Okune 53:06
Okay. So in such an example, would looking at the transcripts that had already been done have been beneficial you think?
BC-MW-F-S-02 53:13
Definitely. So that's also the question of... we have a current project that's going on here...research has been conducted. And we have the results, but we don't know what informed those results, and when you look at the results... You're meant to design an intervention. So let's imagine if ummm ... I'm trying to think of a good one they were covering...They're trying to get people to...women who are pregnant to go and deliver in a clinic rather than relying on traditional birth attendants. So in that example, we, we have...the report tells us a couple of things that people like traditional birth attendants, because they can be in their home, they have the comforts of their home, and they don't necessarily like the clinic. We don't know why they don't like the clinic, and we just have that final... "they don't want to go to the clinic, they want to go to the TVA." And so for us designing the interventions, we don't know what we're designing for. There may be 10 reasons why they don't want to go to the clinic. Is it because it's not a nice space? Is it because it's too far away from them? Is it because they can't have anyone else in the room? Is it because they're forced to give birth in certain positions that they don't think are suitable. And so if we knew more around the why, then you could design better interventions. So someone has already conducted that research. So it's hard to face the client now, if they won't give us the transcripts because they don't have them, we need to go back and do qual research, because in their mind, it's been done, you can just work off the report. So it would be useful to have it in those cases, because you can identify gaps and know what things you should go research rather than starting from scratch.
Angela Okune 54:40
That is a really good use case of why we should share research data and why it's not just enough to have the final report."
AO: Here, the interlocutor takes the time to carefully distinguish between "impact" (which she views as the "results") versus "reach" (how many people read something). Various publics are increasingly interested in the "impact" of research with diverse interpretations of what it means. Funders ask researchers to articulate the "broader impact" of research work (one of the categories of being awarded NSF funding for example) and "impact factor" is often used to rate and rank academic journals.
BC-MW-F-S-02 50:57
“So because then you're not showing impact in terms of how you've impacted your end users. So for example, if I was doing a project on sexual health targeted towards adolescent girls, the impact there would be how many people...what the results are from the study. That's how I would show impact. But in this case, I'm just showing impact in terms of how many people have read my study, and that that's a different type of impact and I wouldn't necessarily call that impact. That's just reach, I guess.”
AO: The interview excerpts below demonstrate the need to carve out and establish a research company's unique value proposition in Nairobi's research market. It is important to also develop clear brand recognition for that value in order to develop collaborations across different groups. If you are going to be a good collaborator within the Nairobi research market, you don't want to be seen as stepping on someone else's established turf.
BC-MW-F-S-02 39:44
“…the second element, I guess is you don't necessarily want a client to think that you're incapable of doing this yourself. So by citing another person, it raises the question of "why are we hiring you? Are you able to carry out this work to completion? Or do you need to revive another organization's knowledge in order to get to this stage?" And so I think when you have those two things together, it prevents you from from really acknowledging things. But it just depends on how established the company is as well. So if you're a very established company, and everyone trusts you to be the expert, then you're probably going to be more comfortable with acknowledging people because it's fine, everyone knows you're great. Whereas if you're starting off, and you have more of that insecurity, of "I'm in competition with these different people, we haven't yet found our niche. People don't necessarily know that we're the experts in this area," then you don't want to flag up other companies that could in theory be contracted instead of you. So I guess it depends on the company as well.”
BC-MW-F-S-02 41:14
“...the new technical division around qualitative research and design...the reason why it doesn't have a name yet is because you don't want to call it a certain thing so that other people think you're now stepping in their territory. So it's meant to be an internal resource. But if people then see [REDACTED ORG NAME] is opening up a design department, then all the design or HCD firms will start thinking, are we going to be slowly creeping into their market and that's not the intention, but we're not going to have those conversations with them beforehand...”
AO: The interlocutor raised the double bind of anonymization of qualitative data:
"BC-MW-F-S-02 6:01
So internally I would want all of this information. So when even I can see anonymized male, anonymized ministry...I would probably want to know that information so that I know that if we've gone to this person more than once with different research projects, or if I want to do something on a particular ministry, I know how to identify that. So that would be for internal use. But for external use then I think...this is tricky because the location that you have that could be of interest to someone so taking away is a loss of... so I don't know whether you could... hmmm... this is a very tricky one because the more information you give the easier it is identify that person, but then the less information you give them...
Angela Okune 6:44
the less value it is for the researcher.
BC-MW-F-S-02 6:46
Exactly. Yeah."
AO: This interlocutor raised the following challenges/risks of data sharing:
- Who is responsible/has the mandate/time to do the labor required to surface the information (e.g. make a summary page with various links to different data points/resources; add the context to the data that was collected; sort through data request emails and approve/disprove; etc.)
- Uncertainty about what the client is okay with in terms of sharing because this is not an explicit part of the client contracting. The person writing report/working on the data may not be the person directly liaising with the client.
- How much data context is needed - naming all the people who helped make the data? How they were trained, etc. Etc. It could quickly get very overwhelming.
- How easy is it to download the data and get into the wrong hands; how easy to manipulate it and use it in ways that were not intended?
- Plagiarizing of data - not acknowledging source.
AO: This interlocutor raised the following perceived benefits of sharing:
- Internal knowledge management (being able to make sure those in the organization can access each others' work/data)
- Inspiration: Being able to see the types of questions asked previously and how effective they were.
- Literature review: knowing what kind of work has been done previously.
- Reduced need for large sample size (= reduced cost to run the research); ability to build on existing work that has been done rather than redo the whole thing again
- Ability to test if a claim holds; increase the validity/credibility of a qualitative claim
- (With PECE), ability to get a sense of the perspectives and background context that goes into the creation of qualitative data (e.g. by sharing instrument; notes; stripping sheets; etc.)
AO: The interlocutor indicates lack of time or being busy as a key determining factor for whether or not data would be uploaded and shared by others in the organization. She notes that currently, there is little consistency in how people share existing data within the org, noting the multiple technical platforms being used. Later in the interview, she also mentions that limited time might also impact on how much the actual transcripts are in fact used, noting: "I have a feeling that they wouldn't necessarily use the transcripts because people struggle with using our own transcripts. So I think the instruments may be the initial style of...you can see these instruments and the transcripts are there, if you want but I just have a feeling that people wouldn't necessarily read through all of them."
***
"BC-MW-F-S-02 15:31
I'm also wondering whether, depending on how easy it is to upload this, if the person is doing the transcription in the first place can upload it to this platform. So the researcher isn't looking in internal folders for these transcripts, or through things like Slack, but instead they go into a centralized place. And that forces people to use a centralized place. But otherwise, you're just going to have the same problem with knowledge management, where people use their own channels. They have Google Docs floating around and they're not... they're not going to really upload things onto this. So that could be one way of skipping that loop... by having the transcriber do it. And then if that's the case, then I'm thinking who would tag this then? So it's either the researcher tags it, but that's kind of looking at a more thematic analysis where they're going through it in each paragraph and thinking through what's the key themes and then making note of that, I don't know someone's...unless they're thinking of doing that in their own research, they're not going to do that as an aside...to tag things...unless you really buy them into the fact that this is a useful resource. So you may need someone externally to do this...to read through the transcripts and tag it themselves. But I think if you had the researchers doing this, I've think it's not going to happen in terms of uploading it. Unless you make it compulsory, a compulsory component of a project.
Because right now people don't upload things onto Box which is our internal one. So if you can't get them to do that, and then you're adding another one, it's not going to happen. So what I see if you're showing them, "this is where they get the transcripts from," and it makes it easier to find transcripts, and then find other ones that's related, then they would utilize it but I don't think they're going to upload it themselves."
...
"Angela Okune 1:17:00
...How do you feel your thoughts reflect across the organization? Like, do you think that your colleagues here would share a lot of the sentiments you've expressed in this interview or do you feel like you're an outlier?
BC-MW-F-S-02 1:17:28
I think people would share the sentiment that it's useful, in terms of...often especially...So useful in two ways. One, internal resource. I think it would be very easy to get buy-in in that sense. "Here's the place where you can find this information." Because we struggle with it right now. So definitely there. In terms of externally, I think there will be some apprehension on who's seeing this and client perception and then whether things can or can't be shared. And apprehension almost lies in the fact that it's not explicit with many clients on what their wants are. So we may share some things based on our own value judgment but if you're sharing a lot of information then it requires further conversations and depending on where you are in the organization, you have a different level of comfort with that because you may not be the person that has direct access to the client to ask them and those details. So I think there would be some apprehension there. I also think there would be some apprehension in managing it so who's the person that's uploading the information...how much work do they have to do? Generally wouldn't want to do more work so unless they are really bought into this as something that I see a lot of value in they're not going to do it. I could see some people being like yes, I think this is great...I really think we need to expand the knowledge but other people are more like, "this is my project. I don't want anymore work, I'm already overrun with things." So there's kind of two...two groups but I think most people would say it's a really good idea, especially for internal management and I think people would also want to access it for external resources to help them with the literature review and thinking through...
Angela Okune 1:19:05
...And then I think when they would see that value...would then incentivize them to do it now, because it needs to be supported...
BC-MW-F-S-02 1:19:12
But I also don't know whether people would use the transcripts. I have a feeling that they wouldn't necessarily use the transcripts because people struggle with using our own transcripts. So I think the instruments may be the initial style of...you can see these instruments and the transcripts are there, if you want but I just have a feeling that people wouldn't necessarily read through all of them.
Angela Okune 1:19:29
Right.
BC-MW-F-S-02 1:19:29
If the format we're reading through them right now, there are not even transcripts there in the stripped format...then yeah..."
AO: This interlocutor perceived "good quality" qualitative data to be signaled through "thick" documentation which would include background context on the interpersonal relationship between the researcher and the person being researched as well as details that go beyond what is said (e.g. laughter, stuttering, etc.). They indicate that close attention and capture of such details would increase their confidence in the data.
"Angela Okune 30:07
So if you were let's say to assess this data set to determine whether this was quote unquote good quality or bad quality, what things would you look out for?
BC-MW-F-S-02 30:16
Oh gosh. Good quality, bad quality? In what way?
Angela Okune 30:27
I mean, even and this can be outside of this particular platform, but like what kinds of qualitative data ... like if you're looking at a transcript, you're reviewing someone's work from the field or whatever...what would make you feel confident in?
BC-MW-F-S-02 30:42
In interpreting it. Um. So usually, for me, ...how it's written. So this is gonna sound strange unless I give an example. But for example, here, you've got "aahhh... noooo... mmmhh...ummmm..."... So I'm seeing it as this person is transcribing exactly what they hear. I've seen other transcriptions here where you can tell that that's not what the person said they've summarized it in their own words, or they've tried to make it grammatically correct. And so it doesn't match up, you don't capture some of those details of this person is unsure at this point, and then they go on to it. Other ones were liker the dot, dot, dot, etc. But there's other ones where sometimes you'll have in brackets "pause" and how long that pause is for. So if someone's sharing [pause] verbal things that you wouldn't necessarily want to record because they are not correct English, or they're also indicating pauses or laughter or other things then I trust that more than one that looks perfect, because I'm thinking they're summarizing it in their own words, and it's not real transcript. Then the other thing I mean, this one is good because you have the respondent talking a lot, and the questions seem pretty open ended. So that's another thing that I will look out for is it one where the moderator seems to be taking up most of the the written work or is it the respondent, if it's the moderator, then I'm assuming that's really just a quantitative interview, and they're controlling it. And there's probably some other dynamic going on then. Then also in terms of your summary at the beginning, ideally, and this has not happened here. But in an ideal world, if there was more information on the person conducting it, or if there was some kind of reflective notes, where they may say, I am from this particular region, and I was interviewing this person from another region, and there's conflict between our regions, having that context would be really useful or we're from the same region so this person would feel more comfortable with me, that would be useful because especially in the Kenyan context, that is often something that could be quite [inaudible]. So if I conduct an interview, the response that I am going to get is going to be very different from a Kenyan conducting the interview. So even having that like if you just have anonymous female, you're not going to know that this is a... like where I'm from etc, of the age, my age whether this person does or doesn't match me. So I think that information would be really useful as well. Yeah."