Things That Can’t be Coded

by Daniel Kreiss

by Sarah Sobieraj 

Our family photos from the last decade are tucked snuggly in their digital home; I can’t remember the last time I made an actual print. But when it comes to research, I like to touch the data I collect. I still print out transcripts and field notes. They clog my cabinets alongside stacks of blog posts, newspaper articles, print advertisements, and screen shots for content analyses. Flyers, newsletters, and brochures? I have those too. Their files are not only thick, but also disorderly — 11×14 posters, newsprint, and glossy direct mail postcards competing for limited space in their densely packed hanging folders. A dedicated external drive and a scanner could end this, but some part of me needs those hard copies.

Those piles and files are daunting, but your data don’t need to be physically bulky in order to overwhelm; even if you manage them digitally, there is something about the sheer volume of information generated by qualitative research that intimidates.

This is even more true for me now at midcareer than it was when I was a graduate student. As a student, the data gave me validation. They served as proof that I was working as hard as I felt I was working, and also offered a sense of security, because those stacks reassured me that I would make it to the end of the road eventually. I felt I could do it because the answers were there somewhere, I just needed to excavate them.

But as I look at the new heap beginning to accumulate, I have to make peace with the uncomfortable reality that very often the answers are not in there.

I will still go through the pages with a fine-tooth comb. I will underline, highlight, and bookmark. I will repeatedly write words like “interesting” and “important.” There will be asterisks and exclamation points in the margins. I will leave myself pdf sticky notes that say, “think more about this” and “what is going on here?” And, inevitably, I will code the pages within an inch of their lives. But I know from past experience that many of the most valuable insights can’t be coded.

In qualitative research on political communication, much is embedded in things that are left unsaid, conversations that never take place, behaviors that are not part of the repertoire.  Managing the data you have is demanding, but the real challenge is in identifying what’s missing.

Some of my favorite sociology exposes these gaps and silences. I love the way Eduardo Bonilla Silva identifies the verbal gymnastics whites use to avoid speaking about race. And Nina Eliasoph’s fabulous Avoiding Politics, captures the way voluntary association members work to prevent political conversations. I initially thought that such instructive gaps were unique, but have come to realize that they are always there. We always need to fill in the blanks.

My first book, Soundbitten: The Perils of Media-Centered Political Activism, focuses on the relationship between journalists and activists as they wrangle over what will become news. After the field research, I found myself with thickly detailed notes about the media trainings that activist groups planned and implemented for their participants. Organizers also talked about the trainings in our interviews. I had the luxury of knowing the backstories about why they were running them in particular ways and how participants and even journalists felt about the trainings.

As interesting as those details were, the absence of analogous trainings designed to help activists speak with lay people was equally important. There were no sessions to strategize about how to approach pedestrians who happened upon their protests or marches or street theater.  But swimming in a sea of information about what happened – the conversations, body language, behaviors – it was hard to get out of the weeds. It took a long time for it to even cross my mind that there could have been such trainings. My realm of possibility was circumscribed by that of my participants.

Similarly, for our new article in Poetics, my collaborators and I interviewed fans of outrage-based political opinion media to better understand the relationship between someone such as Rush Limbaugh or Glenn Beck and their audience. We asked evocative questions about the programming and the personalities. We asked respondents to parse the differences among the content they enjoyed, and to tell us about the context in which they listened and watched.

The first transcripts were peppered with colorful stores as one would expect with in-depth interviews, but something was incomplete. There were hints. In talking about the hosts, the respondents sounded as though they were describing people they knew. And in reflecting, it started to feel as though their experiences with these “friends” were implicitly being compared to something else, something beyond the abundant comparisons to other political opinion programming or conventional news. We had a hunch that the most significant reference point was not other political media, but unmediated political conversations. But our interview guide contained no questions about everyday political talk.

And so we revised our interview guide, and went back to the well.

Once we added questions about talking politics, it became clear that the warmth, validation, and faux-intimacy that outrage-based programs offer their fans serve as a safe haven, where the risks and discomforts associated with political conversations in everyday life were bracketed. Respondents’ fears about political conversations – trepidations about social conflict, being judged negatively, or appearing uneducated poured out and stood in sharp contrast to the reassurance, camaraderie, and flattery offered by the programs. In the envelope of outrage, the fan is invited into a community and praised for their righteousness – a welcome reprieve from neighbors or co-workers who might belittle their views, poke holes in their arguments, talk about them behind their backs, or write them off as ignorant.

Fandom proves to be about the engaging formats and charismatic hosts, but even more so it is about the value these shows offer in this particular political milieu. And we nearly missed it. Respondents don’t answer the questions we fail to ask.

And so, each new project brings with it a mountain of data. Trying to make sense of what is in there is no small challenge, but knowing that the silences must also be explored sometimes feels like enough to stop me in my tracks. If there is a way to be sure that you’ll be able to see the negative space around your data, I have not found it. I just underline, highlight, bookmark, code, and wait.