Openness in Practice in Qualitative Research

The discipline of political science has been engaged in discussion about when, why, and how to make scholarship more open for at least three decades. Scholars who collect and base their work on qualitative evidence are relative newcomers to the debate. Over the last ten years, a vibrant conversation on these questions has emerged among such scholars and multiple differences of opinion have arisen. This piece argues that the best way to resolve our differences and develop appropriate norms and guidelines for making different types of qualitative research more open is to move from “if” to “how” – for individual political scientists to make their work more open – generating examples from which we can learn and on which we can build. We begin by articulating a series of principles that underlie our views on openness. We then consider the “state of the debate,” briefly outlining the contours of the scholarship on openness in political and other social sciences, highlighting the fractured nature of the discussion. The heart of the piece considers various strategies, illustrated by exemplary applications, for making qualitative research more open. We close by emphasizing that by engaging and collaborating with each other, we will find ways to employ openness to its best purpose – to demonstrate the rigor of qualitative research and the value it delivers to political science. Introduction: Key Principles The discipline of political science has been engaged in vital debate about the issue of research openness for more than three decades. In the abstract, augmenting openness implies the same steps in all types of political science scholarship: making the empirical information that underpins our work meaningfully accessible, and elucidating how that information was collected, generated, interpreted and/or analyzed, and how it supports claims and conclusions in published work.1 1 Here we follow the American Political Science Association’s conceptualization of transparency (see, e.g., APSA 2012).


Introduction: Key Principles
The discipline of political science has been engaged in vital debate about the issue of research openness for more than three decades. In the abstract, augmenting openness implies the same steps in all types of political science scholarship: making the empirical information that underpins our work meaningfully accessible, and elucidating how that information was collected, generated, interpreted and/or analyzed, and how it supports claims and conclusions in published work. 1 Nonetheless, the way in which openness is enacted, and the form and severity of the challenges that pursuing it presents, vary across research traditions. 2 Scholars who collect, generate, and draw on qualitative evidence in their work are relative newcomers to the debate about openness. Their more vigorous engagement in these discussions over the last decade has brought important new voices and viewpoints to the conversation, contributing fresh ideas, and raising new issues and questions. These perspectives were perhaps most clearly and completely brought to the fore through the recently completed Qualitative Transparency Deliberations, directed by Tim Büthe and Alan Jacobs (QTD, www.qualtd.net). Among its many lessons, the QTD clearly demonstrates that practices and strategies for making scholarship more transparent, and the challenges that doing so poses, vary among different forms of qualitative inquiry. 3 Nonetheless, we firmly believe that three key principles underlie the openness enterprise across all political science. First, openness is not an all or nothing pursuit; it is replete with gradations. Making scholarship slightly more transparent is better than not doing so at all. Sharing some of the data that underpin an article benefits authors, readers, and research communities more than does sharing no data. For scholars with serious misgivings about sharing data, clarifying the steps they took to generate and analyze those data is superior to leaving these aspects opaque. We should strive for as much openness as is legally and ethically possible, and any step towards transparency should be recognized and celebrated. Second and relatedly, conducting research ethically trumps conducting it openly. To offer just one self-evident example, when scholars promised the people they involved in their research that they would not share the information they conveyed -or did not discuss sharing that information -the information cannot be shared.
Third and finally, the most important steps forward will be those taken by individual political scientists to make their work more open. Their doing so will generate examples from which research communities can learn and on which they can build to develop appropriate (communityspecific) guidelines for openness, which funders, journals, and other institutions can then adopt. To that end, after a brief review of the rich literature on transparency in qualitative inquiry across disciplines and contexts, we offer a menu of strategies that political scientists who engage in qualitative inquiry can adopt and adapt to make their work more open. None will likely be useful to everyone, but we hope most scholars will identify a handful worthy of exploration.

State of the Debate 4
Over the last decade, political scientists who generate, collect, interpret, analyze, and publish scholarly work based on qualitative data have been engaged in vibrant debate about research openness. 5 That debate has mainly played out in written form in a series journal symposia in which scholars have discussed the intellectual benefits -for authors, readers, and research communities more generally -of making qualitative research more transparent, and the epistemological, ethical, legal, and practical challenges to openness. 6 In addition to these exchanges, political scientists have been published a few individual pieces, some advocating for transparency (e.g., Miguel et al. 2014, Gleditsch and Janz 2016, Elman et al. 2018) and others registering concerns (e.g., Schwartz-Shea and Yanow 2016, Monroe 2018, andTripp 2018).
Perhaps the crowning achievement with regard to qualitative political scientists' consideration of openness has been the Qualitative Transparency Deliberations (QTD, https://www.qualtd.net/). Called for during the business meeting of the Qualitative and Multi-Method Research (QMMR) section of APSA during the annual meeting in 2015, the deliberations involved 13 working groups (and hundreds of political scientists in and beyond those groups) considering the meaning, advantages, and challenges of pursuing transparency in qualitative research. 7 The product -13 thoughtful final reports -were published in early 2019.
All of these discussions have laid very useful groundwork for pursuing transparency in qualitative political science. While in a piece this brief we cannot do such a complicated debate justice, we offer three observations about the discussion. First, our overarching impression is that proponents and opponents of openness are not listening to, absorbing, and responding to each other's arguments as carefully as they might; even in dedicated symposia, we often seem to be speaking past rather than with each other. Second, with important exceptions that we consider in the next section, much of the debate remains focused on the question of "whether" different types of qualitative research can and should be more transparent, rather than on how to enhance transparency. Third, the discussion among qualitative scholars seems to be waning at precisely the moment when vital exchange about openness is needed: the QTD is a critical starting (not ending) point for this conversation.
Fortunately, rich discussions about openness in qualitative research have been proceeding in other disciplines and areas, and other countries -discussions to which political scientists can repair for ideas and inspiration. That broader debate began earlier than did discussions of transparency in qualitative political science -indeed, we found more than a dozen pieces addressing openness in qualitative research written between 1998 and 2009. Further, while much of the political science literature on qualitative transparency originates from the U.S. or at least appears in US-based journals, the lion's share of contributions to the broader literature on transparency emanates from Europe (especially the UK) and Australia. In terms of authors' backgrounds, scholars conducting empirical research in an array of disciplines, in particular Education, Health, and Sociology, have written pieces based on their own experiences and ideas. In addition, practitioners from Information Schools, university libraries, and repositories -i.e., those who assist scholars to make their work more open within ethical and legal limits -have offered their perspectives.
With regard to the content of this broader literature, there is work expressing generalized support for openness and sharing qualitative data in particular (e.g., Corti 2000) as well as work voicing caution or skepticism (e.g., Tsai et al. 2016). There are also pointed exchanges (e.g., Parry and Mauthner 2004, 2005and Bishop 2005. Overall, two key themes stand out. The first is ethics. More than a few scholars discuss ethical challenges to openness (e.g., Carusi andJarotka 2009 andChauvette et al. 2019). Yet ethics are discussed in multiple ways, and invoked by authors who are enthusiastic about data sharing and transparency. For instance, Stein (2010) questions the quest for anonymity; Lester and Anders (2018) explore the notions of ethics in practice and relational ethics; and Bishop (2009) advocates for a philosophical debate on the ethics of sharing that extends beyond the traditional focus on the rights of human subjects to include other actors such as research communities and the public. The second key theme concerns the re-use of qualitative data. Discussion of the practices, challenges, and benefits of secondary analysis are practically absent from the conversation in the U.S., but are topics of vibrant debate in Europe (see, e.g., Heaton 2008, Hammersley 2010, and Irwin and Winterton 2012.
Importantly, this broader literature also discusses an array of promising practices for pursuing and achieving openness in qualitative research. Our stance, and the central message in this piece, is that the best way to devise workable strategies for making qualitative research more transparent is for authors who engage in different forms of qualitative inquiry to try their hand at doing so. Learning from those experiences, scholars in different research communities can begin to identify productive practices for openness.

Openness in Practice
As noted in the introduction to this piece, in one conceptualization, openness comprises three components: data access, production transparency, and analytical transparency. Drawing on the literature on openness in qualitative research discussed in the previous section, as well as our experiences working with scholars pursuing openness, we present here a series of approaches that social scientists and others whose research draws on qualitative data are already using to achieve each component of openness. A key principle underlying our survey -which is merely suggestive, hopefully inspiring, and certainly not comprehensive -is that more openness, in any form, is better than less. Scholars who are hesitant to share their data have available to them an array of other strategies for making their research more open.

Data Access
Data access is the most controversial component of transparency in qualitative inquiry. Two areas of particular concern are sharing information conveyed by individuals whom scholars involve in their research, and sharing documents and other work that are under copyright. Sharing data from human participants research requires the consent of participants. However, scholars sometimes shy away from soliciting that consent, assuming that respondents' vulnerability or the dangerous nature of the research context imply that the information conveyed by respondents is too sensitive to be shared, or fearing that even the request to share data could prevent participants from consenting to be interviewed, or influence their responses, thus negatively affecting the quality of the research (see, e.g., Tripp 2018, 732).
In some situations, however, more agency could be given to the individuals whom scholars involve in their research. When soliciting informed consent, scholars could engage with participants about whether a subset of the information they conveyed could be shared in some form, and/or about the possibility of de-identifying conveyed information prior to sharing it. Contreras (2019) explores disclosure strategies in dangerous research, including semibiographical disclosure, partial spatial disclosure, and invitational disclosure. Alternatively or in addition, researchers and respondents can consider the option of limiting the number or type of people to whom any shared information is available (i.e., placing "access controls" on shared data). For instance, Camp and Dunning (2015) shared transcripts of interviews with political brokers in Argentina, describing the general region but not the specific location where data were collected, and restricted access to the data to researchers with clearly specified research plans.
With regard to copyright, an initial challenge is determining whether a document or other work a scholar may wish to share is under copyright or in the public domain. For example, Hitt (2019) shared papers that U.S. Supreme Court justices had dedicated to the public domain and can thus be freely shared. When work scholars wants to share is under copyright, they can petition the copyright owner to permit its sharing (see, e.g., newspaper articles shared in association with Holland [forthcoming]). Finally, excerpts of copyrighted work may be shareable under the "fair use" exemption outlined in the U.S. Copyright Act (section 107).

Production Transparency
Production transparency, i.e., describing how the data underpinning a study were collected or generated, is perhaps the aspect of transparency with which qualitative researchers are most familiar and comfortable. Per a QTD report, "Many contributors to the [QTD working groups] expressed support for... production transparency" (Shesterinina, Pollack, and Arriola, 2019, 5).
Indeed, in traditions such as ethnography and interpretive research, details about how data were collected and about their context are often considered key aspects of the research and included in the publication proper. Political scientist Cramer (2015) notes, "[T]ransparency in the sense of explaining in detail my data collection and analysis procedures, as well as my epistemological approach, has been a professional necessity for me." Sociologist Reyes (2018) describes "three models of transparency in ethnographic research," while education researchers Lester and Anders (2018) discuss how they pursued transparency ethically in their study of the experiences of Burundians with refugee status resettling in southern Appalachia.
While publications in other research areas have typically left many details of data collection unclear, scholars are developing strategies to address that gap. One approach involves developing methodological appendices. A particularly impressive example is the appendix provided by Shesterina (2016) describing her research in Abkhazia: she describes in detail how she organized her fieldwork, her interview strategy, how respondents were recruited and how she gained their trust, and the setting in which interviews were conducted. Bleich andPekkanen (2013, 2015) propose a more formalized "Interview Method Appendix" comprising a list of all interviews conducted with descriptive information such as the source of the interview contact, structure, and length. Other strategies, pioneered in education research, are generating reflective journals in which the steps of the research process are recorded in detail (Ortlipp 2008) and, with larger research teams, periodically carrying out debriefing interviews in which team members describe their decisions and actions throughout the research process (Collins et al. 2013). The sharing of such journals and interview transcripts greatly enhances transparency.
Production transparency is perhaps least successfully pursued in historically-oriented scholarship. Using primary historical sources without describing how they were originally produced or how they were selected by the researcher remains commonplace (Gaikward, Herrera, and Mickey, 2019, 2), as does social scientists' casual use (and misuse) of historiography in historical scholarship principally based on secondary sources (Møller and Skaaning 2018). Annotation for Transparent Inquiry (ATI, discussed below) offers scholars a vehicle to provide such detail, as do more traditional appendices to scholarly publications. Best practices in archival research typically include keeping a research log reflecting a full, ideally annotated, account of all sources consulted; 8 publishing such documentation in tandem with a publication would provide significant insight into the production of the historical data underlying an argument.

Analytic Transparency
Analytic transparency requires authors to "clearly explicate the links connecting data to conclusions" (APSA 2012). Qualitative researchers have begun to develop a broad range of strategies to achieve analytic transparency, reflecting the widely varying epistemological commitments that underpin, and methodological practices that animate, qualitative inquiry.
For instance, scholars who employ process tracing can generate online appendices to make analytic claims (and their role for the overall argument) more explicit (see, e.g., Fairfield 2013 as well as Bennett, Fairfield, and Soifer 2019). Scholars whose work relies heavily on qualitative coding can enhance transparency by clearly describing how they arrived at their initial coding, and how they resolved questions of inter-coder disagreements and refined that schema. Fuji Johnson (2017) provides a particularly detailed description of how she coded legislative discourse on sex work in Canada. Similarly, in their deliberative research on ex-combatants in Colombia, Jaramillo et al. (2017) discuss procedures for being transparent about coding decisions and provide detailed justifications of every coding decision on a website. Deterding and Waters (2018) describe the flexible approach they used to analyze a large qualitative dataset, subsequently shared, that begins with broad, per-case memos and then narrows down to analytic codes and model building.
There has also been some recent interest in preregistration for qualitative work (see, e.g., Piñeiro and Rosenblatt 2016;Kern and Gleditsch 2017;Haven and Grotel 2019;Jacobs forthcoming). 9 There are good reasons to be skeptical of the need and usefulness of preregistration in qualitative research given its often exploratory nature (cf. Haven and Grootel 2019, 6-8). Yet having a timestamped record of the original research and analysis plan and of the changes made throughout the research process enhances transparency. 10 This is particularly true when qualitative research seeks to advance causal claims. A pioneering example of a preregistered qualitative case study is presented in Christensen, Hartman, and Samii (2019), who use case study evidence to validate (and extend) quantitative findings.
In describing their methodology, researchers, in particularly in health-related areas, increasingly rely on reporting guidelines or checklists that set thresholds for information provision about data collection and analysis. The reporting guidelines most familiar to political scientists are those on public opinion polls by the American Association of Public Opinion Research (AAPOR 2017). Yet similar guidelines exist for in-depth interviews and focus groups (COREQ; Tong, Sainsbury, and Craig 2007); for synthesizing qualitative research (ENTREQ; Tong et al. 2012), and for qualitative research in general (SRQR;O'Brien et al. 2014). These guidelines will likely require some adaption for use in political science given the heterogeneity of qualitative research in our discipline, but they strike us as potentially fruitful tools for transparency.

Integrated Approaches
Some approaches can help authors to achieve all three components of transparency. For instance, Bringer, Johnston, and Brackenridge (2004) highlight how scholars can use qualitative data analysis (QDA) software, in particular the memo/note function, to provide an "electronic audit trail" of the research process and the development of a project. 11 Similarly, Sinkovics and Alfonsi (2012) argue, based on their work in international business and management, that QDA software can help capture the non-linear back-and-forth between data collection and analysis that is characteristic of much qualitative work, thereby improving its transparency and trustworthiness.
Yet QDA software can also provide a coherent image of the data as a whole. Corti and Gregory (2011) have long advocated for the sharing of QDA-generated qualitative data, and some researchers have shared excerpts from their QDA projects (see e.g. Luigjes 2019; O'Neill 2017). While the proprietary nature of QDA file formats had stymied such efforts, an open exchange format for QDA data supported by the major software projects has recently emerged (www.qdasoftware.org). Today, then, researchers can simultaneously satisfy all three areas of transparency by sharing QDA data.
Annotation for Transparent Inquiry (ATI; www.qdr.org/ati), a new approach to transparency in qualitative research, likewise facilitates achieving all three aspects of transparency. Building on Moravcsik's (2014) work on active citation, ATI allows researchers to link specific passages in a publication to digital annotations comprising "analytic notes" and extended excerpts from data sources, as well as to the data sources themselves (e.g., interview transcripts or digital images) (see Karcher and Weber forthcoming). Analytic notes can perform multiple functions, such as elucidating data generation or analysis, making the link between a source and a claim explicit, or discussing other aspects of the research process. When sharing underlying data sources is not possible, extended excerpts allow for meaningful data sharing (see Ellett 2016 and the discussion of this project in Shesterinina, Pollack, and Arriola 2019, 23).

Conclusion: The Next Frontier
In this brief piece we have sought to make several points. First, debates about the challenges and benefits of openness and how to pursue it are proceeding across academic disciplines and geographies. We hope that qualitative political scientists turn to this work for ideas and inspiration. Second, openness is a continuum: most work is neither completely opaque nor completely transparent, but somewhere in between, with a plethora of factors affecting where an author places her work on that spectrum. In our view, any step towards transparency that is taken ethically and legally benefits research and research communities. Finally, the way forward will be shown by individual scholars making their work more transparent, generating examples from which research communities can learn and on which they can draw to develop norms and practices.
Of course, openness is a means to an end, not an end in itself. Openness adds value by facilitating comprehension and assessment of our scholarship. The goal and necessity of assessment are in no way new: our research is assessed informally every day by individual scholars and more formally periodically through the peer review process. But increasing openness facilitates new ways to evaluate qualitative inquiry. What should new forms of evaluation look like? How can they be developed in ways that are appropriate for the epistemological commitments and methodological practices that make qualitative research the immensely powerful form of inquiry that it is?
Ultimately the large and heterogeneous community of qualitative researchers will develop the best answers to these questions, and to all of the questions raised in this piece and the broader symposium, by engaging with each other within and across different types of qualitative scholarship. By working collaboratively, and listening to and learning from each other, we will develop and employ openness to its best purpose: to demonstrate the rigor of qualitative research and the value it delivers to political science. We hope this piece represents a step in that direction.