The exact content of your appendix depends on your research project and the types of data collection in which you engaged. They key is for it to help readers of your published work to understand to the greatest degree possible how your data collection processes produced the data that underpin your work, thus helping them to assess the quality of the data and how well your claims are supported.
Creating a Data Appendix
- Draft the framework for a data appendix for a research product you are currently creating. What set of attributes would work across all your different forms of data (documents, interviews transcripts / notes, etc.) and help your readers understand and evaluate your project’s evidentiary foundation?
- show solution
- Give your draft framework, and the abstract for the research product you are writing, to someone unfamiliar with the piece. Solicit their input on whether the information you are proposing to provide is the type they would want to know when evaluating your work.
Qualitative Data Analysis Software and Transparency
The use of software to assist qualitative data analysis (sometimes referred to as CAQDAS or “Computer Assisted Qualitative Data Analysis Software”) is becoming more and more common across the social sciences. Such software assists researchers with routine tasks such as coding, categorizing, and annotating documents. Different from statistical packages (and as including “assist” in the name suggests), the analysis itself does not take place in the software: you, not an algorithm, make key analytic choices such as which code to assign to a statement. As a result, simply sharing some code or output does not satisfy requirements for analytic transparency. Nevertheless, the software can help you to make your work and your data transparent.
We offer here some suggestions on achieving transparency when working with CAQDAS software
1. Follow General Advice on Data Management
Most advice for managing qualitative data in order to facilitate their subsequent sharing is applicable to CAQDAS data: have a clear organizational structure, document during data collection, etc.
2. Keep Track of Sensitive Information
As you collect your data, keep concerns about privacy and sensitivity in mind. As you see information in your data that may need redacting, use the software to highlight it, so you can quickly identify it later on. Also consider tagging files that you specifically cannot share (e.g., interviews given “off the record” or signed consent forms).
3. Keep Memos about Analytic Decisions
As you analyze your data, your CAQDAS tool will help you make your analytic process transparent. Making coding and analysis decisions explicit in memos will help readers to evaluate your conclusions, and also help secondary users to better understand the application of given codes in your data.
4. Preparing Your Data for Sharing
Make a copy of your project and delete any information you do not want to share, such as private notes or sensitive information. If you have followed our advice above, you can now use the tags you have created to redact potentially identifying information from transcripts following the guidelines we provided previously.
5. Exporting Your Data for Sharing
One of the challenges of sharing CAQDAS-produced data is that every software product has its own, typically proprietary, export format. These formats do not travel between software, may change between software versions, and are thus problematic for sharing and archiving.
One solution is to share data in two different forms. The first form is the raw full export from your software. Once your data are prepared for sharing, first export the whole project into your software’s dedicated export format (e.g., .nvp for NVivo, “Export Data” for Dedoose, or “copy bundle” for atlas.ti). Then, create a second “human-readable” export that anyone, regardless of software, can use: Export all relevant files in widely used formats (such as RTF, PDF, Excel, as well as widely used video, image, and audio formats). Also export all relevant memos as RTF or PDF files.
As of this writing, efforts are underway to provide a standardized format for exchange between different CAQDAS software products. As this exchange format matures and becomes more widely available, we expect it to replace some of these recommendations.
- American Political Science Association. 2012. A Guide to Professional Ethics in Political Science, http://www.apsanet.org/portals/54/Files/Publications/APSAEthicsGuide2012.pdf
- Bleich, Erik, and Pekkanen, Robert. 2015. “Data Access, Research Transparency, and Interviews; the Interview Methods Appendix.” Qualitative & Multi-method Research 13 (1): 8–13. https://doi.org/10.5281/zenodo.892386
- Elman, Colin, Diana Kapiszewski, Andrew Moravcsik, and Sebastian Karcher. 2017. A Guide to Annotation for Transparent Inquiry. https://qdr.syr.edu/ati/guide-to-ati