Research Transparency and Qualitative Data

Go to the full lesson

Exercise

ATI (1) -- Considering Another Author’s Annotations

  1. Find this article on-line and then follow the steps below: O’Mahoney, Joseph. 2017. “Making the Real: Rhetorical Adduction and the Bangladesh Liberation War.” International Organization 71 (2): 317–48. https://doi.org/https://doi.org/10.1017/S0020818317000054
    1. Click on the Adobe Acrobat icon to download the PDF.
    2. Read the abstract.
    3. Scroll to a section called, “The Role of Troop Withdrawal” on p. 332 and read the first two paragraphs of that section.
    4. Given what you have learned about ATI, which passages of those paragraphs would you expect to see annotated and with what type of content? Use your PDF viewer’s annotations functionality to highlight them.
    5. Now view the same article with ATI annotations here. Find the same two paragraphs and compare your expected annotations with those of the article’s author, Joey O’Mahoney.
      1. How do the author’s annotations differ from your expectations?
      2. Why do you think they differ?
      3. How do the annotations affect your assessment of the underlying claims?
  • show solution
    1. It is unlikely that your expectations and the author’s annotations matched exactly. This reflects a challenge in achieving qualitative transparency – there is an almost unlimited number of things about which you can add information and be more transparent in order to better convince readers of your claims. One way to “triage” is to develop explicit rules for yourself about the types of passages or claims you annotate. You can see O’Mahoney’s discussion of his “logic of annotation” here. Perhaps you also found that the annotations raised as many questions as they answered, e.g., why did the author annotate this claim but not that one? Making your work more vulnerable to criticism by exposing more of its internal workings is a very real risk of working more transparently. Yet it is precisely this susceptibility, which facilitates correction and learning, that allows social science to advance.