Catalog Methodology in Depth

How do we choose studies and reports for the ARTT Catalog?

The ARTT Catalog considers different types of engagement around information across disciplines, including:

  • Psychology
  • Media literacy
  • Conflict resolution / transformation
  • Science communication

For the purposes of online conversation, the Catalog distills this interdisciplinary research into different response modes. These modes may overlap, interact, or counteract each other.

Furthermore, learnings from one context may not directly translate to another. Thus, the Catalog also surfaces existing gaps in understanding related to these response modes to create opportunities for future explorations across disciplines.

Each paper and practical report selected for inclusion in the Catalog describes or evaluates a method to engage in polarized online discourse while increasing acceptance and trust in expert knowledge.

The Catalog selection process includes: 

  1. Identifying papers describing interventions around online discourse.
    We sought diversity based on authorship and chose papers that evaluated a unique intervention or presented more recent evidence on a contested one.
  2. Collecting feedback on the structure of the Catalog.
    Three focus group discussions with area experts were part of this feedback collection process.
  3. Mapping connections between the different papers.
    The ARTT team used the Cita Wikidata Zotero add-on to identify clusters of linked literature, and map the relative weightage of representation of specific areas.

The Catalog is a “living collection,” in that the tagging system reflects the themes and methods identified from the body of research at a specific time. As a result, the tagging system and the Catalog content is also expected to evolve over time, as new research and new resources are added to the collection. As a note, the ARTT Catalog is a resource that brings together research from different disciplines; it does not endorse any specific study or report.

How to understand ‘Notes’ and ‘Tags’

A key feature of the ARTT Catalog is the notes and tagging system:

  • Notes explain why a paper or practitioner report was included in the catalog and how it relates to the overall aims of the catalog.
  • Tags allow catalog users to explore the research by searching and filtering through the literature. 
About Notes

The notes section explains why the study or report has been included in the catalog and how it corresponds to one or more of the ARTT tags. This information can be expanded by clicking on the notes tab in the right hand section of the screen.

Image of the expanded ‘Notes’ section from the paper “Analysis Versus Production: Adolescent Cognitive and Attitudinal Responses to Antismoking Interventions” by Banerjee and Greene.
About Tags

Once you have narrowed down your search for relevant papers and articles, the tag section is intended to be a useful feature that provides a quick overview of each paper.

A closer look at the tag section of each paper or other Catalog resource can show you information about the methods used, the outcomes tested, the theories guiding the intervention, as well as the conditions under which the study was conducted, such as how many people participated in the study, or where the study took place, and on what social media platform, how long the study was conducted, and so on. 

All the papers in the Catalog are described by one or more high level ARTT Conversational Response / Intervention tags, but additional tags can vary.

The following image explains what each tag represents and how they can provide a quick view of each paper and report in the Catalog: 

How to interpret tags appearing in the Catalog’s ‘tag’ section

The following table provides a high-level overview of the Catalog tags, and their definitions. These tags and definitions represent the body of research at the specific time; some of the tags may change as the research evolves. For more information about each tag, please refer to the Response Definitions page.

ARTT: The tag describes, at a high level, the response or intervention analyzed within the resource. You can think of an ARTT tag as something completing the following sentence: “In this conversation, I could…”

ARTT tags: Correct, De-escalate, Encourage Healthy Skepticism, Encourage norms, Empathize, Take Perspective, Listen
Theory: The theory tag describes the theory informing the method or intervention. In some studies the theory might be explicitly stated but in others it may be inferred.

Theory tags: Inoculation, Continued Influence Effect, Backfire Effect, Media Literacy, Nudge
Type: Tag for the type of resource

Type tags: Study, Meta, Toolkit, Policy
Method: The method tag describes how an intervention is tested. There can be a number of ways in which an intervention is tested.

Method tags: Post-exposure correction, Preemptive Refutation, Causal Correction, Content Production, Warning of Threat, Lateral Reading, Source Evaluation, Humor
Sample: The sample tags provide context on the research setting.

Sample tags: Size, Pop, Geography, Setting
Outcome: The outcome tags describe the outcomes a study is measuring. For some papers, the effect of the methods on the outcome would also be described through a sign next to the outcome. ‘+’ describes a positive effect, ‘-’ describes a negative effect and ‘~’ describes a neutral/non-significant effect. 

Outcomes tags include: Memory, Attitude, Action, Media Knowledge, Understanding of Others