![Author ORCID: We display the ORCID iD icon alongside authors names on our website to acknowledge that the ORCiD has been authenticated when entered by the user. To view the users ORCiD record click the icon. [opens in a new tab]](https://preprints.apsanet.org/engage/assets/public/apsa/logo/orcid.png){kind=logo}
Abstract
A key limitation in debates about democracy measurement and whether democratic backsliding is occurring is the mismatch between theoretical claims — about specific institutional actions and processes — and available data. Political events are the appropriate unit of analysis: they denote discrete actions by identifiable actors with consequences for democratic institutions, but are difficult to produce at scale. This paper presents an LLM-based pipeline that transforms Freedom House's annual country reports (1990–2024) into structured event data. Our four-stage process produces nearly 200,000 annotated events across 228 countries, each tied to textual evidence. Validation against human coders and existing datasets demonstrates high construct validity. We apply the data to compare democratic backsliding in Hungary and Poland. The primary contribution is auditable interpretation as a standard for LLM-assisted measurement — producing outputs traceable to verbatim source material. Such event-level data enables analyses of institutional sequencing where indices and case studies fall short.
Supplementary materials
Title
Supplementary Materials for "From Text to Events"
Description
This supplementary document accompanies "From Text to Events: Turning Freedom House Reports into Evidence of Democratization and Democratic Backsliding" (Wilson, Martin-Morales, and Nelson, 2026). It provides methodological detail supporting the paper's core pipeline, which uses large language models to extract structured political events from Freedom House country reports spanning 1990–2024. The appendices include diagnostic figures on sentence and highlight extraction rates, inter-coder reliability comparisons between human coders and GPT, stability tests across repeated model runs, and validation results for eight case-study countries. Additional appendices document the complete prompt templates used for highlight selection, event extraction, thematic tagging, and democratic sentiment classification, along with configuration parameters for each pipeline stage. A detailed de-duplication procedure is described, and country-year coverage and formatting changes in Freedom House reports over time are documented. Replication files and the final annotated event dataset will be made available for online access.
Actions