Data Extraction
The corpus was segmented into data extracts (N = 1790) through a four-stage workflow: 1) initial heuristic rule drafting (Researcher 2); 2) LLM-assisted linguistic refinement of extraction rules (Gemini); 3) expert review and ratification (Researcher 1); and 4) automated paragraph-level and intra-paragraph segmentation via LLM (Gemini). Segmentation was strictly governed by explicit rhetorical shift rules, splitting paragraphs whenever arguments transitioned or signpost markers ("Yet," "Therefore") occurred. Each extract was constrained to a 15-45 word unit containing a clear subject, action, and immediate justification. Paragraphs continuously reinforcing a single claim were retained as intact single extracts.
Codebook Development
The codebook was developed within a hybrid deductive-inductive process.
An initial codebook draft created by Researcher 1 was expanded with an inductive, bottom-up subcode generation using Gemini with human supervision (Researcher 2) to draft a modified codebook following pilot coding.
It was then followed by a multi-turn deliberative refinement between the human researcher (Researcher 1) and Claude, such as establishing explicit boundary rules (e.g., D3 vs. D4, M2 vs. M3), changing subcodes and refining definitions until consensus was achieved. Researcher 1 then reviewed the corpus and generated 10 initial coding exemplars per tradition to construct a gold-standard reference dataset to ground the model.
To map the religious arguments directly onto policy mechanisms, a final post-hoc codebook refinement was conducted by Researcher 2. An LLM (Gemini) was utilized to analyze the extracted data and establish standardized policy codes (as well as the granular subcodes) and stakeholder categories in a fully inductive workflow, aligning closely with the original pipeline of Dai et al. (2023), but it did not undergo human-LLM deliberative refinement, as neither researcher had specialized domain expertise in AI policy.
For a summary of the working notes of the deliberative process, please refer to Appendix D.
Hybrid Coding and Verification Pipeline
Data annotation was conducted through a hybrid human-LLM pipeline combining initial automated LLM coding with extensive, multi-round manual verification and audit passes by human researchers.
Automated coding was subsequently executed using an LLM coder prompted with both the deliberation JSON of Researcher 1 and Claude to establish context as well as the frozen codebook. Extracts were processed in batches of 10, with fresh model instances initialized every 50 extracts to prevent context-window degradation or attention drift.
Then, multiple rounds of complete manual verification passes were conducted by Researcher 1 and 2 across every data extract to eliminate hallucinations, formatting errors, or classification misalignments. This iterative review included a complete manual recoding of Gradual Disempowerment (GD) by Researcher 1 following a systematic misalignment between the model and initial human assumptions as well as a targeted manual recoding of M3 Redlines by Researcher 2 to resolve coding disagreements. Specifically, during the manual recoding of GD, extracts were graded on a four-tier interpretive levels by Leech which led to further refinement of the codebook.
For policy codes, to further increase validity and mitigate errors stemming from a lack of specialized domain expertise, Researcher 2 deployed two independent instances of Gemini as automated red-team auditors in a sequential pipeline (i.e., Auditor 1 reviewed and modified the output of the LLM Coder, after which Auditor 2 reviewed and refined the output of Auditor 1).