Methodology

Design

We adopted a sequential qualitative-to-quantitative mixed-methods design. The reason for the choice is that while purely qualitative approaches provide interpretive context, they are less suited for evaluating structural frequency patterns or proportional distribution across extensive textual datasets.

Sampling of Documents

The final analytical corpus comprises 15 primary texts selected across five major Abrahamic traditions (4 Catholic, 4 Protestant, 3 Islamic, 2 Jewish, and 2 Eastern Orthodox) resulting in 1790 extracts. Four documents originating from Buddhist and Hindu traditions were excluded, as it proved methodologically challenging to construct a unified codebook that could meaningfully bridge Abrahamic and non-Abrahamic theological structures alongside the lack of comparable centralized, top-tier institutional statements. To map the institutional standing of each source, the dataset includes descriptive metadata capturing tradition-relative authority tiers (3 levels) alongside discursive genre tags.

Detailed source mappings are catalogued in Appendix A.

Variables

The full description of the definition of each code and sub-codes as well as why they were selected is included in Appendix B. The process of generating and refining the codes can be found in Appendix C.

Researcher Backgrounds

Task allocation between the human researchers was strictly structured around their respective domain expertise and methodological backgrounds:

  • David Leech (Researcher 1): Honarary Research Associate in the Department of Religion and Theology at the University of Bristol

  • Alexandra Cheung (Researcher 2): PhD student in Psychology at the University of Bristol

To maintain full transparency, the researchers are referenced throughout the following methodology section to indicate specific responsibility for each task.

Hybrid LLM and Human Pipeline

The codebook was developed by adapting the deliberative human-LLM collaboration framework proposed by Dai et al. (2023) within a hybrid deductive-inductive process, following a three-stream, hybrid human-in-the-loop pipeline that integrated theoretical derivation with multi-stage automated coding and human auditing.

The workflow began with a methodological setup phase, where initial deductive frameworks (Researcher 1) were expanded through inductive pilot testing on approximately 100 extracts (Researcher 2 using Gemini).

This codebook underwent multi-turn deliberative refinement between Researcher 1 and Claude to establish explicit boundary definitions, culminating in a gold-standard reference dataset of 50 human-authored exemplars (10 per tradition) to ground the model before full execution.

During primary data extraction (N = 1790), automated LLM extraction was conducted under human oversight (Researcher 2) with misaligned extracts returned to the model for re-segmentation.

For the automated coding, human verification branched according to disagreement severity: mild misalignments triggered automated re-coding prompts, whereas strong misalignments (such as Gradual Disempowerment and M3 Redlines) prompted manual recoding by Researcher 1 and Researcher 2.

Finally, to handle post-hoc policy and stakeholder code generation without dedicated policy domain expertise, the pipeline deployed a sequential multi-agent audit framework, passing initial automated outputs through two independent, chained Gemini red-team instances to iteratively review and refine final categories, pending expert consultation.

For a full breakdown of the details of data extraction, codebook creation and refinement and full coding process, please refer to Appendix C.