A Study of Equivalence in Methods of Using AI-Generated Drafts in Bible Translation

The rapid development of Artificial Intelligence (AI) is impacting many disciplines, including Bible translation (BT). For any emerging technology, it is crucial to evaluate its appropriate use, seeking maximum benefit while mitigating unintended adverse effects. Ethical adoption of technology requires consideration not only of efficiency, but also all potential impacts on the beneficiaries. This research explores the use of AI in Bible translation with the goal of providing insights into how AI-generated drafts can best be used.

Scripture Forge, a companion tool for Paratext projects, includes a feature that can produce

AI-generated drafts. This tool is particularly well suited to projects in which a significant portion of the Bible has already been translated and used as training data. Prescience Labs partnered with the developers of Scripture Forge to study different methods of utilizing AI-generated drafts in BT.

Experts with experience in using AI-generated drafts were interviewed to gain insights prior to implementing field research. Three methods of producing first drafts of translations were then compared in a field study – Manual method without use of AI technology; Direct Edit method where an AI-generated draft was directly edited by the Mother Tongue Translators (MTTs); and Reference method in which the MTTs used the AI-generated draft as a reference.

Using a mixed methods research approach, data was collected on the time spent and quality of the draft. Additionally, an internalization progression tool (a set of questions ranging from basic recall and comprehension to interpretation and application) was developed to measure how well the translators could demonstrate comprehension and internalization of the passage after translating. Results of this research are presented along with insights regarding the use of AI-generated drafts in Bible translation.