Beta
384369

Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths

Article

Last updated: 25 Dec 2024

Subjects

-

Tags

-

Abstract

The surge of advanced neural machine translation (NMT) tools like Google Translate and large language models (LLM) like ChatGPT and Gemini has fueled their application across diverse religious texts, like prophetic hadiths. However, the effectiveness of these tools, particularly LLMs, in the domain of religious translation and their understanding of classical Arabic (CA) remains largely unexplored. This study bridges this gap by comprehensively assessing the machine translation (MT) accuracy of these three systems in translating 25 succinct, two-word prophetic hadiths from Arabic into English. These two-word hadiths, categorized as Jawāmiʿ al-kalim (comprehensive words), encapsulate profound meanings despite their brevity. This study delves deeper by evaluating the ability of Google Translate, ChatGPT (GPT-3.5), and Gemini to capture the essence of these two-word hadiths, considering their unique linguistic features and cultural context. Through meticulous evaluation by Subject Matter Experts (SMEs)/translators, the study reveals recurring challenges in MT, including word sense disambiguation, handling out-of-vocabulary terms, part-of-speech tagging, morphological segmentation, and comprehending the contextual, cultural, lexico-semantic, and pragmatic dimensions. These limitations were particularly evident with GT, whose reliance on literal translation often resulted in inaccurate renderings in the target language. Despite these challenges, the potential of LLMs for religious translation remains promising, especially with AI tools like Gemini showing promise. Their ability to learn and adapt finds that with further development, including domain-specific fine-tuning, incorporation of diacritics and contextual information via accurate prompts, development of religious text-specific evaluation metrics, and human post editing, MT systems can significantly be improved. It also underscores the vital role of human expertise while paving the way for future research to develop more accurate and culturally sensitive MT systems for religious communication across languages and faiths. The study concludes that LLMs displayed promise in capturing the essence of certain hadiths.

DOI

10.21608/opde.2024.384369

Keywords

Natural Language Processing (NLP), Neural Machine Translation (NMT), large language models (LLM), Classical Arabic (CA), Google Translate (GT), ChatGPT, Gemini, Prophetic hadith

Authors

First Name

Muhammad A. A. ,

Last Name

Taghian

MiddleName

-

Affiliation

Lecturer of Applied Linguistics/Translation, Faculty of Arts, Helwan University, Egypt.

Email

-

City

-

Orcid

0000-0003-3078-0645

Volume

87

Article Issue

1

Related Issue

50844

Issue Date

2024-07-01

Receive Date

2024-10-06

Publish Date

2024-07-01

Page Start

217

Page End

262

Print ISSN

1110-2721

Online ISSN

2735-3591

Link

https://opde.journals.ekb.eg/article_384369.html

Detail API

https://opde.journals.ekb.eg/service?article_code=384369

Order

7

Type

Original Article

Type Code

1,140

Publication Type

Journal

Publication Title

CDELT Occasional Papers in the Development of English Education

Publication Link

https://opde.journals.ekb.eg/

MainTitle

Assessing the Accuracy Criteria of AI Tools-aided Translation: A Case Study of Two-word Prophetic Hadiths

Details

Type

Article

Created At

25 Dec 2024