ManageChatDiscover
422937

ChatGPT's Potential in Navigating the Complexity of the Polish Anaesthesiology Specialist Examination

Article

Last updated: 27 Apr 2025

OverviewSimilar Items

Subjects

-

Tags

-

Abstract

Purpose: This study aims to assess the capability of an artificial intelligence (AI) model, specifically ChatGPT-3.5, in answering questions from the test section of the Polish National Specialist Examination (PES) in anaesthesiology and intensive care.
Materials and Methods: A pool of 118 questions from the spring 2023 PES exam was utilized. Bloom's classification was employed to categorize questions based on comprehension, critical thinking, and memory. The questions were then presented to ChatGPT-3.5 in five independent sessions to evaluate its performance. Statistical analyses were conducted to assess correlations between the model's confidence, question difficulty, and correctness of answers.
Results: ChatGPT-3.5 achieved an overall accuracy of 47.5%, with variations observed across different question types and subtypes. Significant correlations were found between the model's confidence and answer correctness. However, no correlation was observed between the certainty index and question difficulty or answer correctness based on category or subcategory.
Conclusions: While ChatGPT-3.5 exhibited moderate performance, it fell short of the 60% threshold required to pass the PES exam. Comparison with similar AI studies in Japan suggests superior performance by the Polish AI model, albeit with limitations in expertise level. Human candidates consistently outperformed the AI model, indicating the current superiority of human expertise in this domain. Despite current limitations, continued research and collaboration offer promising prospects for AI integration in medical practice, supporting diagnostics, therapeutics, and patient care.

DOI

10.21608/asja.2024.290716.1111

Keywords

Anaesthesiology, artificial intelligence, ChatGPT, Intensive care, Medical education, specialty examinations

Authors

First Name

Michał

Last Name

Bielowka

MiddleName

-

Affiliation

Student Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice

Email

michalbielowka01@gmail.com

City

-

Orcid

0009-0007-0609-1678

First Name

Jakub

Last Name

Kufel

MiddleName

-

Affiliation

Department of Radiodiagnostics, Interventional Radiology and Nuclear Medicine

Email

jakubkufel92@gmail.com

City

-

Orcid

0000-0001-7633-3600

First Name

Marcin

Last Name

Rojek

MiddleName

-

Affiliation

Student Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice

Email

marcinarojek@gmail.com

City

-

Orcid

0009-0005-0807-5423

First Name

Adam

Last Name

Mitręga

MiddleName

-

Affiliation

Student Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice

Email

adam.mitrega2306@gmail.com

City

-

Orcid

0009-0004-3383-6477

First Name

Dominika

Last Name

Kaczynska

MiddleName

-

Affiliation

Student Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice

Email

dominikakaczynska01@gmail.com

City

-

Orcid

0009-0004-9448-7856

First Name

Łukasz

Last Name

Czogalik

MiddleName

-

Affiliation

Student Scientific Association of Computer Analysis and Artificial Intelligence at the Department of Radiology and Nuclear Medicine of the Medical University of Silesia in Katowice

Email

lukczog@gmail.com

City

-

Orcid

0000-0003-3886-9708

First Name

Dominika

Last Name

Kondoł

MiddleName

-

Affiliation

Dr B. Hager Memorial Multi-specialty District Hospital, Pyskowicka 47-51, 42-600 Tarnowskie Góry, Poland

Email

domika1313@gmail.com

City

-

Orcid

0009-0009-8150-9426

First Name

Kacper

Last Name

Palkij

MiddleName

-

Affiliation

Dr B. Hager Memorial Multi-specialty District Hospital, Pyskowicka 47-51, 42-600 Tarnowskie Góry, Poland

Email

palkij.kacper@gmail.com

City

-

Orcid

0009-0004-3738-3161

First Name

Wiktoria

Last Name

Bartnikowska

MiddleName

-

Affiliation

Faculty of Medical Sciences in Katowice, Medical University of Silesia, 40-752 Katowice, Poland

Email

wiktoriabart96@gmail.com

City

-

Orcid

0000-0002-9303-946X

Volume

17

Article Issue

1

Related Issue

52828

Issue Date

2025-01-01

Receive Date

2024-05-18

Publish Date

2025-01-01

Page Start

1

Page End

4

Print ISSN

1687-7934

Online ISSN

2090-925X

Article File

ASJA_Volume 17_Issue 1_Pages 1-4.pdf

PDF . 387.2kB

Link

https://asja.journals.ekb.eg/article_422937.html

Detail API

http://journals.ekb.eg?_action=service&article_code=422937

Order

422,937

Type

Original Article

Type Code

2,871

Publication Type

Journal

Publication Title

Ain-Shams Journal of Anesthesiology

Publication Link

https://asja.journals.ekb.eg/

MainTitle

ChatGPT's Potential in Navigating the Complexity of the Polish Anaesthesiology Specialist Examination

Details

Type

Article

Created At

27 Apr 2025