Beta
135871

*Improving Natural Language Queries Search and Retrieval through Semantic Image Annotation Understanding

Article

Last updated: 22 Jan 2023

Subjects

-

Tags

-

Abstract

Retrieving images using detailed natural language queries remains a difficult challenge. Traditional annotation-based image retrieval systems using word matching techniques cannot efficiently support such query types. Significant improvements for this problem can be achieved with a semantic understanding for those query sentences and image annotations. This paper presents a two-stage semantic understanding approach for natural language query sentences. At the first stage, the Stanford parser and a designed rule-based relation extraction tool are used in triple extraction process to efficiently extract the objects attributes, instances and natural language annotations relationships involving these objects. The second stage integrates the extracted relations with external commonsense knowledge source in a mapping process to provide high-level semantic meanings to the extracted triples. Experiments are conducted for evaluating the benefit of the proposed semantic understanding against a testing set of natural language sentences from the Flickr8k dataset. The results show that the proposed approach succeeds to extract relational triples with average accuracy value of 97% for the different types of annotations relationships: attributes and instance relations, multiword dependence relations, and semantic relations.

DOI

10.21608/ijicis.2021.50161.1041

Keywords

Image retrieval, natural language queries, semantic understanding, commonsense knowledge sources

Authors

First Name

Haitham

Last Name

Samih

MiddleName

-

Affiliation

Faculty of Computer and Information Technology, Egyptian E-Learning University, Cairo, Egypt

Email

hsamih@eelu.edu.eg

City

-

Orcid

-

First Name

Sherine

Last Name

Rady

MiddleName

-

Affiliation

Ain Shams University

Email

srady@cis.asu.edu.eg

City

Cairo

Orcid

0000-0003-4991-966X

First Name

Manal

Last Name

Ismail

MiddleName

-

Affiliation

Computer Engineering and Systems Department, Faculty of Engineering, Helwan University, Cairo, Egypt

Email

manal_shoman@yahoo.com

City

-

Orcid

-

First Name

Tarek

Last Name

Gharib

MiddleName

-

Affiliation

Information Systems Department, Faculty of Computer and Information Sciences, Ain Shams University, Cairo, Egypt

Email

tfgharib@cis.asu.edu.eg

City

-

Orcid

0000-0003-0780-782X

Volume

20

Article Issue

2

Related Issue

19789

Issue Date

2020-12-01

Receive Date

2020-11-17

Publish Date

2021-01-05

Page Start

67

Page End

78

Print ISSN

1687-109X

Online ISSN

2535-1710

Link

https://ijicis.journals.ekb.eg/article_135871.html

Detail API

https://ijicis.journals.ekb.eg/service?article_code=135871

Order

12

Type

Original Article

Type Code

494

Publication Type

Journal

Publication Title

International Journal of Intelligent Computing and Information Sciences

Publication Link

https://ijicis.journals.ekb.eg/

MainTitle

-

Details

Type

Article

Created At

22 Jan 2023