Beta
266291

Real-Time Facial Expression Recognition and Speech Tran-scripts over an on-premise Video Conference Application

Article

Last updated: 29 Dec 2024

Subjects

-

Tags

-

Abstract

Since Covid-19 pandemic outbreak, organizations and individuals have had to use vid-eo conference applications increasingly. However, the commercial video conference applications are expensive, and feature limited. This paper discusses how to enable organizations to host on-premise video conference applications. Then, it explores assisting organization's stakeholders with making decisions based on facial expressions of video conference attendees. Moreover, it facili-tates transcribing speech into text to enable deaf persons to participate in online conferences. Technologies and tools used in addressing these challenges respectively are: (i) Web Real Time Communication (WebRTC) project, (ii) Tensorflow.js library, (iii) and Web Speech Application Programming Interface (API). This paper depends on integration between a collection of technol-ogies, libraries, standards, and protocols. Most of them can be managed using JavaScript frame-work. Hence, load of the performance is distributed on each client-side device. The proposed on-premise video conference application has been enhanced through including facial expression recognition with 66% high accuracy while the speech-into-text feature with Word Error Rates (WER) are 0 and 0.12 for British English and Egyptian Arabic, respectively

DOI

10.21608/ijt.2022.266291

Keywords

WebRTC, Video conferencing, Facial Expression Recognition, Speech Recog-nition, Computer Vision, ML, TensorFlow.js, OpenVidu, Speech-to-Text

Authors

First Name

S

Last Name

Eltenahy

MiddleName

-

Affiliation

Mansoura University Electronics and Communication Engineering Department, Faculty of Engi-neering, Mansoura University

Email

sallyahmed2011@gmail.com

City

Mansoura

Orcid

-

First Name

Nihall

Last Name

Areed

MiddleName

-

Affiliation

Mansoura University Electronics and Communication Engineering Department, Faculty of Engi-neering, Mansoura University

Email

nahoolaf@mans.edu.eg

City

-

Orcid

-

First Name

Marwa

Last Name

obayya

MiddleName

-

Affiliation

Mansoura University Electronics and Communication Engineering Department, Faculty of Engi-neering, Mansoura University

Email

omnya@mans.edu.eg

City

-

Orcid

-

First Name

Fahmi

Last Name

Khalifa

MiddleName

-

Affiliation

Mansoura University Electronics and Communication Engineering Department, Faculty of Engi-neering, Mansoura University

Email

fahmikhalifa@mans.edu.eg

City

-

Orcid

-

Volume

02

Article Issue

02

Related Issue

37273

Issue Date

2022-12-01

Receive Date

2022-04-26

Publish Date

2022-08-21

Page Start

1

Page End

14

Online ISSN

2805-3044

Link

https://ijt.journals.ekb.eg/article_266291.html

Detail API

https://ijt.journals.ekb.eg/service?article_code=266291

Order

266,291

Type

Original Article

Type Code

2,522

Publication Type

Journal

Publication Title

International Journal of Telecommunications

Publication Link

https://ijt.journals.ekb.eg/

MainTitle

Real-Time Facial Expression Recognition and Speech Tran-scripts over an on-premise Video Conference Application

Details

Type

Article

Created At

23 Jan 2023