Beta
73486

Survey of Apache spark optimized job scheduling in big data

Article

Last updated: 25 Dec 2024

Subjects

-

Tags

-

Abstract

Big data have acquired big attention in recent years. As big data makes its way into companies and business so there are some challenges in big data analytics.  Apache spark framework becomes very popular for using in distributed data processing. Spark is an analytic machine for big data processing with various modules for SQL, streaming, graph processing and machine learning. Different scheduling algorithms vary with its behavior, design and also the goal required solving a problem like data locality, energy and time. The main goal in this research is to represent a comprehensive survey on job scheduling modes using in spark, the types of different scheduler, and existing algorithms with advantages and issues. In this paper, various adaptive ways to schedule jobs on spark and development algorithms to improve performance in Spark will be discussed, analyzed and evaluated. A comparison between different scheduling algorithms, strength and weakness points of them are provided. This can aid to the researchers understanding of which scheduling mechanisms best applied for Big Data.

DOI

10.21608/ijisd.2020.73486

Keywords

Big Data, Spark, Scheduler, Scheduling algorithm

Authors

First Name

Walaa

Last Name

Khalil

MiddleName

Ali

Affiliation

Computer Science and engineering dept. Faculty of Electronics Engineering, Menofiua university, Egypt

Email

walaaali412@gmail.com

City

-

Orcid

-

First Name

Hanaa

Last Name

Torkey

MiddleName

-

Affiliation

Computer Science and engineering dept. Faculty of Electronics Engineering, Menofiua university, Egypt

Email

-

City

-

Orcid

-

First Name

Gamal

Last Name

Attiya

MiddleName

-

Affiliation

Computer Science and engineering dept. Faculty of Electronics Engineering, Menofiua university, Egypt

Email

-

City

-

Orcid

-

Volume

1

Article Issue

1

Related Issue

11124

Issue Date

2020-01-01

Receive Date

2019-10-01

Publish Date

2020-01-01

Page Start

39

Page End

48

Print ISSN

2682-3993

Online ISSN

2682-4000

Link

https://ijisd.journals.ekb.eg/article_73486.html

Detail API

https://ijisd.journals.ekb.eg/service?article_code=73486

Order

5

Type

Original Article

Type Code

1,141

Publication Type

Journal

Publication Title

International Journal of Industry and Sustainable Development

Publication Link

https://ijisd.journals.ekb.eg/

MainTitle

Survey of Apache spark optimized job scheduling in big data

Details

Type

Article

Created At

22 Jan 2023