*Result*: An adaptive spark-based framework for querying large-scale NoSQL and relational databases.

Title:
An adaptive spark-based framework for querying large-scale NoSQL and relational databases.
Authors:
Khashan E; Department of Computers and Systems, Faculty of Engineering, Mansoura University Mansoura, Egypt., Eldesouky A; Department of Computers and Systems, Faculty of Engineering, Mansoura University Mansoura, Egypt., Elghamrawy S; Department of Computer Engineering, MISR Higher Institute for Engineering & Technology, Mansoura, Egypt.
Source:
PloS one [PLoS One] 2021 Aug 19; Vol. 16 (8), pp. e0255562. Date of Electronic Publication: 2021 Aug 19 (Print Publication: 2021).
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: Public Library of Science Country of Publication: United States NLM ID: 101285081 Publication Model: eCollection Cited Medium: Internet ISSN: 1932-6203 (Electronic) Linking ISSN: 19326203 NLM ISO Abbreviation: PLoS One Subsets: MEDLINE
Imprint Name(s):
Original Publication: San Francisco, CA : Public Library of Science
MeSH Terms:
Algorithms* , Databases, Factual* , Software*, Cloud Computing/*statistics & numerical data , Data Management/*methods , Database Management Systems/*standards , Information Storage and Retrieval/*statistics & numerical data
References:
J Vis Exp. 2018 Mar 19;(133):. (PMID: 29608174)
Entry Date(s):
Date Created: 20210819 Date Completed: 20211126 Latest Revision: 20211126
Update Code:
20260130
PubMed Central ID:
PMC8376024
DOI:
10.1371/journal.pone.0255562
PMID:
34411131
Database:
MEDLINE

*Further Information*

*The growing popularity of big data analysis and cloud computing has created new big data management standards. Sometimes, programmers may interact with a number of heterogeneous data stores depending on the information they are responsible for: SQL and NoSQL data stores. Interacting with heterogeneous data models via numerous APIs and query languages imposes challenging tasks on multi-data processing developers. Indeed, complex queries concerning homogenous data structures cannot currently be performed in a declarative manner when found in single data storage applications and therefore require additional development efforts. Many models were presented in order to address complex queries Via multistore applications. Some of these models implemented a complex unified and fast model, while others' efficiency is not good enough to solve this type of complex database queries. This paper provides an automated, fast and easy unified architecture to solve simple and complex SQL and NoSQL queries over heterogeneous data stores (CQNS). This proposed framework can be used in cloud environments or for any big data application to automatically help developers to manage basic and complicated database queries. CQNS consists of three layers: matching selector layer, processing layer, and query execution layer. The matching selector layer is the heart of this architecture in which five of the user queries are examined if they are matched with another five queries stored in a single engine stored in the architecture library. This is achieved through a proposed algorithm that directs the query to the right SQL or NoSQL database engine. Furthermore, CQNS deal with many NoSQL Databases like MongoDB, Cassandra, Riak, CouchDB, and NOE4J databases. This paper presents a spark framework that can handle both SQL and NoSQL Databases. Four scenarios' benchmarks datasets are used to evaluate the proposed CQNS for querying different NoSQL Databases in terms of optimization process performance and query execution time. The results show that, the CQNS achieves best latency and throughput in less time among the compared systems.*

*The authors have declared that no competing interests exist.*