SQL to NoSQL Transformation System using Data Adapter and Analytics

  • Ganesh B Solanke university of pune
  • Sudarshan S Deshmukh
Keywords: NoSQL, Big data, Data adapter, Hybrid database, Data analytics

Abstract

Popularity of big data in cloud computing is growing tremendously nowadays. Data production rate has become explosive. Many existing systems need to expand their service to support explosive data generation rate. NoSQL database are suitable to handle large volume of data. This paper proposes a data adapter which supports hybrid database systems, NoSQL databases and relational databases. Also, a seamless mechanism is provided for hybrid database system construction, which enables access to both databases at same time. This paper focuses on the automated transformation of multi-structured data hybrid systems. Hybrid database systems provide access to either relational database or NoSQL database, per data size. The proposed data adapter supports application queries and database transformation at same time, which in turn speed up the process. Three different approaches are proposed for queries on database. Blocking transformation mode, blocking truncation mode, direct access mode. This paper also discusses the integration of Pentaho business analytics tool with proposed data adapter, for big data predictive analytics. This integration focuses on empowering users to prepare, model and visualize and explore data sets stored in NoSQL databases. This paper describes the design of the data adapter in detail.

References

[1] “Cassandra ODBC and JDBC drivers with SQL connector”, Available: http://www.simba.com/drivers/cassandra-odbc-jdbc/ [2] “Datastax enterprise 2.0”, Available: http://www.datastax.com/2012/03/how-to-move-data-fromrelational-databases-to-datastax-enterprise-cassandra-usingsqoop [3] “Apache Sqoop”, Available: http://sqoop.apache.org/ [4] OpenStack Available: https://www.openstack.org/ [5] I.A.T. Hashem, I. Yaqoob, S. Mokhtar, N.B. Anuar, A. Gani, S.U. Khan, “The rise of ‘big data’ on cloud computing: Review and open research issues”, Information Systems 47 (2015) 98–115. [6] C.L. Philip Chen, C. Y. Zhang, “Data-intensive applications, challenges, techniques and technologies: A survey on Big Data”, Inform. Sci. 275 (2014) 314–347. [7] J. Han, E. Haihong, J. Du, G. Le, “Survey on NoSQL database”, 6th International Conference on Pervasive Computing and Applications, ICPCA, 2011, pp. 363–366. [8] F. Chang, S. Ghemawat, J. Dean, W.C. Hsieh, D.A. Wallach, M. Burrows, et al., “Bigtable: A distributed storage system for structured data”, ACM Transaction Computational Systems 26 (2008). [9] “Apache HBase”, Available: http://hbase.apache.org/ [10] M. Vora, “Hadoop-HBase for large-scale data”, International Conference on Computer Science and Network Technology”, 2011, pp. 601–605. [11] Kristina Chodorow, “MongoDB: The Definitive Guide”, O’Reilly Media, Inc., 2013. [12] Avinash Lakshman, Prashant Malik, “Cassandra: a decentralized structured storage system”, ACM SIGOPS Operations System Rev. 44 (2010) 35–40. [13] Neal Leavitt, “Will NoSQL databases live up to their promise?” Computer archival 43, Issue 2(February 2010) pp. 12–14. [14] James Manyika, M. Chui, B. Brown, J. Bughin, R. Dobbs, C. Roxburgh A. H. Byers, “Big data: The next frontier for innovation, competition, and productivity”, 2011. Mckinseyglobal institute. [15] Apache Cassandra. Available http://cassandra.apache.org/ [16] Rick Cattell, “Scalable SQL and NoSQL data stores”, ACM SIGMOD Rec. 39 (2011), pp 12–27. [17] M. Fazio, A. Celesti, M. Villari, A. Puliafito, “The need of a hybrid storage approach for IoT in PaaS cloud federation”, 28th International Conference on Advanced Information Networking and Applications Workshops, WAINA, 2014, pp. 779–784. [18] K.A. Doshi, T. Zhong, Z. Lu, X. Tang, T. Lou, G. Deng, “Blending SQL and NewSQL approaches: Reference architectures for enterprise big data challenges”, 2013 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC, 2013, pp. 163–170. [19] J. Rith, P.S. Lehmayr, K. Meyer-Wegener, “Speaking in tongues: SQL access to NoSQL systems”, In Proceedings of the 29th Annual ACM Symposium on Applied Computing, SAC’14, 2014, pp. 855–857. [20] J. Roijackers, “Bridging SQL and NoSQL (Master’s thesis)”, Eindhoven University of Technology, 2012. [21] O.V. Joldzic, D.R. Vukovic, “The impact of cluster characteristics on HiveQL query optimization”, 21st Telecommunications Forum, TELFOR, 2013, pp. 837–840. [22] T. Kim, H. Chung, W. Choi, J. Choi, J. Kim, “Cost-based join processing scheme in a hybrid RDB and hive system”. [23] J. Cho, H. Garcia-Molina, “Synchronizing a database to improve freshness”, ACM SIGMOD, 2000, pp. 117–128. [24] Y Lio, J. Zhou, C. Lu, S. Chen, C. Hsu, W. Chen, M. Jiang, Y. Chung, “Data adapter for querying and transformation between SQL and NoSQL database”, Elsevier, Future Generation Com. Sys., 65, 2016, pp. 111-121.
Published
2018-03-20
How to Cite
Solanke, G., & Deshmukh, S. (2018). SQL to NoSQL Transformation System using Data Adapter and Analytics. Asian Journal For Convergence In Technology (AJCT) ISSN -2350-1146, 3(3). Retrieved from http://www.asianssr.org/index.php/ajct/article/view/223
Section
Article

Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.