Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization

[{“box”:0,”content”:”[if 992 equals=”Open Access”]

n

Open Access

n

[/if 992]n

n

Year : | Volume : | [if 424 equals=”Regular Issue”]Issue[/if 424][if 424 equals=”Special Issue”]Special Issue[/if 424] [if 424 equals=”Conference”][/if 424] : | Page : –

n

n

n

n

n

n

By

n

    n t

    [foreach 286]n

    n

    Rajesh Yadav

  1. [/foreach]

    n

n

n[if 2099 not_equal=”Yes”]n

    [foreach 286] [if 1175 not_equal=””]n t

  1. Assistant Professor, Department of Computer Science, Sies College of Arts, Science & Commerce (Autonomous), Maharashtra, India
  2. n[/if 1175][/foreach]

[/if 2099][if 2099 equals=”Yes”][/if 2099]nn

n

Abstract

nAs the technology is in progress the accumulation of data is also increasing. As a result of which a lot of organisations are constantly seeking new yet innovative solutions to process and analyse this huge accumulation of data. Well Hadoop proved a gamе-changеr in thеrеalm of big data procеssing whereas Avro proved a solution provider to data Serialization. In this review articlе, wе’lldеlvе into thе world of Avro and Hadoop, еxploring its basics, features, kеycomponеnts, Connection between Avro and Hadoop, Real-world examples of Avro’s integration with Hadoop.

n

n

n

Keywords: Avro , Data Serialization, Hadoop HFS , MapReduce,big data.

n[if 424 equals=”Regular Issue”][This article belongs to International Journal of Data Structure Studies(ijdss)]

n

[/if 424][if 424 equals=”Special Issue”][This article belongs to Special Issue under section in International Journal of Data Structure Studies(ijdss)][/if 424][if 424 equals=”Conference”]This article belongs to Conference [/if 424]

n

n

n

How to cite this article: Rajesh Yadav Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization ijdss ; :-

n

How to cite this URL: Rajesh Yadav Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization ijdss {cited };:-. Available from: https://journals.stmjournals.com/ijdss/article=/view=0

nn


nn[if 992 equals=”Open Access”] Full Text PDF Download[else] nvar fieldValue = “[user_role]”;nif (fieldValue == ‘indexingbodies’) {n document.write(‘Full Text PDF‘);n }nelse if (fieldValue == ‘administrator’) { document.write(‘Full Text PDF‘); }nelse if (fieldValue == ‘ijdss’) { document.write(‘Full Text PDF‘); }n else { document.write(‘ ‘); }n [/if 992] [if 379 not_equal=””]n

Browse Figures

n

n

[foreach 379]n

n[/foreach]n

nn

n

n[/if 379]n

n

References

n[if 1104 equals=””]n

  1. Vohra, D., & Vohra, D. (2016). Apache avro. Practical Hadoop Ecosystem: A Definitive Guide to Hadoop-Related Frameworks and Tools, 303-323.
  2. Mayer-Schonberger, Viktor; Cukier, Kenneth (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think . Houghton Mifflin Harcourt.
  3. Sandy Ryza, Uri Laserson et.al (2014). Advanced-Analytics-with-Spark. OReilley.
  4. White, Tom (2014). Mastering Hadoop. OReilley
  5. Marz, Nathan, and James Warren (2015). Big Data: Principles and Best Practices of Scalable Real time Data Systems. Manning Publications.
  6. Maheshwari, Anil.Big Data. McGraw-Hill Education, 2019.
  7. Bhosale, H. S., & Gadekar, D. P. (2014). A review paper on big data and hadoop. International Journal of Scientific and Research Publications, 4(10), 1-7.
  8. Phaneendra, S. V., & Reddy, E. M. (2013, April). Big Data-solutions for RDBMS problems-A survey. In 12th IEEE/IFIP Network Operations & Management Symposium (NOMS 2010)(Osaka, Japan, Apr 19 {23 2013).
  9. Mukherjee, A., Datta, J., Jorapur, R., Singhvi, R., Haloi, S., & Akram, W. (2012, December). Shared disk big data analytics with apachehadoop. In 2012 19th International Conference on High Performance Computing (pp. 1-6). IEEE.
  10. Gupta, P., & Tyagi, N. (2015, May). An approach towards big data—A review. In International Conference on Computing, Communication & Automation (pp. 118-123). IEEE.
  11. Patel, A. B., Birla, M., & Nair, U. (2012, December). Addressing big data problem using Hadoop and Map Reduce. In 2012 Nirma University International Conference on Engineering (NUiCONE) (pp. 1-5). IEEE.
  12. Hukill, G. S., & Hudson, C. (2018). Avro: Overview and Implications for Metadata Processing.
  13. Abdelouarit, K. A., Sbihi, B., & Aknin, N. (2016). Towards an approach based on hadoop to improve and organize online search results in big data environment. In Communication, Management and Information Technology (pp. 557-564). CRC Press.
  14. Apache Hadoop Resources: https://hadoop.apache.org/docs/r2.7.2/
  15. Apache HDFS: https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
  16. Hadoop API site: http://hadoop.apache.org/docs/current/api/
  17. https://www.linkedin.com/pulse/unraveling-power-hadoop-journey-big-data-success-stories-akshat-jain?trk=article-ssr-frontend-pulse_more-articles_related-content-card.
  18. https://data-flair.training/blogs/avro-serialization/amp
  19. https://www.baeldung.com/java-apache-avro

nn[/if 1104][if 1104 not_equal=””]n

    [foreach 1102]n t

  1. [if 1106 equals=””], [/if 1106][if 1106 not_equal=””],[/if 1106]
  2. n[/foreach]

n[/if 1104]

nn


nn[if 1114 equals=”Yes”]n

n[/if 1114]

n

n

Subscription Review Article

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

n

Volume
[if 424 equals=”Regular Issue”]Issue[/if 424][if 424 equals=”Special Issue”]Special Issue[/if 424] [if 424 equals=”Conference”][/if 424]
Received November 22, 2023
Accepted November 29, 2023
Published

n

n

n

n

n

nn function myFunction2() {n var x = document.getElementById(“browsefigure”);n if (x.style.display === “block”) {n x.style.display = “none”;n }n else { x.style.display = “Block”; }n }n document.querySelector(“.prevBtn”).addEventListener(“click”, () => {n changeSlides(-1);n });n document.querySelector(“.nextBtn”).addEventListener(“click”, () => {n changeSlides(1);n });n var slideIndex = 1;n showSlides(slideIndex);n function changeSlides(n) {n showSlides((slideIndex += n));n }n function currentSlide(n) {n showSlides((slideIndex = n));n }n function showSlides(n) {n var i;n var slides = document.getElementsByClassName(“Slide”);n var dots = document.getElementsByClassName(“Navdot”);n if (n > slides.length) { slideIndex = 1; }n if (n (item.style.display = “none”));n Array.from(dots).forEach(n item => (item.className = item.className.replace(” selected”, “”))n );n slides[slideIndex – 1].style.display = “block”;n dots[slideIndex – 1].className += ” selected”;n }n”}]