Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization

Year : 2024 | Volume :02 | Issue : 01 | Page : 8-13
By

    Rajesh Yadav

  1. Assistant Professor, Department of Computer Science, Sies College of Arts, Science & Commerce (Autonomous), Maharashtra, India

Abstract

As the technology is in progress the accumulation of data is also increasing. As a result of which a lot of organisations are constantly seeking new yet innovative solutions to process and analyse this huge accumulation of data. Well Hadoop proved a gamе-changеr in thеrеalm of big data procеssing whereas Avro proved a solution provider to data Serialization. In this review articlе, wе’lldеlvе into thе world of Avro and Hadoop, еxploring its basics, features, kеycomponеnts, Connection between Avro and Hadoop, Real-world examples of Avro’s integration with Hadoop.

Keywords: Avro , Data Serialization, Hadoop HFS , MapReduce,big data.

[This article belongs to International Journal of Data Structure Studies(ijdss)]

How to cite this article: Rajesh Yadav , Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization ijdss 2024; 02:8-13
How to cite this URL: Rajesh Yadav , Unravelling the power of Avro and Hadoop: Revolutionising Big Data Processing and Serialization ijdss 2024 {cited 2024 Feb 21};02:8-13. Available from: https://journals.stmjournals.com/ijdss/article=2024/view=133422


References

  1. Vohra, D., & Vohra, D. (2016). Apache avro. Practical Hadoop Ecosystem: A Definitive Guide to Hadoop-Related Frameworks and Tools, 303-323.
  2. Mayer-Schonberger, Viktor; Cukier, Kenneth (2013). Big Data: A Revolution That Will Transform How We Live, Work, and Think . Houghton Mifflin Harcourt.
  3. Sandy Ryza, Uri Laserson et.al (2014). Advanced-Analytics-with-Spark. OReilley.
  4. White, Tom (2014). Mastering Hadoop. OReilley
  5. Marz, Nathan, and James Warren (2015). Big Data: Principles and Best Practices of Scalable Real time Data Systems. Manning Publications.
  6. Maheshwari, Anil.Big Data. McGraw-Hill Education, 2019.
  7. Bhosale, H. S., & Gadekar, D. P. (2014). A review paper on big data and hadoop. International Journal of Scientific and Research Publications, 4(10), 1-7.
  8. Phaneendra, S. V., & Reddy, E. M. (2013, April). Big Data-solutions for RDBMS problems-A survey. In 12th IEEE/IFIP Network Operations & Management Symposium (NOMS 2010)(Osaka, Japan, Apr 19 {23 2013).
  9. Mukherjee, A., Datta, J., Jorapur, R., Singhvi, R., Haloi, S., & Akram, W. (2012, December). Shared disk big data analytics with apachehadoop. In 2012 19th International Conference on High Performance Computing (pp. 1-6). IEEE.
  10. Gupta, P., & Tyagi, N. (2015, May). An approach towards big data—A review. In International Conference on Computing, Communication & Automation (pp. 118-123). IEEE.
  11. Patel, A. B., Birla, M., & Nair, U. (2012, December). Addressing big data problem using Hadoop and Map Reduce. In 2012 Nirma University International Conference on Engineering (NUiCONE) (pp. 1-5). IEEE.
  12. Hukill, G. S., & Hudson, C. (2018). Avro: Overview and Implications for Metadata Processing.
  13. Abdelouarit, K. A., Sbihi, B., & Aknin, N. (2016). Towards an approach based on hadoop to improve and organize online search results in big data environment. In Communication, Management and Information Technology (pp. 557-564). CRC Press.
  14. Apache Hadoop Resources: https://hadoop.apache.org/docs/r2.7.2/
  15. Apache HDFS: https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
  16. Hadoop API site: http://hadoop.apache.org/docs/current/api/
  17. https://www.linkedin.com/pulse/unraveling-power-hadoop-journey-big-data-success-stories-akshat-jain?trk=article-ssr-frontend-pulse_more-articles_related-content-card.
  18. https://data-flair.training/blogs/avro-serialization/amp
  19. https://www.baeldung.com/java-apache-avro

Regular Issue Subscription Review Article
Volume 02
Issue 01
Received November 22, 2023
Accepted November 29, 2023
Published February 21, 2024