Prashant D. Sawant,
- AI Research Director Ai-Discovery Consultancy, Melbourne, Australia
Abstract
The rapid progress of the Fourth Industrial Revolution has been largely driven by the evolution of artificial intelligence (AI), with notable contributions from technologies such as Generative Pre-trained Transformers (GPT). This revolution has seen the convergence of physical, digital, and biological technologies, leading to transformative impacts across various sectors. Data science, serving as a crucial enabler, has enabled the development of intelligent value chains. However, the application of data science in this domain often remains limited to isolated elements of the data pipeline. The concept of Full Stack Data Science (FSDS) addresses this limitation by providing a comprehensive approach to managing all aspects of a data science project. This paper explores the challenges faced by Full Stack Data Scientists, such as data integration, quality, scalability, security, privacy, tool complexity, stakeholder communication, and regulatory compliance. It further discusses how the Microsoft Intelligent Data Platform (MIDP) offers a suite of tools and services that support FSDS, particularly in the healthcare industry. The roles of Full Stack Data Scientists in healthcare, including data integration, analysis, visualization, model deployment, and governance, are examined. The paper also highlights the significance of data processing frameworks and machine learning services provided by Azure, emphasizing their utility in the data science workflow.
Keywords: Artificial Intelligence, Generative Pre-trained Transformers, Full Stack Data Science, Microsoft Intelligent Data Platform, Healthcare
[This article belongs to Journal of Advanced Database Management & Systems(joadms)]
References
- D. Sawant, Artificial Intelligence: The era of new industrial revolution, eBook, Amazon.com.au: Kindle Store, (2022).
- Prashant D. Sawant. A Real-time Visualization Framework to Enhance Prompt Accuracy and Result Outcomes Based on the Number of Tokens. Journal of Artificial Intelligence Research & Advances. 2024; 11(01):45-53.
- Schwab K, World Economic Forum. The Fourth Industrial Revolution: what it means and how to respond. 2016. Available from: https://www.weforum.org/agenda/2016/01/the-fourth-industrial-revolution-what-it-means-and-how-to-respond/
- ODSC Community. Data Science in Manufacturing: An Overview. Open Data Science – Your News Source for AI, Machine Learning & more. 2020. Available from: https://opendatascience.com/data-science-in-manufacturing-an-overview/
- The Click Reader. Full-Stack Data Science: A Complete Overview. Medium. 2021. Available from: https://medium.com/@theclickreader/full-stack-data-science-a-complete-overview-9d66df1ac771
- Perfect eLearning. Building End-to-End Data Solutions: Full Stack Data Science. 2023. Available from: https://perfectelearning.com/blog/building-end-to-end-data-solutions-full-stack-data-science
- How to Become a Full Stack Data Scientist – StrataScratch. 2020. Available from: https://www.stratascratch.com/blog/how-to-become-a-full-stack-data-scientist/n
- What is a Full Stack Data Scientist?. 2022. Available from: https://shopify.engineering/what-is-a-full-stack-data-scientist
- Institute of Data. What is Full Stack Data Science?. 2024. Available from: https://www.institutedata.com/us/blog/what-is-full-stack-data-science/
- Kumar R. Introducing the Microsoft Intelligent Data Platform | Microsoft Azure Blog. 2022. Available from: https://azure.microsoft.com/en-us/blog/introducing-the-microsoft-intelligent-data-platform/
- Microsoft Intelligent Data Platform | Microsoft Cloud. Microsoft.com. 2023. Available from: https://www.microsoft.com/en-us/microsoft-cloud/solutions/intelligent-data-platform
- Hawk J. The Microsoft Intelligent Data Platform—Unleash your data and accelerate your transformation | Microsoft Azure Blog. 2024. Available from: https://azure.microsoft.com/en-us/blog/the-microsoft-intelligent-data-platform-unleash-your-data-and-accelerate-your-transformation/
- Thomas S. Microsoft Cloud for Healthcare: Transforming patient care through data and insights – Microsoft Industry Blogs. 2022. Available from: https://www.microsoft.com/en-us/industry/blog/healthcare/2022/04/05/microsoft-cloud-for-healthcare-transforming-patient-care-through-data-and-insights/
- About health data in Australia. Australian Government Department of Health and Aged Care. 2024. Available from: https://www.health.gov.au/topics/health-data-and-medical-research/about-health-data
- Guide to health privacy. OAIC. oaic.gov.au. 2019. Available from: https://www.oaic.gov.au/__data/assets/pdf_file/0011/2090/guide-to-health-privacy.pdf
- Media B. Data Masking vs De-Identification: Understanding the differences and relevance in healthcare – Orion Health. 2023. Available from: https://orionhealth.com/us/blog/data-masking-vs-de-identification-in-healthcare/
- Kadam S. Data Masking : Concept, Tools, Masking polices & Healthcare Data Masking. Medium. Petabytz; 2019. Available from: https://medium.com/petabytz/data-masking-concept-tools-masking-polices-healthcare-data-masking-3c28ca03a30a
- Jain K. Data Anonymization and Data Masking in Healthcare – Karen Jain – Medium; 2023. Available from: https://karen-j.medium.com/data-anonymization-and-data-masking-in-healthcare-db2650f04213
- Privacy, Security, and HIPAA | HealthIT.gov. 2019. Available from: https://www.healthit.gov/topic/privacy-security-and-hipaa
- bhatia jasdeep. The Role Of Data Science In Healthcare Industry In 2024. Learn Coding Anywhere Anytime -PW Skills Blog. 2024. Available from: https://pwskills.com/blog/the-role-of-data-science-in-healthcare-industry-in-2024/
- Guide to Becoming a Full Stack Data Scientist in 2024 – Explore the comprehensive pathway to becoming, covering skills, tools, and strategies for success. 2024. Available from: https://sqlpad.io/tutorial/guide-to-becoming-a-full-stack-data-scientist-in-2024/
- DS Roadmap. Samwestby.com. 2023. Available from: https://www.samwestby.com/2023/data-science-roadmap
- Sinha V. Building a Data Science Tech Stack: A Comprehensive Guide. Learn | Hevo. Hevo Data; 2020. Available from: https://hevodata.com/learn/building-a-data-science-tech-stack/
- What is Data Science? Become a Data Scientist | Microsoft Azure. Microsoft.com. 2024. Available from: https://azure.microsoft.com/en-ca/resources/cloud-computing-dictionary/what-is-data-science/
Journal of Advanced Database Management & Systems
| Volume | 11 |
| Issue | 03 |
| Received | April 27, 2024 |
| Accepted | June 3, 2024 |
| Published | August 31, 2024 |