Data Engineering for Predictive Analytics in Healthcare: Challenges and Solutions

  • Authors

    • Sophia White Operations Director, Amazon, USA. Author

    DOI:

    https://doi.org/10.67228/30715717/IJDEIC-2019PII9V6N

    Published 12-11-2019

  • Data Engineering, Predictive Analytics, Healthcare, Electronic Health Records, Data Integration, Big Data, Machine Learning, Data Security, Interoperability, Cloud Computing

    Issue

    Vol. 2 No. 2 (2019)

    Section

    Articles

    How to Cite

    [1]
    S. White, “Data Engineering for Predictive Analytics in Healthcare: Challenges and Solutions”, IJDEIC, vol. 2, no. 2, pp. 01–12, Dec. 2019, doi: 10.67228/30715717/IJDEIC-2019PII9V6N.

  • Abstract

    This document provides detailed guidelines for the preparation and submission of manuscripts to World Comet Predictive analytics in healthcare has revolutionized medical decision-making by enabling early disease detection, risk stratification, and personalized treatment plans. However, the implementation of predictive analytics relies on robust data engineering processes to handle the vast amounts of structured and unstructured healthcare data. The integration of electronic health records (EHRs), genomic data, and real-time patient monitoring systems presents significant challenges related to data quality, interoperability, security, and computational efficiency. This paper explores the critical role of data engineering in predictive analytics, addressing key challenges such as data acquisition, cleaning, storage, and real-time processing. Furthermore, it discusses various solutions, including data integration frameworks, cloud-based infrastructures, and artificial intelligence (AI)-driven data processing techniques. The research highlights emerging trends such as federated learning, blockchain for data security, and automated data pipelines that enhance the scalability and accuracy of predictive models. The paper concludes by emphasizing the need for standardized data governance policies, cross-institutional collaborations, and advanced machine learning algorithms to overcome data engineering challenges and improve healthcare outcomes.

  • References

    [1] Jensen, P. B., Jensen, L. J., & Brunak, S. (2012). Mining electronic health records: Towards better research applications and clinical care. Nature Reviews Genetics, 13(6), 395–405.

    [2] Raghupathi, W., & Raghupathi, V. (2014). Big data analytics in healthcare: Promise and potential. Health Information Science and Systems, 2(1), 3.

    [3] Bates, D. W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014). Big data in health care: Using analytics to identify and manage high-risk and high-cost patients. Health Affairs, 33(7), 1123–1131.

    [4] Murdoch, T. B., & Detsky, A. S. (2013). The inevitable application of big data to health care. JAMA, 309(13), 1351–1352.

    [5] Belle, A., Thiagarajan, R., Soroushmehr, S. M. R., Navidi, F., Beard, D. A., & Najarian, K. (2015). Big data analytics in healthcare. BioMed Research International, 2015, 370194.

    [6] Ristevski, B., & Chen, M. (2016). Big data analytics in medicine and healthcare. Journal of Integrative Bioinformatics, 13(3), 293.

    [7] Wang, Y., Kung, L., & Byrd, T. A. (2015). Big data analytics: Understanding its capabilities and potential benefits for healthcare organizations. Technological Forecasting and Social Change, 126, 3–13.

    [8] Krumholz, H. M. (2014). Big data and new knowledge in medicine: The thinking, training, and tools needed for a learning health system. Health Affairs, 33(7), 1163–1170.

    [9] Luo, J., Wu, M., Gopukumar, D., & Zhao, Y. (2016). Big data application in biomedical research and healthcare: A literature review. Biomedical Informatics Insights, 8, BII-S31559.

    [10] Herland, M., Khoshgoftaar, T. M., & Wald, R. (2014). A review of data mining using big data in healthcare. Journal of Big Data, 1(1), 2.

    [11] Saria, S., Butte, A., & Sheikh, A. (2014). Better medicine through machine learning: What’s real, and what’s artificial? PLoS Medicine, 11(12), e1001746.

    [12] Chawla, N. V., & Davis, D. A. (2013). Bringing big data to personalized healthcare: A patient-centered framework. Journal of General Internal Medicine, 28(S3), 660–665.

    [13] Feldman, B., Martin, E. M., & Skotnes, T. (2012). Big data in healthcare hype and hope. Dr. Bonnie 360.

    [14] Cios, K. J., & Moore, G. W. (2002). Uniqueness of medical data mining. Artificial Intelligence in Medicine, 26(1–2), 1–24.

    [15] Koh, H. C., & Tan, G. (2011). Data mining applications in healthcare. Journal of Healthcare Information Management, 19(2), 64–72.

  • Downloads