Real-Time Analytics and Cloud-Native Data Engineering for Smart Manufacturing Systems

  • Authors

    • Dhanaraj Sathiri Independent Researcher, USA. Author

    DOI:

    https://doi.org/10.67228/30715717/IJDEIC-2023PI9T2L

    Published 01-05-2023

  • Cloud-Native Manufacturing, Smart Manufacturing Systems, Industrial IoT Analytics, Real-Time Data Pipelines, Scalable Cloud Architecture, Elastic Compute Resources, Multi-Tenancy Support, Stateless Service Design, Service Mesh Integration, Observability Frameworks, Data Ingestion Pipelines, Streaming Analytics, Edge To Cloud Integration, Manufacturing Data Engineering, Decision Support Analytics, Cost-Efficient Deployment, Small And Medium Enterprise Enablement, Big Data Infrastructure, Cloud-Native Design Patterns, Industrial Analytics Governance

    Issue

    Vol. 6 No. 1 (2023)

    Section

    Articles

    How to Cite

    [1]
    D. Sathiri, “Real-Time Analytics and Cloud-Native Data Engineering for Smart Manufacturing Systems”, IJDEIC, vol. 6, no. 1, pp. 01–18, Jan. 2023, doi: 10.67228/30715717/IJDEIC-2023PI9T2L.

  • Abstract

    Although cloud computing is not yet widely embraced by the industrial domain, its growing popularity in Information Technology suggests it will soon be adopted in this area as well. Smart manufacturing advocates see potential in the cloud-native ecosystem due to its capability for rapid development and cost savings. At the same time, the Internet of Things (IoT) and Big Data are fundamental components of smart manufacturing. A cloud-native environment significantly lowers the barrier for small- and medium-sized enterprises that strive to take the first step toward integrating these technologies. Yet cloud-native technology brings its own challenges, and suitable architecture design patterns must be adopted to ensure scaling and optimization during and after development. Based on an extensive body of knowledge and well-proven design patterns, these aspects have been incorporated into a cloud-native, data-engineering-oriented end-to-end solution design that supports real-time analytics in a manufacturing context. The solution design ultimately enables data from IoT devices to flow all the way through to the delivery of insights with decision-making impact. It covers the entire process—from data ingestion and storage to data-engineering pipelines and enabling cloud-native tooling—and satisfies six aspects important for cloud-native design: scalability, elasticity, multi-tenancy, support for stateless services, service mesh integration, and observability. The results, streamlined, synthesized, and presented in an integrated manner, provide a cloud-native solution-architecture perspective aligned with the real-time analytics demand of IoT-enabled manufacturing systems. They also enable or facilitate further, more specialized design decisions in the context of analytics and data engineering and serve as a knowledge repository for related explorations.

  • References

    [1] Mangalampalli, B. M. (2022). Automated Invoice Validation Systems Using Advanced SQL Analytics in Healthcare Insurance. Front Health Inform, 11.

    [2] Davenport, T., & Kalakota, R. (2019). The potential for artificial intelligence in healthcare. Future Healthcare Journal, 6(2), 94–98.

    [3] Puli, V. O. R., & Maguluri, K. K. (2022). Deep learning applications in materials management for pharmaceutical supply chains. Migration Letters, 19(6), 1144-1158.

    [4] Inala, R. (2022). Cross-Domain MDM Integration Using AI-Driven Data Governance: A Case Study In Financial Technology Architecture. Migration Letters, 19(2), 280-304.

    [5] Vankayalapati, R. K., Pandugula, C., Ganti, V. K. A. T., & Mishra, G. (2022). AI-powered self-healing cloud infrastructures: A paradigm for autonomous fault recovery. Migration Letters, 19(6), 1173-1187.

    [6] Srinivas Kalisetty (2020). Intelligent Supply Chain Ecosystems: Cloud-Native Architectures and Big Data Integration in Retail and Manufacturing Operations. Open Journal of Educational Research, 1(1), 1-19. https://doi.org/10.31586/ojer.2020.1343

    [7] Mashetty, S. Data-driven insights for community investment through mortgage-backed securities. International Journal of Innovative Research in Electrical, Electronics, Instrumentation and Control Engineering (IJIREEICE), DOI, 10.

    [8] Paleti, S. (2022). Financial Innovation through AI and Data Engineering: Rethinking Risk and Compliance in the Banking Industry. Available at SSRN.

    [9] Yandamuri, U. S. (2022). Cloud-Based Data Integration Architectures for Scalable Enterprise Analytics. International Journal of Intelligent Systems and Applications in Engineering, 10, 472-483.

    [10] Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine, 25, 24–29.

    [11] Loganathan, R. (2021). Integrated Risk and Compliance Frameworks for Global Data Center Operations: A Governance-Centric Approach. Universal Journal of Computer Sciences and Communications, 1(1), 1-26.

    [12] Mangala, N. (2022). Real-Time Data Quality Monitoring and Gating Frameworks in Cloud-Based Data Pipelines. International Journal of Research and Applied Innovations, 5(6), 8197-8219.

    [13] Mukesh, A., & Aitha, A. R. (2021). Insurance Risk Assessment Using Predictive Modeling Techniques. International Journal of Emerging Research in Engineering and Technology, 2(4), 68-79. https://doi.org/10.63282/3050-922X.IJERET-V2I4P108

    [14] Mattaparthi, R. (2022). Engineering Predictive Industrial Systems Through IoT-Driven Asset Monitoring and Machine Learning Prognostics. International Journal of Future Innovative Science and Technology (IJFIST), 5(1), 7790.

    [15] Chakilam, C., Suura, S. R., Koppolu, H. K. R., & Recharla, M. (2022). From Data to Cure: Leveraging Artificial Intelligence and Big Data Analytics in Accelerating Disease Research and Treatment Development. Journal of Survey in Fisheries Sciences. https://doi. org/10.53555/sfs. v9i3, 3619.

    [16] Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021). The false hope of current approaches to explainable artificial intelligence in health care. The Lancet Digital Health, 3(11), e745–e750.

    [17] Reddy, V. A. R. (2022). Designing Fault-Tolerant Data Ingestion Pipelines for High-Volume Healthcare Transactions. Frontiers in Health Informatics, 11, 861-889.

    [18] Nandan, B. P. (2022). AI-Powered Fault Detection In Semiconductor Fabrication: A Data-Centric Perspective. Lebcir, I., Mageswari, SU, Bhosale, YH, Nagubandi, AR, & Mahabooba, MM Agile Strategic Management in the Age of Disruption: Leveraging AI and Data Analytics for Competitive Advantage.

    [19] Adusupalli, B. (2021). Multi-Agent Advisory Networks: Redefining Insurance Consulting with Collaborative Agentic AI Systems. Journal of International Crisis and Risk Communication Research, 45-67.

    [20] Pamisetty, A. (2022). A Comparative Study of AWS, Azure, and GCP for Scalable Big Data Solutions in Wholesale Product Distribution. International Journal of Scientific Research and Modern Technology, 71-88.

    [21] Mangala, N. (2021). CI/CD Pipeline Automation for Enterprise Data Artifacts Using Azure DevOps. Universal Journal of Business and Management, 1(1), 1-18.

    [22] Inala, R. (2020). Building Foundational Data Products for Financial Services: A MDM-Based Approach to Customer, and Product Data Integration. Universal Journal of Finance and Economics, 1(1), 1-18.

    [23] Maguluri, K. K., Yasmeen, Z., & Nampalli, R. C. R. (2022). Big Data Solutions For Mapping Genetic Markers Associated With Lifestyle Diseases. Migration Letters, 19(6), 1188-1204.

    [24] Peddi, R. K. (2021). Optimizing Case Management Workflows in Global Data Center Colocation Services. Universal Journal of Computer Sciences and Communications, 1(1), 1-21.

    [25] Polineni, T. N. S., Pandugula, C., & Ganti, V. K. A. T. (2022). AI-driven automation in monitoring post-operative complications across health systems. Global Journal of Medical Case Reports, 2(1), 32-46.

    [26] Lakkarasu, P., & Kalisetty, S. (2022). Hybrid Cloud and AI Integration for Scalable Data Engineering: Innovations in Enterprise AI Infrastructure. Math. Stat. Eng. Appl., 71(4), 16774-16784.

    [27] Challa, K., Burugulla, J. K. R., Pandiri, L., Pamisetty, V., & Paleti, S. (2022). Optimizing Digital Payment Ecosystems: Ai-Enabled Risk Management, Regulatory Compliance, And Innovation In Financial Services. Migration Letters, 19, 1748-1769.

    [28] Pamisetty, A. (2022). Big Data can Generate Major Opportunities for Manufacturing Supply Chains. International Journal of Scientific Research and Modern Technology, 1(12), 238-251.

    [29] Mashetty, S. (2022). Innovations in mortgage-backed security analytics: A patent-based technology review. Available at SSRN 5238910.

    [30] Aitha, A. R. (2022). Cloud Native ETL Pipelines for Real Time Claims Processing in Large Scale Insurers. Universal Journal of Business and Management.

    [31] Mangalampalli, B. M. (2021). Scalable Data Warehouse Architecture for Population Health Management and Predictive Analytics. World Journal of Clinical Medicine Research, 1(1), 1-18.

    [32] Reddy, V. A. R. (2022). Data-Driven Healthcare Operations: Architecting Unified Member, Provider, and Claims Intelligence Platforms. International Journal of Science, Research and Technology, 5(5), 8511-8521.

    [33] Pamisetty, V. (2022). Transforming fiscal impact analysis with AI, big data, and cloud computing: A framework for modern public sector finance. Big Data, and Cloud Computing: A Framework for Modern Public Sector Finance (November 30, 2022).

    [34] Recharla, M. (2020). Targeted Gene Therapy for Spinal Muscular Atrophy: Advances in Delivery Mechanisms and Clinical Outcomes. International Journal of Science and Research (IJSR), 9(12), 1921-1934.

    [35] Botlagunta, P., & Chitta, S. (2022). Advanced optical proximity correction (OPC) techniques in computational lithography: Addressing the challenges of pattern fidelity and edge placement error. Glob. J. Med. Case Rep, 2, 58-75.

    [36] Sriram, H. K., Adusupalli, B., Singreddy, S., & Malempati, M. (2021). Revolutionizing risk assessment and financial ecosystems with smart automation, secure digital solutions, and advanced analytical frameworks.

    [37] Murali, Revolutionizing Risk Assessment and Financial Ecosystems with Smart Automation, Secure Digital Solutions, and Advanced Analytical Frameworks (December 27, 2021).

    [38] Grote, T., & Berens, P. (2020). On the ethics of algorithmic decision-making in healthcare. Journal of Medical Ethics, 46(3), 205–211.

    [39] Paleti, S. (2021). Cognitive core banking: A data-engineered, AI-infused architecture for proactive risk compliance management. AI-Infused Architecture for Proactive Risk Compliance Management (December 21, 2021).

    [40] Kalisetty, S., & Kothpalli Sondinti, L. R. (2022). AI-Native Cloud Platforms: Redefining Scalability and Flexibility in Artificial Intelligence Workflows. Linguistic and Philosophical Investigations, 21(1), 1-15.

    [41] Yandamuri, U. S. (2021). A Comparative Study of Traditional Reporting Systems versus Real-Time Analytics Dashboards in Enterprise Operations. Universal Journal of Business and Management, 1(1), 1-13.

    [42] Mashetty, S. (2022). Enhancing Financial Data Security And Business Resiliency In Housing Finance: Implementing AI-Powered Data Analytics, Deep Learning, And Cloud-Based Neural Networks For Cybersecurity And Risk Management. Migration Letters, 19(6), 1302-1818.

    [43] Haleem, A., Javaid, M., Khan, I. H., & Vaishya, R. (2022). Significant applications of artificial intelligence in healthcare: A review. Current Medicine Research and Practice, 12(3), 128–134.

    [44] Maguluri, K. K., & Ganti, V. K. A. T. (2019). Predictive Analytics in Biologics: Improving Production Outcomes Using Big Data.

    [45] Mangala, N. (2021). Optimizing Large-Scale ETL Pipelines Using Medallion Architecture on Azure Data Lake. Journal of Artificial Intelligence and Big Data, 1(1), 1-20.

    [46] Sheelam, G. K., & Nandan, B. P. (2022). Integrating AI And Data Engineering For Intelligent Semiconductor Chip Design And Optimization. Migration Letters, 19, 2178-2207.

    [47] Paleti, S. (2022). Fusion bank: Integrating AI-driven financial innovations with risk-aware data engineering in modern banking. Decision Making, 2326, 9865.

    [48] Pamisetty, A. (2022). Integrating Big Data, AI, and Financial Modeling in Cloud-Based Insurance and Banking Ecosystems. AI, and Financial Modeling in Cloud-Based Insurance and Banking Ecosystems (December 05, 2022).

    [49] Pandugula, C., & Yasmeen, Z. (2019). A Comprehensive Study of Proactive Cybersecurity Models in Cloud-Driven Retail Technology Architectures. Universal Journal of Computer Sciences and Communications, 1(1), 1253.

    [50] Inala, R. (2021). A New Paradigm in Retirement Solution Platforms: Leveraging Data Governance to Build AI-Ready Data Products. Journal of International Crisis and Risk Communication Research, 286-310.

    [51] Aitha, A. R. (2021). Dev Ops Driven Digital Transformation: Accelerating Innovation In The Insurance Industry. Journal of International Crisis and Risk Communication Research.

    [52] Loganathan, R. (2022). Converging Security Architecture and Compliance Management in Enterprise Data Center Ecosystems: A Unified Control Framework. International Journal of Scientific Research and Modern Technology, 1(12), 295-312.

    [53] Reddy, V. A. R. (2021). Challenges in Standardizing Member Eligibility Data Across Multi-Payer Healthcare Ecosystems. International Journal of Medical Toxicology and Legal Medicine, 24(3), 1-19.

    [54] Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17, 195.

    [55] Mattaparthi, R. (2021). Unified Data Lineage and Quality Governance Framework for Multi-Source Sensor Streams in Heavy-Duty Powertrain Manufacturing. Online Journal of Mechanical Engineering, 1(1), 1-15.

    [56] Chava, K., Chakilam, C., & Recharla, M. (2021). Machine Learning Models for Early Disease Detection: A Big Data Approach to Personalized Healthcare. International Journal of Engineering and Computer Science, 10(12), 25709-25730.

    [57] Adusupalli, B. (2021). Multi-Agent Advisory Networks: Redefining Insurance Consulting with Collaborative Agentic AI Systems. Journal of International Crisis and Risk Communication Research, 45-67.

    [58] Pamisetty, V. (2021). A cloud-integrated framework for efficient government financial management and unclaimed asset recovery. Available at SSRN.

    [59] Krittanawong, C., Johnson, K. W., Rosenson, R. S., Wang, Z., Aydar, M., & Baber, U. (2021). Deep learning for cardiovascular medicine. Journal of the American College of Cardiology, 78(6), 589–600.

  • Downloads