Designing Real-Time Streaming Microservices Using AI and ML for Distributed Systems

  • Authors

    • Dr. Lucas Martin Professor, Sorbonne University, France Author
    • Dr. Chloe Bernard Associate Professor, University of Paris-Saclay, France. Author

    DOI:

    https://doi.org/10.67228/3142788X/IJMLPA-2019PI2K9X

    Published 02-03-2019

  • Microservices Architecture, Artificial Intelligence (AI), Machine Learning (ML), Real-Time Streaming, Distributed Systems, Intelligent Microservices, Data Processing, Scalability, Flexibility, Automation

    Issue

    Vol. 2 No. 1 (2019)

    Section

    Articles

    How to Cite

    [1]
    L. Martin and C. Bernard, “Designing Real-Time Streaming Microservices Using AI and ML for Distributed Systems”, IJMLPA, vol. 2, no. 1, pp. 01–09, Feb. 2019, doi: 10.67228/3142788X/IJMLPA-2019PI2K9X.

  • Abstract

    The evolution of microservices architecture has significantly enhanced the scalability and flexibility of distributed systems. Integrating Artificial Intelligence (AI) and Machine Learning (ML) into real-time streaming microservices further augments their capability to process and analyze vast data streams efficiently. This paper explores the design and implementation of such intelligent microservices, focusing on the synergy between AI/ML and microservices architecture. We discuss the benefits, challenges, and best practices of incorporating AI and ML into microservices, particularly for real-time data processing in distributed environments. Case studies in sectors like e-commerce, finance, and healthcare illustrate the practical applications and advantages of this integration. The paper concludes with future directions for research and development in this domain.

  • References

    [1] Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2016). TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI).

    [2] Babazadeh, M., & Pautasso, C. (2014). The Stream Software Connector Design Space: Frameworks and Languages for Distributed Stream Processing. Proceedings of the IEEE/IFIP Conference on Software Architecture (WICSA), 201–208.

    [3] Cui, X., Yu, X., Liu, Y., & Lü, Z. (2015). Distributed Stream Processing: A Survey. Journal of Computer Research and Development, 52(2), 318–332.

    [4] Gulisano, V., Jiménez-Peris, R., Patiño-Martínez, M., Soriente, C., & Valduriez, P. (2012). StreamCloud: An Elastic and Scalable Data Streaming System. IEEE Transactions on Parallel and Distributed Systems, 23(12), 2351–2365.

    [5] Kamburugamuve, S., Fox, G., Qiu, J., & Leake, D. (2014). Survey of Distributed Stream Processing for Large Stream Sources. Technical Report, Indiana University.

    [6] Querzoni, L., & Rivetti, N. (2017). Data Streaming and its Application to Stream Processing: Tutorial. Proceedings of the 11th ACM International Conference on Distributed and Event-Based Systems.

    [7] Val, P. B., Fernández-García, N., Sánchez-Fernández, L., & Arias-Fisteus, J. (2017). Patterns for Distributed Real-Time Stream Processing. IEEE Transactions on Parallel and Distributed Systems, 29(6), 1384–1397.

    [8] Gedik, B., Schneider, S., Hirzel, M., & Wu, K.-L. (2014). Elastic Scaling for Data Stream Processing. IEEE Transactions on Parallel and Distributed Systems, 25(6), 1447–1463.

    [9] Ramachandran, U., Nikhil, R. S., Rehg, J. M., Angelov, Y., Paul, A., Adhikari, S., et al. (2003). Stampede: A Cluster Programming Middleware for Interactive Stream-Oriented Applications. IEEE Transactions on Parallel and Distributed Systems, 14(11), 1140–1154.

    [10] Sergeev, A., & Del Balso, M. (2018). Horovod: Fast and Easy Distributed Deep Learning in TensorFlow. arXiv preprint arXiv:1802.05799.

  • Downloads