Federated MLOps: Secure CI/CD for Distributed Model Training and Deployment

  • Authors

    • Dr. Ibrahim Yusuf Professor University of Lagos, Nigeria. Author
    • Amina Bello HR Manager Dangote Group, Nigeria. Author

    DOI:

    https://doi.org/10.67228/3142788X/IJMLPA-2019PII4D7P

    Published 10-04-2019

  • Federated Learning, MLOps, CI/CD, Secure Model Deployment, Privacy-Preserving Machine Learning, Distributed Model Training, Pipeline Automation, Model Versioning

    Issue

    Vol. 2 No. 2 (2019)

    Section

    Articles

    How to Cite

    [1]
    I. Yusuf and A. Bello, “Federated MLOps: Secure CI/CD for Distributed Model Training and Deployment”, IJMLPA, vol. 2, no. 2, pp. 01–11, Oct. 2019, doi: 10.67228/3142788X/IJMLPA-2019PII4D7P.

  • Abstract

    Federated Machine Learning (FL) has emerged as a promising approach for collaborative model training without sharing raw data, thereby preserving privacy. However, integrating FL with modern MLOps practices poses unique challenges in automating and securing the Continuous Integration and Continuous Deployment (CI/CD) pipelines. This paper proposes Federated MLOps, a framework that combines CI/CD principles with federated model training to enable secure, automated, and efficient deployment of distributed ML models. We describe the system architecture, security mechanisms, and pipeline orchestration strategies, and demonstrate the framework through a case study evaluating performance, scalability, and privacy preservation. Our results highlight the potential of Federated MLOps to enhance model reliability, reproducibility, and security in distributed learning environments.

  • References

    [1] McMahan, B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. (2017).Communication-Efficient Learning of Deep Networks from Decentralized Data.Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 1273–1282.

    [2] Bonawitz, K., Ivanov, V., Kreuter, B., et al. (2017). Practical Secure Aggregation for Privacy-Preserving Machine Learning.

    [3] Baylor, D., Breck, E., Cheng, H.-T., et al. (2017). TFX: A TensorFlow-Based Production-Scale Machine Learning Platform. KDD 2017. One of the earliest production-grade ML pipeline architectures and a precursor to modern MLOps practices.

    [4] Chen, T., Moreira, S., et al. (2015). MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems. arXiv preprint arXiv:1512.01274.

    [5] Kreps, J., Narkhede, N., & Rao, J. (2011). Kafka: A Distributed Messaging System for Log Processing.

    NetDB Workshop. Provides the streaming backbone commonly used in CI/CD and ML deployment pipelines.

    [6] Newman, S. (2015). Building Microservices: Designing Fine-Grained Systems.

    O’Reilly Media. Important for understanding microservice-based deployment architectures used in MLOps.

    [7] Humble, J., & Farley, D. (2010). Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Addison-Wesley. The foundational reference for CI/CD methodologies applicable to ML systems.

    [8] Kim, G., Humble, J., Debois, P., & Willis, J. (2016). The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution Press.

    [9] Zaharia, M., Das, T., Li, H., et al. (2013). Discretized Streams: Fault-Tolerant Streaming Computation at Scale.

    ACM Symposium on Operating Systems Principles (SOSP). Introduces Spark Streaming, widely used in ML data pipelines.

    [10] Dean, J., Corrado, G., Monga, R., et al. (2012). Large Scale Distributed Deep Networks.Advances in Neural Information Processing Systems (NeurIPS).

    [11] Meng, X., Bradley, J., Yavuz, B., et al. (2016). MLlib: Machine Learning in Apache Spark. Journal of Machine Learning Research, 17(34), 1–7.

    [12] Dragoni, N., Dustdar, S., Larsen, S. T., & Mazzara, M. (2017). Microservices: Migration of a Mission Critical System.

    IEEE Software, 35(3), 70–75.

    [13] Sattler, F., Wiedemann, S., Müller, K.-R., & Samek, W. (2019). Robust and Communication-Efficient Federated Learning from Non-IID Data. arXiv:1903.02891. Extends federated learning with communication-efficient mechanisms for distributed environments.

  • Downloads