Automated Test Oracle Generation for ML-Driven Microservices

  • Authors

    • Dr. Lucas Martin Professor, Sorbonne University, France. Author

    DOI:

    https://doi.org/10.67228/3142788X/IJMLPA-2018PII5C1Q

    Published 08-03-2018

  • Test Oracle Automation, ML-Driven Microservices, Software Testing, Metamorphic Testing, Continuous Integration And Deployment (CI/CD), Model Behavior Verification, Data-Driven QA, Microservice Reliability, Automated Validation Framework, Explainable AI (XAI) for Testing

    Issue

    Vol. 1 No. 2 (2018)

    Section

    Articles

    How to Cite

    [1]
    L. Martin, “Automated Test Oracle Generation for ML-Driven Microservices”, IJMLPA, vol. 1, no. 2, pp. 01–15, Aug. 2018, doi: 10.67228/3142788X/IJMLPA-2018PII5C1Q.

  • Abstract

    Modern software systems increasingly integrate Machine Learning (ML) components into microservice architectures to enable intelligent decision-making and autonomous operations. However, testing such ML-driven microservices remains a significant challenge due to the stochastic behavior of ML models, evolving data distributions, and the absence of explicit expected outputs creating what is known as the oracle problem. This paper proposes an automated framework for generating test oracles tailored to ML-based microservices, leveraging statistical metamorphic relationships, behavioral invariants, and automated data characterization techniques. The approach dynamically constructs oracles by combining domain-specific constraints, model interpretability outputs, and service-level behavioral patterns to validate ML predictions and microservice interactions. Experimental results across multiple domains demonstrate significant improvements in fault detection, scalability, and test coverage compared with traditional rule-based and manual testing strategies. The findings highlight the feasibility and necessity of automated oracle generation for ensuring reliability, correctness, and maintainability of ML-driven microservice ecosystems.

  • References

    [1] Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015). The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering, 41(5), 507–525. https://doi.org/10.1109/TSE.2014.2372785

    [2] Xie, T. (2006). Augmenting automatically generated unit-test suites with regression oracle checking. In Proceedings of the 20th European Conference on Object-Oriented Programming (ECOOP 2006) (pp. 380–403). Springer. https://doi.org/10.1007/11785477_23

    [3] Chen, T. Y., Kuo, F.-C., Liu, H., Poon, P.-L., Towey, D., Tse, T. H., & Zhou, Z. Q. (2018). Metamorphic testing: A review of challenges and opportunities. ACM Computing Surveys, 51(1), 1–27. https://doi.org/10.1145/3143561

    [4] Ding, J., Zhang, D., & Hu, W. (2016). A machine learning approach for developing test oracles for testing scientific software. In Proceedings of the 28th International Conference on Software Engineering and Knowledge Engineering (SEKE 2016) (pp. 247–252).

    [5] Zhou, Z. Q., Sun, L., Xiang, Y., & Chen, T. Y. (2004). Metamorphic testing and its applications. In Proceedings of the 8th International Symposium on Future Software Technology (pp. 346–351).

    [6] Harman, M., Jia, Y., & Zhang, Y. (2015). Achievements, open problems and challenges for search based software testing. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST 2015) (pp. 1–12). https://doi.org/10.1109/ICST.2015.7102590

    [7] Newman, S. (2015). Building Microservices. Sebastopol, CA: O’Reilly Media.

    [8] Dragoni, N., Giallorenzo, S., Lafuente, A. L., Mazzara, M., Montesi, F., Mustafin, R., & Safina, L. (2017). Microservices: Yesterday, today, and tomorrow. In M. Mazzara & B. Meyer (Eds.), Present and Ulterior Software Engineering (pp. 195–216). Springer. https://doi.org/10.1007/978-3-319-67425-4_12

    [9] Li, N., & Offutt, J. (2017). Test oracle strategies for model-based testing. IEEE Transactions on Software Engineering, 43(4), 372–391. https://doi.org/10.1109/TSE.2016.2597136

    [10] Goffi, A., Gorla, A., Ernst, M. D., & Pezzè, M. (2016). Automatic generation of oracles for exceptional behaviors. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016) (pp. 213–224). https://doi.org/10.1145/2931037.2931054

    [11] Zhou, Z. Q., Xiang, Y., & Chen, T. Y. (2009). Metamorphic testing for software quality assessment: A study of search engines. IEEE Transactions on Software Engineering, 40(4), 388–404.

    [12] Wang, W., & Zeller, A. (2016). Automated test oracle generation using denotational semantics. Computer Languages, Systems & Structures, 45, 204–219. https://doi.org/10.1016/j.cl.2016.01.006

  • Downloads