Machine Learning Techniques for Automated Database Performance Tuning

  • Authors

    • Mr. Noah Wright Data Engineering Lead, SAP, Germany. Author
    • Ms. Isabella Moore HR Director, Siemens, Germany. Author

    DOI:

    https://doi.org/10.67228/30713498/IJADSMC-2024PII2X3R

    Published 10-04-2024

  • Database Performance Tuning, Machine Learning, Automated Database Administration, Reinforcement Learning, Query Optimization, Self-Tuning Databases, Artificial Intelligence, Database Management Systems, Predictive Analytics, Autonomous Databases

    Issue

    Vol. 7 No. 2 (2024)

    Section

    Articles

    How to Cite

    [1]
    N. Wright and I. Moore, “Machine Learning Techniques for Automated Database Performance Tuning”, IJADSMC, vol. 7, no. 2, pp. 01–15, Oct. 2024, doi: 10.67228/30713498/IJADSMC-2024PII2X3R.

  • Abstract

    Database Management Systems (DBMSs) are essential for modern applications such as cloud computing, e-commerce, finance, healthcare, and scientific research. As data volumes continue to grow, traditional manual database tuning methods have become inefficient, time-consuming, and unable to adapt to dynamic workloads. Machine Learning (ML) offers an intelligent solution by enabling databases to learn from historical workload patterns, resource utilization, query performance, and system metrics. This study explores the application of ML techniques for automated database performance tuning. Various algorithms, including Random Forest, Support Vector Machines, Artificial Neural Networks, Gradient Boosting, and Deep Reinforcement Learning, are utilized to optimize database parameters such as indexing, memory allocation, buffer management, query execution plans, and storage configurations. A systematic framework is proposed that incorporates performance monitoring, feature engineering, predictive modeling, and automated decision-making components to achieve self-managing database operations. Experimental results demonstrate that ML-based tuning significantly improves query response time, throughput, resource utilization, and adaptability compared to static configurations. The study also discusses challenges related to model interpretability, scalability, data quality, and real-time deployment. Furthermore, future research directions including autonomous databases, cloud-native optimization, federated learning, and AI-driven database administration are highlighted. Overall, the findings indicate that machine learning is a transformative technology for developing intelligent, self-tuning database systems with enhanced performance, reliability, and reduced operational costs.

  • References

    [1] S. Chaudhuri and G. Weikum, “Rethinking Database System Architecture: Towards a Self-Tuning RISC-Style Database System,” in Proc. VLDB Conference, Berlin, Germany, 2000, pp. 1–10.

    [2] S. Chaudhuri and V. Narasayya, “An Efficient Cost-Driven Index Selection Tool for Microsoft SQL Server,” in Proc. VLDB Conference, New York, NY, USA, 1997, pp. 146–155.

    [3] S. Chaudhuri, V. Narasayya, and R. Ramamurthy, “Estimating Progress of Long Running SQL Queries,” ACM SIGMOD Record, vol. 33, no. 2, pp. 803–814, 2004.

    [4] J. O. Kephart and D. M. Chess, “The Vision of Autonomic Computing,” Computer, vol. 36, no. 1, pp. 41–50, Jan. 2003.

    [5] M. Stillger, G. M. Lohman, V. Markl, and M. Kandil, “LEO—DB2's Learning Optimizer,” in Proc. VLDB Conference, Cairo, Egypt, 2001, pp. 19–28.

    [6] D. Van Aken, A. Pavlo, G. J. Gordon, and B. Zhang, “Automatic Database Management System Tuning Through Large-scale Machine Learning,” in Proc. ACM SIGMOD International Conference on Management of Data, Houston, TX, USA, 2017, pp. 1009–1024.

    [7] D. Van Aken, A. Pavlo, G. J. Gordon, and B. Zhang, “An Inquiry into Machine Learning-Based Automatic Configuration Tuning Services on Real-World Database Management Systems,” Proceedings of the VLDB Endowment, vol. 11, no. 10, pp. 1242–1255, 2018.

    [8] J. Zhang, Y. Liu, K. Zhou, G. Li, Z. Xiao, B. Cheng, J. Xing, Y. Wang, T. Cheng, L. Liu, and M. Zeng, “An End-to-End Automatic Cloud Database Tuning System Using Deep Reinforcement Learning,” in Proc. ACM SIGMOD Conference, Amsterdam, Netherlands, 2019, pp. 415–432.

    [9] H. Marcus, P. Negi, H. Mao, N. Tatbul, M. Alizadeh, and T. Kraska, “Neo: A Learned Query Optimizer,” Proceedings of the VLDB Endowment, vol. 12, no. 11, pp. 1705–1718, 2019.

    [10] T. Kraska, A. Beutel, E. H. Chi, J. Dean, and N. Polyzotis, “The Case for Learned Index Structures,” in Proc. ACM SIGMOD International Conference on Management of Data, Houston, TX, USA, 2018, pp. 489–504.

    [11] X. Zhang, H. Wang, S. Li, and Y. Zhou, “Deep Reinforcement Learning for Database Knob Tuning,” IEEE Access, vol. 8, pp. 181457–181469, 2020.

    [12] A. Pavlo, G. Angulo, A. Arulraj, H. Lin, P. Leis, and M. M. Zukowski, “Self-Driving Database Management Systems,” in Proc. Conference on Innovative Data Systems Research (CIDR), California, USA, 2017, pp. 1–10.

    [13] H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource Management with Deep Reinforcement Learning,” in Proc. ACM Workshop on Hot Topics in Networks (HotNets), Atlanta, GA, USA, 2016, pp. 50–56.

    [14] V. Hamscher, U. Buyya, and J. M. Schopf, “Economic Models for Resource Management and Scheduling in Grid Computing,” Concurrency and Computation: Practice and Experience, vol. 14, no. 13–15, pp. 1507–1542, 2002.

    [15] A. Shukla, P. Deshpande, and S. Kumar, “Machine Learning Techniques for Database Performance Optimization: A Survey,” IEEE Access, vol. 10, pp. 76891–76915, 2022.

    [16] Gajula, S. (2023). A review of anomaly identification in finance frauds using machine learning system. International Journal of Current Engineering and Technology, 13(6), 568–575. https://ijcet.evegenis.org/index.php/ijcet/article/view/820.

  • Downloads