Adaptive Query Routing for High-Speed Distributed Data Systems

  • Authors

    • Dr. Norbert Wiener Professor, Massachusetts Institute of Technology, United States. Author
    • Dr. Fernando J. Corbató Professor, Massachusetts Institute of Technology, United States. Author
    • Dr. Douglas Engelbart Research Director, SRI International, United States. Author

    DOI:

    https://doi.org/10.67228/30715717/IJDEIC-2024PII6M2K

    Published 10-10-2024

  • Adaptive Query Routing, Distributed Systems, Load Balancing, Data Locality, Query Optimization, Network Latency, Distributed Databases, High-Speed Systems

    Issue

    Vol. 7 No. 2 (2024)

    Section

    Articles

    How to Cite

    [1]
    N. Wiener, F. J. Corbató, and D. Engelbart, “Adaptive Query Routing for High-Speed Distributed Data Systems”, IJDEIC, vol. 7, no. 2, pp. 01–12, Oct. 2024, doi: 10.67228/30715717/IJDEIC-2024PII6M2K.

  • Abstract

    Adaptive query routing is essential in high-speed distributed data systems because static routing cannot effectively handle dynamic changes in network conditions, workloads, and system heterogeneity; instead, it dynamically selects optimal nodes or paths based on real-time factors such as latency, node availability, workload distribution, and past query performance, thereby improving efficiency, scalability, and reliability. Unlike early deterministic or heuristic methods, adaptive approaches use feedback mechanisms and probabilistic models to continuously update routing decisions, enhancing throughput and reducing query response time. The main strategies include centralized routing (with a global view but limited scalability and single-point failure risk), decentralized routing (better scalability but coordination challenges), and hybrid models (balancing both approaches). Additionally, techniques like caching, replication, and indexing help reduce unnecessary data transfer and improve execution speed, while mathematical cost models guide optimal routing decisions. Experimental results show that adaptive routing can improve query response time by 40–60% under dynamic workloads, making it a crucial component for modern distributed systems, especially in cloud computing and big data environments.

  • References

    [1] Özsu, M. T., & Valduriez, P. (2011). Principles of Distributed Database Systems (3rd ed.). Springer.

    [2] Tanenbaum, A. S., & Van Steen, M. (2017). Distributed Systems: Principles and Paradigms (2nd ed.). Pearson.

    [3] Stonebraker, M., et al. (2010). "The Case for Shared Nothing." IEEE Database Engineering Bulletin, 9(1), 4–9.

    [4] Kossmann, D. (2000). "The State of the Art in Distributed Query Processing." ACM Computing Surveys, 32(4), 422–469.

    [5] Yu, C., & Meng, W. (1998). "Principles of Database Query Processing for Advanced Applications." Morgan Kaufmann.

    [6] Hellerstein, J. M., Stonebraker, M., & Hamilton, J. (2007). "Architecture of a Database System." Foundations and Trends in Databases, 1(2), 141–259.

    [7] Bernstein, P. A., & Newcomer, E. (2009). Principles of Transaction Processing (2nd ed.). Morgan Kaufmann.

    [8] Abadi, D. J., et al. (2009). "A Comparison of Approaches to Large-Scale Data Analysis." Proceedings of the VLDB Endowment, 2(2), 1654–1657.

    [9] Dean, J., & Ghemawat, S. (2008). "MapReduce: Simplified Data Processing on Large Clusters." Communications of the ACM, 51(1), 107–113.

    [10] Ghemawat, S., Gobioff, H., & Leung, S. T. (2003). "The Google File System." ACM SIGOPS Operating Systems Review, 37(5), 29–43.

    [11] Castro, M., et al. (2002). "SCRIBE: A Large-Scale and Decentralized Application-Level Multicast Infrastructure." IEEE Journal on Selected Areas in Communications, 20(8), 1489–1499.

    [12] Stoica, I., et al. (2001). "Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications." ACM SIGCOMM Computer Communication Review, 31(4), 149–160.

    [13] Ratnasamy, S., et al. (2001). "A Scalable Content-Addressable Network." ACM SIGCOMM, 161–172.

    [14] Gajula, S. (2023). A review of anomaly identification in finance frauds using machine learning system. International Journal of Current Engineering and Technology, 13(6), 568–575. https://ijcet.evegenis.org/index.php/ijcet/article/view/820

    [15] Rowstron, A., & Druschel, P. (2001). "Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems." IFIP/ACM Middleware Conference.

  • Downloads