Multi-Tenant Data Architecture for Real-Time AI Workloads with Role-Based Compliance Enforcement
-
DOI:
https://doi.org/10.67228/30715717/IJDEIC-2023PII3A9MPublished 10-10-2023
Multi-Tenant Architecture, Real-Time AI Workloads, Role-Based Access Control (RBAC), Data Compliance, Data Governance, Secure Data Sharing, Streaming Data Architecture, AI/ML Infrastructure, Regulatory Enforcement, Scalable Data Systems Issue
Section
ArticlesHow to Cite
[1]P. B. Hansen and B. Diderichsen, “Multi-Tenant Data Architecture for Real-Time AI Workloads with Role-Based Compliance Enforcement”, IJDEIC, vol. 6, no. 2, pp. 01–12, Oct. 2023, doi: 10.67228/30715717/IJDEIC-2023PII3A9M.Abstract
In the age of real-time AI workloads, scalable and secure data infrastructure is paramount—especially in multi-tenant environments where multiple users or organizations share resources. This paper presents a novel architectural blueprint for designing a multi-tenant data system optimized for real-time AI processing. It addresses the dual challenge of performance and compliance by introducing a layered, role-based compliance enforcement mechanism. This approach enables dynamic access control, data isolation, lineage tracking, and auditability without compromising on latency or throughput. Through practical design patterns and reference implementations, the paper outlines how to balance architectural efficiency with rigorous regulatory obligations such as GDPR, HIPAA, or SOC 2. We also examine case studies and real-world implementations to validate the proposed framework.
References
[1] Armbrust; M.; et al. (2018). "Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark." Proceedings of the VLDB Endowment; 11(12); 2025-2036. https://doi.org/10.14778/3229863.3236256
[2] Chen; T.; et al. (2019). "Tecton: A Platform for Building Production Data Systems for Machine Learning." Proceedings of the VLDB Endowment; 12(12); 2114–2126. https://doi.org/10.14778/3352063.3352113
[3] Kreps; J.; et al. (2011). "Kafka: A Distributed Messaging System for Log Processing." Proceedings of the NetDB; 11.
[4] Lakshmanan; K.; et al. (2019). "Feast: A Feature Store for Machine Learning." arXiv preprint; arXiv:1909.03299.
[5] Neha; S.; et al. (2020). "Open Policy Agent: Unified Policy Enforcement Across the Cloud Native Stack." Proceedings of the 2020 ACM SIGCOMM Workshop on Hot Topics in Networking; 7.
[6] Zhou; X.; et al. (2021). "Multi-Tenant Data Lakes for Machine Learning: Architecture and Compliance." IEEE Transactions on Cloud Computing; 9(3); 1227-1239. https://doi.org/10.1109/TCC.2021.3057643
[7] Xu; X.; et al. (2018). "Secure and Efficient Data Sharing in Multi-Tenant Cloud Environments." Journal of Network and Computer Applications; 113; 15-28.
[8] Zhang; Y.; et al. (2020). "Real-Time Fraud Detection at Scale: Architectures and Techniques." IEEE Big Data Conference Proceedings; 411-420.
[9] O’Reilly; T.; et al. (2019). "Designing Cloud-Native Multi-Tenant Architectures." O’Reilly Media.
[10] Sabelfeld; A.; & Myers; A. C. (2003). "Language-based Information-flow Security." IEEE Journal on Selected Areas in Communications; 21(1); 5–19.
[11] Shafique; K.; et al. (2022). "Policy-as-Code for Dynamic Access Control in Cloud Platforms." Proceedings of the 27th ACM Symposium on Access Control Models and Technologies; 145-156.
[12] Chen; R.; & Zhao; J. (2021). "Cross-Cloud Data Governance and Compliance: Challenges and Solutions." IEEE Cloud Computing; 8(5); 35-43.
[13] GDPR.eu. (2020). "General Data Protection Regulation (GDPR) Compliance Guidelines." https://gdpr.eu/
[14] HIPAA Journal. (2022). "HIPAA Compliance and Cloud Computing." https://www.hipaajournal.com/hipaa-compliance-cloud-computing/
Downloads
How to Cite
[1]P. B. Hansen and B. Diderichsen, “Multi-Tenant Data Architecture for Real-Time AI Workloads with Role-Based Compliance Enforcement”, IJDEIC, vol. 6, no. 2, pp. 01–12, Oct. 2023, doi: 10.67228/30715717/IJDEIC-2023PII3A9M.