Engineering Data for Explainable AI in Financial Forecasting Systems

  • Authors

    • Dr. Fatima Zahra El Idrissi Department of Business Studies, Atlas Valley University, Fez, Morocco. Author
    • Marta Silva Department of Business Studies, Atlas Valley University, Fez, Morocco. Author

    DOI:

    https://doi.org/10.67228/30713315/IJAIDT-2019PII2P24

    Published 12-05-2019

  • Data Engineering, Explainable Ai (Xai), Financial Forecasting, Feature Lineage, Shap, Stock Market Prediction, Transparent Ai, Metadata Management, Regulatory Compliance, Data Pipeline

    Issue

    Vol. 2 No. 2 (2019)

    Section

    Articles

    How to Cite

    [1]
    F. Z. El Idrissi and M. Silva, “Engineering Data for Explainable AI in Financial Forecasting Systems”, IJAIDT, vol. 2, no. 2, pp. 01–14, Dec. 2019, doi: 10.67228/30713315/IJAIDT-2019PII2P24.

  • Abstract

    In high-stakes financial forecasting systems, the demand for both predictive accuracy and interpretability has elevated the role of Explainable AI (XAI). However, achieving meaningful explainability requires more than just applying model-agnostic techniques it demands a robust data engineering foundation that ensures transparency, traceability, and consistency across the data lifecycle. This paper presents a novel data engineering framework tailored for explainable AI applications in financial forecasting. We explore how pipeline design, feature provenance, and metadata management influence the quality and reliability of XAI outputs. Through a case study on stock market prediction using SHAP-enabled XGBoost models, we demonstrate how engineered data pipelines contribute to regulatory compliance, model transparency, and trustworthiness. Our findings suggest that data engineering is not just a support mechanism but a critical enabler of explainability in AI-driven financial decision-making systems.

  • References

    [1] Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30.

    [2] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). "Why should I trust you?": Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD, 1135–1144.

    [3] Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.

    [4] Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019). Interpretable machine learning: Definitions, methods, and applications. PNAS, 116(44), 22071–22080.

    [5] Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 1–42.

    [6] Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.

    [7] Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015). ModelTracker: Redesigning performance analysis tools for machine learning. CHI, 337–346.

    [8] Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society, 4(2), 2053951717743530.

    [9] Breck, E., Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2019). Data validation for machine learning. Proceedings of SysML Conference.

  • Downloads