Reinforcement Learning for Smart Grid Energy Optimization

  • Authors

    • Dr. Fatima Zahra El Idrissi Department of Business Studies, Atlas Valley University, Fez, Morocco. Author

    DOI:

    https://doi.org/10.67228/30713498/IJADSMC-2021PI1C4Q

    Published 06-04-2021

  • Smart Grid, Reinforcement Learning, Energy Optimization, Demand Response, Energy Management Systems, Deep Learning

    Issue

    Vol. 4 No. 1 (2021)

    Section

    Articles

    How to Cite

    [1]
    F. Z. El Idrissi, “Reinforcement Learning for Smart Grid Energy Optimization”, IJADSMC, vol. 4, no. 1, pp. 01–14, Jun. 2021, doi: 10.67228/30713498/IJADSMC-2021PI1C4Q.

  • Abstract

    Frequent network of renewable energy sources, electric cars, and distributed generation stations has changed traditional power systems into complicated smart grids. This change puts in place considerable uncertainty, non-linear and dynamic decision-making problems regarding energy management. Conventional optimization methods are usually unable to adapt effectively to the stochastic and time sensitive nature of contemporary smart grids. A recent development in machine learning has been presented as a means of solving these problems by use or Reinforcement Learning (RL), a branch of machine learning, which allows intelligent agents to acquire an optimal control policy by interacting with the environment. The paper will be a detailed report on the implementation of reinforcement learning to solve smart grid optimized energy. The framework proposed is based on the demand-side control, scheduling of energy storage and integration of renewable energy to reduce the operational cost without affecting the grid stability and reliability. The different RL paradigms such as Q-learning, Deep Q-networks (DQN) as well as Policy Gradients are discussed in their applications in the context of the smart grid. An elaborated methodology is constructed, with its system modelling, the design of state space, design of reward functions, and processes of training. The simulated experiments prove that the RL-based management strategy is much more effective in terms of minimization of costs and peak loads and its use of renewable energy sources in comparison with traditional rule-based and optimization-based strategies. The findings indicate the versatility and scability of reinforcement learning techniques in complex power system settings. The study concludes that reinforcement learning will be a highly robust and versatile solution to next-generation optimization of cyber grids with regard to data-based and autonomous, data-based grid management systems.

  • References

    [1] Wood, A. J., & Wollenberg, B. F., Power Generation, Operation, and Control, 2nd ed., Wiley, 1996.

    (Traditional optimization techniques in power systems)

    [2] Conejo, A. J., Carrion, M., & Morales, J. M., Decision Making Under Uncertainty in Electricity Markets, Springer, 2010.

    (Mathematical optimization under uncertainty)

    [3] Momoh, J. A., Smart Grid: Fundamentals of Design and Analysis, Wiley-IEEE Press, 2012.

    (Smart grid optimization foundations)

    [4] Abido, M. A., “Multiobjective evolutionary algorithms for electric power dispatch problem,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 3, pp. 315–329, 2006. (Genetic algorithms in energy optimization)

    [5] Kennedy, J., & Eberhart, R., “Particle swarm optimization,” Proceedings of IEEE ICNN, pp. 1942–1948, 1995.

    (Particle swarm optimization)

    [6] Dorigo, M., & Gambardella, L. M., “Ant colony system: A cooperative learning approach,” IEEE Transactions on Evolutionary Computation, vol. 1, no. 1, pp. 53–66, 1997. (Ant colony optimization)

    [7] Hatziargyriou, N., et al., “Microgrids,” IEEE Power & Energy Magazine, vol. 5, no. 4, pp. 78–94, 2007. (Microgrid control and optimization)

    [8] Watkins, C. J. C. H., & Dayan, P., “Q-learning,” Machine Learning, vol. 8, pp. 279–292, 1992. (Q-learning fundamentals)

    [9] Sutton, R. S., & Barto, A. G., Reinforcement Learning: An Introduction, 2nd ed., MIT Press, 2018. (Reinforcement learning theory)

    [10] Pipattanasomporn, M., Kuzlu, M., & Rahman, S., “An algorithm for intelligent home energy management,” IEEE Transactions on Smart Grid, vol. 2, no. 1, pp. 92–99, 2011. (RL for demand response)

    [11] Glavic, M., “Reinforcement learning for electric power system decision and control,” European Journal of Operational Research, vol. 241, no. 3, pp. 646–661, 2015. (RL applications in power systems)

    [12] Mnih, V., et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015.

    (Deep Q-Networks)

    [13] Zhang, Y., Wang, J., & Wang, X., “Review on probabilistic forecasting of wind power generation,” Renewable and Sustainable Energy Reviews, vol. 32, pp. 255–270, 2014. (Uncertainty in smart grids)

    [14] Francois-Lavet, V., et al., “Deep reinforcement learning solutions for energy microgrids management,” European Workshop on Reinforcement Learning, 2016. (DRL for microgrid energy management)

    [15] Wei, T., Zhu, Q., & Li, N., “Deep reinforcement learning for building HVAC control,” Proceedings of the 54th Annual Allerton Conference, pp. 1078–1085, 2016. (Actor–Critic and DRL in energy systems)

  • Downloads