LAUSR

Abstract This study presents a novel approach called scenario-based Fitted Q-Iteration (sFQI) for controlling water reservoir systems under climate uncertainty. In these problems, robust control frameworks, based on worst-case realization, are usually adopted. Yet, these might be overly conservative. In this paper, we use sFQI to design adaptive control policies by enlarging the state space to include the space of the uncertain system’s parameters. This allows obtaining a control policy for any scenario in the uncertainty set with a single learning process. The method is demonstrated on a simplified model of the Lake Como system, a regulated lake operated for ensuring reliable water supply to downstream users. Numerical results show that the sFQI algorithm successfully identifies adaptive solutions to operate the system under different inflow scenarios, which outperform the control policy designed under historical conditions. Moreover, the sFQI policy generalizes over inflow scenarios not directly experienced during the policy design, thus alleviating the risk of mis-adaptation, namely the design of a solution fully adapted to a scenario that is different from the one that will actually realize.