Utilizing reinforcement learning tools for real-time optimization of aging gas turbines

Although gas turbines are built to last, there is no hiding the fact that as they age their performance deteriorates causing a loss of power. To date, this has been managed through traditional maintenance strategies and manual adjustments, but by combining digital twin technology and artificial intelligence (AI), Siemens Energy has made it possible to overcome this age-related performance loss in real-time.

This novel solution was implemented for the first time in a control system upgrade in a power station of Dubai Electricity and Water Authority (DEWA) that has seen the power output of the turbines increase by up to 3.5 MW each, with a reduction in NOx emissions by as much as 10%. Five gas turbines in DEWA’s Jebel Ali’s M Station, which is the biggest power station unit in the United Arab Emirates, were equipped with this controller. The solution at Jebel Ali consists of two separate modules. One focuses on the gas turbine performance while the other one focuses on the emissions and combustion stability. The first one is called the performance module and the second one is the stability and emissions module.

The key to controlling the complex operations of a gas turbine is a digital twin in combination with AI. At its research and development centers in Germany, Siemens Energy has been investigating the advantages of AI in controlling complex systems from wind turbines to traffic management systems. The work on how to utilize AI, neural networks and Bayesian networks for gas turbines began way back in 1996 with work on the 4000 series of turbines. Whatever the application, AI relies on data to deliver accurate results. When a wealth of reliable data is available, it is a relatively simple undertaking to apply AI, however when the data available is patchy then it becomes more interesting.

Feeding data for artificial intelligence calculations

When it came to applying AI to improve gas turbine operation (reducing emissions and improving stability), the data available was less than usually required and this was compounded by intricacies of the gas combustion. In the process, the concoction of gas and air travels from the valves into the combustion chamber, where it is ignited with temperatures climbing as high as 1,600oC. There are myriads of parameters that determine the performance of a gas turbine such as how the gas burns, whether powerful combustion dynamic occurs, how much nitrogen oxide is created and how long a gas turbine can ultimately stay in operation. The quality of the gas plays a role as well, just as much as the outside temperature and the required combustion performance.

To enable the program to learn to control the unit during all possible situations by employing the amount of data generally used for reinforcement learning at the time, a turbine would have had to run around 100 years to supply the necessary information. Faced with this problem, the Siemens Energy team pioneered a data-efficient method for reinforcement learning. To ensure that a gas turbine runs optimally, you must always search for a balance in which undesired effects in terms of combustion dynamics and emissions are kept as low as possible. If you improve one variable, you will worsen a different one. Artificial intelligence knows how to find the sweet spot.

Strictly speaking, this is a particular type of AI known as ‘reinforcement learning. In this case, the algorithms imitate a learning behavior that is found in activities like children who are learning a new game by trial and error and continuously improving their playing strategy. By feeding an AI system with actual measured operating data from the turbine and the temperature readings supplied by the digital twin, the system can develop control strategies that were previously unknown and are better than the other traditional, model-based control solutions available on the market.

One important aspect of the solution is the ability that allows the algorithms to be structured so they can develop their strategies using far less prior empirical knowledge than was previously needed. Without this step, which Siemens Energy has protected with numerous patents, AI could not have formed part of the controller for DEWA. If we could not incorporate AI, the system would have been lacking an important element needed to automatically optimize engine operation in regard to emissions and combustion stability. The interaction between AI and the digital twin does not just guarantee more efficient combustion with lower emissions, it also monitors the performance to ensure that adjustments to offset the aging process never jeopardizes combustion stability.

Excellent results at Jebel Ali

At the Jebel Ali M power station each of the plants’ five SGT5-4000F gas turbines s an intelligent controller that provides power plant operators with a complete and continuous overview and scenario-based assessment of operations. Soon, the L station will also be equipped with the intelligent controller, with the station operated by an upgraded SPPA-T3000 control system. The latest system delivers an intuitive and safe operation solution and is adaptable to meet the individual company requirements during daily routines as well as during critical situations. This will improve operator effectiveness and efficiency – and directly affects the availability of the entire plant’s operations.

The digital twin of intelligent controller works by accurately estimating the firing temperature in real time. The aim is to compensate the power degradation (performance recovery) in combination with a continuous real-time optimization of emissions and combustor dynamics. The use of AI and machine learning allows better combustion control. It also minimizes the effect of ambient conditions by supplementing the controls with a trained reinforcement learning control policy together with a direct engine temperature control method. Reinforcement learning is essentially an AI discipline in the sense that it tries to realize content that normally can only be realized by utilizing human intelligence. The difference to other Machine Learning Methods is that no explicit target in the sense of “what to do in which situation” exists. Instead, the optimal control strategy is learned out of the operational data of the turbine, based on a reward or “cost” information. So, we are gaining an understanding from data and act based on this understanding in the manner that a skilled human would do.

Rationale behind the upgrade

The project at Jebel Ali began in November 2017 shortly after Siemens Energy had completed a major thermodynamic examination of the turbines to counteract age-related performance losses. This process was initially a manual process with thermodynamic specialists analyzing the data to calculate the compensation factors. The results were excellent, although time-consuming, so the decision was made to attempt to automate the process.

The R&D team in Mülheim an der Ruhr and Munich soon discovered an automated solution already to hand that had been developed by a team from Siemens Energy’s Orlando operations that could calculate a turbine’s operating status. This solution was based on a digital twin that utilized readings from the turbine and its environment to produce an accurate physical simulation of the turbine’s thermodynamic behavior.

The digital twin within the new controller makes computations to provide values at a level of quality that was not previously available, especially for the turbine inlet temperature of the gas, which is a key parameter for turbine control. Not only does the simulation enable the aging performance of the turbine to be accurately determined, but because the temperature readings are available in real time, age-related performance losses can now also be offset in real-time. The system would have been improved even more if we could measure the temperature directly, but we do not have measuring capabilities that can function in the high inlet temperatures of these gas turbines.

Such has been the success of this collaboration that we are already working with DEWA to advance further digital innovations. Our efforts are aimed at extending the control upgrade to a broader range of components of Jebel Ali M Station in the next phase. The fruits of this collaboration will mean that not only will the gas turbines benefit, but we will also enhance the steam production process that is used to generate electricity and to recover drinking water from seawater. The goal is to optimize the overall operation automatically.