AI keeps fusion plasma in check
A reinforcement-learning algorithm has managed to manipulate and control a high-temperature plasma in a tokamak. Edwin Cartlidge reports
Artificial intelligence (AI) can help control the delicate process of confining an ultrahot plasma within a fusion reactor. That’s according to a collaboration between physicists and computer scientists, who used a technique known as reinforcement learning to operate the magnetic coils in a tokamak. The work could enable more speedy development of reactors with novel geometries (Nature 602 414).
Tokamaks are doughnut-shaped chambers designed to produce energy by fusing light nuclei in the form of a plasma. The nuclei are heated to hundreds of millions of degrees to overcome their mutual repulsion, while the plasma is held in place using the fields from a series of magnetic coils. Those fields keep the plasma away from the chamber’s walls, where it would otherwise lose heat and damage the tokamak.
One aim of fusion scientists is to understand how the spatial distribution of a plasma in the tokamak chamber influences its stability and spatial confinement. This task is complicated by the need to design a new feedback scheme for each configuration so that the magnets can be tuned in response to the plasma’s highly nonlinear behaviour. The design process usually involves calculating an initial set of coil currents and voltages, then using a combination of plasma-reconstruction algorithms and feedback controllers to adjust the voltages – and with them the magnetic field. The result is effective control over a plasma’s vertical and radial positions as well as its current, but only via significant effort.
Optimum outputs
In the latest work, scientists from Google’s London-based DeepMind subsidiary and the École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland swapped these multiple feedback controllers for a single controller based on reinforcement learning. This algorithm is a form of machine learning that is designed to arrive at an optimum set of output values by adjusting the weights on its nodes in a step-like fashion.
In this case, the algorithmic “agent” receives measured and target values of a long list of plasma parameters – including numerous spatial dimensions, electrical currents and magnetic fluxes within the tokamak chamber. After processing these inputs using several layers of nodes, it issues outputs corresponding to the voltage levels on each of the magnets. Once the magnets are adjusted and the new feedback from sensors is received the cycle starts again – being repeated some 10,000 times a second.
This process relies on a computational object known as a reward function, which compares the measured and target values of the plasma parameters at each step and combines the disparity among all parameters into a single value. Its value determines how great a “reward” the agent receives for getting closer to the desired values. Conversely, if the plasma crashes into the chamber wall, the agent receives a penalty and the process stops.
Working out this reward function is the first stage of setting up a new reinforcement-learning-based controller, since each function corresponds to a specific plasma configuration. Next, the agent is trained by exposing it to input data from a simulated tokamak, which must realistically describe the plasma’s changing shape and current while limiting its computational demand to keep the learning process manageably short. Finally, the agent’s optimized control scheme is tested on a real tokamak.
To carry out the experimental stage of the work, the collaboration turned to the EPFL’s Variable-Configuration Tokamak. Initially, members of the team used traditional feedback controllers to create and maintain the plasma in an initial state. At a certain pre-agreed time, they switched to their own control scheme, adjusting 19 separate magnetic coils to tune the plasma so that it ended up with the correct shape and current while keeping the neural network’s weights fixed.
The researchers then showed they could keep control over the plasma’s current and shape – with deviations of no more than a few percent from the intended values – while making the kind of changes needed for a full plasma discharge. They were also able to manipulate the plasma into a variety of shapes, including one similar to that proposed for the ITER reactor being built in France, and another known as a snowflake configuration that helps to spread a tokamak’s heat and particle exhaust over a larger surface. In addition, they showed it was possible to set up two separate plasmas within the same tokamak, which they say is a first step to study more advanced plasma configurations.
The researchers say that their new AI-based scheme could improve tokamak performance, with its open-ended nature perhaps allowing power output to be maximized. The technology might also lead to new reactor designs by allowing the joint optimization of several device parameters – including plasma shape, wall design and heat load.
Other scientists warn that the simulator used to train the AI agent will have to be improved before the technique is more widely adopted. According to Karel van de Plassche of the Dutch Institute for Fundamental Energy Research, “essential components” of more accurate simulation will include detailed physics on turbulence and magnetohydrodynamics. It is also not clear how the new approach can be applied to tokamaks based on superconductors rather than copper coils, given that the former introduce time delays when regulating current.