Guest essay by Eric Worrall
Scientists claim deep mind’s reinforcement learning AI has demonstrated superior control of fusion plasmas. But they may also have inadvertently revealed a critical weakness in their approach.
DeepMind Has Trained an AI to Control Nuclear Fusion
The Google-backed firm taught a reinforcement learning algorithm to control the fiery plasma inside a tokamak nuclear fusion reactor.
AMIT KATWALASCIENCE FEB 16, 2022 11:00 AM
THE INSIDE OF a tokamak—the doughnut-shaped vessel designed to contain a nuclear fusion reaction—presents a special kind of chaos. Hydrogen atoms are smashed together at unfathomably high temperatures, creating a whirling, roiling plasma that’s hotter than the surface of the sun. Finding smart ways to control and confine that plasma will be key to unlocking the potential of nuclear fusion, which has been mooted as the clean energy source of the future for decades. At this point, the science underlying fusion seems sound, so what remains is an engineering challenge. “We need to be able to heat this matter up and hold it together for long enough for us to take energy out of it,” says Ambrogio Fasoli, director of the Swiss Plasma Center at École Polytechnique Fédérale de Lausanne in Switzerland.
That’s where DeepMind comes in. The artificial intelligence firm, backed by Google parent company Alphabet, has previously turned its hand to video games and proteinfolding, and has been working on a joint research project with the Swiss Plasma Center to develop an AI for controlling a nuclear fusion reaction.
…
DeepMind has developed an AI that can control the plasma autonomously. A paperpublished in the journal Nature describes how researchers from the two groups taught a deep reinforcement learning system to control the 19 magnetic coils inside TCV, the variable-configuration tokamak at the Swiss Plasma Center, which is used to carry out research that will inform the design of bigger fusion reactors in the future. “AI, and specifically reinforcement learning, is particularly well suited to the complex problems presented by controlling plasma in a tokamak,” says Martin Riedmiller, control team lead at DeepMind.
…
Read more: https://www.wired.com/story/deepmind-ai-nuclear-fusion/
The abstract of the paper;
Magnetic control of tokamak plasmas through deep reinforcement learning
Jonas Degrave, Federico Felici, Martin Riedmiller
Show authors
Abstract
Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A core challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires high-dimensional, high-frequency, closed-loop control using magnetic actuator coils, further complicated by the diverse requirements across a wide range of plasma configurations. In this work, we introduce a previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils. This architecture meets control objectives specified at a high level, at the same time satisfying physical and operational constraints. This approach has unprecedented flexibility and generality in problem specification and yields a notable reduction in design effort to produce new plasma configurations. We successfully produce and control a diverse set of plasma configurations on the Tokamak à Configuration Variable1,2, including elongated, conventional shapes, as well as advanced configurations, such as negative triangularity and ‘snowflake’ configurations. Our approach achieves accurate tracking of the location, current and shape for these configurations. We also demonstrate sustained ‘droplets’ on TCV, in which two separate plasmas are maintained simultaneously within the vessel. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.
Read more: https://www.nature.com/articles/s41586-021-04301-9
What is this critical weakness I mentioned?
Deep mind is a tremendously powerful AI. But by virtue of its architecture the Deep Mind system is stateless. It has no memory of the past.
Deep Mind accepts an input, like the current state of the reactor. It processes the input. Then it provides an output, and forgets everything it has just done. Every day is the first day for Deep Mind.
This is fine for playing a game of chess, because it is perfectly possible to evaluate the current state of the chess board, and generate a technically perfect move. A chess AI does not have to know the past, all it has to do is evaluate the present, and decide the best move based on the current layout of the pieces.
This living in the moment model starts to break down when you attempt to control real world processes.
Imagine trying to train an AI to catch a baseball, by showing the robot pictures of the field. You can’t catch a ball by moving the robot hand to where the ball is, you have to intercept the ball in flight, by predicting where the ball will be in the time it takes to move the robot’s hand to the right position. This requires not just knowledge of where the ball is now, but the ability to evaluate the flight of the ball, the velocity and direction it is moving. This is knowledge which the robot can only obtain by remembering where the ball was, and how quickly and from which direction it moved towards its current position.
Only knowledge of the past can give the robot the ability to truly manage a real world process.
I’m not dissing what Google and the Swiss Plasma Center achieved – they demonstrated AIs have a role in managing fusion plasmas. What I am questioning is whether the deep mind reinforcement learning architecture is the best solution.
Because there is another class of AI architectures which can learn by their mistakes just like Deep Mind, but which can also evolve to have memory of the past. For example, NEAT, or Neuroevolution of augmenting topologies.
Unlike Deep Mind style architectures, which have a finite, well defined path from stimuli to response, after which the neural network forgets everything until presented with a new stimulus, NEAT systems are messy. They evolve their own network of connections, even adding new neurones and layers if needed, which can include connections which flow backwards. A signal can enter a NEAT network, and kind of bounce around, affecting the process, never truly being forgotten until the information is no longer relevant. Unlike Deep Mind, NEAT networks can respond differently to the same stimuli, depending on the NEAT network’s memory of the past. NEAT can catch the baseball.
But NEAT style architectures do not fit well with Google’s AI business model.
Stateless neural net AI architectures are much easier to manage from a business perspective, they allow Google to create vast arrays of independent computers which are all perfect clones of each other, and assign the next inbound processing request, the next set of plasma sensor reading, to any computer in their array.
NEAT systems by contrast need a dedicated computer. If the specific NEAT solution has a feedback loop, the current state matters. The current state of the NEAT network cannot be purged after the task is complete, and rehydrated back to initial state on whatever computer is available, it has to remember what happened before.
In Google’s world, this would be an absolute nightmare – either each client would need their own dedicated computer. This would completely mess up their business model, because if the experiment ends, the client won’t necessarily tell Google they no longer need the dedicated computer. Assigning dedicated computers ties up resources, depleting the pool of computers available for other clients.
Either that or the current state of the NEAT network would have to be saved somewhere, and passed around Google’s network – which would require an enormous increase in storage and network capacity, over what is required for Google’s current stateless business model.
It is going to be very interesting to see how far Google can carry their stateless Deep Mind model when it comes to process control. I’m impressed they got it to work at all. Perhaps they simulated holding state somehow, by asking the client to remember the previous result, and pass those results back through their network.
Or perhaps plasmas are almost like a chess board – most of the time, the current state of the plasma is enough information to calculate the next step required to maintain control.
But the Google Deep Mind experiment was not a complete success. I suspect that last missing piece of the puzzle will be to reject Deep Mind’s stateless architecture, and embrace a neural network architecture which can catch the baseball.
Simulated state, in which a human tries to guess what memory of the past is required to catch the baseball, cannot match the flexibility of a NEAT style neural net architecture which can evolve its own memory of the past, which is capable of making its own mind up about what state it has to keep to perform its task, and how long that state remains relevant.
Has there been a technology developed in the modern era that was commercially successful that took longer than 40 years to go from concept to market? It seems that something that takes longer will be impractical or uneconomic in real world conditions.
Exactly, it doesn’t matter which AI design is used, fusion power generation will always be ten years away. At least Google is wasting their own money not the taxpayer’s.
I quoted a French Nobel physics prize winner in my ‘Going Nuclear’ essay cited below. He said (I am paraphrasing from memory rather than looking it up in the book):
” The idea of fusion is pretty. We put the Sun in a box. The problem is, we do not know how to build the box.”
” We put the Sun in a box. ”
The real problem is that in fact, we don’t do even that.
The Sunny Boy is, due to its dimensions and the resulting pressure / temperature mix inside of it, able to perform H+H fusion.
We aren’t.
Due to the Lawson criterion defining requirements for minimal temperature, confinement time and density of the plasma, the D+T fusion is the only one currently possible to use (D+D for example needs five times higher temperatures).
This means that while we have an abundance of deuterium, tritium does not occur naturally (about 5kg of it lurks over the oceans) and therefore has to be bred from lithium.
Means the same amount of waste as for 4G reactors breeding U233 out of Th232, or Pu239 out of U238.
Moreover, D+T generates a neutron flux with extremely high energy, something no one seems to care about.
The very best is for the end: tritium probably is the most volatile element on Earth. It even escapes from special steel/zirconium alloys.
I’ll bet if you look hard enough there is some taxpayer money involved.
It’s well known that google, FB, etc received seed money and technical support from the NSA/CIA/etc. They are all very well connected as they say…
The only technology that has a similar unfruitful development period is caseless ammunition. The Germans had something that almost worked, but was never adopted.
I suspect the reasons involve the safety of bulk ammunition,
One of the three new designs for the new 6.7mm US assault rifle (replacing 5.56 because of body armour advances) is caseless. Very interesting, with a lot of advantages beyond weight saving. Lots been written about it last couple of years.
There was a caseless 22 rifle sold in the late 60’s an early 70’s.
The real reason for replacing the 5.56 is because enemies, e.g., Taliban, stay outside of the 5.56’s effective range. Forcing troops to designate riflemen to carry and use 7.62 NATO rifles that have the power to engage distant enemies.
The new 6.7mm (.264) utilizes a bullet with better aerodynamics that enable lethal shots at distance. Depending on the rifle/barrel up to 1,000 meters.
Nor is the new ammunition “caseless”. It is only caseless in that it uses a plastic case instead of brass.
“A key advantage of the 6.7x35mm round is its unusual design: it is a cased telescoping round, meaning that the actual projectile is set within the case, with the powder all around it. This creates a much shorter, more volume efficient round, one that is ideal for usage with the AVIR’s 50-round helical magazine.”
The new rifles are supposed to cycle extremely fast. Fast enough that 2 or 3 round bursts form tight groups, allowing successive impacts to the same point on ceramic armor which causes failure of the armor.
“Taliban, stay outside of the 5.56’s effective range”
Not very sporting of them!
Depends on who determines what is sporting. If I was an Afghanistan, I’d watch from a distant mountaintop.
Even then, I’d spook easily, as Afghanistan is where most of the extremely long rang sniper kills have taken place.
That is, kills around a mile distant or further, using .50BMG, .338 Lapua, .300 Winchester Magnum and the .308 (7.62) M118 Long Range cartridge.
The latter cartridge is the round deployed to designated marksman.
Don’t we have an over-the-horizon mantra?
mRNA vaccines are just one example. The idea of delivering mRNA into human cells for medical purposes goes back to the 70s. It’s all a question of money, and the cost of fusion research is massive. Nixon began putting money into fusion research in the US but since then government funding has steadily declined. Public funded research is increasingly captured by rent seekers, and bureaucrats always have a strong incentive to dole out money to a lot of small projects where they can claim a high success rate and more “clients” that will support an increasing administrative structure. It’s only now that the fusion research has progressed to the point that private industry is willing to risk large amounts of money.
We are still waiting on a reliable system that will prevent everyone except the owner of a firearm from firing it. Then there is the issue of cost.
What that going to fix, most criminal homicides are by the often illegal firearm owner.
That would prevent a soldier with a broken weapon to use that of a fallen companion …
That would prevent an owner from selling his gun to someone else.
If there is a way to transfer ownership, then criminals who steal guns would be able to make use of that method to register stolen guns to themselves.
A proven waste of research.
Actually, I think it would be really easy to build a reliable system that prevented everyone but the owner of a firearm from firing it under all common circumstances, and it wouldn’t cost the moon. But is there a market, really?
Imagine a bracelet, worn on your gun hand, with a unique code that is pingable at very short range. This technology all exists — it is used to get me into the physics building parking lot, for example, without having to do anything beyond hang a special card on the mirror. The bracelet itself wouldn’t even need a battery.
In the gun, the safety has a single modification. Internal electronics — and yeah, the gun would need periodic charging, although one could PROBABLY put solar cells on the stock or butt and develop the needed charge without having to actually plug the gun in if it is exposed to enough daylight every day — pings the bracelet with a very short range, very weak pulse WHEN you try to release the safety, and only does so if it receives the correct response. Hell, you could probably get by with a ring on a finger of your gun hand. Or two rings, one on each hand, if you want to be able to shoot ambidextrously.
Mass produced, the technology required would probably cost less than a fancy pistol safe, maybe twice the cost of existing combination locks. Only you can take off the safety. If you want to loan the gun to somebody, you loan them both the gun and the bracelet (provide one bracelet and one ring with the gun — this would really need to be built INTO the gun, note well, and this wouldn’t necessarily work of a Nefarious Person specifically targeted YOUR gun with high tech tools, but it sure as hell would keep cop guns from working if it was picked up by a bad guy in a fight or your own gun working if your two year old grandson picked up your pistol and pointed at your three year old granddaughter, if you are dumb enough to leave it out so that this is even possible).
Problem solved. Next problem?
Similarly, if someone were to break into your house, and responding slowly after being woken from a deep sleep, you get shot. Your device would keep your wife from using your gun to defend herself.
To prevent your child from using your gun, get a .45 Auto. As an experiment, I found that an adolescent girl did not have enough hand strength to pull the slide back and charge the chamber. There is no way even a younger boy could do it.
Television was such a technology. This was because in addition to developing the TV camera and receiver, a network of customers and whole new business had to be developed. TV was invented in 1926!
And commercial TVs were available in the late 1940s. Well within 40 years.
Commercially successful after 40 years? Frequency-hopping spread spectrum?
When was it first conceived? And when did anyone start a serious effort to develop it?
G’Day Guest;
“When was it first conceived?”
You might research the following names:
Paul Nipkow – first “TV” patent – 1884
Boris Rosing
Vladimir Zworykin – TV receiver 1929, TV camera 1931.
Added: 1884? First came tele-graph – then tele-phone, then tele-vision – all transmitted by wire. 1884 was the first successful transmission of an image by wire.
https://en.wikipedia.org/wiki/Frequency-hopping_spread_spectrum
Described in 1908… Wi-Fi used it in 1997, although some other commercial devices used it in the 900MHz band.
The actress Hedy Lamarr is listed on a June 1941 patent for radio frequency hopping (in this case for use in torpedoes).
Fans of classic film know the name Hedy Lamarr for her memorable performances in Algiers,H.M. Pulham Esq., and Ziegfeld Girl, among others. But the actress also made a small contribution to wartime technology with her co-invention of an early form of spread spectrum communication technology, in which a noise-like signal is transmitted on a much-larger bandwidth than the frequency of the original information. It is a staple today in modern wireless communications.
LINK
Deep Mind is still going to have to use a greater amount of energy than that produced by the process if it is to control it.
You can’t control and direct a quantity of energy with a lesser amount of energy.
I would be more impressed if they could deal with that issue.
Then triode power tubes and Jfet transistors don’work? How about SCR’s?
Depends on circumstances. We control all the vast electrical energy on the grid using a very small amount of electricity. Because it is all confined inside conductors at high voltage and well behaved. Magnetically confined plasmas are NOT well behaved
Every hydraulic control system in the world disagrees.
You’re wrong about that. However, computation takes energy. In theory it’s quite possible that a chaotic system might require so much computation that it would use more energy than it could possibly generate.
Why, for instance has nobody created a working version of Maxwell’s Demon?
“Why, for instance has nobody created a working version of Maxwell’s Demon?
‘This would decrease the total entropy of the two gases, without applying any work, thereby violating the second law of thermodynamics. ”
Wrong, the demon is supplying the “work”.
Two reasons for no demon..
1/ The particles would simply ignore the gate. They’d give their energy to the dividing wall and it would hand it out on its other side.
(Like AOC, the corn-cob and the chain-link fence)
2/ I’d guess that the guy Heisenberg put the brakes on it via his Uncertainty Principle – in combination with Schrödinger’s cat.
You’d have to take the energy away from the particles to know how much they had and then give it back to them.
Then Carnot would get in the act telling you that in the absence of a thing/place at a temp of zero Kelvin, to do such a thing would always be a lossy business.
= exactly why the Green House Gas Effect, as commonly described, is thermodynamic nonsense.
IOW Maxwell’s Demon cannot work because we live in a “cooling universe”
Back in the days of hydraulic gold mining in California (circa 1860) a simple mechanical device called a “monitor” directed a stream of water 8″ in diameter with enough velocity that it went well over 100 yards. I have read a description of it relating that a large boulder dropped in the stream of water would travel almost the full distance of the stream before rolling off. It was controlled by one man with one hand.
Then there is the trigger on a firearm that takes only a couple of pounds of pressure to release the firing pin that detonates the primer, that ignites the powder in the cartridge case, and discharges a bullet with several hundred foot-pounds of energy.
Triggers have been used by mankind for most of man’s hunter-gatherer existence.
A trigger does not control the mechanism! It simply releases the latent energy allowing the mechanism to fully work. Whether that mechanism is a bent tree, leaning rock or a firearm.
How about the slave master with 30 slaves?
I note that commenters are missing the point by using illustrations involving mechanical processes.
As someone said:
” The idea of fusion is pretty. We put the Sun in a box. The problem is, we do not know how to build the box.”
His point is that the plasma required to create energy from fusion is so hot that it will break down any containment structure of a mechanical nature.
Thus it is proposed to use magnetic fields but the energy required to generate a sufficiently powerful magnetic field is necessarily greater than the energy contained in the plasma.
I would like to know how they can create such a magnetic field which works successfully whilst requiring less energy to create it than is contained in the plasma.
Only then would one get a usable surplus.
The magnetic field deal changes dramatically with superconductors. A new high power type tested last year can hold the amperage of a lightning bolt at 30 watts draw
Once a plasmas energy gain gets over 10 you start to approach commercial levels as at that point it is 2/3rds self heated. Based on the trend to date you can make a net energy reactor with the new magnets with a plasma gain of 13 and a total system gain of 3
Nonsense. You have to provide some numbers in order for your claim to be believable: 1) How much power does Deep Mind use? 2) How much power could fusion provide?
I don’t know (1), but I do know it has gone down by a factor of over 3 in the past 5 years.
Neither of us knows (2). It is currently negative, and if that continues to be the case, then obviously an LED light bulb would use more energy than a fusion reactor would provide, much less Deep Mind. But the question is whether this will always be the case. Predictions are difficult, especially about the future.
Sigh. This is simply not true. Look, all of the energy required by Deep Mind is to TRAIN THE NN IN THE FIRST PLACE. It’s a one-time investment. Once you have built the NN, it is just a fancy table of numbers, and can be computed amazingly quickly with a very modest (by comparison) computer. It won’t even be “on the books” of the energy balance equation — it’s a fixed part of the overhead, on the order of the cost of the electric lights, the heating and air conditioning, and the gasoline used by people driving to work at the power station every day. The scalable energy consumption is all in the giant, superpowerful magnets being used to control the plasma.
Besides, your basic premise — that you can’t control and direct a quantity of energy with a lesser amount of energy — is so incredibly false, with so many counterexamples, that it almost isn’t worth addressing. Turn on your lights much? A two year old child can do the work of pressing a trigger sufficiently far to release enough directed energy to punch a hole right through a human, one that her little finger could never, ever produce on its own. You steer a car, probably every day, with a tiny, tiny fraction of the energy being released, directed, and controlled in its engine.
I’m sure you are thinking of something like the first or second laws of thermodynamics, but please, in the current problem the THERMODYNAMIC losses are all associated with controlling the plasma itself and converting the output “raw” energy into useful electrical energy. “Break even” is when the combination of energy required to confine the plasma plus energy lost converting output energy to useable energy equals the energy produced, so that the system could just run forever as long as you supply it with fuel, ultimately turning the fuel into heat since none is left over for practical use. ITER is shooting for 10 to 1 past this on the confinement side, so that at least 90% of the energy released can be turned into usable electricity with only conversion inefficiencies (which you’ve got no matter what the fuel source). They’re looking to make gigawatts, sustained — the control computer probably won’t use more than kilowatts, sustained, totally negligible.
More interesting are some of the OTHER fusion approaches. Some of them are in principle capable of nearly “direct” conversion of a fraction of the output energy into electricity PLUS the heat engine conversion of the rest, and may actually beat the usual thermodynamic limits for a cyclic heat engine. I’m very interested in seeting what both the MIT crowd and Skunk Works comes up with — Skunk Works announced three years ago that they would release a working, scalable, fusion generator in five years, and while the do NOT advertise exactly what they are doing or their progress, again, they are pretty damned smart folks working with a corporate reputation on the line — I’m pretty sure they had model computations that produced a pretty wide window for eventual success to release such a specific timeline.,
It’s fine to be skeptical about people building fusion, but make sure that any argument that they WON’T is well-founded. Saying “nothing takes 40 years that is possible” isn’t even an argument, and ignores ALL of the specifics of fusion as a damned hard problem and why it couldn’t be done back when all of the computers in the world put together were slower in aggregate than your cell phone computer is today. Claiming that something will eat all of the output energy so none will be left over makes out all of the physicists involved to be complete idiots, and they just aren’t — they not only know the laws of thermodynamics, they can DERIVE them, they have TAUGHT them, they have USED THEIR PRINCIPLES IN COMPUTATIONS, and THEY UNDERSTAND THEM. What do you expect them to get out there and go “Doh! We forgot to include the energy used by Deep Mind! Holy Crap!”?
If it works fusion power could solve our energy problems forever.. or at least until the greens find a reason to hate it.
AOC says that AI taught her how to eat corn on a cob through a chain link fence, so she’s for it.
Easy. Turn the cob 90 degrees and shove it through.
They already hate it.
link
The solution to the world’s environmental problems is prosperity for everyone. The biggest despoilers of the environment are those who are desperate to put their next meal on the table.
Or those who are desperate to make sure the poor remain desperate to put their next meal on the table.
You can’t steal resources from a rich country without major effort but the poor country only takes the bribery of the few rich ones at the top of the food chain.
“You can’t steal resources from a rich country without major effort” Funny the US is a very rich country and yet we are unable to use the resources we have within our borders, since the uber rich have bribe our politicians and brainwash the masses not to use said resources.
The Chinese are burning USA coal…whilst we shut down our coal fired plants…Chens get the coal and the jobs, we get the shaft…no major effort required on the part of those running the heist
think greens and clearing forests for solar projects
Fission power used to produce electricity is the safest power and cheapest we have. Yet the world does not know that.
As Douglas Adams said:
“the secret is to bang the rocks together, guys.”
” … the safest power and cheapest we have. ”
Maybe you think about a simple detail.
Imagine the nuclear industry would be made responsible for any devastation of the environment around its plants – including a consequence of a natural disaster like earth quakes, tsunamis etc.
Do you think that even one electricity energy producer would spend even a minute in planning any nuclear plant anywhere?
How much and for how long time, do you think, will the Japanese population have to pay for the absolutely insane decision to build nuclear plants in a region like Japan?
Interesting point, Bindidon. One remembers the millions who died of radiation poisoning from Fukushima, and the millions who didn’t because 50 engineers died to shut down those reactors. The whole island now requires its topsoil to be replaced because nothing can grow on it now – plant or animal – much like the area around Chernobyl that is devoid of life of any kind, and where one scientist estimated that 400% of the population of the country died of radiation poisoning.
Or so I’ve heard from the MSM.
I can’t help but look at the small numbers of people who have died in coal energy-related accidents and wish we still were limited to that technology.
You’re right but I don’t think it will be the greens who will object to fusion but the followers of that old religion of ancient Egypt who worshipped Ra. I can just see it now, all those Sun worshipping activists dressed like extras from the film The Mummy trying to “free” the trapped Ra in the box and getting melted in the process.
Egyptian deities number far more than ra.
They’ll say that the workers at the fusion plant must represent all ethnicities and gender types.
Will all ethnicities have to have all gender types represented as well?
Fortunately they will always have 10 years to solve the problems.
They have though invented the world’s first “perpetual promotion machine.”
The Borg.
Independent Thought.
A very interesting dichotomy.
“The Borg.”
Generic Modulation platform… once knowledge gained… memory not needed much.
Sharing and exchange of performance, experience and knowledge, further boosts the platform through, at any given scale.
For example: QR code, the fastest, cheapest, easier, vastly widespread and the most secured authentication platform… or maybe not. 🙂
“vast arrays of independent computers which are all perfect clones of each other,”
Somehow that makes me think about the cells in my body.
🙂
cheers
whiten
Other computer examples in this theme are “H.A.L. 9000” from 2001 A Space Odyssey, and my favourite by Douglas Adams “Deep Thought” from Hitchhikers Guide.
Yes of course… 🙂
Following this theme;
Via sci-fi, is more like the Cube than the Bumblebee.
And
Via science fantasy, is more like Skynet than the Terminator robot.
But still, Dave in the end makes it…
😝
Thanks
cheers
Nuclear power from Molten Salts Reactors powered by thorium are available now. What good would fusion power be if costs $100/kwhour? Can a fusion reactor be made in a small modular form so that even a small town could afford it’s own electric power plant? I don’t think Scotty will ever give Cap’n Kirk all the power he wants.
Lockheed-Martin Skunk Works was working on it. However, they have been very quiet lately.
The Chinese will son having working models. Oh by the way what is wrong with the reactors used in out subs and aircraft carriers. They are small, safe and go from basically idle to full power in seconds? We have the solution, why are we not using it?
who are we?
Those reactors use highly enriched uranium, problematic in a civilian setting. And they are ‘small’, on order of 100MW equivalent, so not economically viable for grid applications. ‘Refueling’ a carrier at 20 years by replacing its either 2 or 4 reactors costs almost $2 billion.
There is something basic missing from this report. So, they can do neat shapes using AI. The critical question is for how long? Plasma pinches, touches the wall of the tokomac, and quenches. In less than a second. The theory behind ITER is simple, Make the tokomac big enough, and the confined plasma small enough, and you can sustain it long enough. I have my doubts, because the confining magnetic field drops off in 3D as a radius cubed function. So you get a little away from the superconducting magnets in the walls, you lose a lot of magnetic confinement force. Illustrated this for ITER in essay Going Nuclear in ebook Blowing Smoke,
Indeed. If this were a real thing, they would be announcing that they had created a functional fusion reactor.
That would conveniently make smaller reactors better in that sense. I think there could be considerable time between the concept and a working reality. Do they even have fast reacting magnetic fields now or is it more about brute force?
“Detect” the disturbance early enough and the change in the magnetic field would only need to be small but its potentially a very different way of doing containment and might need considerable rework of the design.
“magnetic field drops off in 3D as a radius cubed function.”
Depending on a geometry. Not true for a long coil.
Perhaps they’ll design a NEAT architecture that can “leave it at work” when it goes from one job to another like humans do when they go to their 2nd job…
That would be pretty amazing. Unfortunately even NEAT doesn’t know how to fix the catastrophic forgetting problem which plagues artificial neural networks. Artificial neural networks can learn how to do one job well, but they can’t learn how to do two jobs, if you teach the network how to do the second job, it forgets how to do the first job.
When I was trying to use this type of neural network for real-world applications, I encountered the same problem. My solution, which showed some promise, was to apply three input sets to the input of the network. Current state, state at t-1, and state at t-2. Each input cycle I would shift the inputs to the right and drop off the oldest input. This makes position, velocity, and acceleration available for the network to learn on and process. There is no reason to limit the time-shifted inputs to only 3 other than processing power and time. Adding t-4 gives change in acceleration, and t-5 gives rate of change in acceleration, etc.
So, in principle, it could still all be done with the feed-forward, multi-layer network shown in this article.
My concern is, what about processing delay due to the computers being off-site? How fast will the control system have to react to changes? Micro-seconds, nano-seconds, femto-seconds? At some point speed of propagation and switching delays in data transmission become imortant.
When you control a magnet you are controlling the field the magnet produces and that field is energy. To change the field in the magnet you have to move energy in or out of it. Scaling to a significant magnetic field is going to limit how nimble the magnet will be with reasonable power electronics.
Dont know ietr specifics but the TF coils in a fusion experiment I helped with years ago had 200MJ in each of 6 super conducting coils. Try moving that around in microseconds with all the parasitic magnetically coupled structures. (1 kWh = 3.6 MJ). In our case it took hours to get the coils to full current.
That makes sense. This is the first I’ve ever heard about AI controlling a powerful magnetic field, and I really was thinking more about the computing side of it. Still, I’d think that super-hot plasma reaching the outer wall of the container would be fairly catastrophic.
The density of the plasma is pretty low.
Processing delay could be an issue.
I agree creating artificial state by retaining previous inputs is a workaround, my point is NEAT is potentially more flexible, because it can decide what state and what dimension of state it wants to retain.
This is the first I’ve reard about NEAT, but from what little I’ve read so far, yes, NEAT could get very messy.
My prejudice is that for any real time control, the computer should be local and not multi-tasking any more than necessary. I’d want to know a lot about the system before I’d even consider using someone else’s computer for control of my process–whatever it is. Like I said, just my prejudice.
NEAT can get verbose, though arguably a verbose solution which works is acceptable.
For example the optimal XOR problem only takes 4 neurones to solve, but a NEAT solution can take 4 neurones, 6 neurones or 20 neurones.
Having said that NEAT rarely misses, it usually gets there in the end.
There have been attempts to prune NEAT networks as they evolve, by say introducing the occasional deleting a neurone and seeing what that does to the fitness of the network, but as you say it is frequently a little messy.
Dr. Otto Octavius has the solution!
“Deep Mind accepts an input, like the current state of the reactor. It processes the input. Then it provides an output, and forgets everything it has just done. Every day is the first day for Deep Mind.”
They must have taken their inspiration from the movie ” Groundhog Day”
Or, 50 First Dates.
Deep mind does understand what its just done. It uses Transformers. Even the simplest check from Eric would have shown he got the Deep Mind architecture wrong.
https://deepmind.com/blog/article/building-architectures-that-can-handle-the-worlds-data
As I’ve come to understand fusion reactors, the energy from the reaction is collected via hot neutrons coming away from the reaction core
Neutrons are far from what their name suggests, they are not only hideously radioactive, but are an embodiment of the Philosopher’s Stone.
i.e. They (will) change whatever material the energy collector is made of into some other material.
= some new material which, in absolute certainty, would not be in possession the sorts of properties you’d desire it to have.
(Is that where Plutonium comes from – Uranium 238 being bombarded by spare neutrons coming off a U235 fission reaction. Also at the core of the classic Hydrogen Bomb = a small fission bomb blasts some Hydrogen with a big mess of neutrons in order to ‘set off’ the Hydrogen)
So Mr Google Brain Box Hyper Intelligent, what are you going to make the reactor out of – what substance in this universe is immune from transmutation via neutron bombardment?
In fact, never mind a flaw in the computer, is that not how El Sol itself works and exposes an epic flaw in the whole idea of fusion reactors.
Ain’t that how it makes Helium and higher elements – exactly via neutron bombardment and why El Sol and all stars are, have to be = ‘big‘ or especially, very dense near their centres.
If stars weren’t ‘big’ and thus dense, the neutrons would escape before they decayed into protons & electrons and did their magic.
(You have the half-life of a neutron to do something with it, about 12 minutes I think)
Thus stars would never ‘burn’ or make any energy whatsoever and there’s be = No Universe. Apart from a vast and uniform cloud of protons and electrons
https://medium.com/fusion-energy/reactor-component-blanket-48dd47e309b5
Peta, you have beclowned yourself, again. The helium that results from fusion of hydrogen doesn’t come from neutron bombardment, it comes from ….well, …fusion of hydrogen. DUH. However, first generation fusion reactors won’t use hydrogen, they’ll use deuterium and tritium as they are the easiest elements to fuse. Since tritium doesn’t occur naturally, it will have to be formed by …well…neutron bombardment of lithium in the fusion blanket.
Just another post from you that would have benefited from doing two minutes of research. Are you ever going to learn?
I wish I knew where you come up with this nonsense. Not a single thing you wrote comes even close to being accurate.
eh??
WTF?
Neutrons they are not only hideously radioactive, (!)
( at the core of the classic Hydrogen Bomb = a small fission bomb blasts some Hydrogen with a big mess of neutrons in order to ‘set off’ the Hydrogen)
If you had read any physics at all, you would know the first hydrogen bombs used liquid deuterium, and then Teller-Ulam’s break through like Sakharov in Russia consisted of setting off the secondary by X ray compression, nothing to do with your so called “radioactive neutrons”.
AI has multiple ways of managing state. You really think deep mind is stateless??
No, there are ways state can be maintained. But as far as I know the deep mind engine itself is stateless. This means state has to be managed by likely suboptimal methods, such as feeding previous state back through the engine as part of the next set of inputs.
Deepmind is based on Transformer technology which is arguably the best way to understand a flow of information in AI today. Its extremely effective.
Modelling plasma states and corresponding magnetic corrections wouldn’t even need a stateful model. In principle It could be trained with nothing more than an objective goal of maximising plasma burn time by controlling the magnet field. In practice they’d find ways to maximise the effectiveness of the training without having to have thousands and thousands of runs.
Sounds like right after a big lunch in a warm conference room at a corporate training session…Deep Stateless
I don’t know about that, but deep state is mindless…
I been enjoying the comments and have learned much.
I’m not seeing the problem in this application with the AI not ‘remembering’ history.
I’m thinking of the old Pong video game. If you set the AI program’s task to just keep the Pong ball in the playing field, all it has to know is “Is Now” then “Adjust”.
The first problem I see with that is if the AI program can’t adjust fast enough. Then something will get loose from the fusion field, and someone will have a bad hair day or worse.
The second problem I see is if the AI loses track of “Is Now”. Then “Adjust” will be useless.
Now if the objective is for AI to ‘win’ at Pong, then you need memory of the past. The two Pong paddles are no longer working together, but in opposition. The AI only controls one paddle. It is going to get suckered here and there and so the program has to remember the times it got suckered to keep it from happening again. It also has to remember the opponent’s moves in order to itself, devise something to sucker the opponent in order to win.
At the moment, I’m more concerned about the rest of the kit than the AI.
“This requires not just knowledge of where the ball is now, but the ability to evaluate the flight of the ball, the velocity and direction it is moving. This is knowledge which the robot can only obtain by remembering where the ball was, and how quickly and from which direction it moved towards its current position.”
If you know a baseball’s position and velocity, plus aerodynamics, you can calculate a its future trajectory to as much precision as you want. Doesn’t matter if it was thrown or hit, has bounced, anything else.
Don’t you also need to know which way the wind is blowing and a few other things?
Funny deep mind is much of an AI, the Saturn 5, booster flight computer was much the same, it ask a simple question where should I be an than corrected an deviation from that. I may be understating what the said Google AI is doing, but it looks to me as a simple feedback loop that responds to changes, the amount and speed has nothing to do with intelligence.
Solar fusion simply provides the ignition required to create the coronal plasma. The electric dynamo created by the sun’s rotating magnetic field surrounded by the conducting plasma is responsible for the vast majority of the sun’s light and heat.
The sun’s core is only 10,000° Celsius. By comparison, the corona is 1 million degrees Celsius generated by Joule heating. This heat and light is generated by the resistance of the coronal plasma to the electric current induced by the solar dynamo. Sunspots are dark because of local disruptions in the coronal plasma.
The question is not how to recreate solar fusion, rather, it is how to recreate the solar dynamo.
https://www.jpl.nasa.gov/nmp/st5/SCIENCE/sun.html#:~:text=Core%3A,make%20up%20the%20Sun's%20core.
Core:
the temperature at the very center of the Sun is about 27 million degrees Farenheit (F). The temperature cools down through the radiative and convective layers that make up the Sun’s core.
“previously undescribed architecture for tokamak magnetic controller design that autonomously learns to command the full set of control coils.”
A shortcoming I see is that Deep Mind is stuck with the 19 mag coils as they exist. It can’t advise whether the existing setup is the best. I suppose it has some utility to optimize the givens of the setup.
Do we know unreservedly that the donut itself is the best possible “container”. That its static orientation is the best. I recall from Popular Mechanics in the 1950s pretty much the same diagrams one sees today. Maybe a way to “tighten” the torus of plasma to a stable central position would be to excite magnetization in a way to cause a helical twisting of the “rope” of plasma. I realize mechanical rotation is out of the question for such a huge device.
There is a way to “evolve” new designs, an evolutionary algorithm. But this relies on having a mathematical model which is good enough to predict how well the computer generated designs perform. I don’t believe models of fusion plasmas are good enough to do this yet, though they may be.
The following is a demonstration of an evolutionary algorithm. In this case it is solving the travelling salesman problem, the shortest line which joins all the dots. But the model for evaluating the quality of each proposed solution is easy in this case, all you have to do is measure the length of the line.
We certainly have more mental assist tools than in the 1950s. I’m impressed that the present configuration is so similar. When I graduated back in the day the sliderule was my computer. There was a new digital computer in the science building, but it took up a whole airconditioned room and didn’t have the computing power of a scientific handheld calculator.
This is probably a silly question: instead of heating water for a conventional steam turbine, is it a physically thinkable process to ‘bleed off’ electrons from the torus directly onto a conductor (maybe at the end of a long tube) and feed fuel in to replace electrons drawn off. The conducting element could be a hollow metal tube with a circulating coolant inside if needed.
It’s a good thought, but the neutrons get thrown off anyway, so you can’t ignore them, they carry much of the generated heat.
Also the problem as I understand with plasmas is they really want to lose energy, so they don’t need a bleed mechanism, they need to be kept hot, only extracting energy which cannot be recirculated.
One could use a blackboard area as both an output and an input.
For tracking a baseball, it would have to figure out that the baseball was of interest, emit something in its blackboard output, then correlate that input and the new baseball info with progress toward the desired state.
There are whole classes of AI systems that have state (or “memory”), that have been around for some time. They are particularly useful in dealing with things that are time based like signal processing.
Examples include Long Short Term Memory Networks and Recurrent Neural Networks:
e.g. https://www.cs.bham.ac.uk/~jxb/INC/l12.pdf
I think that the real impact of AI systems with respect to things like controlling plasma within a Tokamak is that it may be very hard to come up with a successful algorithm that you can use to create a conventional computer program. AI, being based on training from real data, may be able to learn and encode complex patterns and so create a useful system that would be difficult to create any other way. It is certainly worth trying out AI as a technique in this complex area.
Computer architectures are not that simple. It is perfectly possible for a system to be simultaneously both stateless and stateful – depending on the level at which you are considering this description.
This is one of the most interesting articles and comments that I’ve seen. Perhaps because I’m currently reading “The Alignment Problem”-Machine Learning and Human Values, by Brian Christian.
In my 75 year old mind, it seems that the baseball analogy is more like a steady state situation where time to recognize and adjust to the solution is obvious to compared to a mostly chaotic field situation.
In baseball, even 10 year-olds (well ahead of knowing of physics) can see the hit ball and identify where and when to get to the catch. Apparently a control time frame is available.
From the comments, identifying and reacting to chaotic fluctuations of the magnetic fields are currently well-beyond available solutions.
So, as in the above referenced book….perhaps we all need to ask “What problem are we trying to solve?”
Problem: provide energy needs of a growing population within a successful economy and clean environment (lowest real cost is a bonus we once enjoyed)
We’ve already achieved the ability to generate and distribute available, stable and reliable electricity with fossil fuels, hydro and nuclear in a clean and safe way.
Now we just need to quell the panic created and sustained by the politically, self-interested.
We can adapt and use what we have.
Hell with chaos.
The UK JET, Joint European Torus, a fusion record setter, lost funding after Brexit. Looks like BoJo does not care….
The LLNL, Lawrence Livermore Nat. Lab, NIF, Nuclear Ignition Facility laser outfit was almost confined only to warhead testing until it set a record last August.
As discussed in essay Going Nuclear, the NIF is a complete waste of money in re fusion for grid generation. The laser optics take over 4 hours to cool down between 2 MW (net, 4MW gross) shots. For grid power, you would need a shot about every second. And you cannot afford 14,400 $billion plus NIFs focused on 14,400 hohlraums just for one fusion generating station.
Quoting Feynman when pointing at a box with a cross in it on a blueprint at Oak Ridge (trying to find out what the symbol meant without looking silly)–“What happens when this fails?” It just might apply to such computer AI.
skeptical about these claims, but AI may be able to infer new details of physics that could lead to better designs
note this already happens in biology via tools like AlphaFold, where AI’s ability to predict protein structure exceeds our direct knowledge of the particular chemistry of the protein interactions
protein folding and plasma physics in the realm of interest to fusion both have a lot of unknowns, and what we do know can be very situational
e.g. even PhDs with decades of career work around toks will struggle to describe an IEC system operating at the quasineutral limit, where electrons don’t immmediately collapse into a Debye sheath, even both designs have been around for half a century
AIs working directly with masses of experimental data might have better luck than theorists and engineers in some circumstances
In theory you can measure the velocity and direction of a ball and include it in the current state of the system. I built a radar tracker that used the measured distance and velocity to inform its update step. You will, however, get a better answer by iterating toward the answer on both position and velocity.
Seems sort of odd to make AI look like a 1st order FIR filter
I haven’t commented for a long time, but this hits one area of my personal expertise, and one are I’ve been tracking closely for my entire life. IMO — which I’d argue is a pretty well informed one — fusion power is finally just around the corner. Not just one solution, but very likely two or three distinct solutions. The kicker was the development of scalable parallel computing back in the 90’s and early 2000’s, which at this point is so ubiquitous a model for large scale supercomputing that it is almost uninteresting, but back then was a revolutionary end run around the big supercomputer companies that charged literally 10 to 100x what you could build one yourself for a la beowulf. The superhot plasma has a number of instabilities that killed early attempts to just “guess” a geometry for magnetic confinement and tweak it until it worked, but computers can now do precise modeling of magnetohydrodynamics at a scale enabling systematic searches for confinement regimes that reach the critical region, and they’ve succeeded.
To comment more directly on the model above, I “invented” the NEAT approach for neural nets twenty odd years ago (I’m guessing lots of people did — the ideas are straightforward) but never implemented it in code because it wasn’t necessary for the problems we were interested in. I filed for an actual patent on it as a basis for “true AI” — think NEAT plus a particular integration of I/O plus careful training — well over a decade ago, but funding for developing it failed to materialize from DARPA as the project we proposed wasn’t a good match for their then-RFPs, and I ultimately let the patent filing lapse. The article above is quite right that in principle, at least, adding a memory of any sort to an AI stack (NN is my own preference, but the CONCEPT is catholic enough) enables one to model non-Markovian dynamical systems with a non-Markovian model (one with a memory). Most straight up FFNNs are fundamentally Markovian, and do not use any part of of the past time series of the “real time” inputs as an input. It is not at all clear, however, that this will make any difference in the particular problem at hand. Lemme ‘splain.
Consider driving a car. Your eyes are on the road, steering the car and simultaneously integrating all sorts of data — where the road goes, condition of the road surface, instantaneous location of all of the other cars in the moving, possibly accelerating frame of your own car, road hazards. If you limit your inputs to just positions — the instantaneous state in position space — your driving would be purely Markovian and you’d immediately crash because obtaining velocities or accelerations is non-Markovian and requires at least two timesteps or three timesteps of past data to compute. Similarly, a neural net with no memory can’t distinguish between a car parked on a side street that might cross your path from one in the same position that has a velocity/momentum that makes it certain that it will cross your path. Doubtless the same is true for NN’s trying to “drive” containment.
But that is almost certainly not the way it is implemented, because the people setting this up aren’t idiots. You don’t need much non-Markovian data to add in velocities and accelerations — your brain does it with less than one second worth of data and it is slow, slow, slow compared to computers and sensors providing input. You also don’t want or need to clog the jets of your NN by making it find some way of computing it from the input data stream. If I were building the modeling system running ITER, I would simply make sure the input stream is preprocessed to provide the missing derivatives as inputs in their own right. That way the car on the side street is identified on the input stream not just by its position, but by its position, velocity, acceleration if any, and I personally would multiply out by size-estimated mass and convert velocity into momentum and acceleration into change of momentum all in the input preprocessor, as that’s what we KNOW the physics REALLY depends on and now all the net has to do is learn to solve the physics problems involved based on instantaneous state the same way we actually solve them.
Personally, I’d go further and run the model forward umpty times (where umpty depends on how fast the computer is relative to the process actually being modeled and controlled), build a statistical map of projected outcomes compared to the desired outcome plus a risk assessment, and adjust inputs accordingly to stay in the sweet spot of the cone of future probabilities, but maybe that is just me. Here being Markovian can actually be an advantage, because one thing about a trained NN sensibly implemented — they are very fast. Fast enough to run millions of times in a second on a big (but not “huge”) parallel machine. How that compares to the time granularity of the controls involved at ITER I don’t know, but I am quite certain that they people that are doing this are really, really smart and aren’t making any stupid mistakes.
Would NEAT be better than Deep Mind at this? Who knows? Maybe, out of the box so to speak. But give ME unlimited access to Deep Mind, a “who cares” budget for preprocessing hardware and computer, a detailed description of the actual magnetohydrodynamic processes being controlled and their relevant timescales, and I’m pretty sure that by the time I finished, the advantage would disappear AND I’d have a precise idea of what was going on EXCEPT for the multivariate Markovian integration and decisioning embedded in the actual NN. And NEAT, Deep Mind, my own personal NN with its bells and whistles, over the counter NNs from e.g. Matlab/Octave or R, that’s the real key. CAN it, can ANY, NN solve the problem?
Personally I would like to think so — NNs are incredibly powerful multivariate nonlinear function approximators — but that’s why they call it “research”. To find out.