By David Wojick
I recently had the honor to write CFACT comments in response to an Energy Department (DOE) request for information (RFI). The topic is “Transformational Artificial Intelligence Models” where the transformation in question is that of improving scientific and engineering research. DOE is one of the world’s biggest funders of physical and computer science research so this is potentially an important opportunity to make a big difference.
The focus of their RFI is on using AI models to do research. But we pointed out that they can also be used to understand what is going on in research across the huge number of publications that are often published on a major topic. Our point is on the “I” side of AI. It is a specific form of intelligence that AI models can emulate.
Here are the CFACT comments:
Transformational AI Models for Analyzing Science and Engineering Research
By David Wojick, Senior Analyst, CFACT
Here are CFACT comments in response to the U.S. Energy Department’s “Request for Information (RFI) on Partnerships for Transformational Artificial Intelligence Models.“
DOE proposes “to curate DOE scientific data across the National Laboratory complex for use in artificial intelligence (AI) models and to develop self-improving AI models for science and engineering using this data.“
Our basic comment is that DOE’s Transformational AI Program should include the use of AI models to analyze ongoing research. A major research area typically produces far more publications than a human can read. Those models focused on language and reasoning capabilities can be configured to effectively analyze these massive bodies of information, where the data is textual.
For example, plasma physics is a major topic of DOE research, both scientific and engineering. According to Google Scholar the number of journal articles specifically addressing plasma is on the order of 40,000 a year or more. Articles at least mentioning it number in the hundreds of thousands a year. These numbers are well beyond what a human can read and analyze.
A properly designed and trained AI model can read all of these articles and answer well formulated questions about the entire corpus. Here are some of the sorts of questions the answers to which could dramatically assist American researchers.
1. Who is doing what, including by country? For example China now rivals America in research output. In which topics are they most active?
2. How do the articles on a given topic all fit together? This includes mapping the cognitive structure of the research.
3. How does that cognitive structure changes over time? Examples include the dynamics of the findings, methods, applications and questions being addressed. All change over time, sometimes rapidly emerging. Early identification could be crucial.
4. What are the competing hypotheses and schools of thought regarding a specific problem or question? The frontier of science and engineering is a land of conjecture and debate. These debates often generate central questions for new research.
A model or models that can answer these crucial questions will give new meaning to the concept of keeping up with one’s field of research. This new power can both speed up the diffusion of knowledge and lead to new discoveries.
Regarding curation this AI research analysis will of course require access to a large number of journals, conference proceedings, research reports and other sources. DOE’s Office of Scientific and Technical Information (OSTI) has already made considerable progress in this regard. DOE should also consider partnering with major journal publishers. A small number of leading publishers account for a large fraction of the journals.
Initial efforts could focus on a specific research topic, or a specific community like DOE funded research, or both.
By way of background I was Senior Consultant for Innovation at OSTI for almost ten years under director Walt Warnick, where I did a lot of research on mapping the cognitive structure and dynamics of scientific and engineering research. This research has continued at CFACT and CFACT has also done research on AI cognition. I will be happy to provide additional information on these comments.
The singular ability of language and reasoning models to analyze huge bodies of technical information opens up new ways to understand and advance research.
I would say this piece was written using AI…it’s been gaining ground for years…what happened to thinking things out..? Is it because it’s a race to get things out into the forum..?
If I wanted pictures of negro nazis etc I’d use [Google] AI etc
The image at the top reminds me of Avatar.
i have been mapping research for many years and when people asked me if a computer could do it I said “no because a computer cannot read.” Now they can.
No AI was injured in writing this piece.
I was unaware AIs were equipped with eyes. 🙂
China now rivals America in research output.
China has been known to perform outsourced American research, research that was outlawed in the US by the Obama administration. So, the question “Who is doing what, including by country” is a whole lot more complicated than might be assumed.
“this AI research analysis will of course require access “
And ‘there’s the rub’.
If taxpayer funds from any level of government are being used to perform research, then ALL of the data, methods, and conclusions should be put on a public server for access by the taxpayers.
Government “partnering” with private publishers will only further divide the research into “I like this” and “I don’t like that”. Guess what gets published?
In the UK “I like this” and “I don’t like that” has been the norm for at least 20 years.
To quote our dear leader: “We’re still all in“.
Partnering in this case means giving the AI model access to the publisher’s journals.
There is one exception, Jim. Classified research.
China produces a huge number of English language journal articles a year, rivaling the U.S., so small specialized numbers do not matter much. My understanding is the govt has been giving cash rewards for getting top journal articles.
Good points here.
“A major research area typically produces far more publications than a human can read. Those models focused on language and reasoning capabilities can be configured to effectively analyze these massive bodies of information, where the data is textual.”
The power to rapidly perform expansive search, summary, and cataloguing of references, directed by well-formulated prompts, is indeed new and potentially useful for honest purposes. There’s the thing.
This seems like a good area to apply AI.
No one person could read 40,000 studies in a year.
Will the AI be able to tell junk and fraud from genuine research? This would seem to be something only humans can do.
And , across languages .
It is absolutely necessary
+ an optimized use of language/words is necessary,
as even a summary may be too eventually too much with so much output =
Reduction of superfluous words and phrases.
Maybe even a 1- 10 scale to classify things in terms of importance/potential(yes, here is massive downside in potential abuse, and it will happen ).
This way much of the useless or barely helpful stuff can be filtered out and experts can save hundreds of hours and brain memory by instantly getting to the core.
How many articles will the AI hallucinate?
I have this question for the AI Bot: How do I convince Mad Ed of the UK that carbon dioxide in the air does not cause any global warming?
Vey funny, Harold. How can you convince the High[est] Priest of the faith to recant his beliefs?
The simple answer is you cannot.
Here is what the AI Bot would say:
“Tell Minister Mad Ed to go to the late John L. Daly’s website:
“Still Waiting For Greenhouse” available at: http://www.john-daly.com. From the home page, go to the end and click on “Station Temperature Data”. On the “World Map”, click on “Australia”. There is displayed a list of weather stations.
Click on “Adelaide”. The chart (See below) shows a plot of average annual temperature from1857 to 1999. In 1857 the concentration of CO2 in the air was 280 ppmv (0.55 g of CO2/cu. m. of air), and by 1999 it had increased to 368 ppmv (0.72 g CO2/cu. m. of air), but there was no corresponding increase in air temperature at this port city. Instead there was a cooling that began in ca.1940. The Tavg in 1999 was ca.16.7° C.
For more recent temperatures in Adelaide, go to:
https://www.extremeweatherwatch.com/cities/adelaide/average-temperature-by-year. The Tmax and Tmin data from1887 to 2025 are displayed in table. The computed Tavg for 2025 was 15.9° C, a 0.8° C decline since 1999. Adelaide is still cooling. The reason CO2 has no influence on temperature
in Adelaide is that there is too little CO2 mass in the air to absorb out-going long wavelength IR light to cause warming of the air.
The above empirical data and calculations falsify the claim by the IPCC that CO2 causes warming of air and hence global warming. John Daly found over 200 weather stations that showed no warming up to 2002″.
If I had a sit down with Mad Ed, walked him thru the above, and showed him many more charts from John Daly’s website, would he abandon his draconian climate agenda and the goal of Net Zero by 2050?
NB: If you click on the chart, it will expand and become clear. Click on “X” in the circle to contract the chart and return to comments.
You cannot trump blind faith.
I know. DANGER WILL ROBINSON….
But Trump is trumping blind faith or at least making appreciated progress in that endeavor.
Interesting question. Try it.
As a geologist with both AI processed imagery research experience and now exposure to AI (every time you google ™ some geology question the top response is AI, and it is around 50/50 correct. So, David, to your “A properly designed and trained AI model…”, I would add, “monitored” like “designed, trained, and monitored”. AI needs adult supervision, then it can do the routine stuff.
AI uses the preponderance of the evidence and does not filter for duplications and repetitions (aka media reports).
Before AI can effectively be used, the data sources need some kind of scrubbing. AI uses the internet. Enough said.
Very true. In fact one might have two models do the same query. Or have one check the other.
“A model or models that can answer these crucial questions will give new meaning to the concept of keeping up with one’s field of research. This new power can both speed up the diffusion of knowledge and lead to new discoveries.”
Makes sense!
On the flip side, it can confound, confuse, and cause a misdirection.
AI is not conscious. AI is not intuitive.
Humans do all this too so use the same precautions.
https://www.cfact.org/2024/03/16/ai-chat-bots-are-automated-wikipedias-warts-and-all/
Concur, especially those that claim to be “climate scientists.”
Not yet anyway.
While not impossible, knowing what I know about computers and software, it is highly unlikely.
Lots of videos on YouTube talking about AI. Lots of opinions. I haven’t yet played with AI but I’m tempted. Also, lots of discussion about consciousness- what it is and how it works. Apparently not much is understood about it- so until we get a better grip on what it is- it won’t be possible to know if and when AI takes on some of its qualities. My unqualified opinion is that it won’t be conscious until it can experience pain and suffering. Just being smart ain’t necessarily consciousness. It’s probably impossible that we could ever know for sure if it’s conscious. But if it acts as if it is- we could go along and pretend it is. There’s probably no particular reason why we need to know- until the nut jobs start talking about giving AI rights. Maybe when the climate nut jobs realize their game is over, they’ll move to “protecting AI souls”. 🙂
One current theory I see on YouTube by many neurologists and philosophers is that consciousness isn’t in our brains- it’s in the “spiritual dimension” – whatever the hell that is- and that the brain is just a receiver. Nice theory but I don’t think it’s correct. Lots of discussion about this on the “Closer to Truth” podcast with neurologist Robert Lawrence Kuhn- who also talks to all sorts of very smart people about deep subjects. One of my favorite podcasts of the over 100 I subscribe to.
I had a lengthy interaction with AI.
I finally got the AI to confess that IR is electro-magnetic energy and heat is the flow of thermal energy across a temperature gradient (due to molecular kinetic interactions).
It went on and then AI concluded that IR is what transfers heat from the planet’s surface to space.
My pet goldfish can give a better answer.
Interesting but that is a lot to know. People almost never get heat right. They consistently confuse it with energy. CO2 “traps heat” and all that noise. Your bot may have been trained on that fallacy.
Not trained. All it had to do was go on the internet and grab stuff.
I had to point out contradictions and used various phaseologies to get it to admit the truth.
I did another. I posted all matter conducts electricity. It told me I was wrong that insulators did not conduct. Back and forth until it confessed that insulators can conduct electricity just not much and that there were no perfect insulators.
One has to know the subject to get anything useful out of AI, but because it has such a refined language module, the general public (as throughout history) accepted it as intelligent.
My experience with AI is that they cannot escape their algorithmic constraints. They are not critically trustworthy.
Same for people so use the same cautions.
I would have thought that the first question for AI would be: Is the research solid, or are there signs of bias, or unsupported findings, etc. That would include looking for omissions, ie, lines of research that are not being published.
You are anticipating AI can reach the point where it can make valid judgements?
I suspect the first answer would be, “Yes. It is solid. It is peer reviewed.”
Very interesting topic. Two points which may be relevant.
First, the major weakness of AI software is that because of its method of “reasoning” it cannot explain how it arrives at any decision. This makes it susceptible to a skilled interrogator convincing the software to give whatever answer the interrogator wants.
Second, and related to the first, is that that AI programs are neither fish nor fowl. A mathematician is entitled to say that he has given his solution and the axioms on which it was based. A scientist is entitled to give his answer and show that his experiment can be replicated.
AI can do neither. It’s “reasoning” is neither based on deductive nor inductive reasoning. The only justification for any output is to protest that is the way it was trained.
Absolutely correct.
AI is basically a weighted decision tree with limited alterations of the weightings hidden behind an excellent language interface.
AI is good for repetitive tasks as is any computer program.
AI is good for pattern recognition but is unable to tell if the data used is correct.
AI generally uses the preponderance of the evidence (or data or reports) when weighing the validity of a report, but lacks the ability to impartially judge.