Last week, Defense Secretary Pete Hegseth announced a major new command to speed the use of drones and artificial intelligence across the military.
AI already pores over satellite and drone data to find targets, and it can take over attack drones on their final approach so electronic jamming can't stop them. But it's expected to soon dominate battlefields and speed everything from targeting decisions to supply chains.
Meanwhile, Congress is grappling with the military version of a question that employees at the major AI companies also have been raising: Is AI technology moving too quickly to be sure it's safe?
"These are questions we haven't yet dealt with here in Congress," said Arizona Sen. Mark Kelly during a Senate Armed Services Committee hearing earlier this year. "Whether or not a human has to make the final decision to strike the target, or can a system execute the engagement autonomously once it's been activated?"
For the private sector, President Trump just signed an agreement between the major AI companies that essentially has them policing themselves.
Congress, though, is considering restrictions on military AI.
The committee added a host of provisions to the pending annual military policy and spending bill. Among other things, they would require human oversight of AI-controlled weapons, ban the use of AI for nuclear launch or detonation, and bar its use for domestic surveillance.
The stakes, experts say, are high.
"The consequences of getting it wrong could be really bad," said Paul Scharre of the Center for a New American Security, who has written two books on military AI and led the working group that drafted the Pentagon's first policy on autonomous weapons in 2012.
"You bomb the wrong target, which could be really bad, could cause harm to civilians," Scharre said, "but it also could escalate a conflict, could create a geopolitical crisis if people are trusting AI, and maybe it's not trustworthy yet."
The U.S. uses an AI platform called the Maven Smart System to help find and identify targets. In the first hours of the Iran war, it did so at ten times the speed previously possible, according to analysts at the Center for Strategic and International Studies.
But that speed — and an overreliance on AI — may have played a role in the U.S. missile strike that destroyed an Iranian school, killing more than 160 people, many of them young children.
Scharre notes that the military has had more basic automated targeting for years, typically defensive systems that can quickly stop fast-moving threats like incoming missiles. Today's AI will speed things up even more.
"I think what we're likely to see over time is a sort of slow expansion of this domain of machine speed and warfare into other areas where the scale of activity is such that humans can't really keep up," Scharre said. "So maybe a human is authorizing a target list of a whole bunch of targets, but the human doesn't really have time to pore through each one individually and analyze it."
Even with a human involved, things can go wrong. CNN reported recently that an analyst at a special operations command used a chatbot to help write an intelligence report that mistakenly claimed a Chinese ship in the Middle East was carrying nuclear weapons components. Officials caught the error just before a planned operation that could have set off a major international incident.
Joe Chapa, a retired Air Force Lieutenant Colonel, helped establish AI policy for the Air Force and Space Force. He is now with the University of Massachusetts Boston's Applied Ethics Center and the Institute for Ethics and Emerging Technologies. He says the misidentified ship underlines the importance of training humans in how to use AI.
"If we don't get AI education right in the military, there will be consequences for that down the road," he said. "Every person who's going to touch an AI system, which very soon is going to be every person, needs at least a baseline education in how statistical AI works.
"There could be something that goes terribly wrong, and everybody looks at each other and goes, 'Well, there's no way for us to have known,'" he said. "Well, the way for you to have known might be to have had a better understanding of how statistical AI works in advance."
Chapa says many commonly discussed scenarios for AI going rogue in a conflict center on unintended escalation. He expects nations will work to avoid that in how they incorporate AI into the military. But even if they succeed, non-state actors such as terrorist groups may do the opposite.
The Senate bill also calls for the Pentagon to keep data on decisions made by or with the help of AI. Chapa and Scharre both said that's crucial so the military can understand mistakes and prevent them from happening again. Today's AI models are so complex that even their own developers sometimes struggle to understand their behavior.
"We've had airline autopilots do some kind of surprising things and cause accidents, but at least it's kind of understandable after the fact," Scharre said. "Oftentimes, with AI systems, it's like, why did it do that? And there's not an obvious answer."
Some parts of the military were developing autonomous systems aggressively even before Hegseth began pushing.
On a recent day on a remote part of Fort Bragg, soldiers and contractors assembled to see the Sandhills Project, which started three years ago with the goal of automating one of the most dangerous tasks in combat: clearing a path through a minefield.
Helping manage it is the Bragg-based 20th Engineer Brigade, whose soldiers' duties include such route clearance.
The demonstration featured off-the-shelf Ford pickup trucks, which are cheap by military standards and have inexpensive maintenance and replacement parts. But they were equipped with robot guns on their roofs to defend against drones, and they were covered with sophisticated sensors to help them drive themselves through rough terrain. A complex AI platform honed on the battlefields of Ukraine also aided their movements, incorporating visual data from previous, unrelated drone flights over the routes.
"It's a transparent battlefield and there are more sensors than ever before, new ways to strike us from all sides, and so, how do we adapt quickly for this new character of war that we see?" said Col. Jeff Burges.
Atop the cabs of two of the trucks were 12-gauge shotguns on robotic pivots, controlled by AI. In the demonstration, two drones simulated an attack on one of the trucks, and the gun destroyed both. The soldiers in charge said a human was "on the loop," ready to intervene if anything went wrong.
Then the other truck drove off on its own, picking its way for several minutes through rugged terrain. As the soldiers and contractors watched a live drone video feed, the truck reached a simulated minefield, turned and fired a rocket that traveled about 100 yards, trailing a hose like those firefighters use. The truck filled the hose with an explosive slurry, which was detonated to clear a path through the mines.
Given the pace of change in technology and automation, Pentagon leaders have been trying to speed up the development and acquisition of such systems. The Army also was using the program as a model for that by having constant interaction in the field between the troops and the companies developing the different pieces of hardware and software.
Colonel Tom Monaghan is director of the Joint Innovation Outpost at Bragg, which is working with the engineer unit on the project. He said in terms of acquisition, it's a test case for the Army.
"In our normal program, these individual products would all be separate programs and with separate program offices and separate funding profiles, with separate leads, with separate industry partners," he said. "So this is a first shot at doing something like this, where all the industry partners are out here with the warfighter at the speed of need, at the speed of industry, with direct warfighter input."
The minefield breaching system appeared to be nearly ready for combat. Chapa said several such programs across the services are testing autonomy in various systems.
Scharre said the pace of AI adoption in the military could be a plus.
"Military adoption takes some time," he said. "I give a lot of credit to the current Pentagon leadership that they're working really hard to move fast and tear down parts of the bureaucracy that move slowly and cut through red tape, but it's still military procurement, and bringing in new technologies out of the civilian space is really slow."
That means some of the military's dangers are likely further off than the more immediate challenges the private sector is seeing, such as the recent high-profile hacking incidents involving AI agents. That, he said, could give the Pentagon and Congress more time to get AI right.
This story was produced by the American Homefront Project, a public media collaboration that reports on American military life and veterans.
