Special Forces, Marines Embrace Palantir Software

DCGS-A demo

Special forces and Marines are embracing the commercial software Palantir for analyzing battlefield intelligence even as the Army seeks to downplay its effectiveness, according to a new report from government auditors.

Both U.S. Special Operations Command, based at MacDill Air Force Base in Tampa, and the Marine Corps have opted to field the product to troops after finding it “easy to use” and “effective” on missions in Afghanistan in recent years, according to a June report from the Government Accountability Office and obtained by Military.com.

The command, which oversees operations involving Navy SEALs and other elite troops, last year added the software to its version of a military-wide intelligence system called the Distributed Common Ground System, or DCGS, according to the document, which is labeled “for official use only” and hasn’t been previously released. The Marine Corps next year may do the same.

“Users indicated it was a highly effective system for conducting intelligence information analysis and supporting operations,” the report states. “The software had gained a reputation for being intuitive and easy to use, while also providing effective tools to link and visualize data.”

The commercial software is made by Palo Alto, Calif.-based Palantir Technologies Inc. The company was founded in 2004 with seed money from In-Q-Tel, the venture capital arm of the Central Intelligence Agency. The military has spent about $35 million on versions of the product, according to government auditors.

Palantir’s non-defense customers include civilian agencies such as the Recovery Accountability and Transparency Board, which used the software to investigate fraud in stimulus spending, and investment banks such as JPMorgan Chase & Co., which installed it to analyze trends in mortgage and financial data.

While SOCOM and the Corps have pushed for wider adoption of the software, the Army has only approved limited use of the product and instead encouraged soldiers to rely on its existing system.

The service’s reluctance to adopt the commercial software has angered lawmakers such as Rep. Duncan Hunter, R-Calif., who got into a heated exchange over the issue with Army Chief of Staff Gen. Raymond Odierno in April during a hearing on Capitol Hill.

The congressman, a former Marine who served tours in Iraq and Afghanistan, criticized the Army system for what he said was its increasing cost and lackluster performance, and questioned why a commander who requested Palantir never received it.

Odierno defended the Army’s actions, saying his staff members are doing “all they can to help.”

The Distributed Common Ground System was conceived in the 1990s as a better way to analyze and share intelligence by shifting from “stovepipe” systems, in which data can’t be easily modified or shared, to an open architecture based on common standards set by the larger intelligence community.

The military-wide system is estimated to cost at least $10.6 billion. More than half of that, or about $6 billion, has already been spent, according to the GAO, the investigative arm of Congress.

Yet troops in Afghanistan as late as 2010 were asking for better tools to sift through the torrent of digital information being captured by surveillance aircraft and other platforms, according to the GAO report.

The number of aerial drones used to collect intelligence in Iraq and Afghanistan surged from 150 to 7,500 aircraft from 2002 to 2010, the document states. A single Predator made by San Diego-based General Atomics can loiter over the battlefield for hours taking photographs, capturing live video, collecting infrared and other imagery, it states.

The sheer volume of intelligence data has overwhelmed analysts, who complain of having to search multiple databases to find relevant information, according to government auditors.

The Army’s version of the system draws on more than 600 sources of information, from Global Hawk drones and GPS satellites to ground sensors and biometric scanners, officials said. It uses a mix of commercial and military software applications to link the vast amounts of data, including Google Earth made by Google Inc. and i2 Analyst’s Notebook made by IBM Corp., they said.

During a three-day demonstration of the Army’s Distributed Common Ground System in May at Fort Belvoir, Va., just south of Washington, D.C., officers and soldiers criticized Palantir for not being fully compatible with the service’s technology.

“It’s essentially a difficult problem because Palantir uses a different data anthology and data structure than we do on the DCGS side and the [intelligence community] side,” Col. Charles Wells, who manages the Army program, said at the event. “It’s not a trivial change or a trivial problem that we’re trying to work through. It actually requires some fundamental adjustments to the data structure.”

Yet Special Operations Command seems to have overcome that challenge, according to the GAO report.

“Although the two systems are not fully interoperable, SOCOM has worked out a means to integrate the systems so users can gain the benefits of both,” the document states. “Users will have to import and export data across the systems, but SOCOM has developed a user-friendly way to do this.”

Army Lt. Col. Jerome Pionk, a spokesman for the Army at the Pentagon, said in an e-mail the service entered into a cooperative research agreement with Palantir in May 2012 and that, as of now, ingestion of data from Palantir to DCGS can only be done manually.

In another e-mail, Pionk said the report makes clear that while the Army has more troops and a different mission than Special Operations Command or the Marine Corps, the services are working to support the Distributed Common Ground System to share information across the military, intelligence community and coalition forces.

The document acknowledges the Army’s gains in developing common intelligence standards, and in testing and certifying software compatibility, Pionk said.

It also cites the importance of using an open-architecture approach so the Defense Department “can make improvements to the system without having to commit to a specific product or developer,” he said.

About the Author

Brendan McGarry
Brendan McGarry is the managing editor of Military.com. He can be reached at brendan.mcgarry@military.com. Follow him on Twitter at @Brendan_McGarry.
  • hibeam

    The federal government tried to put cameras on poles along the Southern Border and threw in the towel 300 Million Later. Too tough a problem. Poles and all. It is good that they are buying products from real companies operating in the free market sector. Writing a check for a finished product they can figure out. Barely.

  • Dfens

    What, this program can’t be any good. It was developed using private funds. There was no bureaucratic over sight. There were no US taxpayer dollars spent on endless meetings. There were no documents for thousands upon thousands of civil servants to review. This can’t possibly be good. Everyone knows capitalism doesn’t work, after all that’s why we believe deep in our hearts that paying a government contractor more to fail will produce the best possible product.

    • yogiberra111

      If you read the article you would see that the company was funded by the CIA,i.e., taxpayer dollars.

  • voodkokk

    The service’s reluctance to adopt the commercial software has angered lawmakers such as Rep. Duncan Hunter, R-Calif., who got into a heated exchange over the issue with Army Chief of Staff Gen. Raymond Odierno in April during a hearing on Capitol Hill.

    We don’t care if you will use it or not. You will like it and buy it.

  • Carl

    The headline “Special Forces Embrace Palantir Software” misleads the reader into thinking Special Forces means everyone in USSOCOM when in fact Special Forces means Army Green Berets who are not even mentioned in this article.

    If Navy SEALS, MarSOC and Air Force Commando’s are using it than its Special Operations.

    Let the former Marine who’s now a Congressman scream. General Odierno will do whatever he wants.

  • Jane
  • Jane
  • Michelle

    Isn’t that document you published still a classified document? It says For Official Use Only. Anyway the Army’s mission is larger than the marines or special forces for that matter, maybe that’s why they have different requirements? Duh!

    • Boo

      Unclassified//FOUO is not a classified document.

  • OD375

    Not a well written article as one reader already pointed out the misuse of the title ‘Special Forces’ which is US Army Green Berets of whom there was no mention, but also leads the reader to inaccurately or unjustifiably think the US Army is disingenuous or is ambiguous in objecting to the use of this software. My immediate concern is that this is slanted journalism and that mil.com is allowing it.

    • blight_

      Indeed, it’s probably that USSOCOM is using the software, but not Army Special Forces.

      I feel like DefTech should stop using images from other articles, since it’s essentially as bad as using random stock images. I’m not sure what other media graphic would be appropriate in this case. Sometimes, I think DID’s interface is more appropriate, clustering articles by system and updating as needed, vs multiple redundant blog posts without the “big picture” of other articles, just “tagging” and a requirement that the reader put the pieces together.