Subscribe via RSS

Archives by Date
April 2009
March 2009
February 2009

See all Archives
Newsletters
Archives by Date
'Canes
Afghan Update
Ammo and Munitions
Armor
Around the Globe
Av Week Extra
Axe in Iraq (and Elsewhere)
Bizarro
Blimps
Blog Bidness
Body Armor Blues
Bomb Squad
Brownshoes in Action
Bubbleheads, etc.
Cammo Green
Catch the "Buzz"
Chem-Bio
Civilian Apps
Cloak and Dagger
Commandos
Comms
Contingency Ops
Cops and Robbers
Cyber-warfare
Data Diving
Defense Tech Poll
Defense Tech Radio
Dissent Tech
Door Kickers
Drones
DT Administrivia
Eat DT's Dust
Extra! Extra!
Eye on China
Fast Movers
FCS Watch
Fire for Effect
FOS Files
Friday Funnies
Gadgets and Gear
Going Green
Grand Ole Osprey
Ground Vehicles
Guns
Homeland Security
In the Weeds with Eric
Info War
Iraq Diary
Jarhead Jazz
JSF Watch
Just War Theories
Lasers and Ray Guns
Less-lethal
Logistics
Los Alamos and Labs
M4 Monopoly
Medic!
Mercs
Missiles
Money Money Money
Most Wanted
MRAP Edge
Net-Centric
Nukes
Old Skool
Our Shrinking Planet
Planes, Copters, Blimps
Podcast
Politricks
Polmar's Perspective
Popular Mechanics
Rapid Fire
Raptor Watch
Red Team
Retro-Futuro
Robots
Roll Your Own
Sabra Tech
Ships and Subs
Snipertech
Soldier Systems
Space
Special Ops
Star Wars
Strategery
Stray Trons
Tactical Development
Terror Tech
The Deadlies
The Defense Biz
The Peoples' Site
The Sunday Paper
The Tanker Tango
The View from Av Week
Those Nutty Norks
Training and Sims
Trimble on the Case
Video Lounge
War Update
Ward'z Wonderz
You can run...

See all Archives
Related Links
News and Intel
Military.com News
From The Front: Christian Lowe
Aviation Week
Natl Defense Mag
Strategy Page
Global Security Newswire
Soldiers for the Truth
Security News
Defense Review
Fed Comp Week

Security Sources
GlobalSecurity.Org
Fed of American Scientists
Ctr for Strategic & Intl Studies
Ctr for Defense Info
Defense and the National Interest
Instit for Sci & Intl Security
Secrecy News
POGO
Cryptome
The Memory Hole
Natl Security Archive

Geeks and Mad Scientists
Slashdot
Wired News
Security Focus
The Register
Gizmodo
Geek Press
Robots.Net
Cosmic Log
Space Daily
New Scientist
TechCentralStation
Engadget
Space.Com
Technology Review
Gyre
Near Near Future

Bloggers and Buddies
Phil Carter
Global Guerillas
Jeffrey Lewis
Belmont Club
Back to Iraq
Laura Rozen
Juan Cole
Ryan Singel
Josh Marshall
Cursor
Boing Boing
InstaPundit
Winds of Change
Tapped
Steve Gilliard
TalkLeft
Brad DeLong
Max Sawicky
Gene Healy
Clive Thompson
Greg Djerejian
Workbench
Electrolite
Jim Henley
Kathryn Cramer
Sensors blog
Tom Shachtman
PoliceLink.com
NursingLink.com

Official Dispatches
DARPA
AF Research Lab
Marine War Lab
Soldier Systems Ctr
Naval Research
Army Research Lab
UK Def Sci Lab
NASA News
DoJ Cybercrime

Military Network
Military Benefits
Veteran Employment
GI Bill Express
Personnel Locator
Free ASVAB
The Few
Fred's Place
Army Insider
Navy Insider
Air Force Insider
Marine Corps Insider
Coast Guard Insider



Edited by Christian Lowe | Contact

LMCO Not So Good DCMA Says

dcma.gif

Our freind Nick Schwellenbach over at the Project on Government Oversight dredged up a pretty damning report from the Pentagon's Defense Contract Management Agency that calls Lockheed Martin's aircraft division to the carpet for not keeping close track of costs.

Lockheed Martin, the world’s largest defense contractor, does "not provide the requisite definition and discipline to properly plan and control complex, multibillion dollar weapon systems acquisition programs," states the executive summary of a November 2007 Pentagon report obtained by the Project On Government Oversight. Questions about this report are likely to be raised this morning at a Senate Armed Service Committee hearing on weapons acquisition.

The report by the Defense Contract Management Agency found that Lockheed Martin’s military aircraft division based in Fort Worth, Texas, is not compliant with contractually-required industry guidelines for tracking and managing costs called the "Earned Value Management System." EVMS helps contractors and the government spot potential cost problems before they balloon out of control. This April the GAO reported $295 billion in cost growth for the 95 major weapons systems it reviewed bringing their estimated total price tag to $1.6 trillion.

The report will be highlighted today at a hearing of the Senate Armed Services Committee which will ask questions about "acquisition of major defense weapons systems" of John Young -- who needs no introduction -- and Katherine Schinasi, the GAO's Managing Director of Acquisition and Sourcing Management (whatever the heck that means)...Our boy Colin Clark will be there to hear what's what and he'll have some follow-up gouge for you on what goes on.

The decline of Pentagon and contractor emphasis on EVMS was “an unintended consequence of 1990s acquisition reform,” Dr. James I. Finley, the Deputy Under Secretary of Defense for Acquisition and Technology, told POGO. “EVM is getting more attention throughout industry now that the DoD is stressing compliance.”

Ouch...
-- Christian

New Spy Chief's "Total Information" Ties

"John Michael McConnell, the retired vice admiral slated to become America's new top spy, [has some] longtime associations [which] may cause him headaches during Senate confirmation hearings," Newsweek.com notes."One such tie is with another former Navy admiral, John Poindexter, the Iran-contra figure who started the controversial 'Total Information Awareness' program at the Pentagon in 2002."

iaologo.gif


The international consultancy that McConnell has worked at for a decade as a senior vice president, Booz Allen Hamilton, won contracts worth $63 million on the TIA "data-mining" program, which was later cancelled
[kinda sorta -- ed.] after congressional Democrats raised questions about invasion of privacy... While his role in the TIA program is unlikely to derail McConnell's nomination, spokespeople for some leading Democratic senators such as Russ Feingold of Wisconsin and Ron Wyden of Oregon say it will be examined carefully.

McConnell was a key figure in making Booz Allen, along with Science Applications International Corp., the prime contractor on the project, according to officials in the intelligence community and at Booz Allen who would discuss contracts for data mining only on condition of anonymity because of the sensitivity of the subject. "I think Poindexter probably respected Mike and probably entrusted the TIA program to him as a result," said a longtime associate of McConnell's who worked at NSA with him...

Intel experts agree that McConnell will need all the good will he can get from the intelligence and defense communities. "It's a good appointment for a bad office," says John Arquilla, who teaches intelligence at the Naval Postgraduate School in Monterey, Calif. "The directorate of national intelligence should not exist. It's very redundant." Insiders say Negroponte was frustrated by his lack of budgeting control over Pentagon intelligence, and the resistance of the CIA to his direction since his office was created in 2004 as part of the Bush administration's post-9/11 reforms.

And by the way, Rutty asks in the comments (I'm paraphrasing heavily here): What was McConnell's role in Echelon -- the NSA's massive information sweeper, which got some much attention during the Clinton years? (The project had been around for decades, remember.)

Data Diver Disses Terror-Mining

Jeff Jonas is one of the country's leading practitioners of the dark art of data analysis. Casino chiefs and government spooks alike have used his CIA-funded "Non-Obvious Relationship Awareness" software to scour databases for hidden connections.

nyt_mag_terror_diagram.jpgSo you'd think that Jonas would be all into the idea of using these data-mining systems to predict who the next terrorist attacker might be.

Think again. "Though data mining has many valuable uses, it is not well suited to the terrorist discovery problem," he writes in a new study, co-authored with the Cato Institute's Jim Harper. "This use of data mining would waste taxpayer dollars, needlessly infringe on privacy and civil liberties, and misdirect the valuable time and energy of the men and women in the national security community." Are you listening, NSA?

Jonas doesn't have a problem cobbling together information on suspects from various databases. It's using these databases to forecast a terrorist's behavior -- think market research, but for Al-Qaeda -- that Jonas hates. "The possible benefits of predictive data mining for finding planning or preparation for terrorism are minimal. The financial costs, wasted effort, and threats to privacy and civil liberties are potentially vast," he writes.

One of the fundamental underpinnings of predictive data mining in the commercial sector is the use of training patterns. Corporations that study consumer behavior have millions of patterns that they can draw upon to profile their typical or ideal consumer. Even when data mining is used to seek out instances of identity and credit card fraud, this relies on models constructed using many thousands of known examples of fraud per year.

Terrorism has no similar indicia. With a relatively small number of attempts every year and only one or two major terrorist incidents every few years—each one distinct in terms of planning and execution—there are no meaningful patterns that show what behavior indicates planning or preparation for terrorism. Unlike consumers’ shopping habits and financial fraud, terrorism does not occur with enough frequency to enable the creation of valid predictive models. Predictive data mining for the purpose of turning up terrorist planning using all available demographic and transactional data points will produce no better results than the highly sophisticated commercial data mining done today [with results in the low single-digits – ed.]. The one thing predictable about predictive data mining for terrorism is that it would be consistently wrong.

Without patterns to use, one fallback for terrorism data mining is the idea that any anomaly may provide the basis for investigation of terrorism planning. Given a “typical” American pattern of Internet use, phone calling, doctor visits, purchases, travel, reading, and so on, perhaps all outliers merit some level of investigation. This theory is offensive to traditional American freedom, because in the United States everyone can and should be an “outlier” in some sense. More concretely, though, using data mining in this way could be worse than searching at random; terrorists could defeat it by acting as normally as possible.

Treating “anomalous” behavior as suspicious may appear scientific, but, without patterns to look for, the design of a search algorithm based on anomaly is no more likely to turn up terrorists than twisting the end of a kaleidoscope is likely to draw an image of the Mona Lisa.

Civil libertarians and bloggers have talked 'til they're blue in the face about how lame this kind of terror-predicting is. But I don't think I've ever heard a giant of the field, like Jonas, come out against the practice -- at least not on-the-record. Let's hope this is one conversation that the feds are monitoring.

(Big ups: Daou)

UPDATE 11:49 AM: Shane Harris here. Die-hard proponents of pattern-based 'data mining' to catch terrorists will remain unconvinced by Jonas' and Harper's argument. While it's true that data mining in the commercial sector is based upon "training patterns," backers of systems such as Total Information Awareness will say, yes, and that's why data mining for terrorists has to start with hundreds -- maybe thousands -- of known or potential terrorist patterns to look for. A major part of TIA research was the creation of terrorist attack templates through red teaming exercises, in which experts were paid to come up with devious and clandestine plots that a terrorist might conceivably attempt. Their various machinations would, presumably, leave a set of digital footprints -- airline tickets purchased, money wired, hotels paid for, and so on -- and THAT data would be mined for clues.

What's also interesting about this paper is the combination of the authors. Jim Harper is a well-known and articulate activist, and has long since staked out central territory in the security vs. privacy debate. But Jonas has stayed out of politics. Indeed, those who've met him will know that he sticks out like a sore West coast thumb among Washington gear heads, being unafraid to use the word "dude" in formal conversation and happily acknowledging his ignorance of most Beltway insider baseball. But those who know Jonas and have heard him speak about electronic terrorist hunting know that, like his co-author Harper, he has a strong libertarian streak. Maybe Jonas wouldn't put it quite that way -- dude -- but it's there.

Traveling Americans Get Terrorism Score

Do you know your official terrorism score? U.S. Customs agents will with a new database system that uses algorithms to figure out which international travellers warrant closer search.

The system, announced in the Federal Register today, is called the Automated Targeting System, which will use the Treasury's watchlist (.pdf), data provided to it by the airlines, your I-94 form and other data sources to compute your terrorism risk when you cross the border.

Here's what I had to say over at 27B/6:

The data -- which includes all the information you give to an airline such as medical conditions, frequent flier number, special meal requests, home and email addresses, payment information and your travel agent's names -- will be held for up to 40 years. The data can be shared with any government agency or local law enforcement agency for civil or criminal matters, and can even be shared with foreign governments as data to test other data-mining programs, even ones not related to border security.

What happens if you have a name that's similar to a suspected terrorist or drug smuggler? Conceivably, you could have your car torn apart every time you drive to Canada or have a blue-gloved agent checking your anus for dope every time you go to Cancun.

But surely, you'll be able to remedy such mistakes using the Privacy Act, which prevents secret databases? Actually, no.

Full story and links to other bloggers here.

Hat Tip: JQP

On another note: This post concludes my week-long takeover of DefenseTech. Thanks for humoring me over here at Noah's house. It's been quite fun and I'm jealous of his great readers, tippers and commenters. He'll return soon, but feel free to stop by my blog-house occasionally.

- Ryan Singel

Military Ballots' Privacy Risks

American troops could be putting their most personal information at risk -- just by voting in next week's elections.

Members of the armed forces, stationed overseas, can cast their vote with a Federal Write In Absentee Ballot, or FWAB, if they can't get one from their local election boards. But that federal ballot, "Standard Form 186 (Oct 95)," comes with a major privacy risk, at least in some editions. The ballot has to be mailed in a special return envelope, in order to be properly processed. On military bases in the Pacific, Special Form 186 requires a service member to include his address, social security number, date of birth, and signature on the outside of that envelope.

envelopeback_crop2.jpg
In other words, everything needed to steal a soldier or sailor's identity is on public display, for anyone to see (full pics: back, front). .

"You'd think the people running this program would've noticed. It's a joke they didn't, and it's obvious no one was paying attention," a Navy aviation electrician, attached to the 7th Fleet, tells Defense Tech.

Online editions of the FWAB seem to be more security-conscious, warning servicemembers "NOT [to] WRITE ANY PERSONAL IDENTIFYING INFORMATION ON THE ENVELOPE" -- an envelope that's largely blank.

But the paper ballots aren't the only source of privacy concerns in the military voting system. An e-mail balloting program has been called into question, for using unencrypted data. "E-mail traffic can flow through equipment owned and operated by various governments, companies and individuals in many countries," the Washington Post quotes an August report prepared for the Pentagon as saying. "It is easily monitored, blocked and subject to tampering."

But even easier to monitor is a paper ballot, with personal data scrawled right on the outside of the envelope. Which is why the Navy aviation electrician refused to use the form.

"I wasn't the only person who didn't send the ballot in. It wasn't worth the risk," he notes. "I gave some money to the candidates instead."

UPDATE 7:02 AM: What are the absentee ballots like where you're stationed? Tell us here or write in.

Citizen's Guide to Getting the Goods

The Freedom of Information Act isn't just for journalists or activist groups -- citizens (with and without blogs) can also petition the federal government to turn over documents. While it's rather simple to file a request, it's a bit more complicated to file one that actually gets you information.

The Electronic Frontier Foundation, which hired two of the best FOIA filers in the country this summer, just updated its legal guide for bloggers with a FOIA primer.

How do I know what to ask for?

News articles, government reports, press releases, and Congressional hearings are good starting points for thinking up FOIA request ideas.

How do I make a FOIA request?

You can make a FOIA request by mailing or faxing a letter to the agency. You may also be able to submit your request by email. Check the agency's web site for information about how and where to send requests.

Are there any step-by-step guides for writing and submitting FOIA requests?

Yes. Reporters Committee for Freedom of the Press has published a guide called How To Use the Federal FOI Act, and also has a FOI Letter Generator. The National Security Archive also has helpful guidance for FOIA requesters.

It's a bit simplified since government agencies vary widely in their attitude towards requests. The best advice is to make your request very narrow. Ask for a report by name (for instance, ask for the Pentagon's Inspector General's report on the Iraqi National Congress), instead of asking for all agency records about Chalabi and the INC. (BTW, there's a good possibility that report exists and hasn't been published).

Another fun place to start would be to follow on Michael Ravnitzky's FOIA work, which unearthed the indexes to four internal NSA publications, whose articles have tantalizing titles like "Was a Cryptologic Corporal." All you have to do is look through the indexes, find a title or two that interests you and ask for it. You just might get it.

Another place to get inspired is Russ Kick's The Memory Hole, a collection of documents he's built with FOIA requests he's filed after reading news articles. For instance, he's the one who got official pictures of the coffins of soldiers killed in Iraq when they landed at Dover Air Force base, after the photography ban was debated in the news.

You could be charged a small amount, but generally if it's going to be more than $25 dollars or so in fees, the agency will let you know.

And if an agency stonewalls you or ignores you, well, you can either sue yourself (not a good idea and even if you win, you don't get attorney's fees) or ask a group like EPIC or the First Amendment Center or a public interest law clinic to help.

Think of it like a letter to the editor or your congress critter, it's something every citizen should try at least once.

On an unrelated note, I'm pretty honored that Noah handed me the keys and I'll likely be focusing mostly on anti-terrorism and government database stuff since that's my normal beat.

But keep the tips and comments coming and together we'll keep DefenseTech humming while Noah racks up speeding tickets in 10 different states.

-- Ryan Singel

LifeLog Trials Begin

Those kooky, possibly-creepy defense programs are awfully hard to kill. Take LifeLog, Darpa's controversial project to archive almost everything about people -- where they've gone, what they've said, how they're feeling. The agency seemed to pull the plug on the program, after some pesky reporters started looking into it. But seven months later, large portions of the electronic diary effort were back, under a new name: Advanced Soldier Sensor Information System and Technology, or ASSIST.

06MSEL015_soldsens01_LR.jpgNow, Darpa is showing its LifeLog ASSIST handywork off, at the Aberdeen Proving Grounds. Soldiers there, wearing a ton of cameras and sensors, are going on mock-patrol through a simulated Iraqi village -- and recording the whole thing.

The sensors are expected to capture, classify and store such data as the sound of acceleration and deceleration of vehicles, images of people (including suspicious movements that might not be seen by the soldiers), speech and specific types of weapon fire.

A capacity to give GPS locations, an ability to translate Arabic signs and text into English, as well as on-command video recording also are being demonstrated in Aberdeen. Sensor system software is expected to extract keywords and create an indexed multimedia representation of information collected by different soldiers. For comparison purposes, the soldiers wearing the sensors will make an after-action report based on memory and then supplement that after-action report with information learned from the sensor data.

(Big ups: Boing Boing)

Watch List Snags Fellow Feds

How bad are the feds' enemy-of-the-state databases? So bad, they can't even keep fellow terror-hunters off their blacklists, Ryan Singel reports.

airlinetoy7.jpgThe Transportation Security Administration's airline screening system "tends to mistake government employees and U.S. servicemen for foreign terrorists," he writes in today's Wired News. "Newly released government documents show that even having a high-level security clearance won't keep you off the Transportation Security Administration's Kafkaesque terrorist watch list, where you'll suffer missed flights and bureaucratic nightmares."

According to logs from the TSA's call center from late 2004 -- which black out the names of individuals to protect their privacy -- the watch list has snagged...

* A high-ranking government employee with a better-than-top-secret clearance who is also a U.S. Army Reserve major...

* An active-duty Army officer who had served four combat tours (including one in Afghanistan) and who holds a top-secret clearance.

* A retired U.S. Army officer and antiterrorism/force-protection officer with expertise on weapons of mass destruction who was snared when he was put back on active-duty status while flying on a ticket paid for by the Army.

Now, I'm sure there have been improvements to the watch lists since 2004. But, as
Justice Department Inspector General Glenn Fine told Congress earlier this week, database managers still "had not ensured that the information in that database is complete and accurate. For example, the OIG found instances where the consolidated database did not contain names that should have been included on the watch list and inaccurate or inconsistent information related to persons included in the database."

The OIG's June 2005 report offered 40 recommendations to the TSC [Terrorist Screening Center] to address areas such as database improvements, data accuracy and completeness, call center management, and staffing. The TSC generally agreed with the recommendations and in some cases provided evidence that it has taken action to correct the weaknesses that the audit identified.

Since issuance of the audit, the TSC has initiated a record-by-record review of the terrorist screening database to ensure accuracy, completeness, and consistency of the records. TSC staff informed the OIG it is focusing first on the records deemed most important. According to the TSC, review of the entire database, which contains more than 235,000 [uh, make that 325,000] records, will take several years.

UPDATE 9:57 AM: Slashdot sez, "The Guardian newspaper has a great story about how the gathering of information for 'anti-terrorist' passenger screening databases allowed a reporter and security guru Adam Laurie to lay the groundwork for stealing the identity of a business traveller by using his discarded boarding-pass stub."

Stroke Me, Stroke Me

Oh, this is gonna be good. Ryan Singel, the man behind a zillion data-mining scoops, and cracker-legend-turned-editor Kevin Poulsen have teamed up for a new blog over at Wired News. 27B Stroke 6 (named for Brazil's most famous form) will "scare peace-loving people with phantoms of lost liberty, in a daily briefing on security, freedom and privacy in the wired world," according to Poulsen. I can't wait.

How AT&T; Helped the NSA Snoop

wiretap.jpgRyan Singel has himself a big, fat scoop. We already knew that telecom companies were cooperating with the NSA to eavesdrop on domestic and international communications. Now, Ryan reveals how it was done.

AT&T; provided National Security Agency eavesdroppers with full access to its customers' phone calls, and shunted its customers' internet traffic to data-mining equipment installed in a secret room in its San Francisco switching center, according to a former AT&T; worker...

According to a statement released by Klein's attorney, an NSA agent showed up at the San Francisco switching center in 2002 to interview a management-level technician for a special job. In January 2003, Klein observed a new room being built adjacent to the room housing AT&T;'s #4ESS switching equipment, which is responsible for routing long distance and international calls...

"While doing my job, I learned that fiber optic cables from the secret room were tapping into the Worldnet (AT&T;'s internet service) circuits by splitting off a portion of the light signal," Klein wrote.

The split circuits included traffic from peering links connecting to other internet backbone providers, meaning that AT&T; was also diverting traffic routed from its network to or from other domestic and international providers, according to Klein's statement.

The secret room also included data-mining equipment called a Narus STA 6400, "known to be used particularly by government intelligence agencies because of its ability to sift through large amounts of data looking for preprogrammed targets."

UPDATE 04/10/06 9:10 AM: Lots more on Naurus' data-sniffing products here, including one "capable of monitoring 10 billion bits of data per second."

NSA Wiretap Tips: Lame

There are a ton of problems with data mining for potential enemies of the state. Privacy is one, of course. But another is its questionable utility. It doesn't make you a jihadist, because you've e-mailed Chris Allbritton, who interviews guerillas sometimes. Or because you've said "bomb" and "trainwreck" in the same overseas call. Just look at all the hijinks with our "no-fly" lists, to see what an imprecise science we're talking about here.

eavesdrop.jpgSo I guess I'm not surprised to learn from tomorrow's New York Times that the NSA's domestic eavesdropping project -- which some seem to think is awfully similar to a rather infamous data mining program -- produced a "flood" of tips, and "virtually all of [which] led to dead ends or innocent Americans."

More than a dozen current and former law enforcement and counterterrorism officials, including some in the small circle who knew of the secret eavesdropping program and how it played out at the F.B.I., said the torrent of tips led them to few potential terrorists inside the country they did not know of from other sources and diverted agents from counterterrorism work they viewed as more productive.

"We'd chase a number, find it's a schoolteacher with no indication they've ever been involved in international terrorism - case closed," said one former F.B.I. official, who was aware of the program and the data it generated for the bureau. "After you get a thousand numbers and not one is turning up anything, you get some frustration..."

Officials who were briefed on the N.S.A. program said the agency collected much of the data passed on to the F.B.I. as tips by tracing phone numbers in the United States called by suspects overseas, and then by following the domestic numbers to other numbers called. In other cases, lists of phone numbers appeared to result from the agency's computerized scanning of communications coming in and out of the country for names and keywords that might be of interest. The deliberate blurring of the source of the tips caused some frustration among those who had to follow up.

F.B.I. field agents, who were not told of the domestic surveillance programs, complained they often were given no information about why names or numbers had come under suspicion. A former senior prosecutor, who was familiar with the eavesdropping programs, said intelligence officials turning over the tips "would always say that we had information whose source we can't share, but it indicates that this person has been communicating with a suspected Al Qaeda operative." He said, "I would always wonder, what does 'suspected' mean?"...

Aside from the director, F.B.I. officials did not question the legal status of the tips, assuming that N.S.A. lawyers had approved. They were more concerned about the quality and quantity of the material, which produced "mountains of paperwork" that was often more like raw data than conventional investigative leads.

"It affected the F.B.I. in the sense that they had to devote so many resources to tracking every single one of these leads, and, in my experience, they were all dry leads," the former senior prosecutor said.

Of course, any wide-spread investigation is going to mean a ton of dead ends. But, under normal circumstances, if there's a problem with the information you get, you can go back to your sources, ask more questions, hit them up again. If all you're getting is a list of names and numbers, however, there's no follow-up possible. No chance to prioritize the information. No way of telling whether this run of the algorithm is actually going to work, this time.

UPDATE 01/07/06 12:03AM: Does it strike anybody else as odd that the NSA's "unofficial ambassador," author James Bamford, is now suing to stop the domestic spying program? Do you think he'd be doing that without the tacit approval of at least some of his contacts within the agency?

UPDATE 01/01/06 12:29 PM: Al Gore was one of my least-favorite presidential candidates of all time. But he's got this NSA thing nailed.

President Lincoln, of course, suspended habeas corpus during the Civil War, and some of the worst abuses prior to those of the current administration were committed by President Wilson during and after World War I, with the notorious red scare and "Palmer Raids."

...But in each of these cases throughout American history, when the conflict and turmoil subsided, our nation recovered its equilibrium and absorbed the lessons learned in a recurring cycle of excess and regret.

But there are reasons for concern this time around that conditions may be changing so that this cycle may not repeat itself. For one thing, we have for decades been witnessing the slow and steady accumulation of presidential power....

A second reason to believe that we may be experiencing something new, outside that historical cycle, is that we are, after all, told by this administration that the war footing upon which he has tried to place the country is going to last, in their phrase, "for the rest of our lives."

And so we are told that the conditions of national threat that have been used by other presidents to justify arrogations of power will in this case persist in near perpetuity.

Third, we need to be keenly aware of the startling advances in the sophistication of eavesdropping and surveillance technologies with their capacity to easily sweep up and analyze enormous quantities of information and then mine it for intelligence. And this adds significant vulnerability to the privacy and freedom of enormous numbers of innocent people at the same time as the potential power of those technologies grows.

Those technologies do have the potential for shifting the balance of power between the apparatus of the state and the freedom of the individual in ways that are both subtle and profound.

Don't misunderstand me. The threat of additional terror strikes is real and the concerted efforts by terrorists to acquire weapons of mass destruction does indeed create a real imperative to exercise the powers of the executive branch with swiftness and agility.

Moreover, there is an in fact an inherent power conferred by the Constitution to any president to take unilateral action when necessary to protect the nation from a sudden and immediate threat. And it is simply not possible to precisely define in legalistic terms exactly when that power is appropriate and when it is not.

But the existence of that inherent power cannot be used to justify a gross and excessive power grab lasting for many years and producing a serious imbalance in the relationship between the executive and the other two branches of government.

NSA Spying: Two Views

What's behind the NSA domestic eavesdropping program? And how bad it is, really? Defense analyst Willliam Arkin and law professor Orin Kerr have competing theories.

wiretap_cover.jpgArkin takes a peek at section 126 of the USA PATRIOT Improvement and Reauthorization Act Of 2005, which requires the Attorney General to submit a report to Congress "on any initiative of the Department of Justice that uses or is intended to develop pattern-based data-mining technology." He wonders if that data-mining might be what the NSA is up to.

Patterns of activity associated with actual terrorists in the past are derived from investigations and debriefings -- let's say, for example, visas from certain countries, calls from public phone booths to Pakistan, renting of cars with newly acquired driver's licenses, one-way airline tickets. Patterns are used to trigger "tip-offs."

Massive amounts of collected data -- actual intercepts of phone calls, e-mails, etc. -- together with "transaction" data -- travel or credit card records or telephone or Internet service provider logs -- are mixed through a mind-boggling array of government and private sector software programs to look for potential matches...

The law says "the search does not use personal identifiers of a specific individual or does not utilize inputs that appear on their face to identify or be associated with a specified individual to acquire information," I take it to mean the new computer-based data mining isn't looking for an individual per se, it is looking at information about all individuals (at least all who make international telephone calls or send e-mails overseas or travel to foreign countries according to the government) to select individuals who may be worthy of a closer look.

In other words, with the digitization of everything and new computer and software capabilities, the government couldn't go to the Court or the Congress and say, "hey, we'd like to monitor everyone on a fishing expedition to find the next Mohamed Atta."

Senator Jay Rockefeller and others have made noises that the NSA project reminds them of the most notorious of data-mining efforts, Total Information Awareness, or TIA.

But Kerr, leafing through James Risen's new book, says that "it seems less likely to me than it did before that this is a TIA-like data-mining program."

"As best I can tell, the NSA program was not actually recording domestic Internet traffic, putting it in a database, and then 'mining' it for key words and the like," he writes. Instead, what went on is packet-sniffing -- "installing a monitoring device on a steam of traffic that looks for specific sequences of letters, numbers, or symbols... [like] phone numbers and e-mail accounts... For those with criminal law experience, this was basically a large-scale pen regsister/trap-and-trace or wiretap, depending on how the filters are configured."

Which, of course, would be a whole lot less scary than some ginormous profiling project. We'll see.

(Big ups: David)

UPDATE 10:50 AM PST: FBI whistle-blower Coleen Rowley calls BS on claims that the courts somehow got in the way of catching Zacarias Moussaoui, the so-called "20th hijacker." NSA whistle-blower Russ Tice, says he wants to talk about the agency's "highly classified Special Access Programs." A little birdie tells me that he won't be the last.

Navy Wants Insurgent-Predicting Program

It was senior year, and I had just taken a semester off to work for the Clinton campaign in Philadelphia. So I figured it'd be the easiest A ever if I signed up for an urban politics class.

Carson-Karnak.jpgThe professor, a pearl-wearing blond fresh out of grad school, confessed she had never actually lived in a city before. But that didn't stop her from having all kinds of theories about how urban politics really worked. And that included a formula --- a mathematical formula -- that she said described how mayors and aldermen made their decisions. I think I laughed out loud when she first wrote it on the blackboard.

This Navy proposal (scroll down) is way more serious, of course. And they claim that it's already worked before. But I couldn't help thinking of that professor back at Georgetown, when I read about the Navy's idea to use a computer program to predict insurgent attacks in places like Iraq.

In current U.S. operations, terrorist and insurgent forces enjoy a significant advantage by being able to launch surprise attacks, whether by small arms, mortar, or improvised explosive devices (IEDs), against weakly defended or undefended targets and disappearing before U.S. forces can concentrate for a counterstrike. Better prediction of where and when such attacks are most likely to occur would therefore be of great benefit, allowing smart allocation of defensive resources as well as preparation for quick counteroffensive operations in response to terrorist and insurgent attacks. This task is significantly complicated by the fact that modern terrorist groups demonstrate an ability to learn and adapt quickly, making it difficult to predict future actions on the basis of past actions.

Recent work has applied and extended discrete choice models originally developed for use in econometrics to predicting the spatial probability of criminal activity. These point-pattern based density models have also been applied to the military domain for prediction of terrorist strikes and IEDs. The result is that the geographical patterns established by past events can be used to build threat maps showing where future strikes are most likely to take place, with accuracies notably better than hot-spotting techniques. The same basic strategy seems likely to be applicable to prediction of the timing of such activities as well as their location.

The technique utilizes as inputs a series of IED incidents... The models typically contain large numbers of attributes, such as population density, proximity to a police station, distance to a mosque, etc. From case to case different attributes and different numbers of attributes are important. For example, when this technique was applied to bombings in greater Jerusalem, it was found that a single attribute, the distance to a controlled intersection, was an accurate predictor.

A fundamental limitation of the techniques as they stand, however, is that they do not model changes in the subjects' decision-making processes; they must currently assume that the subjects' preferences are static. This limits the time horizon over which predictions are of use, and can cause periods of very poor prediction performance when a significant change in strategy occurs. An extension of discrete choice models that allows for learning-directed evolution in the subjects' decision-making processes would greatly improve their applicability to dynamic military situations.

The program is part of a larger effort to address the "human element" of the IED problem, National Defense reports.

"I'd like to be able to pick the terrorist out. I'd like a detector 'tricorder' for intent or evil. I'd like to know ahead of time that this person is planning to hurt other people with the use of IEDs," Office of Naval Research chief scientist Starnes Walker told the magazine.

This project won't do that, of course. But getting it right "will not only contribute to defensive operations, saving lives of civilians and U.S. servicemen, but will also contribute to quick and effective counterstrikes to weaken and eliminate enemy forces," the Navy notes. "The same techniques can be applied to civilian law enforcement to counter gangs, organized crime, and other groups with the capacity to adapt their patterns of behavior through experience."

Maybe it could even predict politicians' behavior, too.

NSA "Tapping Into... Telecom's Main Arteries"

nsa_hq.jpg"The National Security Agency has traced and analyzed large volumes of telephone and Internet communications flowing into and out of the United States... by tapping directly into some of the American telecommunication system's main arteries," the Times is reporting.

The volume of information harvested from telecommunication data and voice networks, without court-approved warrants, is much larger than the White House has acknowledged...

As part of the program approved by President Bush for domestic surveillance without warrants, the N.S.A. has gained the cooperation of American telecommunications companies to obtain backdoor access to streams of domestic and international communications.

When the NSA domestic spying story broke last week, I had a hunch that the eavesdropping technology at work was a whole lot different than what you'd find in an average wiretap. A former signals intelligence specialist wondered whether the NSA "may have compromised... a telecom carrier."

That guess looks to be dead-on.

Since the Sept. 11 attacks, the leading companies in the industry have been storing information on calling patterns and giving it to the federal government to aid in tracking possible terrorists.

"All that data is mined with the cooperation of the government and shared with them, and since 9/11, there's been much more active involvement in that area," said the former manager, a telecommunications expert who did not want his name or that of his former company used because of concern about revealing trade secrets.

The Times article also makes clear why Senator Jay Rockefeller compared the program to Total Information Awareness, the Pentagon's uber-database project.

The N.S.A. has sought to analyze communications patterns to glean clues from details like who is calling whom, how long a phone call lasts and what time of day it is made, and the origins and destinations of phone calls and e-mail messages. Calls to and from Afghanistan, for instance, are known to have been of particular interest to the N.S.A. since the Sept. 11 attacks, the officials said.

This so-called "pattern analysis" on calls within the United States would, in many circumstances, require a court warrant if the government wanted to trace who calls whom.

The use of similar data-mining operations by the Bush administration in other contexts has raised strong objections, most notably in connection with the Total Information Awareness system... [which was] ultimately scrapped after public outcries over possible threats to privacy and civil liberties.

But the Bush administration regards the N.S.A.'s ability to trace and analyze large volumes of data as critical to its expanded mission to detect terrorist plots before they can be carried out, officials familiar with the program say. Administration officials maintain that the system set up by Congress in 1978 under the Foreign Intelligence Surveillance Act does not give them the speed and flexibility to respond fully to terrorist threats at home.

Some will say this story is old news. The NSA has long been rumored to have the ability to vacuum up huge swaths of data at once.

"The NSA is intercepting huge streams of communications, taking in 2 million pieces of communications an hour," James Bamford, the author of two books on the NSA, told the Boston Globe on Friday.

"They have a capacity to listen to every overseas phone call," added Tom Blanton, director of the National Security Archive at George Washington University."

But the question has been: how do you turn all that data into something useful? You've got to find a realtively simple way to get rid of 99.99999% of the calls and e-mails quickly. Otherwise, it's like drinking from a firehose.

But as link analysis and data mining programs have become more sophisticated, that sifting process has gotten easier. And, I'll bet, it is simpler still when the telecom companies are playing ball.

No-Fly List Follies

line.jpgIt's been a while since we've tuned in to the long-running comedy "Secure Flight." That's the one where the feds try to screen airline passengers based on their data trails -- and wind up breaking the law and falling on their faces in the process. Defense Tech pal Ryan Singel catches us up on all the new plotlines.

First up is the story of Sister Glenn Anne McPhee, the Catholic education chief who was mistaken for an Afghani terrorist -- and put on the Transportation Security Administration's "no-fly" list. A similar screw-up just cost a pilot his job.

"Collecting full names and birth dates will reduce false matches by 60%," a top TSA data-miner says. So will snagging "marriage and birth certificates, credit-card records, court filings, [and] newspaper clippings," supposedly. (Cue laugh track.)

In a rare break with character, the TSA decided in last week there might, in fact, be some "privacy concerns" in harvesting all that commercial data. So the administration will knock it off, for now. Of course, this is after the TSA "secretly tested this procedure" on 100 million passenger records.

The privacy worries are one reason why a Secure Flight advisory panel has recommended that all live testing of the system be stopped. There are one or two other minor concerns, as well. Small stuff, like "What is the goal or goals of Secure Flight?" and "What is the architecture of the Secure Flight system?"

Jeez. Now I remember why I never bother to watch this show. Somebody, hand me the remote.

THERE'S MORE: Last month, BJ notes, the feds supposedly trashed three million of its suspicious passenger records. Bill wonders whether that was housecleaning effort or "destruction of evidence?"

"MATRIX" UNLOADED

Score one for the good guys. A project to find enemies of the state in the credit card records, marriage licenses, and vehicle registration data of avergage citizens appears to be over.

carrieanne.jpgAt one time, at least a dozen states had jacked into the Multistate Anti-Terrorism Information Exchange, or MATRIX. Dozens more were considering participation in the four billion-record database. But then came the howls from privacy advocates. And the revelations that MATRIX's founder had been linked to Bahamian drug smugglers in the '80s.

In the end, only Pennsylvania, Florida, Ohio and Connecticut were left. And now that $12 million in federal grant money has run out, it seems unlikely that these states will continue with the project.

“This may be the biggest victory for privacy since we and our allies from across the political spectrum shut down Total Information Awareness,” the ACLU's Barry Steinhardt said in a statement, referring to a similarly Owrellian Pentagon program shuttered by Congress in 2003.

GRAMMAR FOR SPYBOYS

Maybe sixth-grade English was more helpful than we thought. One of the dullest grammar exercises is being used to help find potential terrorists, and save companies a bundle.

03next.jpgDiagramming sentences - picking out subject, verb, object, adjective and other parts of speech - has been a staple of middle and high school grammar lessons for decades. Now, with financing from the Central Intelligence Agency, a California firm is using the technique to comb through e-mail messages and chat room talks, which can be a rich lode of corporate and government information, and a tough one to mine.

Figuring out the connections among people, places and things is something computer algorithms do pretty well, as long as that information is structured, or categorized and put into a database. Looking through a company's customer file for a person named Bonds, for example, is fairly simple. But if the data is unstructured - if the word "bonds" hasn't been classified as the name of a ballplayer or as an investment option - searching becomes much more difficult.

For people in business or in public service, only 20 percent or so of their information is kept in formal databases, noted Nick Patience, an analyst with the 451 Group, a technology research firm. The rest is unstructured, tucked away in e-mail messages, call logs, memos and instant messages.

Attensity, based in Palo Alto, Calif., and financed in part by In-Q-Tel, the C.I.A.'s investment arm, has developed a method to parse electronic documents almost instantly, and diagram all of the sentences inside. ("Moby-Dick," for instance, took all of nine and a half seconds.) By labeling subjects and verbs and other parts of speech, Attensity's software gives the documents a definable structure, a way to fit into a database. And that helps turn day-to-day chatter into information that is relevant and usable.

My article in today's New York Times had details.

"CARNIVORE" CHEWED UP

carnivore-small.jpgBefore Total Information Awareness, before MATRIX, before Secure Flight, and before CAPPS II, the government data-diving project that gave civil libertarians fits was the FBI's Carnivore. Used in tandem with other Bureau tools, Carnivore could monitor a target's Internet traffic, piecing together e-mail messages and web-surfing history.

But Carnivore has been abandoned, according to Security Focus' Kevin Poulsen. And it's not because the Feds have decided that it's no longer cool to peek into a person's inbox. Rather, Carnivore has been outpaced, it appears. The Bureau is now using "commercially-available products to conduct Internet surveillance" instead.

THERE'S MORE: "If you're among the millions of Americans who took airline flights in the months before the Sept. 11, 2001 terrorist attacks, the FBI probably knows about it - and possibly where you stayed, whom you traveled with, what credit card you used and even whether you ordered a kosher meal."

CAPPS' EVIL STEPBROTHER

The eerily invasive passenger-screening program CAPPS II may be dead, "but its evil stepbrother, 'Secure Flight,' will live if we don't complain loudly enough," says Defense Tech pal Bill Scannell. "If something isn't done soon, the passenger records of over 54 million Americans will be handed-over to the [Homeland Security Department] by the airlines. The time to file your comments is now. We've built an interface that links directly into the 'Secure Flight' comments database. The Bill of Rights you save may be your own.

PENTAGON BOARD: SPY ON EVERYONE

The only way to win the war on terror is to track everyone, and everything, that moves.

That, according to ISR Journal, is the conclusion of an influential group of Pentagon advisers, the Defense Science Board. "Technologies that can identify people by unique physical characteristics — fingerprint, voice, odor, gait or even pattern of iris — must be merged with new means of 'tagging' so that U.S. forces can find enemies who escape into a crowd or slip into a labyrinthine slum," says a DSB study, completed over the summer.

“The global war on terrorism cannot be won without a ‘Manhattan Project’-like TTL [tagging, tracking, and locating] program,” briefing charts summarizing some of the study’s findings say...

This tagging and tracking could be used for:

• People or groups such as enemy leaders or sympathizers, nuclear weapons or explosives experts, and terrorist paymasters.

• Things such as weapons of mass destruction, materials or components, precision machinery, pharmaceutical plants, specialized instruments, pathogens and seed stocks or vehicles.

• Activities such as recruiting, financial transactions, Internet activity, pathogen genome sequencing or organizational activity or meeting.

How much would it cost to bring these sci-fi technologies to the real world? Doesn't matter, the Board declares. "Cost is not the issue; failure in the global war on terrorism is the real question."

Long-time Defense Tech readers will find this whole thing terribly familiar. Last year, Pentagon mad science arm Darpa introduced a plan to use security cameras to monitor an entire city at once. The program will receive $4 million in the fiscal year '05 budget. And Mayor Daley is trying to do something similar in Chicago.

Like the Darpa effort, the DSB plans to track all these irises, and all this Internet activity, in places like Iraq and Afghanistan, where insurgents have a nasty habit of melting away into the background. But, of course, if these technologies were ever successfully developed, the temptation to use it to track enemies of the states here at home would be mighty strong, too.

JUNE FLIERS BECOMING ANTI-TERROR GUINEA PIGS

If you flew on a plane in June, your personal information is about to be dumped into the Department of Homeland Security's new terror-screening database.

"The Transportation Security Administration will use passenger data from June 2004 from 77 domestic carriers to test the Secure Flight program, which is designed to check airline passenger names against a centralized terrorist watch list," Defense Tech pal Ryan Singel writes.

The program is a scaled-back successor to CAPPS II, which the TSA scuttled after months of criticism from privacy advocates and disclosures that early CAPPS II contractors secretly got data from major U.S. airlines.

Secure Flight will expand on the current use of watch lists by using a centralized terrorist watch list run by the Terrorist Screening Center housed at the FBI.

The center's director, Donna A. Bucella, told Congress in March the list is now 120,000 names long.

The data poured in Secure Flight "will vary by airline," the Times notes. "It will include each passenger's name, address and telephone number and the flight number. It may also include such information as the names of traveling companions, meal preference, whether the reservation was changed at any point, the method of ticket payment and any comment by airline employees, like whether a passenger was drunk or belligerent in encounters with airline personnel."

PENTAGON'S LIFELOG REVIVED?

It's been seven months since the Pentagon pulled the plug on LifeLog, its controversial project to archive almost everything about a person. But now, the Defense Department seems ready to revive large portions of the program, under a new name.

Using a series of sensors embedded in a G.I.'s gear, the Advanced Soldier Sensor Information System and Technology (ASSIST) project aims to collect what a soldier sees, says, and does in combat zone – and then to weave those events into digital memories, so commanders can have a better sense of how the fight unfolded.

That's similar to what planners at Pentagon research arm Darpa had in mind for LifeLog, its ultra-ambitious electronic diary effort. But ASSIST's aspirations are more modest, its battlefield focus is clearer, and its privacy concerns are more manageable, military analysts and computer scientists say. All of that combines to give the project a better chance of taking off where LifeLog crashed.

"Welcome to the wacky ways of contracting at the Defense Department. If it doesn't fly the first time around, you can be sure it'll be back. And so it is," said Steven Aftergood, with the Federation of American Scientists. "This time around, though, the work has a slightly more plausible context. And more of an effort has been made to connect it to a military application."

My Wired News article has details.

FBI IT = FUBAR

The FBI's "Trilogy" computer-upgrade project has come to be known as one of the great information technology disasters of all time -- the "Gigli" of computing. Now, the New York Times reports, a key part of Trilogy -- the Virtual Case File -- won't be able to deploy by the end of the year, as promised. And FBI officials "could not predict when the entire system would be in place. As a result, an important technological component of the administration's domestic security effort remains in limbo."

The Virtual Case File system, which would allow agents to share information easily — a critical shortcoming of the present system — is already two years behind schedule and one bureau official who spoke on condition of anonymity went so far as to suggest that the program might ultimately have to be abandoned...

In the aftermath of the hijackings, Robert S. Mueller III, the F.B.I. director, told a Senate panel that the bureau's computer system was so limited that it could not search its files for combinations of terms like "flight" and "schools," precisely the kind of combination that might have helped to discern the patterns of activity leading up to the attacks. Instead, Mr. Mueller said, the system could search for words like "flight" and "school" only one at a time...

According to a staff report from the bipartisan commission investigating the Sept. 11 attacks, the F.B.I.'s primary information system, which was designed using 1980's technology, was "already obsolete when installed in 1995." The commission report said that "field agents usually did not know what investigations agents in their own office, let alone in other field offices, were working on."

For now -- and for the forseeable future -- that's how things will stay.

ALASKANS SUE FEDS OVER CAPPS II

Alaskans depend on planes to go just about anywhere. Mess with their ability to fly, and they tend to get pretty pissed off.

So maybe it was only a matter of time before a bunch of Alaskans got together to sue the Transportation Security Administration over CAPPS II, the feds' controversial airline passenger screening program.

"Outside government bureaucrats think we need their permission before we can get on a plane. We think they're wrong, so we're turning to the US District Court for help," the plaintiffs say on their website.

CAPPS II ran into a brick wall of bad press after it came out that JetBlue and other airlines turned passenger information over to the government. That ended a fairly cozy relationship between the TSA and the carriers. Now, "the airline industry has made it clear that it will not participate in CAPPS II unless ordered to do so," reports Defense Tech homie Ryan Singel in today's Wired News.

"Out of frustration, Adm. James Loy, then head of the Transportation Security Administration, threatened in September to issue a secret directive to force hesitant airlines to share the data," Singel continues. "If it follows through, the TSA would require airlines to forward all passenger information to the system, including date of birth, home phone numbers and addresses."

"We think the Feds need to tell us what they're planning before they start turning every flight we take into an excuse to snoop," the Alaskans respond. "The TSA didn't bother responding to a letter we sent, so we're asking the US District Court in Anchorage to help us find out the truth."

MATRIX GOT WHITE HOUSE SHOWING

Hoiw did the creators of the norotious MATRIX database project get the federal government to pony up $8 million for the system? By showing it off to top officials in the Roosevelt Room of the White House in January 2003, the Washington Post reports.

Accompanied by Florida Gov. Jeb Bush and the state's top police official, [MATRIX creator Hank] Asher showed his creation to Vice President Cheney, FBI Director Robert S. Mueller III and Tom Ridge, who was about to be sworn in as secretary of the new Department of Homeland Security, according to people at the meeting.

The demonstration startled everyone in the room who had not seen it before. Almost as quickly as questions could be asked, the system generated long reports on a projection screen: names, addresses, driver license photos, links to associates, even ethnicity. At one point, an Asher associate recalled, Ridge turned toward Cheney and nudged him with an elbow, apparently to underscore his amazement at the power of what they were seeing. A few months later, Ridge approved an $8 million "cooperative agreement" from his department to help states link to the computer system.

MATRIX GAUGES "TERROR QUOTIENT'

AP: "Before helping to launch the criminal information project known as Matrix, a database contractor gave U.S. and Florida authorities the names of 120,000 people who showed a statistical likelihood of being terrorists - sparking some investigations and arrests."

DATA MINER SAYS NO TO CAPPS II

If you've been screened for a new job, hassled by a telemarketer, or asked to fill out an insurance claim, chances are the data aggregation company ChoicePoint had something to do with it. So the firm isn't exactly shy about collecting lots of information about lots of people.

But even for this notoriously invasive company thinks the Homeland Security Department is going too far in its attempts to snoop on airline passengers.

ChoicePoint has dropped out of CAPPS II, the government's controversial passenger-screening program, according to GovExec.com. What's more, the company's CEO threw cold water on the whole idea that the feds could find potential terrorists in the data trails of ordinary people.

Smith said CAPPS II is too much like the Terrorism Information Awareness program once proposed by the Defense Advanced Research Projects Agency to mine commercial data because CAPPS II attempts to ferret out data about 280 million individual Americans.

Smith termed that approach "probabilistic theory" and said law enforcement and private businesses seeking to verify individuals' identities should instead take advantage of "link analysis." The latter approach concentrates first on suspected terrorists and seeks information about anyone who might be connected to them.

"Today, we are looking for small groups of people, or needles in a haystack," he said. "The last thing you want to do is put more hay on the haystacks."

TSA TO AIRLINES: PAPERS, PLEASE

The Transportation Security Administration has decreed that it will soon order airlines to turn over passengers' personal records for the hotly-contested CAPPS II traveller screening program.

MATRIX SHRINKS -- MAYBE

New York and Wisconsin are the latest states to pull out of the notorious MATRIX data mining effort. That means only five states are officially left in the program, according to Wired News.

But "whether they know it or not, at least 33 states have released government and commercial records on residents to MATRIX," the Salt Lake Tribune reports.

THERE'S MORE: The New York Times looks at MATRIX's rapid decline in Monday's edition.

BLACK "TIA"-LIKE RESEARCH REVEALED

We've been saying for months not to get too happy over the supposed "death" of Total Information Awareness. Now, Defense Tech Pal Mike Sniffen reports that when Congress moved to defund TIA, it "quietly agreed to continue paying to develop highly specialized software to gather foreign intelligence on terrorists."

In a classified section summarized publicly, Congress added money for this software research to the "National Foreign Intelligence Program," without identifying openly which intelligence agency would do the work.

It said, for the time being, products of this research could only be used overseas or against non-U.S. citizens in this country, not against Americans on U.S. soil.

Congressional officials would not say which Poindexter programs were killed and which were transferred. People with direct knowledge of the contracts told the AP that the surviving programs included some of 18 data-mining projects known in Poindexter's research as Evidence Extraction and Link Discovery.

THERE'S MORE: Phil Carter has a nice analysis of the risks of TIA-like efforts.

CONGRESS THINKS CAPPS II STINKS

The CAPPS II passenger screening system is seriously screwed up, a new Congressional report finds. The General Accounting Office, Congress' investigative arm, has spent the last four months studying the controversial program. Its conclusion (according to the L.A. Times):

"Uncertainties surrounding the system's future functionality and schedule alone result in the potential that the system may not meet expected requirements, may experience delayed deployment, and may incur increased costs."

The GAO report found that the Transportation Security Administration, which runs CAPPS II, hadn't adequately addressed seven of eight concerns raised by Congress about the system.

The Times says that "these include preventing abuses, protecting privacy, creating an appeals process, assuring the accuracy of passenger data, testing the system, preventing unauthorized access by hackers and setting out clear policies for the system."

THERE'S MORE: The GAO report is now online here.

ACXIOM & "TIA": MATCH MADE IN HELL

By now, regular Defense Tech readers are familiar with Acxiom, the data aggregation company that's supplying your personal information to government data-mining efforts like CAPPS II. Today, it came to light that the company was being considered as a supplier for Total Information Awareness, Darpa's uber-database project.

The Electronic Privacy Information Center has obtained internal Darpa e-mail about using Acxiom in TIA experiments. According to one message, Jennifer Barrett, Acxiom's Chief Privacy Officer, gave Darpa advice on how to keep objections to TIA to a minimum.

"One of the key suggestions she made is that people will object to Big Brother, wide-coverage databases, but they don't object to use of relevant data for specific purposes that we can all agree on. Rather than getting all the data for any purpose, we should start with the goal, tracking terrorists to avoid attacks, and then identify the data needed (although we can't define all of this, we can say that our templates and models of terrorists are good places to start)," wrote Darpa's Lt. Col. Doug Dyer. "Already, this guidance has shaped my thinking."

CLARK: FATHER OF CAPPS II?

Did Gen. Wes Clark push the federal government into the controversial CAPPS II passenger screening system? That's what Farhad Manjoo suggests in today's Salon.

THERE'S MORE: The Washington Post is reporting that Ben H. Bell III, who's heading up the CAPPS II program for the Transportation Security Administration, has just resigned. The Post article doesn't give a reason for the resignation. But a spokesperson for the agency says Bell's departure won't impact the roll-out of the passenger screening effort.

LIFELOG DEAD

The Pentagon has pulled the plug on LifeLog, its stunningly ambitious effort to build a database tracking a person's entire existence.

Run by Darpa, the Defense Department's research arm, LifeLog aimed to gather in a single place just about everything an individual says, sees or does: the phone calls made, the TV shows watched, the magazines read, the plane tickets bought, the e-mail sent and received. Out of this seemingly endless ocean of information, computer scientists would plot distinctive routes in the data, mapping relationships, memories, events and experiences.

LifeLog's backers said the all-encompassing diary could have turned into a near-perfect digital memory, giving its users computerized assistants with an almost flawless recall of what they had done in the past. But civil libertarians immediately pounced on the project when it debuted last spring, arguing that LifeLog could become the ultimate tool for profiling potential enemies of the state.

Researchers close to the project say they're not sure why it was dropped late last month. Darpa hasn't provided an explanation for LifeLog's quiet cancellation. "A change in priorities" is the only rationale agency spokeswoman Jan Walker provided.

However, related Darpa efforts concerning software secretaries and mechanical brains are still moving ahead as planned.

LifeLog is the latest in a series of controversial programs that have been canceled by Darpa in recent months. The Terrorism Information Awareness, or TIA, data-mining initiative was eliminated by Congress -- although many analysts believe its research continues on the classified side of the Pentagon's ledger. The Policy Analysis Market, which provided a stock market of sorts for people to bet on terror strikes, was almost immediately withdrawn after its details came to light in July.

"Darpa's pretty gun-shy now," added Lee Tien, with the Electronic Frontier Foundation, which has been critical of many agency efforts. "After TIA, they discovered they weren't ready to deal with the firestorm of criticism."

My Wired News article has details on LifeLog's cancellation.

THERE'S MORE: LifeLog may be dead, but Darpa still has plenty of creepy data-mining programs, the BBC notes.

Imagine being able to pinpoint someone's location anywhere in the world simply by typing a few keywords on your PC. That is what software partly funded by the US military is trying to do.

The MetaCarta program works by analysing thousands of documents and cross-checking the results with a massive geographical database...

The software automatically extracts geographic references from text documents such as e-mails or webpages. Millions of documents can be searched using keywords, place names or a time reference. Search results appear as points on a map instead of as a list of documents. The company says this information can be used, for example, to track patterns of criminal activity and identify spots of intensity.

(via /.)

BASTARD CHILDREN OF "TIA"

Congress supposedly killed Total Information Awareness, Darpa's far-flung effort to comb databases in search of terrorists. But that doesn't mean the authorities are finished sorting through the records of Americans to expose evildoers. Many analysts think bits of TIA still exist on the covert, "black" side of the Pentagon's ledger. This month's Wired magazine has my rundown of the unclassified efforts, with straight out of government and corporate documents.

THERE'S MORE: Former Utah Governor Mike Leavitt signed his state up to the notorious MATRIX (Multistate Anti-Terrorism Information Exchange) data mining effort, according to the Deseret News. That's the TIA-esque project run by former drug smugglers for eight state governments.

AND MORE: According to the Deseret News, Utah's current governor, Olene Walker, "has pulled the plug on the state's participation in the controversial MATRIX database — at least until a joint governor- legislative oversight committee can hold public hearings about the program that collects comprehensive dossiers on every resident."

AND MORE: Nice! Georgia Governor Sonny Perdue "ordered the GBI [Georgia Bureau of Investigation] on Friday to sever all ties with the controversial Matrix crime-fighting database," the Atlanta Journal-Constitution reports.

This is the second time Perdue has made such a declaration. After the first one, the Georgia authorities continued to "pump information into the database," the Associated Press notes.

AND MORE: Despite all this, many state governments are still showing an interest in jacking into MATRIX, according to the AP:

Mark Zadra, chief investigator for Florida state police, which runs the Matrix project, said organizers have given presentations to more than 10 Northeastern and Midwestern states in recent weeks, arguing at each stop that the database is an invaluable law enforcement tool.

Officials in Iowa and North Carolina said Friday that they are exploring the system. And documents obtained through a public-records request in Florida indicate Arizona and Arkansas also may have interest in the quick-access information repository, which combines state records with 20 billion pieces of data held by a private company.

AIR SAFETY INFO IN NASA TERROR DATABASE

NASA researchers are using flight-safety records -- including reports of sick passengers, bad weather and sleepy pilots -- to build an anti-terror database.

Under the generic name Data Mining and Aviation Security, computer scientists at NASA's Ames Research Center are developing a program for predicting terrorist threats by integrating "the Internet and classified intelligence data" with information from two flight-safety databases.

The program is the second recent example of a NASA effort to mine information storehouses for enemies of the state. Over the weekend, the Electronic Privacy Information Center (EPIC) revealed that Northwest Airlines forked over millions of passenger records to the space agency for a terrorist-screening project, an effort enhanced with data from the 1990 U.S. census.

Although the new program's budget is undersized -- less than $1 million, according to Ames spokesman David Morse -- civil libertarians are troubled by the effort. Such projects are a waste of resources, they say, especially at a time when the space agency is gearing up for a return to the Moon.

"This is 21st-century phrenology," said privacy advocate Bill Scannell, referring to the discredited art of reading people's personalities from the bumps on their heads. "You might as well stick a couple of employees in a sub-basement and have them read tea leaves."

My Wired News article has details.

THERE'S MORE: Using flight safety reports for homeland defense is a "pretty wild experiment," said Barry Steinhardt, director of the ACLU's technology and liberty program. "They're literally attempting to monitor us through our entrails, rather than focusing on the physical security measures that we know work."

For example, Steinhardt notes, airport tarmacs still aren't secure. Just last month, a dead body was discovered at New York's Kennedy Airport, wedged into the wheel of a British Airways 747. How the person got on to the tarmac – and how the corpse managed to go undetected through 15 take offs and landings in eight days – remains a mystery.

AND MORE: Sandra Hart, with the Human Factors Research and Technology Division of NASA Ames, wants to fight terrorists -- bad. She makes the following plea on the American Psychological Association's website to let her and her colleagues get involved with stopping evil-doers:

We can develop system-wide baseline and trend information to identify gaps and vulnerabilities in the security system. Data mining and visualization tools can be adapted to convey security information clearly and unambiguously. Data acquisition and analysis tools can identify patterns in routes of flight as well as passenger profiles. We can help establish policies, design technologies, and develop procedures to ensure that the people in the system are even more effective. We can predict the potential impact of new ideas on the reliability and effectiveness of the system and then evaluate them as they are developed and fielded. Basic knowledge of human vision, cognition, attention, and so on can improve the design of security technologies. Expertise in organizational and team behavior might be applied to the formation of more effective security teams and mitigate the proliferation of ad hoc responses by pilots and controllers in response to perceived threats. Human Factors expertise in task analysis, modeling, and simulation can offer insights into the skills required to perform crucial tasks, identify functions that are candidates for automation, and develop training. We can work with the front line - - security personnel, ticket agents, pilots, flight attendants and their employers - - to identify security gaps and figure out how to ensure that humans are part of the solution in the future, not the problem. (emphasis mine)

Just in case you thought only a couple of NASA mad scientists wanted to find Osamas in our information...

AND MORE: EPIC will file a suit against NASA tomorrow in U.S. District Court in San Jose, to force the space agency to spill the beans about all of its anti-terror efforts.

CENSUS, NORTHWEST AIR GAVE FLIERS' INFO TO FEDS

Northwest Airlines and the Census Bureau teamed up to supply personal information on millions of people to a clandestine government travel security project -- one that's awfully similiar to the notorious Total Information Awareness and CAPPS II data-mining efforts.

Last fall, JetBlue Airways forked over millions of passenger records to a defense contractor, who used it to test a terrorist-screening program. Around the same time, Northwest, the country's fourth-largest airline, claimed that they would never, ever do such a thing themselves.

"But Northwest acknowledged Friday that by that time, it had already turned over three months of reservation data to the National Aeronautics and Space Administration's Ames Research Center," notes the Washington Post.

Using the travel data, government-funded researchers were able to track down the 1990 Census records of 439,381 passengers. This information was added to refine the data-mining project.

"Information given by American citizens for reasonable demographics information has been turned around and used to spy on people. This sounds like East Berlin, circa '74," privacy advocate Scannell tells the Washington Times.

The Northwest discovery came after the Electronic Privacy Information Center filed Freedom of Information Act requests with the government. The documents unveiled from that request are here.

CAPPS II LIVES

Congress is edgy. Privacy advocates are apoplectic. And the airlines won't play ball.

But despite the resistance, reports the Washington Post, the U.S. government is going ahead with CAPPS II -- the far-flung database program that crunches information on every passenger headed into the sky.

The government will compel airlines and airline reservations companies to hand over all passenger records for scrutiny by U.S. officials, after failing to win cooperation in the program's testing phase. The order could be issued as soon as next month. Under the system, all travelers passing through a U.S. airport are to be scored with a number and a color that ranks their perceived threat to the aircraft...

It will collect travelers' full name, home address and telephone number, date of birth and travel itinerary. The information will be fed into large databases, such as Lexis-Nexis and Acxiom, that tap public records and commercial computer banks, such as shopping mailing lists, to verify that passengers are who they say they are. Once a passenger is identified, the CAPPS 2 system will compare that traveler against wanted criminals and suspected terrorists contained in other databases.

THERE'S MORE: Boing Boing founder Mark Frauenfelder says his six-year old daughter has now been flagged as a potential terrorist on the original CAPPS system, and there doesn't seem to be anything he can do to get her off.

STATE-RUN "TIA" EXPOSED

Long-time Defense Tech readers will be familiar with MATRIX -- the data-mining project, run by 10 state governments, that's eerily similar to Total Information Awareness.

Now, the ACLU has a rundown of this creepy program, which combs through credit card transactions, marriage records, and vehicle registration data to find alleged evil-doers. This report based mostly on newspaper articles. But it's the most complete profile to date of this largely-hidden program.

MATRIX (short for "Multistate Anti-Terrorism Information Exchange") "is designed not only to build dossiers on all of our lives so they will be a keystroke away for police and other government officials, but also to search through our dossiers and those of others in a hunt for patterns indicative of terrorist or other criminal activity," the report notes.

"It’s scary," says Phil Ramer, the intelligence chief for Florida, which is taking the lead on MATRIX. "It could be abused. I mean, I can call up everything about you, your pictures and pictures of your neighbors."

KEEPING THE SONS OF "TIA" IN CHECK

On this, everyone in the gold-tinged, eagle-frescoed Senate conference room agreed: Federal authorities badly want to be able to comb the data trails of ordinary people in order to spot terrorists. But what -- if any -- limits should be put on that frighteningly invasive power? A panel of lawmakers, think tankers, data miners and civil libertarians assembled here Tuesday couldn't even begin to make up their minds.

Congress has yanked the funding for Terrorism Information Awareness, the Pentagon's notorious überdatabase effort. But research into TIA-like projects continues, essentially unrestricted. Tomes of regulations tell spooks and cops and g-men how they can amass intelligence and gather evidence. But much of the data mined by these children of TIA -- like itineraries, school transcripts and credit card receipts -- might not fall under those traditional definitions. There's only a vague sense that these database-combing programs can't be allowed to grow out of control.

"When somebody buys a ticket on Delta Airlines in Munich, Germany, if there's any potential for (that person to have) a suspicious background, I want bells and whistles to go off on that computer," Sen. Saxby Chambliss (R-Ga.) told the group of 25 or so policy makers assembled in the Russell Senate Office Building's third floor by the Potomac Institute for Policy Studies, a Washington think tank. But Congress "won't allow (intelligence) agencies" to "truly gather information on people's personal lives."

Nice words. But as Jim Dempsey, executive director of the Center for Democracy & Technology, notes, "none of us really have the answer" for how to put them into action.

My Wired News report from Washington has more.

THERE'S MORE: As usual, Phil Carter has got some interesting things to say about this. Check 'em out here.

JETBLUE: WHY THE FUSS?

The fear of flying wasn't some abstract, idle concern for Joshua Gruber. It was as tangible as the pile of concrete and steel and flesh and ash, smoldering at Manhattan's southern end on 9/11, the day he was in the North Tower of the World Trade Center.

But flying home to California for Christmas on JetBlue -- his first cross-country trip after the tragedy -- made the whole thing easier to take. The staff seemed like human beings, not corporate automatons. The planes were brand-new. Best of all, as he flew, Gruber could watch the Food Network on his own private television screen.

"You'd sit down, watch Food TV, and, before you’d know it, you’d be there," Gruber said. "It made it easier to fly after September 11 to have that distraction."

Although the airline is known for its cheap fares, he added, "I'd pay more to fly JetBlue. I had, in fact. And I had encouraged my friends to try it."

All of which makes JetBlue's decision to hand its passenger records over to a firm doing a government terrorist-screening study even more maddening to Gruber.

"It made it sort of like I had been betrayed by a friend, rather than by a big company," he said.

Businesses sell, trade, and swap their customers' data with each other all the time. That's why every product registration card includes information about income, age, occupation. That's why web-based companies – even privacy-savvy ones like TerraLycos (which owns Wired News) -- "will sometimes share personally identifiable information with third-party companies and organizations."

But the JetBlue privacy debacle has unleashed unusual passions in the public. Already, there's a class action lawsuit against the carrier for its data handover. Already, Gruber has received more than a thousand e-mails from outraged JetBlue customers. And already, the Department of Homeland security is beginning to conduct an internal investigation into how passenger data is used.

Why the fuss? Passengers, privacy advocates and airline analysts all sound a common theme: fliers like Joshua Gruber developed powerful ties to JetBlue, ties that were unusual in business – and especially rare the notoriously nasty airline industry.

When the company turned over its customers' private records without their knowledge -- in violation of JetBlue's own privacy policy -- that sense of corporate love quickly exploded into rage.

My Wired News story has more.

THERE'S MORE: Defense Tech reader KH writes, about "an interesting popup ad I saw on the computerworld.com site. The text is: 'We helped JetBlue Airways do something unique with their data: treat customers like people. Unisys.'"

Ah, the irony...

AND MORE: Gen. Wesley Clark was on the board of one of the companies involved in the JetBlue data mess, Glenn Reynolds notes.

AND MORE: The ACLU now has a web site where JetBlue passengers can file a request to find out what the government may be holding on them.

BRIDE OF "TIA" LIVES

Congress may have driven a stake through Total Information Awareness. But there are lots of other government data-mining programs -- eeriely similar to TIA -- that are still very much alive.

One TIA-like project is Novel Intelligence from Massive Data (NIMD), an initiative of the little-known Intelligence Community Advanced Research and Development Activity, notes secrecy guru Steven Aftergood, with the Federation of American Scientists.

"Pursued with a minimal public profile and lacking a polarizing figure like Adm. Poindexter to galvanize opposition, NIMD has proceeded quietly even as TIA imploded," Aftergood writes.

The NIMD effort aims to comb through "structured text in various formats, unstructured text, spoken text, audio, video, tables, graphs, diagrams, images, maps, equations, chemical formulas, etc." to help "intelligence analysts to spot the telltale signs of strategic surprise."

By now, we all know what that means.

CONGRESS PUTS BRAKES ON CAPPS II

Congress is delaying the planned takeoff of CAPPS II, the controversial new airline passenger-profiling system, for about four months, until a privacy study can be completed. Wired News has the story.

HOUSE AGREES TO DARPA CUTS

Last week, Defense Tech reported that the Senate was looking to cut off funding for most of Darpa's Information Awareness Office -- the group of minds, formerly headed by John Poindexter, that was responsbile for the Total Information Awareness uber-database and the "terror market" mess.

Now, the House has agreed to the Senate's position, notes Associated Press writer -- and Defense Tech pal -- Mike Sniffen. And so it looks like many of the creepiest Pentagon surveillance programs will have their purse-strings cut -- or will at least be driven to the classified side of the Pentagon ledger.

THERE'S MORE: Some of the less creepy Darpa programs, previously cut by the Senate, have now been restored. The $35 million Continuous Assisted Performance program -- an attempt to help soldiers go long periods without sleep -- is back, for example. Now, according to one of the scientists working on the effort, its budget has been cut only by a sixth, the $24 million.

AND MORE: Darpa's information technology research budget should be boosted, according to a new report from the National Academies' Computer Science and Telecommunications Board.

FORMER DRUG SMUGGLERS RUN ANTI-CRIME DATABASE

Defense Tech has highlighted dozens of fear-inducing government efforts to find bad guys in the data-trails of ordinary citizens. But, if this Associated Press report is accurate, it's enough to make the iron-willed hide under the bed and suck their thumbs.

The existence of the Matrix (Multistate Anti-Terrorism Information Exchange) database has been known for a while, now. It's an effort by a few state governments, including Florida, to set up a kind of local Total Information Awareness – looking for crooks, as well as bomb-throwers.

But new details about the program are coming to light. And they are not pretty.

The project is billed as a tool for state and local police, but organizers are considering giving access to the Central Intelligence Agency, said Phil Ramer, special agent in charge of the Florida Department of Law Enforcement’s intelligence office...

Matrix houses restricted police and government files on colossal databases that sit in the offices of Seisint Inc., a Boca Raton, Fla., company founded by a millionaire whom police say flew planeloads of drugs into the country in the early 1980s…

Criminal history files in the database are maintained by 15 Seisint employees, watched over by Florida state police, Ramer said.

Yet a Florida Department of Law Enforcement memo obtained by The Associated Press shows potential lapses in oversight. The memo says background checks on Seisint’s Matrix workers took place only last month, more than a year into the program, and a privacy policy governing the database’s use has yet to be finalized…

California and Texas (have) dropped out (of the Matrix program), citing, among other things, worries over housing sensitive files at Seisint. And a competing data vendor, ChoicePoint, decided not to bid on the project, saying it lacked adequate privacy safeguards."

The Register notes that "Alabama, Connecticut, Florida, Georgia, Kentucky, Louisiana, Michigan, New York, Oregon, Pennsylvania, South Carolina, Ohio, and Utah have signed on to the scheme. Residents of other states are safe, for now."

(emphasis mine)

BRAVE NEW SKIES: WHY?

"If Sept. 11 could have been avoided by four pilots with four .22s, why is the response to that to profile everybody?" asks conservative activist Grover Norquist.

It's the central question Washington power brokers are putting to the Transportation Security Administration, as it attempts to push its creepily invasive CAPPS II passenger-screening program forward.

Everything you wanted to know about the database effort -- and a whole lot you probably didn't -- is in this fine story from Salon's Farhad Manjoo.

LIFELOG GETS A FACELIFT

Monday is the deadline for researchers to submit bids to build the Pentagon's all-encompassing, über-diary project, LifeLog.

But while teams of academics and entrepreneurs are jostling for the 18- to 24-month grants to work on the program, the Defense Department has changed the parameters of the project to respond to a tide of privacy concerns.

My Wired News story has more.

PENTAGON BUILDING SNITCH DATABASE

When Attorney General John Ashcroft proposed last year that a civilian army be recruited to snoop on their neighbors, privacy advocates and Congressional leaders gagged.

Now, it seems, Ashcroft's Operation TIPS is back -- and now, it's being run by the Pentagon.

Brian McWilliams writes in Wired News:

To track domestic terrorist threats against the military, the Pentagon is creating a new database that will contain "raw, non-validated" reports of "anomalous activities" within the United States.

According to a Department of Defense memorandum, the system, known as Talon, will provide a mechanism to collect and rapidly share reports "by concerned citizens and military members regarding suspicious incidents."

Talon was described in a May 2 memorandum to top Pentagon brass from Deputy Secretary of Defense Paul Wolfowitz. In the memo, Wolfowitz directed the heads of military departments and agencies to begin producing Talon reports immediately.

SAFIRE LATEST ON LIFELOG TRAIN

The big time, I suppose. New York Times columnist William Safire is now writing about LifeLog -- sixteen days after my Wired News story on the DARPA program.

BOB GRAHAM: LIFELOGGER?

If the Pentagon is looking for volunteers for its LifeLog uber-diary project, they may want to think about giving Florida Senator (and Democratic presidential hopeful) Bob Graham a ring.

Since 1977, Graham had meticulously documented his life -- from the cereal he eats to the television interviews he gives.

The New York Times has a sample page from Graham's diaries here.

LIFELOG BANDWAGON ROLLS ON

The AP's Mike Sniffen has taken my suggestion and done a story of his own on LifeLog, the creepy Pentagon "diary" effort.

In Sniffen's piece, James X. Dempsey, of the Center for Democracy and Technology, sees a "silver lining" in the otherwise black program:

"If government weren't doing this, it would still be done by companies and in universities all over the country, but we would have less say about it." Because the government is involved, "you can read about it and influence it."

Peter Coffee, over at eWeek, writes about LifeLog's technical and business implications:

Declining costs of data collection, storage and analysis form a seductive force that encourages us to hope that we'll understand more if we collect more. But knowledge of your customer, and insight into customers' needs, has not suddenly become a proposition of quantity rather than quality.

DARPA's goals for LifeLog are only superficially similar to your goals in building a business intelligence system. Follow DARPA's example, and you'll be able to draw on the masses of data that tomorrow's technologies will allow you to collect—but unless that collection effort is guided by a creative vision of your business, the results will be either irrelevant or misleading. And it will be a futile exercise to try to apply your business vision after the fact to an indiscriminate archive.

LIFELOG'S LEGS

You may have read about the Pentagon's eerie LifeLog proposal here first. But now, the rest of the press is starting to take interest in the project, which aims to gather up everything in a person's life, index it, and make it searchable.

Reuters has a story on LifeLog here. The Register and the Washington Post's online edition chime in here and here.

But the most interesting analysis comes from Reason's Charles Freund, who compares LifeLog to "the CIA's Cold War fascination with the chimera of mind control." He also questions the biographical impulse behind the project.

"The notion that peoples' lives actually have had the narrative shape... is one of our more pleasant cultural delusions," Freund writes.

THERE'S MORE: Ten days after my story on LifeLog, the New York Times is running the Reuters take on the system.

DARPA WANTS YOUR LIFE INDEXABLE AND SEARCHABLE

It's a memory aid! A robotic assistant! An epidemic detector! An all-seeing, ultra-intrusive spying program!

The Pentagon is about to embark on a stunningly ambitious research project designed to gather every conceivable bit of information about a person's life, index it and make it searchable.

What national security experts and civil libertarians want to know is, why the hell would the Defense Department want to do such a thing?

The embryonic LifeLog program would take every e-mail you've sent or received, every picture you've taken, every web page you've surfed, every phone call you've had, every TV show you've watched, every magazine you've read, and dump it into a giant database.

All of this -- and more -- would be combined with a GPS transmitter, to keep tabs on where you're going; audio-visual sensors, to capture all that you see or say; and biomedical monitors, to keep track of your health.

This gigantic amalgamation of personal information could then be used to "trace the 'threads' of an individual's life," to see exactly how a relationship or events developed, according to a briefing from the Defense Advanced Projects Research Agency, LifeLog's sponsor.

Someone with access to the database could "retrieve a specific thread of past transactions, or recall an experience from a few seconds ago or from many years earlier … by using a search-engine interface."

On the surface, the project seems like the latest in a long line of DARPA's "blue sky" research efforts, most of which never make it out of the lab. But Steven Aftergood, a defense analyst with the Federation of American Scientists, says he is worried.

With its controversial Total Information Awareness database project, DARPA already is planning on tracking all of an individual's "transactional data" -- like what we buy and who gets our e-mail.

Aftergood said he believes LifeLog could go far beyond that, adding physical information (like how we feel) and media data (like what we read) to this transactional data.

"LifeLog has the potential to become something like 'TIA cubed,'" he said.

My Wired News article has details on the LifeLog program.

THERE'S MORE: The idea of committing everything in your life to a machine is nearly sixty years old. In 1945, Vannevar Bush -- who headed the White House's Office of Scientific Research and Development during World War II -- published a landmark Atlantic Monthly article, "As We May Think." In it, he describes a "memex" -- a "device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility."

Minicomputer visionary Gordon Bell, now working at Microsoft, sees his "MyLifeBits" project as a fulfillment of Bush's vision.

There are other commercial and academic efforts to weave a life into followable threads, including parallel processing prophet David Gelernter's "Scopeware" and "Haystack," from MIT's David Karger.

AND MORE: LifeLog may eventually dwarf Total Information Awareness, DARPA's ultra-invasive database effort. But "TIA" could wind up being pretty damn large on its own, with 50 times more data than the Library of Congress, according to the Associated Press.

AND MORE: Lovers of civil liberties, you now have nothing to fear. Henceforth, the creepy "Total Information Awareness" program will be known as "Terrorism Information Awareness."

Feel better?

AND MORE: DARPA's report to Congress on TIA is online here.