As I never tire of saying, technology is always the easy part. Getting an organization to use a solution - and use it properly - is the real challenge. People will not change how they do things unless they have an incentive to do so (e.g. benefits are high and the costs/risks are low). Also, the problem of organizational "silos" is not limited to the public sector, and certainly not a problem faced by intel agencies alone (although the need for secrecy exacerbates the silo effect).

The computer systems were designed to be “air gapped.” The F.B.I. terminals were connected to one another — but not to the computers at any other agency, and vice versa. Messages written on the C.I.A.’s network (which they still quaintly called “cables”) were purely internal. To get a message to the F.B.I. required a special communication called a “telegraphic dissemination.” Each agency had databases to amass intelligence, but because of the air gap, other agencies could not easily search them. The divisions were partly because of turf battles and partly because of legal restrictions — but they were also technological.

This I find a bit silly, as if the journalist is appalled that solutions built by two different organizations weren't designed to interoperate in the first place. There was no prior requirement to do this, in fact agency mandates might make certain kinds of sharing unlawful. If IBM and Microsoft were suddenly required to share data, no one would be surprised that their systems weren't integrated from the start.

The danger in creating the ability to share information between agencies with different responsibilities is the grey area created with respect to citizen privacy, accountability, information credibility, etc. Part of the reason governments are so hierarchichal is to ensure proper vetting and verification of sensitive information to reduce the risk of errors (and embarassment). Organizational aversion to open or weakly-moderated tools like wikis is understandable from this perspective.

Moving on, a discussion of one "digital pipeline that joined the agencies" - Intelink:

There was already one digital pipeline that joined the agencies (though it had its own limitations): Intelink, which connects most offices in each intelligence agency. It was created in 1994 after C.I.A. officials saw how the Web was rapidly transforming the way private-sector companies shared information. Intelink allows any agency to publish a Web page, or put a document or a database online, secure in the knowledge that while other agents and analysts can access it, the outside world cannot.

This paragraph could have been one sentence. It's a bloody extranet.

Nonetheless, Intelink has grown to the point that it contains thousands of agency sites and several hundred databases. Analysts at the various agencies generate 50,000 official reports a year, many of which are posted to the network. The volume of material online is such that analysts now face a new problem: data overload. Even if they suspect good information might exist on Intelink, it is often impossible to find it. The system is poorly indexed, and its internal search tools perform like the pre-Google search engines of the ’90s.

Again, not unique to intel organizations. Governments, multinationals and companies with many business lines all have information management problems, particularly if there are no central policies around how assets should be organized or retrieved once they're published.

With respect to search engines, it all boils down to relevance. Highly subjective, difficult to measure, even more difficult to improve. That's why this next part is pure fantasy:

With Andrus and Burton’s vision in mind, you can almost imagine how 9/11 might have played out differently. In Phoenix, the F.B.I. agent Kenneth Williams might have blogged his memo noting that Al Qaeda members were engaging in flight-training activity. The agents observing a Qaeda planning conference in Malaysia could have mentioned the attendance of a Saudi named Khalid al-Midhar; another agent might have added that he held a multi-entry American visa. The F.B.I. agents who snared Zacarias Moussaoui in Minnesota might have written about their arrest of a flight student with violent tendencies. Other agents and analysts who were regular readers of these blogs would have found the material interesting, linked to it, pointed out connections or perhaps entered snippets of it into a wiki page discussing this new trend of young men from the Middle East enrolling in pilot training. [Emphasis mine]

A handful of people is now a trend?

If there are 10,000 CIA/NSA/FBI blogs, how do agents come across these clues and put them together? Why would the FBI agent monitoring flight-training in the US bother to visit a spy-wiki page about Al Qaeda planning in Malaysia? Ex post facto relevance isn't useful. Being able to rapidly assimilate and synthesize large amounts of information is useful, and may increase the probability of making arcane connections, but there are no guarantees and only 24 hours in a day. So which information repositories and gathering methods yield the most value?

Near the end of the article, this problem is acknowledged:

Moving quickly, in fact, is crucial to building up the sort of critical mass necessary to make blogs and wikis succeed. Back in 2003, a Department of Defense agency decided to train its analysts in the use of blog software, in hopes that they would begin posting about their work, read one another’s blogs and engage in productive conversations. But the agency’s officials trained only small groups of perhaps five analysts a month. After they finished their training, those analysts would go online, excited, and start their blogs. But they’d quickly realize no one else was reading their posts aside from the four other people they’d gone through the training with. They’d get bored and quit blogging, just as the next trainees came online

Indeed. You can't throw people into a new paradigm and expect them to use it successfully (particularly if its success hasn't even been evaluated or communicated to the organization).

When I raised the idea of collaborative tools like blogs and wikis, Spalding and Russ Travers, one of the [joint CIA-FBI-NSA National Counterterrorism] center's deputy directors, were skeptical. The whole reason the center works, they said, is that experts have a top-down view that is essential to picking the important information out of the surrounding chatter. The grass roots, they’ve found, are good at collecting threats but not necessarily at analyzing them. If a lot of low-level analysts are pointing to the same inaccurate posting, that doesn’t make it any less wrong."

The key is to have very smart people culling the daily tips, Travers told me. In October, for example, nervous rumors that a football stadium in the United States would be subject to a nuclear attack flooded the National Counterterrorism Center; analysts there immediately suspected it was spurious. “The terrorist problem has the worst signal-to-noise ratio,” Travers said. Without the knowledge that comes from long experience, he added, a fledgling analyst or spy cannot know what is important or not. The counterterrorism center, he said, should decide which threats warrant attention. “That’s our job,” he said.

Determining relevance and avoiding the stupid-storm. An elusive problem not solved by the mere existence of wikis, blogs and search engines.