BETA
This is a BETA experience. You may opt-out by clicking here

More From Forbes

Edit Story

NSA Opens Up Data Automation Software For Public Use

Following
This article is more than 9 years old.

The Apache Software Foundation (ASF) has worked with the National Security Agency (NSA) on the release of Niagarafiles (or Nifi, to the initiated) technology designed to “automate data flows” among multiple computer networks. The software is free and open source (so is available to the public) through the Apache Software Foundation. But why is automating data flows important?

Automation for the people

Data flow automation is important because it brings together data streams that need to be aggregated for analysis -- and yes, think big data analytics here if you wish. This can be hard work when data formats and data transmission protocols differ between different sources. But Niagarafiles handles this exact cross-format challenge.

Data flow automation is important to commercial enterprises because they can us it to control, manage and analyze the flow of information from geographically dispersed sites such as international office headquarters and/or customers and partners. If you need to ask why the NSA is good at complex data flow automation and aggregation, then go back to the start of the first paragraph and start reading again.

Situational awareness

So far so good, but why perform data flow automation anyway?

The NSA explains that coalescing information from multiple sources can help “create comprehensive situational awareness” and that this is as relevant to the NSA’s wider work as it is to business. Lead Developer of Nifi Joseph L. Witt explains that this technology, “Provides a way to prioritize data flows more effectively and get rid of artificial delays in identifying and transmitting critical information."

So is the NSA about to start sharing more of its software with the enterprise business sector? The organization confirms that this release is first in a series of releases of in-house software products the NSA Technology Transfer Program (TTP). Posting the code to open source forums allows the private sector and others to examine the agency's research up close, and potentially benefit from it through additional enhancements and applications.

But here’s the point i.e. open source is all about community contributions and shared knowledge, so the at the same time the government can gain from related research advances that occur in the wider world of enterprise software.

"NSA's innovators work on some of the most challenging national security problems imaginable," said Linda L. Burger, director of the NSA TTP. "Their research breakthroughs often have broad, commercial applications, too. We use open source releases to move technology from the lab to the marketplace, making state-of-the-art technology more widely available and aiming to accelerate U.S. economic growth."

The agency says it often uses open source offerings to confront today's complex foreign-intelligence challenges with creativity and agility. It has also contributed to the open source community in recent years; the code for the agency's "Accumulo" project was previously released through the Apache Software Foundation.

NSA: open to global (software) review

This is not the first time the agency has worked at this level and Security-Enhanced Linux (SELinux), Red Hat Enterprise Linux (RHEL) and Debian Linux (used on the International Space Station) have all been through its doors and subsequently shared openly. The NSA’s final word states that “global reviews and critiques” that stem from open source releases can broaden a technology's applications for the U.S. private sector and for the good of the nation at large.

Follow me on Twitter or LinkedIn