, ,

What is Needlebase?

Needlebase is a remarkably easy to use platform for merging data from complex websites and feeds into a clean, queriable database – without any need for programming!

Open data is great, but not always so easy to synthesize and interpret. Needlebase provides the means for

acquiring, integrating, cleansing, analyzing and publishing data on the web…using only a web browser, without programmers or DBAs

Needlebase at work

Needlebase has been used for many excellent data analysis projects. One particularly important use was in the field of data journalism. It concerned a study of peer-reviewed research supporting ‘man-made’ global warming skepticism. Needlebase has no political agenda, but had the means to definitively confirm these findings. In fact, the author of the study welcomed and requested help from Needlebase. Findings included the following:

  • The top 10 most prolific authors contributed 210 of the papers
  • One author was associated with 305 of the 973 papers

The full results of the Needlebase comparison are available as AGW Research Skepticism Analysis (AGW means “Anthropogenic Global Warming”), complete with charts, supporting data and data visualizations.

Another example

The results returned by a simple query on a publicly available data set of all active U.S. nuclear reactors are impressive. Needlebase is useful for geodata visualizaton, as well as descriptive statistics and analysis.


Google acquired ITA Software, an airline flight pricing and e-commerce company, several years ago. As an ITA project, Needlbase was acquired along with ITA. ITA used Needlebase as a data aggregation tool. Sadly, in January 2012, Google announced

the Needlebase data management platform is being evaluated for integration into Google’s other data-related initiatives.

A final Needlebase blog entry confirmed that the application will be permanently retired on June 1, 2012.

I only wish I found Needlebase sooner. Google also owns Freebase, which is highly regarded. Freebase is quite different from Needlebase though. It is a semantic “web of things” type repository of linked data, which I find far less useful for practical data analysis.