IBM Research, with a little help from Violin Memory, today announced it broke a world record for how quickly massive amounts of data can be handled by successfully scanning 10 billion files on a single system in 43 minutes, comprehensively smashing the previous record of three hours.
This is a fun technical achievement for its own sake, but it also has important real-world implications. Consumers are getting used to the idea of cloud computing, and they already take for granted that movies will stream instantly. These and other expectations require companies to handle ever-larger amounts of data, all of which needs to be accessed and crunched simultaneously.
The record was broken using IBM‘s clustered parallel file system, GPFS, running on a cluster of 10 eight-core systems and using Violin Memory‘s 3200 Flash Memory solid state arrays for optimum low latency and high bandwidth.
“Today’s demonstration of GPFS scalability will pave the way for new products that address the challenges of a rapidly growing, multi-zettabyte world,” said Doug Balog, vice president, storage platforms, IBM.
Related Link:
IBM’s research paper (PDF): http://tinyurl.com/3vqakjc
