Data Storage: REST vs. POSIX for Archives and HSM

NoSQL | Tech and Tools   |   
Published August 27, 2013   |   
Henry Newman

Here is my working hypothesis: POSIX (Portable Operating System Interface) can’t scale to meet the demand of clouds and big data, but REST (Representational State Transfer) can’t manage and tier data the way POSIX can, but will likely get those features in the next few years and then take off as the new data interface standard of the cloud era.

There is a lot going in the archive world as archives are becoming far more important, given that companies and researchers are looking back at data to gain a better understanding of our world and help predict the future.

Let’s first address the difference between an archive and (HSM) hierarchical storage management. My view is that the definition of an archive is about storing data that will be needed in the future, while the definition of HSM is managing the archive with hierarchies of storage. There is a lot of money in predicting the future, whether it be commodities traders, the healthcare industry agricultural planning or some other industry. Some use tiers of disk, but the ones that I am taking use tape as one of the tiers, given the high reliability and lower cost.

This a chart that illustrates these concepts:

storage newman


But this is an article on archive interfaces, not the underlying archive. Some industries, such as the geosciences companies that collect seismic information from around the world, have known about the importance of archives for decades. All of the vendors in this market I am aware of use archives with POSIX interfaces.

Read More