Big data is a popular topic these days, not only in the tech media, but also among mainstream news outlets. Executives see Big Data as providing significant business benefits – greater insight and learning, the ability to obtain answers and make decisions faster and in a more informed manner, greater agility and flexibility.
Big data is hence a major business issue and Hadoop is the platform that makes big data easier to manage. Especially after April’s official release of big data software framework, Hadoop 2.7.0 is generating even more media buzz.
There are 3 main reasons why as a business user you need to learn more about Hadoop. So let’s dive into each of those 3 things.
Hadoop has the ability to handle large volumes of data (structured or not):
If, say, the marketing department of your company is generating and storing three billion
records a month. And you expect that in the next three months it will be 10 billion records a month. There are two main limitations for you in this scenario:
- One is unstructured data such as video.
- Amount of data to be processed and stored will grow larger.
You can solve this problem using Hadoop by adding another server to the node. You could complete what your marketing department requires and scale immediately. It is not that this is impossible in RDBMS systems but it’ll be too costly in RDBMS. Hadoop makes it affordable. Also, it makes it scalable (when needed).
In other words, additional hardware could be easily and quickly added on as needed without having to pay extra because Hadoop is open source. That’s dramatically changed the way the company can expand its computing power to meet its needs. You don’t want to spend millions of dollars on infrastructure.
Cost efficient and ability to scale
Previously, enterprises have to keep track of data sets: emails, sales data, customer data, internal data, etc, in a relational data base management system, which was very expensive.
With all of these data coming in, companies would typically down sample the data (reduce the data down to a smaller subset). This reduced data set would automatically be classified based on certain assumptions, the main assumption is that some data would always be more important than other data.
For example, the priorities for e-commerce data would be set on the assumption that debit card would be more important than product data, and product data would be more important than analytics data.
What would happen when the assumptions changed? Because the data was reduced, any new business scenario would have to use the down sampled data still in storage, all the raw data would be long gone.
Because of the expense of RDBMS-based storage, often this data would be siloed with an organization. Finance department would have their data, HR would have theirs, Operations their own and so on. So business decisions would be restricted to each department of the company, not the whole company.
But, using Hadoop, you keep all the data—there are no assumptions. In Hadoop, all data has equal value.
Because all data is equal, and equally available, business scenarios can be run with raw data at any time, without limitation. Moreover, formerly siloed data can be equally accessed and analyzed for organization’s success. So this brings down the cost. Adding to this, Hadoop is open-source and hence its free.
Quick analysis of data
Hadoop allows you to take in and process huge amounts of data in a short amount of time.
One big advantage of Hadoop is its ability to be able to analyze huge data sets to quickly find trends. For a company like Walmart, that could mean analyzing user data to learn what shirt colors were in fashion last season, to be able to compare that information with today’s hot color trends to help determine what will sell this season.
Traditional databases can work for many sorting and analysis needs, but with very large data sets, Hadoop can be a much more efficient way to find things.
Conclusion
Companies considering Hadoop must be sure that it can integrate with their existing IT investments. The massive data aggregation enabled by Hadoop can raise concerns related to security, data access, data entitlement, monitoring, high availability and business continuity. Even though Hadoop saves costs and time, if it is poorly handled, it could explode the cost.
To conclude, a business executive must definitely learn Hadoop to address all these issues.
