Many Global tech companies (EMC, IBM, Microsoft and Oracle) were circling the big data Hadoop elephant of a solution for the onslaught of unstructured data facing them. One has to ask the fundamental question, what sort of companies would face these sorts of problems and whether Hadoop was a solution ? There is a clear and definitive need for managing and analyzing unstructured data beyond searching for alien life or analyzing the human genome, and we can see the significance of that need increasing from an investment perspective.
After six years of development, the Apache Software Foundation (ASF) has announced the official release of the first Hadoop platform (1.0) to address the unstructured data challenge facing network companies which extends beyond traditional telecom operators and includes any operator that has a network with large numbers or users (subscribers or not) and large volumes of traffic. This would include cable and satellite operators, but also social networking companies such as Facebook, Twitter, as well as Sina and Tencent, Microsoft and Yahoo! for messenger as well as companies with transactions such as Alibaba, eBay, PayPal, Flipkart and, of course, the web portals and search engines.
Another area of application is the utilities, which are increasingly in need of collaborating with telcos and tech companies trying to realize smart grids and model load balances for data centers and energy consumption.
In the financial sector, there is a dire need to manage and analyze unstructured data if for no other reason than compliance. The amount of data that a bank generates is already daunting, but layer into that the equities business and compliance needs and being able to understand business, regulatory and IT requirements, and the challenges mount.
What about Hadoop Security in Sensitive areas such as Banking ?
Many banks and telecom operators already operate systems and servers on open-source because it can be inspected and reviewed; lulling in natural lethargy and false sense of security is a problem of IT management and a key performance indicator for telecom operators and banks.
In the telecom and social networking industry the very “open-ness” of the industry is what almost ensures there will be a growing unstructured data challenge and will drive network operators to be a beachhead for adopting unstructured data analytics.
