This is a contributed piece by Vibin Vijay, Solutions Consultant of Data Analytics at OCF
Data analytics is popular. It’s not uncommon for companies, particularly in manufacturing and medical research, to spend millions on Artificial Intelligence (AI), with a view to taking advantage of analytics. However, I don’t think some investments have been that wise.
I am still shocked to find that even after spending significant amounts of money on systems to support analytics, there are still so many companies not getting the basics right. I have seen cases where millions of records are all stored as bits and pieces in Excel, with no system in place to homogenise the data.
In part, this is because an organisation may have started with one or two sites recording data for analytics, using Excel was logical and doable, sharing codes etc. But as more sites and people get involved, you start to lose the data integrity. It’s recorded in different formats for example and the data from different sites and departments varies greatly. What one person sees as valuable to record, another doesn’t. That creates problems.
I think that’s the pressing point now in analytics. I personally believe that big data is hype; it’s got a catchy name. Most of the clients we talk to don’t actually need a ‘big data’ technology solution. They’re dealing with Excel spreadsheets so it’s about helping them to audit the need around database, data warehouse, data mining, in memory, cloud, Hadoop or NoSQL. It’s about going step by step towards analytics rather than just providing a false bulletproof system without any consultancy. There is no one-click intelligent system yet!
Seeing the value
There are certainly some great products out there in analytics, but there are many smaller industries and enterprises that are not always keen to acquire products straight away [and invest the hundreds of thousands that their larger counterparts have done]. Instead they prefer to carry out a Proof of Concept (PoC) for phase one on a cloud platform, such as a product like SAS – currently the leader in analytics, according to Gartner.
That’s an enticing proposition for smaller companies that don’t want to purchase ‘beefy’ solutions that cost a significant amount. Launching on a cloud solution and developing PoC’s, testing out how analytics can work for them and assessing whether they’ll see some value add, then they can press on for purchase orders.
A needed step-change
Companies, in my opinion, don’t need to be told or sold technology; they need consultancy in how to prepare for data analytics to be able to execute a valuable solution. It’s a fast growing market and it’ll be a tough game for even large organisations to really keep an eye on what’s happening in the market. There are some interesting products coming up – Apache Spark is going to lead with in memory analytics, and we’ve got Apache Storm and Flink releases coming too.
In time, we hope that more companies realise the worth in storing and analysing the correct data. We strive to offer multiple vendors and partners help to give a solution based approach rather than product based approach.
PREVIOUS ARTICLE«Databricks has designs on democratising Deep Learning
Jon Collins’ in-depth look at tech and society
Phil Muncaster reports on China and beyond