No algorithm no future but sort out the data first

What is the business value of an algorithm?


While the aptly named TikTok continues its back and forth pursuit of a US deal, a major sticking point has been the withdrawal of its algorithm from the negotiating table. In August, the Chinese government was reported to have changed the rules on the export of technology, meaning that TikTok’s underlying algorithm needed government approval to be part of any sale. The proposed buyout by Oracle and Walmart would, almost certainly, mean that a new algorithm for use in the US would need to be developed, at least according to one report.

No doubt there will be more twists and turns as US President Donald Trump pushes for total US ownership of TikTok but without the algorithm, what is its true value? In fact, what is the true value of any tech business these days without its core technology? Algorithms have increasingly become the backbone and often the differentiator in modern applications. The truth is that algorithms are running our daily, online lives, for good and bad. The data we generate – 2.5 quintillion bytes of data every day, or 1.7MB of data created every second by every person during 2020, according to one source – coupled with our service and software expectations, has lead us to this point.

For Chris Cole, CEO of Headlamp Software, the true business value of algorithms is not so easy to quantify, at least on a broad scale because every company and customer will approach problems differently. Algorithms, he says are not always the answer.

“The basic question is, ‘when should the algorithm directly make decisions about the behaviour of the system, and when should it merely give advice that can be tempered by human judgment?” says Cole. “The temptation to avoid the requirement of human involvement is based on cost. Humans are slow and costly. It is more efficient to take them out of the decision-making process. When is that appropriate?”

Cole adds that while algorithms can be devised to deal with known unknowns, modern machine learning can not only optimise parameters in an algorithm, it can also determine which parameters are to be optimised. Improving algorithms through machine learning, to perhaps help answer those difficult questions on humans and cost, seems like a natural progression but Cole has a warning.

“Machine learning cannot look at all possible algorithms, there are too many,” he says. “So, ML restricts its search to algorithms that are likely to be involved. What does this mean? Here there is guesswork and hence risk. It is possible that the algorithms that are explored do not contain the solution to the problem at hand. It is simple to prove that there is no algorithm that can solve every problem. The only solution is testing. Which involves humans. Which is expensive. But there is, in principle, no alternative.”

So, the business value of an algorithm can be weighed by its ability to reduce human costs, as well as its intelligence in solving complex problems quickly. Certainly, we are already seeing this in data analytics to greater and lesser extents. As Dr Nicolai Baldin, CEO and founder of Synthesized claims, while data collection and management is costly – organisations are estimated to waste $2 million each year trying to handle this process - a machine learning algorithm is the best solution to this problem.

“It can be trained to understand and predict certain scenarios to make better, automated decisions with data, crunching millions of data points in just ten minutes,” says Baldin. “In addition, data is the fundamental element in creating any algorithm, its quality is an essential factor in determining performance. Given that the majority of businesses lack effective processes in place to collect, manage, and provision datasets, an algorithm will likely be built from poor quality data and struggle to show any business value.”

Reputational risk

It's an interesting point because we hear so much about analytics capabilities and much less about improving data quality. This is fundamental. Sorting out data quality and trying to address data bias surely has to be a principle target for any organisation looking to develop algorithmic and ML analytics?

“If inputted data has pre-existing flaws, or worse, unequal representation, the results generated by the algorithm will be biased,” adds Baldin. “The poorly planned use of an algorithm could have serious business and reputational consequences for an organisation.”

As a survey earlier this year found, data quality continues to be an issue, with multiple and varying data sources delivering inconsistencies in quality. If data is the new oil and building block of modern business, then, at the moment at least, the foundations are a little rocky. So, if organisations are going to build value through data and develop algorithms to differentiate and drive analytics and functionality, they have to sort the data first and then decide whether to build their own or buy-in algorithms to manage the data. Easier said than done.

“Of course, every organisation has different needs and internal resources, however, despite rapid advances in technology, creating algorithms is still no easy feat,” says Baldin, adding it’s a time-intensive process that requires an organisation to have deep knowledgeable and capable employees to execute. “Meanwhile, up to 80 percent of the cost of an AI-focused project is estimated to be spent in the areas of collection, cleaning, and organising data, for testing algorithms. By contrast, third-party technology providers can add much-needed expertise and these partners can make a much quicker impact on every aspect of the data challenge.”

With this in mind, watching the on-going saga at TikTok is even more intriguing. Bun fights over intellectual property are not new, but you wonder how much of a future the video sharing network has if its core technology has to be re-written? It may not come to that but what is increasingly clear in these deals is that the algorithm is fundamental. Yes you get access to a brand name and its user base but loyalty counts for nothing if the quality of service falls away. If anything, it may be an opportunity for a newcomer.