No algorithm no future but sort out the data first

What is the business value of an algorithm?


While the aptly named TikTok continues its back and forth pursuit of a US deal, a major sticking point has been the withdrawal of its algorithm from the negotiating table. In August, the Chinese government was reported to have changed the rules on the export of technology, meaning that TikTok’s underlying algorithm needed government approval to be part of any sale. The proposed buyout by Oracle and Walmart would, almost certainly, mean that a new algorithm for use in the US would need to be developed, at least according to one report.

No doubt there will be more twists and turns as US President Donald Trump pushes for total US ownership of TikTok but without the algorithm, what is its true value? In fact, what is the true value of any tech business these days without its core technology? Algorithms have increasingly become the backbone and often the differentiator in modern applications. The truth is that algorithms are running our daily, online lives, for good and bad. The data we generate – 2.5 quintillion bytes of data every day, or 1.7MB of data created every second by every person during 2020, according to one source – coupled with our service and software expectations, has lead us to this point.

For Chris Cole, CEO of Headlamp Software, the true business value of algorithms is not so easy to quantify, at least on a broad scale because every company and customer will approach problems differently. Algorithms, he says are not always the answer.

“The basic question is, ‘when should the algorithm directly make decisions about the behaviour of the system, and when should it merely give advice that can be tempered by human judgment?” says Cole. “The temptation to avoid the requirement of human involvement is based on cost. Humans are slow and costly. It is more efficient to take them out of the decision-making process. When is that appropriate?”

Cole adds that while algorithms can be devised to deal with known unknowns, modern machine learning can not only optimise parameters in an algorithm, it can also determine which parameters are to be optimised. Improving algorithms through machine learning, to perhaps help answer those difficult questions on humans and cost, seems like a natural progression but Cole has a warning.

“Machine learning cannot look at all possible algorithms, there are too many,” he says. “So, ML restricts its search to algorithms that are likely to be involved. What does this mean? Here there is guesswork and hence risk. It is possible that the algorithms that are explored do not contain the solution to the problem at hand. It is simple to prove that there is no algorithm that can solve every problem. The only solution is testing. Which involves humans. Which is expensive. But there is, in principle, no alternative.”

To continue reading this article register now