Transparent vs. Opaque AI: Which is right for you?

Dr. Rob Walker from Pegasystems discusses the common risks and gains of AI in business

This is a contributed piece Dr. Rob Walker, VP of decision management and analytics at Pegasystems

Although Mark Zuckerberg and Elon Musk have famously locked horns over the question of artificial intelligence (AI) and its impact on human civilization, the type of AI they are discussing is a very particular one, relating to ‘human level’ cognitive skills and is known as AGI or ‘Artificial General Intelligence’. Despite its progress in a range of specialities, from playing Go to driving cars, this AI technology is not yet imminent. Organizations looking at AI today are concerned with other more pressing matters.

Indeed, AI is something that’s already in common use by many companies today, and the associated risks are less concerned with the matter of whether or not it will be the destruction of society. Many organizations are focused on the more real risks posed by this technology in the here and now around how it might be used inappropriately. This relates to questions including violation of regulation, reduced business value and brand damage. Though they may not spell the end of humanity, they do impact upon the success or failure of countless organizations.

The two distinct forms of AI, Transparent AI and Opaque, have very different uses, applications and impacts for businesses and users. Insights from Transparent AI can be understood and audited, allowing us to reverse engineer each of its outcomes to understand how it has arrived at each decision. Meanwhile, Opaque AI cannot easily reveal exactly how it has arrived at a particular insight or decision.

Opaque AI can provide many useful benefits. It might even be preferable over an AI that is Transparent, which may be constrained and limited in power and effectiveness. With an Opaque system, however, there is the potential issue of bias. An Opaque AI system may begin to favour policies that break an organization’s brand promise. It’s not so difficult for AI to make use of neutral data to work out information about a customer, which it can then use to make non-neutral decisions. An Opaque AI in a bank may interpret customer data and use it to offer better deals to customers based on factors such race or gender. This, of course, would be a major concern for any business.

Choosing between Transparent and Opaque AI is key in highly regulated industries. In financial services, proper use of Opaque AI in lending will result in fewer errors and improved accuracy. But it becomes a challenge, even a liability, if banks must show how these operational improvements were achieved though reverse engineering the decision process.

This is particularly relevant with the upcoming GDPR in May 2018. GDPR mandates that companies will need to have the ability to explain exactly how they reach certain algorithmic-based decisions about their customers. There is a particular advantage for those who can use a switch to increase transparency by forcing the methods used by AI to make decisions from Opaque to Transparent: they will be more easily able to comply.

By deciding how much they are willing to trust their AI, organizations can determine which type of system is most appropriate. In order for an organization to place total trust in an AI system, either the AI needs to be Transparent so that business management understands its workings, or the AI, if Opaque, would need testing before it is taken into production. Testing must be extensive and go beyond merely searching for viability in delivering business outcomes, searching also for types of unintended bias.

A transparent system might be the preferred choice of many if they could make it unfettered. But, you may hesitate to insist on Transparent AI to diagnose a patient if there was an Opaque alternative which was more likely to diagnose correctly and save a life. In other cases, the deciding factor may be marginal, with issues relating to profitability, customer experience and regulation to consider before the choice can be made.

Many businesses face a dilemma about selecting the right AI system. But, either way, the question is more about customer experience and business value, and less about the life or death of human civilization.