We need to talk about ethics: AI data and the challenge of frameworks

What should enterprises do with AI data? And what would a framework for corporations to manage bias in data look like?

How do you train AI not to misbehave? Scientists in the US and Brazil seem to have come up with an answer, at least according to a combined University of Massachusetts Amherst, Universidade Federal do Rio Grande do Sol and Stanford paper called Preventing undesirable behavior of intelligent machines. In a nod to author Isaac Asimov's character Hari Seldon, the group has developed what it terms a "Seldonian algorithm", a framework for machine learning designers to build behaviour-avoidance instructions into algorithms used in real world products and services.

Asimov's three laws of robotics, from his 1942 short story Runaround, are well referenced. The paper attempts to focus on the first law, to never hurt humans, using a new technique that uses mathematical algorithms to translate goals, such as avoiding gender bias, into instructions for machine-learning algorithms to train AI applications.

"We want to advance AI that respects the values of its human users and justifies the trust we place in autonomous systems," said Emma Brunskill, an assistant professor of computer science at Stanford and senior author of the paper. "Thinking about how we can create algorithms that best respect values like safety and fairness is essential as society increasingly relies on AI."

The idea is that if ‘unsafe' or ‘unfair' outcomes can be defined mathematically, developers can create algorithms that learn from the data to mitigate against potential bad behaviours. Of course, any framework that proposes to ethically cleanse data is still going to be fraught with technical and moral complications but while this may curtail any attempts by AI to end the human race, what can enterprises do about data bias unintentionally skewing AI?

According to Edy Liongosari, a chief research scientist at Accenture Labs in San Francisco, "technology is critical" to solving data bias issues in industry. Speaking at the IoT World Congress event in Barcelona recently, Liongosari outlined a framework for responsible AI, a framework built around business processes and technology that organisations need to consider when handling and using data to build and implement applications.

The problem, says Liongosari, is that data, by its very nature is loaded with bias and as a result can impact AI decisions. He talks about creating a clear and transparent process for data and data processing as being the only way to build trust in AI. To achieve this, he says organisations need to break down processes into three main categories - governance and compliance, system development and runtime management and ethics audit sweep. Each category has its own guidelines, including data ownership, testing, security, modelling and monitoring. The aim is a flow of data that minimises bias input throughout the business cycle.


While only broad foundational categories, Liongosari suggests they are workable and organisations should take responsibility and develop a strategy of transparency. After all, he says, transparency creates trust and trust makes for more productive and satisfied employees and happier, more engaged customers, at least according to Paul J Zak's ideas in The Neuroscience of Trust.

Taking responsibility is a key point here but will businesses actually bother? Speaking during an AI ethics panel discussion at IoT World Congress, Michael Godoy, director of telemedicine and scalable therapeutics at the University of California said that responsibility should lie with both businesses and the technology firms. He told a story about how a lack of data transparency led one business to send out pregnancy product coupons to the wrong person, alerting father that his daughter was pregnant. The problem, says Godoy, is that regardless of the data that is coming into these organisations, they are making it less transparent and less accountable.

While it's important to differentiate between data privacy and data bias, there is certainly overlap in terms of how organisations currently use or intend to use data within deep neural networks. Understanding who is selecting data sets and having some idea of data provenance seem like reasonable requests but not easy.

Speaking on the panel Jennifer Bennett, Google's technical director in the office of the CTO, said that "it can be difficult to explain why some results occur. You have to really understand the weights say a factory applied or how sensors collected the data." Getting this level of transparency throughout the data lifecycle is challenging.

According to David King, CEO of FogHorn Systems, it's almost better to work backwards, or at least identify potential outcomes through a variety of people working in different areas of the business and then work out what is required to achieve those outcomes.

"In manufacturing or industrial, if you don't have the perspective of the operator, and ultimately the business leader, the models you're going to build and what the eventual outcomes are may differ, so you need learning throughout the process to determine how to get the right outcome. I think that's been one of the primary challenges."

It goes back to the idea of having ethical frameworks that structure data flow in such a way as to enable organisations to keep any bias to a minimum and optimise the potential outcomes of the AI decision making process. But they are just that, frameworks. How they are used and implemented is open to interpretation. Of course it is and with that comes consistent challenges in how ethical data is perceived. Who decides what is ethical data? It will mean different things to different people in different scenarios selling different products and services. The variables are huge.

It comes back to transparency. Accenture Labs' responsible innovation lead Steven Tiell suggests an ethics committee. May be this is the only way forward. While we have technical solutions to reduce bias, such as the Seldonian Algorithm, it will still come down to humans as to what is ethical and do we really trust humans?