AI Ethics: With Great AI Data Comes Great Knowledge and Responsibility

We’ve heard it a hundred times when calling a customer service line, “This call may be monitored or recorded for quality assurance purposes.” We might not stop to ask ourselves who — or what — is being trained. We assume that a manager and an agent will review the call together to assess the agent’s performance and look for areas of improvement. For quality, we think that if there is a complaint or the agent does something wrong, the company can pull up a recording of the conversation and check whether the information provided was accurate. But this might not be the case.

As new voice-processing technologies come to light, we shouldn’t assume that “training” is being done with humans. More often, it’s being done with speech transcription, emotion and sentiment detection as well as prediction engines. Your voice, your responses and your behavior are used to understand how you tick.

The 2012 story that brought to light companies’ capabilities to predict what you’ll do next was Target’s directed marketing to a teenage girl. Target predicted she’d be pregnant and started to send marketing material to her home address. Methods and technologies of prediction are now many orders of magnitude more powerful than when that story emerged.

That prediction was based on statistical methods and, since then, new technology and prediction methods have amplified a company’s to learn and affect our decisions. If we extrapolate this further, it’s safe to wonder if there will be a time when we could be completely disarmed by a prediction engine that knows how to manipulate us to make a purchase. Will we then lose all involvement in our own decision making?

The Ethics of the “Tell”
Predictive technologies at play like artificial intelligence (AI) and machine learning can extract more insights about us than just the content of what we say or our personal identifiable information. How we say things, the tone we use, how we elicit emotions to different inputs — these can all be our “tell” in a poker game between us and agents who are selling us things or moving us to a course of action we might not have reached on our own.

Many people question if this ability to learn will create such an imbalance in influence that it will become unethical to deal with any company and how companies must meter their use of this influence. When interacting with companies, we see where there might be some asymmetries in our interactions. These areas include product knowledge, history of interactions with the customer, the ability to try different messaging tactics, price testing and the ability to surface information quickly. These asymmetries are more prevalent in industries where there is more “supplier power,” as described in Porter’s Five Forces.

Typically, a company is much more knowledgeable in its offering as well as those of its competitors. From an optimistic perspective, this means the company can provide value to a customer — even before the customer makes a purchase. Consultative sales processes add even more upfront value. However, when product information is limited or specialized, relying on a specialist to provide this information can create a value gap where a higher price can be charged.

Companies also can systematically track all interactions between themselves and their customers. As individuals, we don’t likely keep a record of all of our calls to the bank, when we visited the website or every email we sent them. But the bank tracks this and can make this information instantly available to an agent. This gives them a better assessment of their relationship with you and what will happen next.

Another advantage is that companies can run tests in every interaction. In a call center, this might mean running different call scripts. In eCommerce, it could mean setting different pricing models. A company with a large number of clients can run these tests with each interaction and know more about how its customers will react. A customer might only interact with a handful of companies versus the thousands or millions of interactions a company might have with different customers.

Companies can also realize an advantage in an interaction by being meticulously organized. Having all information on hand during a discussion, such as previous calls, competitive pricing and product information means that, even if they’re not as knowledgeable of the customer, they have access to more information and resources than the customer could ever have.

Join the discussion on AI Ethics.