The Need for Data Provenance with AI

I recently did my annual pilgrimage to Seattle to attend the Microsoft MVP Summit (I am in the middle, just left of the logo).

While all the talks are under NDA and, therefore, I cannot talk specifics, I do not think I am giving too much away by saying Microsoft are investing heavily in AI services, which they generally refer to as ‘Copilot’. Some of the Copilot capabilities we can already see in demo versions of products like Dynamics 365 Customer Service where AI performs tasks such as:

• Summarizes a Case for us
• Suggests email topics and the content for those emails
• Allows for plain English enquiries of the knowledge base via a chatbot

Broadly speaking, Microsoft’s Copilots either summarise existing data (Case records, KB Articles etc.) or suggest next actions (suggested emails to send). These may be great time savers but, as Microsoft admits, still require a human to verify the summary is correct or the action is appropriate. This is why the bots are called Copilot; the user is still driving.

The reason we cannot put AI in the driver’s seat? In short, AI ‘hallucinates’. With limited or ambiguous data or with questions asked of it without full context, the responses may not be completely accurate. It was pretty easy for me to generate an example of this. I asked Bing’s Copilot who I am on ‘GPT-4 Precise’ mode.

The second area is pretty accurate. The first is pure fantasy and is a description of the actual founder of TRIBE, Leon Burman with the text taken straight from the TRIBE web site. Thankfully, assuming the reader is diligent, they can click through on those numbers to review the sources of the information. Because the provenance of the data is clearly laid out, we have a way of checking the veracity.

For Dynamics, this is not the case and all the disclaimers in the world will not protect Microsoft for the fall out if it goes awry.

Using a Dynamics 365 90-day demo I happen to have lying around, let us explore some examples.

Data Summaries

Here is a Case Description I put on a Case in Dynamics 365 using a File Note format.

The Copilot summary was:

This description is not accurate with Copilot inserting a fictitious Customer and wrapping the description around a fictitious interaction between this Customer and their supplier, rather than ours.

Yes, there is a disclaimer on the summary and, yes, we can give it a thumbs down but there is absolutely no reference to where the data is coming from. In fact, they only way to find out is by being an administrator and heading into Customer Service configuration area.

Without knowing the eight fields being used, a user is compelled to review the entire Case every time they open one to ensure the Summary is accurate. This kills any time-saving provided by the Summary in the first place.

As we can see above, email and conversation data are also considered in the summary. What if multiple people have touched the Case? Will Copilot tell me I spoke to a customer about something when, in fact, it was a completely different user? Will I trust my memory (which may be difficult in a high volume call center), do I trust Copilot, or do I review potentially hundreds of actions in the Timeline to be sure? Provenance, if only for the user’s sanity, is essential.

Action Suggestions

One of the options available for emails is to express empathy to the complainant. As we can see below, the content gets confused in the same way as the Summary. What is worse here is this is going out to the complainant and will likely anger the supplier more, rather than reassure them.

To Microsoft’s credit there is a sources option at the bottom of the suggested email text but, it is not much help and is, in fact, misleading. Rather than say it came from the Case Description, it suggests context came externally.

In the case of Copilot suggesting next best actions, is this based on all sales made to an Account, including those of other departments whose sales processes may be completely different? Is it just my data? Again, provenance would be good to understand the context of the recommendation. What is more, if there is a poor assumption hidden in the recommended action, and the user unwittingly does it anyway, will this reinforce the recommendation for others, training everyone to make the same mistake or the same unnecessary action?

The Need for Provenance

Generally speaking, those of us who have used tools such as ChatGPT or Bing Copilot know we must be careful in trusting the information provided, bringing our own critical thinking into the mix. There is no reason to expect the tools inserted into Microsoft’s products such as Dynamics 365 to be any different. Therefore, it is essential those tools give the user every opportunity to verify what they are seeing is correct by the more efficient means possible.

Without data provenance we are making users’ life harder, not easier as they wade through screens of data to ensure what is being said is correct. We may unwittingly anger our customers and Copilot may try to gaslight us, with no practical way for us to confirm the reality of the situation. Finally, by leaving it up to Copilot to recommend the best way to sell or resolve cases, we could be making users less efficient, carrying out common but unnecessary steps. Transparency is essential provided by clear data provenance for all summaries and recommendations provided by Copilot.