Part III: When Reliance on AI Can Hurt You– The Ugly
Eric Koefoot, President and CEO, PublicRelayIn our line of business, we work with a lot of very smart customers: brilliant communicators and strategists. These professionals are looking for the best way to get information, solve problems, and get ahead of the news cycle and the competition. There is a strong push in their company for them to leverage AI. The operations department uses it. Marketing uses it. Even finance uses it. So why not use it in communications for media analytics?Because it does not work.As we have seen in the study discussed in previous parts of this series, AI alone is not ready to solve this problem – not even close. While many media monitoring and analysis providers are touting their artificial intelligence upgrades, almost none of them are approaching it in a manner that results in trusted analytics for their customers. Most solutions still deliver 25-60% incorrect/irrelevant mentions. And while some claim up to 80% sentiment accuracy, independent studies do not support those claims, especially when multiple topics and multiple sentiments are laced within a single piece of content. Taken together, that means less than 50% of your automated AI analytical data is correct.So without someone to clean up the automated data – and provide a consistent, accurate feedback loop to the AI system – it is virtually impossible to derive sophisticated metrics from these AI-only solutions. Decisions based on inaccurate data can really hurt you – not to mention your efforts to have an impact and prove your results in front of your executive team and your CEO.
What to do?
To make AI work for analyzing human communications, you need enough consistent, superior-quality analysis to feed back to the computer and train it. The training set needs to be updated regularly to stay fresh and keep the analysis current as the conversation and issues evolve. Because language and the public discourse change so much, you need ongoing training for your AI. And you need to tune it specifically to your perspective. A solution tuned to someone else’s perspective just won’t work.It doesn’t mean you have to analyze everything to train an AI system, but you need to analyze enough data so that your computer can learn robustly from them. AI alone can’t teach itself when the answer is not a known ‘yes/no’ answer, so you need human-supervised machine learning to teach it.The fact is that AI ‘thinks’ within fixed boundaries, while humans ebb and flow in the way they think and communicate. Most humans innately and quickly understand shifting context and can get the gist of whether something is positive or negative. Computers and AI are far from understanding gist, which depends on context, time, circumstances, tone, topic, cadence, sarcasm, and double entendre. Because of this complexity, two people can even analyze a news article and disagree whether it’s positive or negative because they think about the world differently. That is the set of problems that computers won’t be solving anytime soon.But on the bright side, humans can work with AI by defining, training, and maintaining a dynamic, accurate, and reliable human feedback loop. This means persistent training, unique for each individual company, with human attention to help AI bridge the gap between what it’s trained on, and what the customer is trying to know. For the foreseeable future, supervised machine learning is almost universally considered to be the leading approach to solving the limitations that AI analytics face in media intelligence.So how will you use AI? Smartly, I hope. PublicRelay delivers a world-class media intelligence solution to big brands worldwide by leveraging both technology and highly-trained analysts. It is a leader on the path to superior AI analytics through supervised machine learning. Contact PublicRelay to learn more.
