Inside Amazon’s method for measuring engagement in the age of AI
Communicators still need to be engaged in active listening.
By Sean Devlin, editor, Ragan Communications
An internal comms dashboard can show leaders how many employees opened a message and clicked through. But it can’t interpret and explain them like a strategic communicator can..
At Ragan’s Employee Experience Conference in Nashville earlier this year, Dan Kneeshaw, director of content and channels for worldwide operations internal communications at Amazon, spoke about how his team answers those questions and how AI fits into the equation.
“The leaders ask, ‘OK, but how is it landing and how do our employees feel about that message?’” Kneeshaw said. “That really gets into sentiment, and it gets into more of the anecdotal feedback. So it’s not always hard numbers.”
Kneeshaw added that with feedback coming in from around the globe, Amazon leans on AI to keep up.
“What it’s done for us is it’s helped to enable real-time dashboards,” Kneeshaw said. “AI gives us that really fast speed on what’s happening, so we can pivot and adjust quickly, versus the old days, where you had data coming in monthly or quarterly. It was so far behind you, you couldn’t really react in the moment.”
Darcie Rosenthal, director of communications planning and performance at ADM, said that communicators needed to take the metrics they’re reviewing with a grain of salt and look for nuance where AI can’t.
“Employees are not necessarily going to put their real opinion, especially if their name’s tied to it. They’re going to filter that,” Rosenthal said. “When we run it through AI to look for sentiment, the AI is not going to know that about our people. But we know that about our people.”
Rosenthal said that ADM added feedback questions to the communications it sends managers in order to get a better read on sentiment analysis.
“They are the ones working with our employees every day, so they have direct line of sight to the things people are complaining about and the great stories that are happening on the ground,” she said.
Amazon uses a similar process with Voice of the Associate, an initiative where frontline workers raise questions and concerns for site managers to answer. Even as an advocate for AI in comms, Kneeshaw said there are still instances in which automation can’t grasp the full picture.
“It can give you the data point, but it can’t give you the why,” Kneeshaw said. “It can’t give you the reason why that data point changed. That’s what takes that human intervention and that human judgment.”
To watch this presentation in its entirety, visit Ragan Training.