Artificial intelligence (AI) has entered various sectors, promising efficiency and innovation. Everyone seems to be chasing that silver bullet to AI utopia and while many firms are making significant progress in the space, every now and again you hear a story which highlights how important it is to make sure the use of AI is implemented with care and caution.
One such headline hit the news last week, with McDonald’s confirming its decision to remove AI-powered ordering technology from its drive-through restaurants in the US after numerous customer complaints and viral videos showcased its comical and frustrating failures. These issues provide a stark reminder of the potential risks and negative impacts of AI. While a customer receiving a bacon-topped McFlurry can easily be rectified and has an element of comedy value, if AI makes a mistake in a more sensitive area of a customer’s life, like their finances, it is unlikely they will still see the funny side.
McDonald’s AI Experiment: A Comedy of Errors
In 2019, McDonald’s introduced an AI ordering system developed by IBM to streamline drive-through operations. However, the reality of the implementation fell short of expectations. The AI’s voice recognition software frequently misinterpreted orders, leading to bizarre and incorrect outcomes such as bacon-topped ice cream and ridiculous amounts of chicken nuggets. Viral videos documenting these mishaps, such as a woman repeatedly receiving stacks of butter instead of a caramel ice cream or multiple unwanted orders of tea, quickly gained traction online.
McDonald’s ultimately decided to scrap their use of AI in over 100 restaurants, citing these reliability issues as the reason for the technology being discontinued. The restaurant chain remains optimistic about future AI usage and no doubt we will see another iteration of the solution being used in the future, but the experiment’s failure underscores the complexities of replacing human roles with AI, especially in real-time, customer-facing scenarios.
Parallels in Financial Advice: The Stakes are Higher
While the missteps of McDonald’s AI ordering system were often met with laughter from customers, it does illuminate the potential dangers which others should consider when considering the implementation of AI and how the consequences of getting it wrong could be more significant.
If we turn our focus to the provision of financial advice, unlike the inconvenience of a botched burger order, errors in financial advice can have severe and long-lasting harmful consequences which customers are less likely to find funny and which could have significant reputational and financial repercussions for firms.
1. Misinterpretation of Client Needs
Just as McDonald’s AI struggled to accurately process simple food orders, AI in financial advisory roles might misinterpret client needs and preferences. Financial advice often requires nuanced understanding and the ability to ask probing questions – capabilities where current AI systems still lag. Misinterpreted financial data or misunderstood client goals could lead to inappropriate financial planning strategies being designed, jeopardising clients’ financial positions and their ability to meet their life goals.
2. Complexity and Context
Financial planning involves a high degree of complexity and context-specific knowledge. McDonald’s AI’s failure to handle varied and context-rich interactions in a fast-food setting highlights a similar risk in financial advice; just think about the volume of rich data which a client is able to provide in a fact finding meeting. AI systems, if not sufficiently trained to interpret this data and handle the complexities of financial advice, might not adequately factor in all relevant variables or adapt to the unique financial situations of clients, leading to suboptimal advice and client outcomes.
3. Trust
Trust is paramount in financial advisory services, not least in the UK where the FCA’s latest Financial Lives survey shows trust remains a key barrier to individuals seeking financial advice. The comedic but frustrating errors of McDonald’s AI eroded customer confidence in the technology. In financial services, breaches of trust can be far more damaging, potentially leading to significant long-term damage to client relationships and ultimately, client retention.
4. Regulatory Concerns
AI in financial services also raises regulatory concerns. While McDonald’s had few regulatory considerations of using AI in their ordering processes, ensuring AI compliance with financial regulations laid down by the FCA, including Consumer Duty regulations, is complex. Missteps in this area could not only damage the firm’s reputation, but also lead to regulatory censure if the FCA determined the AI had been delivering poor client outcomes.
Conclusion: Proceed with Caution
McDonald’s AI experiment serves as a cautionary tale for the integration of AI in areas where the stakes are significantly higher. While AI holds almost untold promise for enhancing efficiency and innovation, its current limitations and the potential for error underscore the need for careful implementation, especially in fields like financial advice, where the impact on consumers isn’t just an inedible McFlurry, but potentially changes the lifestyle which they might be able to live and their ability to achieve their financial goals.
Companies must rigorously test AI systems, ensure robust human oversight, and prioritise the accuracy and reliability of the technology before deploying it in roles that significantly impact individuals’ lives and finances.
As we move forward, the lesson is clear: AI should augment human capabilities, not hastily replace them, particularly where precision, trust, and context are crucial.
If you’d like to discuss your firm’s approach to AI, book in a free initial conversation with one of our consultants here.
We hope you enjoyed our blog McDonald’s AI Experience: A Cautionary Tale for Financial Advice.
Next up: why not read our blog: Understanding the Value of Advice.
