Google Cloud released its new Automotive AI Agent on Monday and has named the Mercedes-Benz CLA as the first car model to offer it later this year. The Agent will enable Mercedes’ MBUX Virtual Assistant to perform a wider array of conversational functions with the vehicle’s passengers.
We got our first look at Mercedes’ next generation assistant a year ago, at CES 2024, though the company did not reveal which large language model underpinned its capabilities at the time. This new assistant differs from the existing MBUX that can activate around two dozen in-car commands and provide information sourced from ChatGPT and Bing. While the current generation assistant can be activated by stating “Hey, Mercedes,” it functions more like Siri or Google Assistant than ChatGPT’s Advanced Voice Mode, offering static responses rather than conversational replies.
Google’s Agent is built atop the Gemini LLM using Vertex AI and is geared specifically to “allow automakers to create highly personalized and intuitive in-car agents that go beyond current vehicle voice control,” per the company’s announcement post. The Agent supports both multimodal and multilingual inputs, as well as can provide answers to follow-up questions. In Google’s example, the AI will be able to tell drivers if there are any Italian restaurants nearby, then offer up reviews of the establishment and even tell you what the most popular dish there is. The system is reportedly robust enough to handle multi-turn dialog with users and remember details from previous conversations.
The new MBUX assistant will reportedly pull “fresh and factual information” from Google Maps in near real time to offer “comprehensive and personalized information” about more than 250 million points of interest worldwide and current traffic conditions. “At Mercedes-Benz, we seek to offer our customers an exceptional digital experience,” said Ola Källenius, CEO of Mercedes-Benz Group AG, in a press statement. “Our partnership with Google Cloud will further enhance in-car navigation, combining sophisticated location data with generative AI.”
Mercedes plans to roll out the new MBUX assistant to additional models in the future, but has not yet specified which ones it will be integrated into.