Apple has set the tech industry on a non-stop chatter mode with its hot Google deal. After missing out on the early AI hype, and stopgap coupling with ChatGPT, Apple is finally putting Gemini (a customized version without any branding) in the driver’s seat. The idea is to fix Siri, Apple Intelligence, and the whole Foundation Models package.
So, what does that mean for you? For starters, Siri will be as chatty and smart as ChatGPT or Gemini. It will no longer run into situations where it asks to offload your queries to ChatGPT. And hopefully, it will finally gain actionable intelligence into other apps. Google just did that with Personal Intelligence.
It’s quite ironic that Apple advertised the “more personal Siri” concept long before Google, but pulled the ad and later claimed that an all-knowing-Siri will take time to perfect. With Gemini coming into the picture, Apple can do a lot more, and I’m pretty excited about it. But what I really want is for Apple to fix the Shortcuts app.
So much wasted potential
The Shortcuts app is one of the most underrated corners of your iPhone. It’s a powerhouse tool that will take you by surprise because of the sheer breadth of multi-step tasks it can accomplish. And now that Apple Intelligence is integrated into the app, it can pull off some extremely cool tricks.
With a single button click, it can screenshot the video playing on my screen, identify the movie or TV series, and tell me where I can stream it. A single tap is all it takes to turn a screenshot into a memory with custom search tags, a brief summary of the content, and the source URL into the Notes app.
These AI-powered tasks can be handled by ChatGPT or even Apple’s own on-device models that run fully offline. You see the immense potential, right? Well, that’s where the rosy picture ends. Creating shortcuts in the eponymous app is a nerve-wracking chore for beginners.
And even for power users, creating a long automation is a test of patience. The scripting system, with its variables and vague control flow, is pretty darn confusing. You will, almost certainly, need to go through a long explainer to get a hang of them. And even after that, you will find yourself lost when you actually start building a shortcut.
The whole system relies on codes and hotwords that you won’t even realize exist in the first place. And only if you look up the specific app where the task must be handled, you will realize “Oh, there’s a connected action available.” Why not use ChatGPT to describe the steps, and simply repeat them in the Shortcuts app?
Well, I wish I could tell you that it works. But the reality is that ChatGPT will land you in an even deeper confusion mess with vague steps, and often misleading directions that simply don’t exist in the Shortcuts app. It’s not surprising that the highest number of reader emails I get in my inbox are about Shortcuts, asking for guidelines or sharing the direct iCloud link.
The ingredients are already here
“Hey Siri, create a Shortcut that automatically enables focus mode at 4PM, silences all notifications except Slack, and sends me a summarized version of the group conversations.” Imagine saying that, and Siri creates a shortcut for you. It’s a pipedream in the current state of the Shortcuts app (and Siri).
But there’s good news, one that builds upon the Gemini deal for Siri. Apple already has a system in place that allows Siri and Apple Intelligence to interact with apps. Simply put, the onboard AI is aware of actions it can take within apps, without actually opening those apps.
“With Apple Intelligence and enhancements to App Intents, Siri will suggest your app’s actions to help people discover your app’s features and gains the ability to take actions in and across apps,” explains Apple.
Gemini can already do that, and not only within Google’s own apps (such as Gmail or Calendar), but also third-party apps such as WhatsApp. You can just tell Gemini to perform a task, and it will handle it as described. Siri can do so, as well. Technically, that is.
But what I really want is this “conversation to action” system applied in the Shortcuts app. Or to put it more accurately, let Siri create shortcuts directly with natural language descriptions. With a framework in place, that shouldn’t be much of a hassle.
I believe Siri has been the bigger challenge so far, no thanks to its language comprehension and reasoning capabilities that are far behind what you experience with ChatGPT or Gemini. Or, Apple can offer an agent that can help create and modify shortcuts using text or voice commands.
We are already in an era where AI agents enable voice-based video editing on phones, build websites, mini-games, and more. With a strong foundation(s) already in place, it’s a fantastic opportunity for Apple to finally give Shortcuts the due credit by simply easing the workflow.
A better (and more ambitious) idea
I just came across the latest announcement from Replit. The company has announced an AI-powered tool that can create a fully-functioning app and push it to the App Store — with just a text description. It’s almost like willing an app into existence.
The idea is ambitious, but not every iPhone user wants to go through the hassle of getting an app vetted and published on Apple’s store. But Shortcuts can definitely benefit from something like this. Look no further than Android upstart Nothing.
The company recently launched a tool called Playground, an AI-powered toolkit where you can simply describe an app and build it. It’s a no-code platform that works entirely on your phone. You don’t have to deal with any complex app-building tools on a desktop.
These mini-apps can be shared with the community, and you can freely pick one from the public dashboard and tune it to your liking. Google, on the other hand, offers a similar tool called Opal to build web-based apps using Gemini.
Apple can borrow that concept and let users build Shortcuts that can instantly be turned into an app instance, as well. Shortcuts already has a wide global community, especially where users share helpful automations as iCloud links.
But for many users, getting these shortcuts to work can be troublesome, especially if they want to customize them. A Shortcut that is shared as a mini-app, and allows conversational building (and customizability) could prove to be a game-changer.
Instead of putting one’s trust in third-party apps, running on a cloud server or tied to a subscription model, users can simply get it done using shortcuts (masquerading as apps). The crucial benefit is that these Shortcuts can run either fully offline, or using Apple’s secure private cloud compute for AI-based chores.
With Gemini coming into the picture, the possibilities are endless. The iPhone experience is going to change dramatically, if Android is any indication. But if Apple can give a similar AI-fueled overhaul to the Shortcuts app, it would not only solve a nagging problem, but also give iPhone users a unique edge that is not available elsewhere at a native level on smartphones.