Chance Miller, reporting last week for 9to5Mac:

Apple released iOS 18.3 beta 3 to developers this afternoon. The update includes a handful of changes to the notification summaries feature of Apple Intelligence.

The changes come after complaints from news outlets such as the BBC. Two weeks ago, Apple promised that a future software update would “further clarify when the text being displayed is summarization provided by Apple Intelligence.”

Here are the changes included in iOS 18.3 for Apple Intelligence notification summaries:

  • When you enable notification summaries, iOS 18.3 will make it clearer that the feature – like all Apple Intelligence features – is a beta.
  • You can now disable notification summaries for an app directly from the Lock Screen or Notification Center by swiping, tapping “Options,” then choosing the “Turn Off Summaries” option.
  • On the Lock Screen, notification summaries now use italicized text to better distinguish them from normal notifications.
  • In the Settings app, Apple now warns users that notification summaries “may contain errors.”

Regarding that note about Apple Intelligence being a beta, here are Apple’s official iOS 18.3 release notes:

For users new or upgrading to iOS 18.3, Apple Intelligence will be enabled automatically during iPhone onboarding. Users will have access to Apple Intelligence features after setting up their devices. To disable Apple Intelligence, users will need to navigate to the Apple Intelligence & Siri Settings pane and turn off the Apple Intelligence toggle. This will disable Apple Intelligence features on their device.

So, in iOS 18.3, Apple Intelligence is no longer in beta.1 But I don’t think the distinction really matters much at all because Apple’s marketing wouldn’t lead anyone to believe Apple Intelligence is anything but a well-built, reliable piece of software. Here on Earth, the truth is far from Apple’s rosy picture painted on billboards across America. Beta or not, Apple Intelligence’s notification summaries are comically unreliable, factually incorrect, and straight-up grammatically awkward (see the headline of this post for an example).

The British Broadcasting Corporation complained to Apple over the holidays because Apple Intelligence incorrectly summarized a BBC headline about Luigi Mangione, the suspect in the UnitedHealthcare chief executive’s killing. The software said Mangione committed and only displayed a small glyph to the right of the blurb indicating that it had been written by artificial intelligence; the BBC app’s logo, however, was prominently displayed next to it, leading readers to believe that the fabricated summary was really from the BBC.

Apple’s response to the debacle was that Apple Intelligence was in beta, but by making it an opt-out feature — i.e., enabling it by default for the millions of iPhone 16 users in supported countries — Apple removed that (debatable) cover it could hide behind. Apple Intelligence isn’t in beta, and it hasn’t been for months — slapping a “Beta” label on it in Settings doesn’t change the fact that it’s heavily advertised when setting up a new compatible iPhone. Removing it further negates any possible excuse for Apple Intelligence summaries not being completely accurate.

It’s not like large language models are bad at summaries. In fact, they’re fantastic at them because LLMs are trained to synthesize the next most logical word in a sentence. When given a snippet of text, they boil it down to some weights, find what other weights correspond to the numbers originally given, and spit out a summary. This is what LLMs are best at. As an experiment, I tried running some botched Apple Intelligence summaries through ChatGPT — both the less-expensive, faster model and the latest 4o one — just to see how a reputable model would do, and ChatGPT aced the text. Its summaries were reliable, short, and grammatically correct.

I’d love to look at the prompt Apple is feeding its so-called foundation models before adding the notification’s content. I presume it’s in some organized data format, not plain text, but that should be fine for a model specifically trained on thousands of summaries. Even low-quality models fare well in summarization tests because this isn’t too difficult of a task for an LLM. I believe Apple’s models — no matter how low-quality they may be to run quickly enough so as not to create a delay from when a notification is sent from a server and when it’s displayed on a user’s device — aren’t what cause Apple Intelligence’s downright disturbing summaries.

The model’s context alters its ability to summarize a notification significantly. For instance, this is how I’ve been prompting ChatGPT to create notification summaries:

Your job is to summarize notifications. A user has received multiple breaking news notifications from The New York Times app. The first one is from 12:56 p.m. and reads, “Eighteen states sued to block an executive order that seeks to deny citizenship to babies born to unauthorized immigrants in the United States.” The latest one is from 4:29 p.m. and reads, “Pete Hegseth’s former sister-in-law made a sworn statement to senators that the secretary of defense nominee was abusive toward his second wife.” Summarize these notifications, with the most importance given to the newest notification, in a maximum of 20 words.

ChatGPT responded with this:

Defense nominee accused of abuse; 18 states challenge executive order denying citizenship to children of unauthorized immigrants.

I wish I could see what Apple Intelligence would’ve cooked up, but I can’t since The New York Times is a news app, and Apple Intelligence summaries are now disabled for them (temporarily, according to Apple) in iOS 18.3. (This is yet another update to address the BBC’s concerns.) Either way, after months of using Apple Intelligence on all my Apple devices, I’m certain it wouldn’t do even half as good as ChatGPT.

Apple Intelligence struggles with two main categories of notifications: short ones that don’t need summarizing and threads of long notifications with details. When presented with a short notification, Apple Intelligence, like any other LLM, just makes up information to fill its character limit. (You can see this in an example Miller posted on Bluesky.) When the software is given tens of notifications from different times and plentiful details, however, it doesn’t understand the contextual difference between a notification sent two hours ago versus a minute prior.

This is most noticeable in delivery notifications, where the status of an order is changing with each notification. Apple Intelligence doesn’t know how to process this, and its insistence on using semicolons to separate notifications into distinct parts creates nonsensical, useless summaries. For instance, three notifications that tell a user that their order is about to arrive, that it’s here, and that they should tip after it’s been delivered turn into one sloppy mess, and Apple Intelligence comes up with, “Order on the way; delivered; rate and tip.”

LLMs speak English well, and with a smidgen of context, iOS could do a much better job. I — a human who writes for a living — would discard the “order on the way” message entirely and summarize the notifications by writing: “Your order has been delivered at [time]. Rate and tip.” There’s no need for semicolons, but because summaries don’t display when each individual notification was sent (tapping on it expands them all but closes the summary), a timestamp could be helpful. If given the time, context, app, and notification title, Apple Intelligence could do this in just a few seconds.

For now, Apple Intelligence summaries aren’t even remotely ready for prime time. I understand the frustration within the company — it needs to iterate to get ahead of OpenAI and Google, and it needs to do so quickly — but shipping incorrect notifications to millions of people is a terrible way of achieving strategic goals. People’s iPhones are lying to them, and Apple can’t even accept minimum fault for its faulty software. The italicized text doesn’t make it clear to me that a summary is generated with AI — it just looks like a sloppy, out-of-place design. Does Apple use italics in any other part of the software? Perhaps that’s why it was implemented here, but it just looks awful and relays little to no information without already knowing italics mean Apple Intelligence.

Instead, I recommend Apple replace the app icon with an Apple Intelligence logo and minimize the icon to be in the lower left corner, almost like iMessage notifications, where the Messages app’s icon is displayed in the corner of a contact’s profile picture. Ultimately, the content displayed on the screen is from Apple Intelligence, not whatever app sent the notification, so that should be obvious. If Apple doesn’t like putting its name on these summaries, perhaps it should reflect why it’s so hesitant. Is it not confident in its software?

One more frustration: Apple Intelligence must stop summarizing spam text notifications. I got one about a toll I allegedly forgot to pay from a random iCloud email address, and Apple Intelligence perfectly summarized it — threat and all. People have asked me previously how I expect AI to detect a scam message, which is an insane question. ChatGPT has the world’s knowledge compacted into one text generation machine, and to think an LLM can’t use that knowledge to detect a scam and choose not to summarize it is ridiculous.

People have an inherent trust in Apple’s products. If Apple summarizes a notification incorrectly — or even worse, marks a scam email as a “priority” in the mail app — people are likely to believe that. “Well, Apple said it’s real, so it must be.” We’ve been teaching people for decades to check if an email or text is really from Apple, Google, the bank, etc., and these summaries are from Apple. Why shouldn’t users trust them? I brought up this same point when Google told its users to put glue on their pizzas last year: If a company has built its reputation around being an arbiter of facts, why is it suddenly acceptable to forgo the truth in favor of shoddy technology?


  1. I’m catching flak for this since Apple Intelligence is still labeled as a beta in iOS 18.3. Here’s the dictionary definition for the phrase “beta test”: “a trial of… software… in the final stages of its development carried out by a party unconnected with its development.” Apple Intelligence is a now-shipping-by-default feature of iOS. It’s not a trial by any definition of the word “trial.” People aren’t trying anything; they cannot make the choice to opt in. No matter what Apple calls it, Apple Intelligence is no longer in beta. ↩︎