According to Bloomberg‘s Mark Gurman, Apple bought a Canadian AI startup earlier this year. As the company promises big AI announcements for 2024, this acquisition might help bring a killer feature that no other competitor has yet: on-device AI.
According to the journalist, Apple bought DarwinAI, which developed AI technology for “visually inspecting components during the manufacturing process (…), but one of its core technologies is making artificial intelligence systems smaller and faster.”
Since Apple is on a big privacy push, being able to run AI on the device rather than entirely in the cloud could be one of the best features Apple could announce at WWDC 2024.
If we combine this report with a paper Apple published last year, it’s almost guaranteed that the company will introduce on-device AI features instead of running them in the cloud.
In this study, Apple researchers say that they can make AI models run up to twice the size of the iPhone’s memory by using windowing and row-column bundling:
- Windowing: It loads parameters for only the past few tokens, reusing activations from recently computed tokens. This sliding windows approach reduces the number of IO requests to load weights.
- Row-column bundling: It stores a concatenated row and column of the up-projection and down-projection layers to read bigger contiguous chunks from flash memory. This increases throughput by reading larger chunks.
Apple is also testing generative AI to help customers with service requests. With iOS 18, the company will likely add a feature for its software for auto-creating presentations, completing blocks of text, and even a new version of Xcode that will help developers write code.
iOS 18 is expected to be announced during the WWDC 2024 keynote. More features should be available later this year when the company unveils the iPhone 16 lineup with A18 chips.
