AIBig Firms

Apple Steps into the AI Era, Siri Finally Gains Intelligence

While Android devices have been making a big fuss about AI, Apple fans have been waiting for Apple’s AI to make its debut. The day has finally arrived.

At 1 a.m. on June 11, Apple’s WWDC 2024 (Worldwide Developers Conference) arrived as scheduled.

Unlike last year, Apple did not launch any hardware products at this conference, but instead focused entirely on operating system updates, sharing new versions of VisionOS, iOS, iPadOS, macOS, watchOS, and other major operating systems in turn, marking a major update of Apple’s family of operating systems.

Afterward, Apple Intelligence made a grand entrance, and Apple officially announced a partnership with OpenAI. However, as in previous years, Apple still couldn’t avoid being criticized for “lacking innovation” and was also “blasted” by Musk with consecutive posts. So, what exactly did Apple’s WWDC 2024 talk about?

A More “Personalized” iOS 18

Following the timeline of the press conference, let’s go through what Apple specifically talked about. First to appear was Apple TV+, then the much-anticipated VisionPro system: VisionOS 2 made its official debut.

VisionOS 2
VisionOS 2

The newly upgraded VisionOS 2 brings the spatial photo function. Through AI computation and analysis, VisionOS 2 can transform all 2D images in the photo library into images with spatial 3D actual size dimensions. Moreover, there have been upgrades in overall interaction, providing more quick gestures.

After VisionPro, the main event, iOS 18, officially made its debut. And if we summarize the updates of this iOS with a keyword, it is “personalization.”

Firstly, iOS 18 has a customizable home screen, where you can freely place application icons anywhere you want.

For instance, you can add favorite wallpapers, apps, and various widgets to the home screen. Moreover, Apple now allows you to set a new dark mode appearance that matches your preferred desktop color scheme and style.

Additionally, you can customize the theme colors of all app widgets, and iOS will automatically recommend matching color tones based on the wallpaper style.

The upgrade to the Control Center also reflects “personalization,” where you can swipe between multiple controls. Aimed at developers, iOS has introduced a new Control API that supports developers in adding their apps to the Control Center. With a long swipe, you can bring up additional control options, and the layout and size are adjustable.

Furthermore, in the locked screen state, the original camera and flashlight functions can be replaced with other control options.

In terms of security and privacy, iOS 18 has also introduced personalized settings.

You can individually lock a specific app, “secure it,” requiring verification to open, and once locked, the app’s internal information will not appear in other parts of the phone system such as search and notifications. Going a step further, you can also hide apps that you do not want others to know about by placing them into a new locked hidden app folder.

In terms of specific app updates, iOS 18 has also put effort into “personalization.” This includes the new features of the Photos app, which the official calls “the most significant design update to date.”

In the upper half of the Photos app, iOS 18 continues the previous grid view, suitable for browsing all images; the lower half adds a “mini album” that categorizes and organizes photos by time and theme. In addition, it includes quick access features such as face recognition categorization and photo album pinning.

Another most commonly used app, Messages, has seen Apple add more emoji in message replies and new formatting displays such as bold, underline, strikethrough, and italics. Moreover, you can add special effects to text.

Here, FirmKnow feels it is particularly worth mentioning that Apple has also added satellite communication capabilities, allowing message sending via satellite without the need for an internet connection.

This feature has begun to be popularized in high-end domestic models, and Apple seems to be keeping up with the trend, but specific details were not announced at the press conference, and the exact effect is still unclear.

In other aspects, iOS 18 also introduced a host of minor updates such as Safari, Maps, Wallet, Game Mode, etc., which will not be described one by one.

https://twitter.com/gregjoz/status/1800272982390284312

Regarding this update of iOS 18, some people have criticized it as “the smallest update in the history of iOS18.” In response, FirmKnow believes that iOS 18 is no longer the main focus of Apple, and expectations should not be too high. Over the years, we should also be accustomed to Apple’s tone in innovation. Being able to optimize the experience on the original basis is already considered passing.

Other Highlights of iOS Updates

After iOS 18, the conference introduced several major systems such as macOS, iPadOS, tvOS, and watchOS in sequence. FirmKnow will highlight the important points.

On the macOS front, Apple officially announced macOS 15 Sequoia.

A feature worth noting here is iPhone Mirroring. With this feature, you can use your Mac to control your phone. For example, you can order coffee, add notes, and the notifications and audio from your iPhone can be shared with your Mac, and the content from your Mac can be more conveniently transferred to your iPhone.

This feature is indeed very convenient. The demonstration shows that with iPhone Mirroring, you can swipe the screen as if you were using a phone, and you can even type on the phone using the Mac keyboard.

In addition, macOS 15 Sequoia also adds a cross-platform Passwords app that can store various new types of account information, including Wi-Fi passwords, keys, verification codes, and shared passwords, which will be used on iOS, iPad, Mac, visionOS, and Windows in the future.

iPadOS 18 also has a significant update, which is the introduction of a calculator and the addition of the Math Note system.

The new system supports recognizing handwritten formulas, automatic solving, and automatically drawing function curves. You can use the Pencil to directly write mathematical formulas, and then it will help you calculate the results directly, even complex formulas, and understand all the variable changes in inequalities.

For example, if you write a mathematical expression with the Apple Pencil, after writing the equals sign, the calculator will automatically calculate the result. If you modify the expression, the result will be updated in real-time.

Moreover, the new feature also supports more complex calculations, such as a physics problem, where you hit a ping pong ball at different angles and speeds, and the highest height the ping pong ball can reach.

You just need to write the numbers of various variables in the math notes, and then the calculator can directly perform calculations in the formula. Write it as a function expression, click the equals sign, and you can directly draw the function curve. Adjust the variables, and the function curve will change accordingly.

Moreover, in handwriting, the system uses machine learning to imitate your handwriting, making writing more natural. At the same time, it also supports generating handwritten notes from keyboard input, spell check, proofreading, and other features.

In summary, FirmKnow feels that Apple has some highlights in the updates of the six major systems, but not many, mainly some improvements in detail experience.

Apple Intelligence saved the day as the finale.

Apple AI Makes Its Debut, Siri Is Reborn

Compared to other manufacturers, Apple did not overly emphasize the grand concept of AI phones but instead applied AI to specific application scenarios.

In terms of text processing, Apple Intelligence introduced a “Priority Notifications” feature. Once enabled, the system can sort notifications based on the importance of the content, with the most critical information, such as emails containing flight boarding passes, displayed at the top.

However, Apple did not clarify the criteria for determining “important information,” which may be based on user habits, app usage frequency, or natural language understanding capabilities.

Text processing is, of course, inseparable from writing. Apple Intelligence also launched a writing tool across the entire system, covering emails, notes, Pages, and various third-party applications. Users can expand, proofread, and summarize texts in various application scenarios, while also adjusting tone and phrasing or receiving intelligent reply suggestions.

In image processing, Apple Intelligence introduced Genmoji and Image Wand features. The former can generate images and emoticons that best match your description, and it can also customize based on people in the photo album, currently supporting only sketch, cartoon, and illustration styles. The latter can help users transform rough hand-drawn sketches into more exquisite images or directly create images from text.

FirmKnow needs to point out that compared to the actual demonstration effects and other large models such as Stable Diffusion, Apple Intelligence’s capabilities still have a certain gap.

In addition, Apple Intelligence also supports removing distractions from photos and using natural language to search for images and videos. For example, if you want to find files sent last week or photos with your girlfriend, just speak out, and the system will automatically search.

In terms of voice calls, Apple Intelligence can automatically generate summaries of recording content and summarize telephone recordings. Although this feature may seem ordinary, FirmKnow believes it can solve a pain point for Apple users and is very practical.

It should be noted that when using this feature, the other party in the call will receive a notification.

In terms of privacy protection, Apple Intelligence offers two solutions. First, the large model functions that can be run locally will only operate through edge AI, with Apple collecting information only locally and not transmitting it to the cloud. Most AI functions can be completed on the edge, mainly thanks to Apple’s powerful chips.

Secondly, if the complexity requires the use of cloud models, to ensure the use of the cloud and information security, Apple has created private cloud computing. These models will run on servers specifically created with Apple Silicon, thus also having very strong privacy protection features.

Apple’s Senior Vice President of Software Engineering, Craig Federighi, said, “The data uploaded to the cloud will never be stored, and even Apple cannot access it; the data is only used to execute requests.”

Compared to the recent Microsoft Recall being hacked incident, Apple’s privacy protection solution is more comprehensive.

The above is the capability of Apple Intelligence in AI, which will be provided for free with iOS 18, iPadOS 18, and macOS Sequoia.

If Apple stopped here, FirmKnow believes that this press conference could only be considered barely passing, but Apple still has a big move, which is to join hands with ChatGPT for a brand new upgrade to Siri, and Siri 2.0 officially debuts.

In fact, when Siri was first launched, Jobs saw it as a product that could disrupt the industry. He believed that through Siri’s intelligent voice interaction, it could make people’s lives more convenient and bring people closer to technology.

As we all know, Jobs’ vision has not been realized for many years.

Now, Jobs’ dream sees hope for realization because Apple has reached a deep cooperation intention with OpenAI. The recently released GPT-4o model has been fully integrated into Siri.

You don’t need to create an account to use most of the functions of this model for free. With the support of GPT-4o, Siri has made a qualitative improvement.

First, in appearance, when Siri is in conversation, there will be a halo around the screen edge, which looks very sci-fi.

In response, Siri truly has the feeling of an “intelligent entity.” In the conversation, even with the added continuous conversation feature, Siri can naturally understand the context. Moreover, Siri can “semantic index library” based on user photos, calendars, files, emails, and other information to achieve more intelligent information search and extraction.

For example, if a user needs to fill in a driver’s license number in a document, Siri can call the driver’s license picture in the photo album and automatically fill in the number into the document.

Moreover, Siri can also support cross-application operations of apps. For example, if you want to find a photo of someone, Siri can find it and then optimize the photo according to your instructions, and then put it into the corresponding person’s profile in the designated memo.

As the most benefited application, this update can be said to have given Siri a new life.

In the future, Apple said that next year, it will launch more new features for Siri, including screen perception that allows Siri to understand the content of the display, and allowing Siri to take actions on behalf of users in applications.

Does it have a bit of the feeling of the artificial intelligence butler “Jarvis” in Iron Man?

In conclusion, in FirmKnow’s view, a series of AI functions such as text summarization, image generation, photo search, AI assistant, etc., at this Apple press conference, Android phones have actually had them for a long time, and the innovation of Apple Intelligence is not high.

Moreover, how much progress Apple will make in AI in the future largely depends on OpenAI. With Apple’s tone, if it can handle it itself, it will never embrace others, let alone OpenAI has countless connections with the old rival Microsoft.

But despite this, compared to competitors, Apple’s advantage lies in the combination of hardware and software, and it has absolute control over its own system ecosystem. This can make AI better integrated with the original system apps or functions, giving users a more seamless and natural user experience.

Good system experience has always been Apple’s trump card.

Finally, it should be pointed out that Apple Intelligence is still in the internal development test stage and will not be open to developers before this fall, and currently, Apple Intelligence only supports English. Some functions, more languages, and platform support need to wait until next year, and domestic users still need to wait patiently.

After the press conference, facing the strong combination of Apple + OpenAI, Elon Musk launched an “attack,” saying “It’s patently absurd that Apple isn’t smart enough to make their own AI, yet is somehow capable of ensuring that OpenAI will protect your security & privacy! ”

https://twitter.com/elonmusk/status/1800269249912381773

He even said that he would ban Apple devices in his own company.

Apple CEO Cook also posted a tweet, saying “Introducing Apple Intelligence—our next chapter in AI.”

https://twitter.com/tim_cook/status/1800251049472839872

Apple has finally embraced AI, and FirmKnow is willing to call it “catching up,” but it is far from leading the competitors. On the road of AI mobile phones, everyone is scrambling, and Apple is undoubtedly one of the strongest players.

Cheryl Fu

Cheryl is adept at capturing the most topical business news and has her own unique analysis. This is rare for a young person and she's offered up some really interesting ideas since joining FirmKnow Media.

Related Articles

Back to top button