Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Top Rated Comments

luvbug Avatar
47 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
47 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
47 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
47 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
47 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
47 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)

Popular Stories

iOS 17

iOS 17.2 Will Add These 12 New Features to Your iPhone

Friday December 1, 2023 12:19 pm PST by
iOS 17.2 has been in beta testing for over a month, and it should be released to all users in a few more weeks. The software update includes many new features and changes for iPhones, including the dozen that we have highlighted below. iOS 17.2 is expected to be released to the public in mid-December. To learn about even more features coming in the update, check out our full list. Journal ...
iOS 16 4 Web Push

Apple Confirms Governments Using Push Notifications to Surveil Users

Wednesday December 6, 2023 5:06 am PST by
Unidentified governments are surveilling smartphone users by tracking push notifications that move through Google's and Apple's servers, a US senator warned on Wednesday (via Reuters). In a letter to the Department of Justice, Senator Ron Wyden said foreign officials were demanding the data from the tech giants to track smartphones. The traffic flowing from apps that send push notifications...
airpods pro 2 pink

Apple Releases New AirPods Pro 2 Firmware

Tuesday December 5, 2023 11:28 am PST by
Apple today released new firmware update for both the Lightning and USB-C versions of the AirPods Pro 2. The new firmware is version 6B34, up from the 6B32 firmware introduced in November. Apple does not provide details on what features might be included in the refreshed firmware beyond "bug fixes and other improvements," so it is unclear what's new in the update, but prior software releases ...
Beyond iPhone 13 Better Blue

'All-Screen' iPhone Under-Display Camera Enters Development

Wednesday December 6, 2023 2:03 am PST by
Apple's Korean suppliers have begun developing smartphone under-display cameras (UDC), paving the way for the first iPhone with a true "all-screen" appearance. According to The Elec, LG Innotek has entered the preliminary development of the UDC, which sits under the display and does not result in a visible hole in the panel when the camera is not in use. A UDC differs from a typical front ...
magsafe blue 2

iOS 17.2 Brings Qi2 Support to iPhone 13 and iPhone 14 Models

Tuesday December 5, 2023 11:04 am PST by
The iOS 17.2 update that Apple is set to release to the public in the near future will bring support for the next-generation Qi2 wireless charging standard to the iPhone 13 and iPhone 14 models. Qi2 was mentioned in the release notes for the RC version of the update that came out today. With the addition of support for the new standard, iPhone 13 and iPhone 14 models will work with Qi2...
iphone se 4 modified flag edges

iPhone SE 4 May Reuse Existing iPhone 14 Battery

Wednesday December 6, 2023 1:17 pm PST by
Recently, MacRumors has received details on the battery currently being tested on the upcoming fourth-generation iPhone SE, and the information corroborates previous findings in relation to the device. The iPhone SE 4, known by its device identifier D59, is expected to use the exact same battery found in the base model iPhone 14. Partially assembled prototypes of the next iPhone SE have been ...
airpods pro bulbs

Black Friday Prices Return for AirPods Pro 2 With USB-C, iPad, and More

Tuesday December 5, 2023 7:30 am PST by
Today we're tracking a collection of deals that are matching - or nearly matching - the same all-time low discounts we saw during Black Friday. This includes the AirPods Pro 2 with USB-C, 9th generation iPad, and M1 MacBook Air. Note: MacRumors is an affiliate partner with some of these vendors. When you click a link and make a purchase, we may receive a small payment, which helps us keep the ...
instagram messenger

Instagram and Facebook Messenger Chats to Disconnect This Month

Tuesday December 5, 2023 1:57 am PST by
Meta has revealed plans to end Instagram users' ability to chat with Facebook accounts later this month, rolling back a feature that it introduced over three years ago. In September 2020, Meta (then Facebook) announced it was merging its Facebook Messenger service with Instagram direct messaging, allowing Instagram users to chat with Facebook users and vice versa using the same platform....