From the moment Apple demoed 3D Touch on stage during the iPhone 6s event, some Android users were quick to point out that Android had it first. But that’s not true. In fact, it’s far from being the case. 3D Touch goes well beyond what a long press can do on Android – and long press is available on iOS too, of course – and a detailed report that examines the math and physics of 3D Touch reveals the complexity of Apple’s new hardware and software tricks.
DON’T MISS: iPhone 6s Plus vs. Galaxy Note 5: Real world performance test (it’s not even close)
3D Touch vs. Long Press
3D Touch is a feature of iOS that allows Apple and developers to offer neat shortcuts that bring up certain functions faster. The feature takes advantage of a new hardware setup, composed of a unique display sensor array capable of detecting the pressure of a tap, a dedicated chip managing it and a Taptic engine that provides tactile feedback, primarily meant to tell the user to either continue pressing on the display or stop.
Long taps, meanwhile, exist on iOS as well. You need them to move the icons around on a home screen or to select text. The same long press gesture exists in Android, allowing users to access certain contextual menus by letting a finger linger on the display.
The complex math in the code
3D Touch and long press have almost nothing in common, and a report on Medium called Exploring Apple’s 3D Touch from R. Kevin Nelson shows the advanced math at play that lets the software discern between taps and touches of different intensities, allowing developers to add 3D Touch functionality to their apps.
Furthermore, iOS 9’s 3D Touch code is also supposed to prevent anyone from misusing 3D Touch and potentially damaging the display – that is, if anyone can even press hard enough on the screen to break it. A recent teardown has shown the screen advancements in the iPhone 6s which support 3D Touch display. There are individual sensors that can sense pressure, and the Corning Gorilla Glass on top is specially made to withstand harder presses.
Nelson explored the software side of things, looking at the new properties of UITouch inside iOS 9. He discovered two properties, including force and maximumPossibleForce that let the device distinguish between different touch pressures.
After some complicated math and coding, Nelson determined the minimum and maximum force values, as well as several increments between them that suggest 3D Touch could offer an even more sophisticated interactions in the future, as capacitive sensors would evolve into even more precise force measurements tools. And Apple didn’t just use a simple number scale to describe various degrees of pressure — like 0 for no force, 1 for a Peek and 2 for a Pop — there are more than just three force tiers.
By mathematically quantifying force using advanced algorithms, Apple allows developers to set pressure sensitivity for various 3D Touch actions, while also seemingly protecting the hardware and future-proofing the features. Most interestingly, the math suggests that if Apple wanted to, it could add even more pressure layers to generate different responses from an app. Nelson speculates that as force sensors get more precise in the future, Apple could create additional more refined, 3D Touch features.
“[You] can probably get away with segmenting your pressure sensitivity into 6 compartments, but more precision than that becomes a burden more than it helps,” Nelson said. “And in fact, if you look at Apple’s own implementations that use force, they’re only really creating three segments: normal touch, and Peek pressure, and Pop pressure.”
But the way iOS 9 currently quantifies force gives Apple the necessary leeway to upgrade 3D Touch in the future without messing up the overall experience of a third-party app. After all, that app would still have to run on older devices in the future when the developer adds extra 3D Touch tricks for new hardware.
Currently, depending on where you are in iOS 9, there are two or three levels of pressures a device recognizes. App icons only support two taps, a regular tap that opens the app and a pressure touch (Quick Action) that brings up a list of shortcuts – we’re not counting the iOS long press here, which “jiggles” all icons so you can easily delete the apps you’re tired of or move apps around.
Inside apps, however, there are three pressure thresholds that iOS 9 can discern: a regular tap, a Peek (slightly more force) and a Pop (even more force). Depending on how you press the screen, 3D Touch delivers an appropriate response. Conversely, you don’t need to go full force on the screen to move from tap to Pop, as both offer the same functionality – opening a message, mail, link, map or photo in full-screen mode. But the Pop is one of the logical conclusions of a Peek preview, so there’s no point in removing the finger from the screen, only to tap again if you plan to view the element in full-screen mode. It’s faster to simply Pop the Peek.
3D Touch and the future
Basically, iOS 9’s 3D Touch software might lay the groundwork for the future of the entire iOS platform, when better hardware enables even better 3D Touch uses – one of them potentially being the removal of the physical buttons and volume rocker keys. Some limitations, however, might prevent Apple from adding more complications to 3D Touch, as users might not be able to perform more than a few force touches on a display.
So, while Nelson didn’t inspect Android’s long press quick as thoroughly, his conclusion should further cement the idea that 3D Touch is far above what an Android long press can offer. Not to mention that unlike 3D Touch, long press doesn’t seem to have what it takes to evolve in future Android builds and offer users even more features.
Let’s not forget that long press – as the name suggests – implies generating on-screen actions based on how long you press a UI element. The only way it can get better is if developers trigger additional actions the longer you keep your finger on the display. But nobody wants to press for 2 seconds to preview an email and 5 seconds to open one.
Meanwhile, Apple’s use of physical force in 3D Touch lets it offer near instant contextual responses by taking into account the variations in force rather than tap length. And that’s even if the user changes the sensitivity of the sensor.
Finally, Craig Federighi did tell Bloomberg not too long ago that 3D Touch is hard to implement right now, without actually going into the algorithms at play.
“It starts with the idea that, on a device this thin, you want to detect force. I mean, you think you want to detect force, but really what you’re trying to do is sense intent,” Federighi said. “You’re trying to read minds. You have a user who might be using his thumb, his finger, might be emotional at the moment, might be walking, might be laying on the couch. These things don’t affect intent, but they do affect what a sensor [inside the phone] sees. So there are a huge number of technical hurdles.”
He continued, “We have to do sensor fusion with accelerometers to cancel out gravity—but when you turn [the device] a different way, we have to subtract out gravity. … Your thumb can read differently to the touch sensor than your finger would. That difference is important to understanding how to interpret the force. And so we’re fusing both what the force sensor is giving us with what the touch sensor is giving us about the nature of your interaction. So down at even just the lowest level of hardware and algorithms—I mean, this is just one basic thing. And if you don’t get it right, none of it works,” the exec added.
But somehow Apple did it. And since Force Touch is already available on the Apple Watch and select Macs, it will probably morph into something even more exciting in the future.
Check out Nelson’s full dive into 3D Touch geekery at this link to see more details about the math and the code necessary to turn a simple physical gesture into magic on the screen.
