The Tech Specs Podcast: Recent Episodes
Tech Specs
There was one question that vexed me above all in 2021. Ghosts n’ Goblins Resurrection was the most fun game I’ve played in many years. I had an absolute blast with this throwback design “resurrecting” the classic series, and it was easily my pick for game of the year last year. So why didn’t everyone else think that?*
You’d never know it was a modern classic from the reviews on Metacritic. For those unfamiliar with video game reviews, an average score of 7.4 essentially means the game is considered borderline bad. Not since Prey (2017) have I seen a great game so critically robbed.** How did this not at least get “Best Action Game” nominations from everyone??
Arthur’s creaky joints are not the problemI will not claim that Resurrection has mass-market mainstream appeal in terms of its visuals or game design. However, that wouldn’t prevent game reviewers from scoring it highly if they thought it was a great game for enthusiasts. Let’s first investigate instead the most obvious potential source of criticism for Resurrection—its controls. Did the game bomb critically and commercially because it was believed to have frustrating controls?
To simplify, the game was commonly panned for having stiff controls.*** Some of the common reasons cited were the fixed jumping trajectories, inability to attack diagonally, and lack of a double jump. Let’s review why I can’t take any of those criticisms seriously.
When you jump left or right in Resurrection, the protagonist Arthur is carried forward in a pre-determined arc trajectory with only the tiniest amount of mid-air control left to the player. His movement speed is also fairly slow, which makes recovering from a mistimed jump and generally evading enemies a little harder.
The old-school jumping is of course intended to be more realistic than in most games: if you jump in real life, you can contort your body in midair but not really change your inherent trajectory after leaping. This also provides the benefit of allowing the player to attack while jumping backwards, as a dual defensive-offensive option.
Regardless, the main intent is to bake-in player commitment into this fundamental action: the player has to learn to be intentional and always look before they leap.
It’s true that most players generally don’t like controls that limit their freedom of movement. They would much prefer having some amount of ability to recover after inputting a control and undo a mistake.
It’s also a big part of why tripping in Super Smash Bros. Brawl is so hated. It’s why all the ailments that can befall you in Far Cry 2 are infamous. Most players hate dealing with sporadic malaria symptoms, having their gun jam, or needing to repair the engine of their car. These kinds of randomness feel unfair, because the average player hates their agency over input being taken away. It also means that players generally hate their movements feeling realistic.
Regardless of one’s preference for “realistic” or “video-gamey” jumping, can any of these developer choices be inherently bad? Absolutely not—everything is completely arbitrary in the realm of artistic or design opinions. It’s the same story for the inability to attack diagonally and lack of a double jump. These are stylistic choices that are no different than for, say, a film with cartoony animations vs. one with grounded physics.
(Even input responsiveness is subjectively debatable. A common criticism leveled against The Last Guardian was that Trico was unresponsive, even though the entire point of the game was to establish him as a separate creature and being with its own will.)
What reviewers get wrong about difficultyBut why ignore that games are not movies, since there is an input-response loop between the player and the content? The vast majority of games do still require a minimum level of responsiveness. Some reviews even claimed that Resurrection’s controls simply are not responsive. That could genuinely be a real problem, but in reality the game’s controls are very responsive, simple, and obvious, and Arthur responds predictably. Outputs always reflect the intended inputs.
I would even say the controls are perfect for the game. They have no objective flaws as far as I can see; they are extremely precise, and input latency is very reasonable.
Sometimes games are unjustifiably criticized for having challenging controls, when the controls actually perform flawlessly, and the gameplay is simply meant to be challenging by requiring fairly exacting control input to make progress. Wave Race: Blue Storm and Bionic Commando: Rearmed are strong examples that come to mind. I struggle to understand why some games get panned for requiring demanding control inputs that have a learning curve, when so many widely-praised games also have very tight input timing requirements (Sekiro, the Nioh series, fighting games, etc.).
Highly-reviewed games like Super Meat Boy and Celeste are similar to Resurrection in that they also require incredibly demanding, often extended sequences of inputs to progress. They have very fast death-respawn cycles and necessitate huge numbers of trial-and-error attempts. Death is constant, and there is little in the way of forgiveness.
Doing difficulty rightIf Resurrection is too hard for most people, surely it does matter if deliberate control constraints are partly at fault, right? Here again I think the critics missed the mark entirely.
It’s true that the old-school controls deliberately limit options, because the player can’t change their mind and audible mid-action based on instantaneous input. But despite Resurrection being very demanding of the player in terms of control input, the game actually isn’t necessarily even that hard! If anything, it deserves praise for handling accessibility right: the developers took a very challenging game and made it as accessible as possible.
To its great credit, Resurrection actually has some of the best difficulty scaling in any game ever. It is quite achievable for most people to beat the game on the easiest difficulty settings. Not only is the game almost never unfair, it has very frequent respawn checkpoints, and learning the enemy patterns and environmental progression takes only a few tries at most. Level sections never feel like they’re dragging out for too long. You learn and progress past small segments of a level at a time, in other words. There are even mandatory hints upon death!****
I say this not as someone who pointlessly insists you must “git gud” at hard games, but half the NES library, F-Zero GX’s Story mode, Ninja Gaiden II (2008), or any of the Souls games are all much harder. Resurrection is not even as hard as Mega Man 11, and I never saw anyone critiquing that game as being too demanding.
What’s actually going onNone of the above is really the root criticism of the game. And while some of the margins for error for the platforming are indeed a little tight, they are also a little forgiving.
The fundamental distinguishing characteristic of the Ghosts n’ Goblins series is its random enemy behaviors, and thus the constant manic energy required to survive their attacks. As a result you are having to make decisions and react on the fly at all times—all the while anticipating several moves in advance—and this is the defining reason why the series is great. As Matthewmatosis succinctly summarized, “in Ghosts 'n Goblins, you are constantly solving a problem.”*
In Resurrection you have to look everywhere and plan ahead in your mind. The player is asked to keep up with the manic pace while also always being deliberate and careful with their every move.
The reason why I am so upset about this game not earning critical praise is because this gameplay is genuinely fantastic. Many of the enemy patterns are even outright brilliant.
So why didn’t this game win over critics and see glowing praise from all comers? Firstly, it’s important to emphasize that the Ghosts n’ Goblins series are 2D games.
In a 3D game, you can evade in three dimensions. You can sidestep, dodge, or back away from enemies. You can generally look on from a distance and plan ahead before crossing any dangerous thresholds. I’m hard-pressed to think of a single 3D game where you are constantly under attack from all sides.
With Ghosts 'n Goblins games, that does not apply. There are essentially no moments to stop and plan or rest. The games are literally survival gauntlet platformers. Enemies are constantly moving towards you, because that’s the entire experiential point of the series—“this is a horror rollercoaster, try to even survive!” Once you get on the ride, it does not stop or relent. Even platformers that have frenetically-paced challenge spots generally have multiple places to pause or rest along the way.
Enduring the gauntletIn essence, it’s this experience of deliberate but rapid decision-making, combined with constant danger that makes or breaks the series for a player. Surviving a challenge in any game is a reward, but each time you survive a threat in Resurrection, a new one is almost immediately upon you again. There is no time to actually savor the reward of besting any individual threat; you are effectively only allowed to enjoy a sense of accomplishment upon reaching a checkpoint or completing an entire level. But the satisfaction is mighty!
This is what reviewers did not understand about Ghosts 'n Goblins Resurrection. They weren’t really complaining about difficulty, or randomness, or fairness. They were unhappy with it because they eventually grew mentally tired of the unrelenting chaos.
This is why the common criticisms leveled at Resurrection do not make sense. A revered series like the original NES Castlevania games had all the same “problems” and then some—not only did you have fixed jumping trajectories and “stiff,” sluggish controls, you even had to deal with knockback damage,** locked movement on staircases, and more.
Random attack patterns, bottomless pits… none of these elements have ever been uncommon in cherished classic games. Yet they’re criticized in modern reviews for games like Resurrection. These are all nonsense reasons without justified specific context.
The real reason why Resurrection didn’t speak to a lot of people wasn’t objective, but subjective. A lot of people don’t enjoy a game that provides nearly unrelenting stress. Almost all games have push and pull mechanics, with frequent breaks to relax and enjoy rewards. But for many, Resurrection’s chaotic pace is simply mentally fatiguing. They endure for so long until they are tired of the game, and give up on liking it. If you’re not sure what I mean, or why this is really so critical, watch this short video from Masahiro Sakurai clearly explaining this aspect of game design.
Game criticism needs to evolveIs a work of art being mentally exhausting bad? No, I don’t think so. It’s not radically dissimilar to games like Dark Souls that have been celebrated for instilling psychological anguish in players in order to provide a deeper sense of reward.
Resurrection is just not everyone’s cup of tea. But neither is City of God.
It is very, very hard for me to think of any significant fair criticisms for Ghosts n’ Goblins Resurrection. That’s not to say it deserved perfect review scores, but I do think reviewers in general need to reassess their views within the lens of art theory.
A review or critique of an artwork has to be fair to the work. For example, criticizing Resurrection or any other game for not having a double jump is almost always pointless—it’s simply an intentional or artistic choice. It’s not that you have to agree with the intention, or believe the execution achieved the intention, it’s that you have to at least try to understand the intention.
Critics of Resurrection weren’t critiquing the philosophy or the execution of the game design in actuality. They were mischaracterizing their points, and instead revealing their own personal preference for different kinds of games than the one they were reviewing.
Of course, no reviewer is expected to enjoy every genre or style of artwork. If a critic simply prefers more relaxed styles of games, that at least would be subjectively fair. But I didn’t see any reviewers stating that subjective preference, only deriding the game for perceived bad design. And that’s truly a shame, as I do think Resurrection was critically robbed.
How you can helpThe Ghosts 'n Goblins series are games designed for action-platformer fans that reward players with deep satisfaction from beating its stages. They provides much of the same profound sense of accomplishment as the newer Souls series which proved to be the most influential games of the last decade.
Capcom is really on a roll these days. If you can afford to, don’t wait for a sale. Support this awesome game, and just buy the darn thing!*
Resurrection takes the series’ extremely manic yet methodical gameplay to new heights. Players feel like they’re on a non-stop conveyor belt ride towards inevitable death. And the constant cheating of death is exactly why the game is so awesome. It’s a hell of a rewarding ride to conquer.
- It Takes Two and Pscyhonauts 2 were awesome as well in 2021. By all means buy those, too!
Here are some other criminally underappreciated games to check out: Eternal Darkness, Alan Wake, and Zack & Wiki: Quest for Barbaros' Treasure*.
I also suspect many reviewers and players used an analog stick, which I would not recommend. Use a good D-pad with this game.
* At least on some difficulty levels.
Please absolutely see also, Matt’s awesome written commentary on the game: https://www.patreon.com/posts/project-diary-55603111.
Seriously, Matt’s article is far better than mine. Go read that one after you’re done with this one.
* Knockback damage in Castlevania* wasn’t just a design mistake. It was there for good reason, even if game designers should be more careful with the surrounding details these days.
* It’s $30 at full price. You’ll probably spend that kind of money on drugs and prostitutes anyway, Timmy. Just buy it.
What is minimalism? Simplicity in form, with nothing to excess, right?
Let’s assert however that there can be two kinds of minimalism: structural, and affective.
Let’s also assert that this distinction reflects two primary concerns of design philosophy. The following quadrant of design choices represents the resulting output space.
The experiential axis signifies the canonical conflict between function and form: do you design for a purpose, or to direct the focus of the user? Do you make something useful, or something auteurially expressive, perhaps beautiful? Something self-expressive, or something evocative?
The subjective* axis signifies the other canonical conflict: do you design something complex but serving specific needs, or something simple and easy to use? Do you design for an individual user, or for mass accessibility? For someone, or for everyone?
Failing to distinguish the separate concerns represented by these two axes is what often clouds design decisions and our reactions to them. This dissonance can be seen in arguments everywhere, again and again and again. As with politics, the opposing sides argue past each other, each side eternally failing to understand the other’s values framework.
An exemplar of misunderstandingConsider the following example. When tech enthusiasts refer to Jony Ive’s minimalist design ethos as a token philosophy, they have often accused him of self-contradiction on occasions when they believe he compromised function on behalf of form. “Design is how it works,” after all. A maximally symmetrical keyboard innately compromises its numerous functions, despite Ive’s suggestions about minimalist form representing functionalism distilled.
But Ive is only truly minimalist regarding structural design considerations. He has always aimed for minimal structural complexity, not minimal affective form. Expanding on his preference for “simplicity,” Ive describes it as the end result of a process “to deeply understand the essence of a product to be able to get rid of the parts that are not essential.” From here I will refer to this sense of “simplicity” as minimalism, as it is more generally described.
The problem is that the vernacular use of “minimalism” happens to conflate experiential and subjective design. Ive’s experiential philosophy is really a centrist position, and not a minimalist one at all. In other words, generally people use the word “form” to mean both structural and affective form, without distinguishing the two meanings. Thus ensues all manner of confusion and false accusations of hypocrisy. Ive is not a design hypocrite, but he could use more distinct terminology to precisely express his philosophy.
When Ive refers to refining a product to its essence and thus effectively marrying form and function, he believes a truly minimalist design is the ideal middle ground between the two. But these concerns are only experiential; structural minimalism is unrelated to both! Ive is in no way opposed to functional design, even if many of his individual decisions and preferences were not to the end of pure function.
Essentially, Ive’s “simplicity” is represented by the vertical axis in the earlier quadrant. This design philosophy* is largely in opposition to individual expression and customization, to conceding design decisions to the user. It’s strongly focused on mass accessibility generally and, orthogonally, opposed to both superfluous and inelegant design. Ive’s products are for the every-person, not the individual. Setting aside manufacturing economics, EarPods come in exactly one size and shape. Idealized, this overall system would thus be represented in the top-middle of the quadrant.
You may wonder if there really are fundamental tradeoffs between these concerns. But even if you seek two different goals and believe you can fully satisfy both without conflict, the order of constraint prioritization still matters. Do you fix a chair to an ideal position, at the cost of making sitting on it more difficult? Or do you optimize ease of sitting on the chair, at the cost of making positioning it more difficult?
Other casesI use Ive’s taste only as a widely known example in the tech industry. Considerations like the number and kind of ports that are included with a computer meanwhile affect both experiential and subjective concerns. Reducing such decisions to simply a tradeoff between complexity and minimalism under-describes the design space! (Thinking through illustrative examples is left as an exercise to the reader, though I’ll suggest user interface animations as another excellent one to reason about.)
Similar confusion arises when discussing open source software. What does “open” mean? What does “free” mean? Hint: “free beer” is maximally accessible, while “free speech” is maximally self-expressive. This is the exact same distinction as positive vs. negative liberty in political philosophy.
Without expounding for thousands of more words, these values represent the extreme ends of the same pair of philosophical axes as described earlier, and their conflation results in all the same problems of misunderstanding value frameworks and false charges of hypocrisy.
Relation to art form, or art theory found usefulIn every artistic discipline there eventually emerges a school of thought that an artist should strive for harmony between its expressive form and the essence of its nature, that the medium should be the message. Artistic purity or honesty of form is sometimes sought by those who believe there’s inherent tension between the possible functions and forms of a specific medium.
Braid, as one example, is a game in which the literal mechanics of its puzzles convey the directed meaning of the game designer.
The goal of such efforts is to find the experiential middle ground between formalism and expressionism, between pure and program approaches. Charging a design with being “overly simplified” cannot be a experiential criticism.
Success at conveying meaning solely by the inherent elements of an art form is often obscured in the minds of the audience. That the central question these efforts seek to address is itself so unclear and variously interpreted reflects how widely confused experiential and subjective considerations really are.
But works can have objective goals, and designers can have clarity instead of confusion. In considering a design, as with all artistic works, you would always do well to distinguish these concerns.
- A design is “subjective” to the degree answered by the question, “What is asked of the user?”
Ive only evolved towards his strict sense of “simplicity” over time. The original iMac was certainly not purely functional in its design.
* Here’s a quote that neatly summarizes the entirety of this philosophy as in reference to Apple’s products: “‘The defining qualities are about use: ease and simplicity. Caring beyond the functional imperative, we also acknowledge that products have a significance way beyond traditional views of function.’”
I really enjoyed this nice, breezy interview with Tyler Pruitt of Portrait Displays on display calibration on the AVForums Podcast.
I especially love that he immediately complained about HDR10 and its lack of a reference tonemapping standard to start the interview. A man after my own heart.
Permalink
There has been a rather interesting shift over the past few years that has gone largely unnoticed by tech and gaming enthusiasts: I don’t believe that PC gaming represents a clearly superior technical proposition compared to console gaming anymore.
Three big changes took place to alter the technical gaming landscape: HDR, W-OLED, and the Xbox One X. To make my case, I will first elaborate about each of these.
HDR is deeply underappreciatedHigh dynamic range display essentially consists of two components: greater contrast, and new math. ST 2084 is the SMPTE’s standardized form of PQ, Dolby’s perceptual quantizer, an electro-optical transfer function (EOTF) that maps video levels to light output and replaces traditional gamma. If you don’t know what any of these words mean, but would like to learn more, see Dolby’s white paper. In short, this function encodes color data far more efficiently than gamma ever could across a wider luminance range and takes into account real-world modeling of human perception, allowing a larger spectrum of colors to be perceived from an otherwise identical source image.
To reduce our focus to the most significant problem that HDR addresses: content such as movies and video games has been effectively constrained to 100 nits of maximum luminance for several decades. Considering that simply walking outside will showcase a distribution of luminance values well into the thousands of nits for reflected surfaces that are perfectly safe to observe, you can begin to appreciate how severely this historical constraint has restricted all of our media and computing interfaces. To quantify the benefits of high dynamic range in combination with support for wider color gamuts on modern displays, color volume is one measure for expressing the expanded range of colors able to be realized.
In other words, HDR provides much more than a leap in contrast alone. Since HD and the “Retina” era of high-DPI displays, HDR offers the single biggest individual improvement to image quality. And it does not apply to video alone, but to any digital display output, including photos and application UI.*
PC monitors have fallen behind for goodThere is however a tragic downside to the HDR era: it’s currently impossible on the PC, and will be for years. This comes down to two factors: power and OLED economics.
Over the past half-decade, OLED TVs quickly superseded plasma TVs in almost all technical regards, thanks to key (heavily patented) technologies that LG Display was able to bring to market with its W-OLED IGZO panels. Briefly, W-OLED displays utilize an RGBW pixel structure, with white subpixels that allow for high light emission in conjunction with filters to convert the colors for final output.
W-OLED technology provides much greater luminance and efficiency, decreases the risk of burn-in, and altogether greatly improves panel yields for production. The alignment precision required for patterning is far easier to achieve than for standard OLED fabrication, enabling the production economics to be viable for even very large panel sizes.
While there was some early hope of bringing good OLED (and even IGZO OLED) panels to laptops, those attempts effectively ended in failure. The economics simply do not work. Worse, it is brutally hard to increase max luminance for monitors compared to for TVs, as TVs are able to draw far greater amounts of power.
That sadly hasn’t stopped a recent sham push by monitor vendors and VESA to peddle “HDR” PC displays to consumers, but I assure you every single last one of these products is effectively a fake, and the overall marketing effort is stupid nonsense.
Meanwhile, the console space has seen widespread HDR adoption by developers, often to magnificent effect. Despite most games doing HDR poorly in various ways, it’s generally a massive improvement to image quality, one that happens to be effectively free from a performance perspective. It’s glorious.
Microsoft has changed the gameWith the Xbox One X, Microsoft has pushed like hell to re-establish its technical dominance over Sony, and it has hugely succeeded in doing so. Despite having a CPU microarchitecture that would best be described as horrendously uncompetitive, the GTX 1070-level of graphics, abundant memory bandwidth, and fixed hardware target with low-level developer access of the One X have combined to set a new benchmark in extracting pure pixel performance from a console. Locked 4K30 and even 4K60 on the One X are far less uncommon than you might think.
I’ve watched hundreds of hours of Digital Foundry and other performance analysis videos, and the One X games to date almost always perform outstandingly well. For the cases with consistent performance drops, all the games nonetheless support variable refresh and are future-proof for running with greatly improved performance on the next Xbox. You basically can’t lose.
This is because Microsoft’s vGPU mastery has brought one of the key advantages of the Windows PC platform, incredible backwards compatibility support in software, to the console space for the first time. And its amazing efforts haven’t stopped there. (If you’re wondering why any company would go to such lengths to resurrect old, existing games, it’s because game preservation is an enormous problem.)
Despite using a fairly terrible SoC, the Switch to an extent has also demonstrated the ever-shrinking gap between the performance of cost-optimized chipsets and silicon designed for powerful PCs. Mark my words, with its ARM efficiency advantage, Nintendo is well-poised to embarrass Microsoft and Sony on hardware technical capabilities over the coming years. Though the less constrained GPU die sizes, memory bandwidth, and especially bill of materials of the latter two companies’ consoles will continue to ensure advantages over Nintendo’s specs, Nintendo will trend ever closer to the x86 platforms in raw performance.
The PC’s remaining performance advantagesSome factors influencing graphics fidelity remain unchanged. Brute-force high quality anti-aliasing is still a major advantage in favor of the PC. 120fps console gaming is currently limited to a literal handful of Xbox One X games that require variable refresh to eliminate stutter. And a precious minority of PC games such as Battlefield V really do still take advantage of PC hardware superiority these days.
You will also always be able to drop huge sums of money on the best-binned, heavily overclocked CPU, with a completely overkill dual-GPU setup and liquid cooling to keep a sky-high power budget in check. This will only get more and more expensive and wasteful, however, with the recent death of Moore’s Law. (It’s not really the lack of competition that has NVIDIA’s GPU pricing soaring through the roof.)
Finally, I won’t neglect to acknowledge that the PC has recently garnered one hell of a trick in its favor: real-time ray tracing. The current, very early, best case for it is demonstrated by Battlefield V, which can just run at 1440p60 on an ~$800 GPU. Even though it’s early days, the ray tracing revolution has indeed already begun.
Weighing the tradeoffs for the gamerBut I don’t believe higher average frame rates and Ultra settings that are pretty poor on performance efficiency are worth quite as much as commonly held. I do however think the advantages of generally better overall image quality thanks to HDR and OLED panels, more consistent frame-times (through micro-optimization), and little required frustration toiling with graphics and motherboard settings heavily weigh the accessibility of high-end graphics fidelity towards console gaming for the overwhelming majority of consumers.
If you truly value stable performance, it’s almost sufficient to note that optimizing settings to get a modern AAA PC game to run with extremely consistent frame delivery is generally an enormous pain requiring a great deal of research, and even then Windows will try to ensure you misery. Heaven forbid you use a workstation CPU with NUMA.
(There’s also more that can be said about the immaturity of Direct3D 12 and Vulkan drivers, and how wildly behind Intel and AMD are competitively on efficiency and hardware platform features compared to the ARM players, but these are secondary matters.)
For any PC gamers who think my overall argument is nonsense, genuinely, show me your frame-time data on a non-monster setup. I’m quite confident that 99+% of PC gamers are not playing AAA games at nearly locked frame rate multiples of 60, but I’m always open to more comprehensive data analysis.
(PC gaming pro-tip: always set a frame rate cap of exactly 120, 60, or 30fps, or do whatever trick you need to to achieve the equivalent with VSync on. And if you’re using a variable refresh display, still set a frame rate cap that ensures nearly 100% frame-time consistency for a given game. See here for further nuance about maximum VRR refresh rates with external frame rate control.)
Lastly, if you haven’t seen it yet and are interested in this topic, take a look at my console gaming optimization reference guide. I put quite a lot of free time into it.
- How well even tech enthusiasts can distinguish HDR from non-HDR content on mobile devices when not displayed side-by-side is another lengthy topic.
Douglas Lanman of Oculus Research gave an awesome talk about reactive displays at the Society for Information Display’s Display Week 2018. Watch it if you want to learn why the future of displays is VR and wearable technology.
Permalink
This is awesome to see. The state of deep learning benchmarking is still dreadful, and I think most observers don't know even the most basic details about how it should be done properly. For more, see Wave Computing's press release.
Unfortunately there are some big names not yet listed among the supporters. That includes NVIDIA, unsurprisingly.
UpdateNVIDIA (and many, many other companies) joined later, thankfully.
Permalink
Cool.
There's also support for Debian. This will obviously be rather useful for developers.
I hope a mobile Gaussian blur implementation would somehow be light on GPU utilization.
Permalink
I've been waiting for someone of sufficient stature to publicly convey this. If you’re not sure what all this means, look at the graph.
While Intel’s 10nm was the canary in the coal mine, it has taken a couple years for the industry to fully grasp the sheer wall it has hit, and how the other foundries would hit it just the same. Cannon Lake’s extreme delay and Apple’s middling A10X and A11 single-threaded performance improvements (despite what it did with the latter's core) were leading indicators.
We're still getting shrinks, but they aren't timely enough to double transistor count every two years anymore.
While there are other areas that can be advanced, we really need materials breakthroughs to be able to push per-core performance again. Until then, we’re mostly stuck.
Permalink
All hardware is degrees of broken. I've unfortunately found, however, that many vendors are happy to advertise their silicon as fully functional despite shipping broken implementations or disabling IP or features outright.
And in the case of the Meltdown and Spectre vulnerabilities, most modern CPUs were deliberately designed with what ultimately proved to be a poor balance between security and performance in regards to their speculative execution implementations.
Permalink
If you want to learn about the state of mobile chipsets, AnandTech’s power and performance overview of HiSilicon’s Kirin 970 is a good place to start. It’s not intended to be a comprehensive overview of the SoC, but it’s also easier to read than such a piece.
While you may not care about Huawei or Chinese silicon vendors — you should! — it’s important to follow HiSilicon’s SoC implementations and results, as they among others serve as a barometer of the state of the ARM ecosystem and all mobile SoCs. The company's best effort to date was Kirin 950, which was a very well-implemented chipset.
Real analyses also can and should convey the things that really matter when it comes to SoCs: interconnect implementation, memory access latency, whether the power management works at all, etc. These are some of the things that most often go wrong, or are the most challenging to implement well. The average mobile observer probably only thinks about CPUs and GPUs, but they’re not even remotely the only things that matter.
Andrei is the only person writing publicly who knows how to measure power properly at the rails (or even publish fuel gauge figures), so you can trust these power figures, unlike everything else you may find on the internet. Additionally, his figures provide a good overview and recap of the state of mobile chipsets in recent years.
This is also the best public data we have on 10nm at the moment, and the results echo what I could gather about the node early on (see: Updates). I particularly appreciate his compiling SPEC2K6 for readers' benefit, since that’s a genuine pain to do.
“An increase in main memory latency from just 80ns to 115ns (random access within access window) can have dramatic effects on many of the more memory access sensitive tests in SPEC CPU. Meanwhile the same handicap essentially has no effect on the GeekBench 4 single-threaded scores and only marginal effect on some subtests of the multi-threaded scores.”
This section as well as the other commentary on benchmarks should sound very familiar to subscribers. SPEC is not exactly the best benchmark in the world in terms of real-world representativeness (read: understatement), but it’s the best we’re going to get publicly.
I should note that interpreting benchmark results beyond the basics is tough. You need to really know what’s going on on a low level. Deep learning benchmarking is also complicated, and I’m not a machine learning researcher so I’ll refrain from commenting about that.
The upcoming SoC to watch right now is Exynos 9810. Samsung’s System LSI division really needs to deliver following recent disappointments that failed to live up to the solid Exynos 7420.
Lastly, if you’re hoping for greatness from 7nm, I would argue that it would probably be better to start accepting that Moore’s Law is dead. I’m not the person to write about that, though.
Permalink
I've mentioned this before for subscribers, but I may as well say it here: it seems obvious to me that we'll see Fuchsia/Zircon devices this year.
Think months, not years.
I heard Intel say many promising things about 22FFL at TechCon 2017, but it has quite a lot to prove when it comes to SoC processes and winning clients for Custom Foundry.
In-depth public analysis of foundry technology is rare, so I'm grateful that David has written about this important topic.
Permalink
If you had to combine Android and Fuchsia, how would you do it?
This article is available for subscribers on Patreon.
Permalink
Welcome back, Andrei.
Now you know why I am no longer worried about quality mobile technical coverage.
Permalink
These standards are abysmal.
Maximum luminance below 1,000 nits for LCDs shouldn't be considered real HDR.
Permalink
I have decided to end subscriptions for Tech Specs. Existing subscribers for this month will be refunded at the end of the month, and all subscriber articles will still be accessible until the end of the month.
Tech Specs will not necessarily end as a blog, but as of now I don’t know what its future will hold. I want to reenter the tech industry for full-time work, and understandably employers generally don’t allow employees to blog. If I can continue to write in some capacity, I will, but I hope you will understand that it is unlikely to be at the same level of depth or weekly commitment.
I deeply, deeply appreciate the support of all my subscribers to date. If not for your support, I would have stopped blogging many months ago, and I honestly kept going as long as I could justify. The comments and questions on subscriber articles were especially great, so my thanks for all of those. Please do feel free to continue sending me any questions or comments via email or Twitter. I always try to respond eventually.
I know $10 a month was not cheap, and I debated launching new pricing options for many months. Ultimately, unless offering cheaper options would have increased the number of subscribers by an enormous multiple, it would have done very little to move the needle. There was no realistic path towards being able to justify the continued time and especially financial expense. I do feel like I could have eventually made the numbers work, at great effort, though it would have taken many years. That kind of time is unfortunately something I do not have.
Public writing was not something I had anticipated doing much of, as it was the benefit of circumstance. While my writing is not very good, I aimed for an intermediate-level of technical depth that was hopefully not too difficult to follow. It’s hard to say if that was the right call. My one regret is that I didn’t make time to write any introductory articles, or even a technical article or two.
Above all, I’m sorry to disappoint everyone.
I will, however, write at least one more article for subscribers on one particular topic. I’ve put it off for a long time because it requires much more research than anything I’ve written about to date. You can probably guess the topic.
My thanks to everyone for reading, and hopefully this is not the end.
Everyone loves GIFs, but they're technically awful. This replacement implementation is awesome. The WebKit/Safari team's work has been absolutely amazing in recent years, especially in regards to energy efficiency improvements.
Also:
Aside from not having a formal standard, animated WebP lacks chroma subsampling and wide-gamut support.
Permalink
Here's the overview of the specification. There's a lot to look into and study.
Quick Frame Transport and Quick Media Switching are of particular interest to me.
Permalink
These are some highlights from Greg Yeric's keynote presentation at ARM TechCon 2017. Greg leads the Future Silicon Technology group within ARM Research and is as qualified as anyone to speak about the industry's current challenges.
It's a bit of a scary time for the silicon industry. Progress is slowing down in many areas, and it's unclear to everyone what technologies and processes will successfully drive advances in the future.
Permalink
With the increasing popularity of UHD TVs in the market and the Xbox One X’s awesome backwards compatibility support, I’ve been working on a small side project over the past few months to make sense of the increasing complexity of console and handheld gaming. Today I’m launching a public spreadsheet which I’m calling The List. Consider it a console gaming “optimization” guide.
The List is linked under a new Reference section of the blog, which I will be expanding over time to include various tech-related reference information. The goal is to list the best version and means of playing almost every (good) console or handheld game ever, within reason. For example, I’m guessing not many people would realize that the best way of playing of some original Xbox games is through an Xbox One X connected to a 1440p FreeSync 2 monitor.
I’m sure this will disappoint some people, but I am not including PC games. This is for a couple reasons. In general, 95+% of multi-platform games will of course run best on the PC (on Windows, specifically), so there isn’t much value to add in that regard. Also, optimizing software settings for PC games and all their potential hardware configurations can be maddeningly complicated if you truly want to play without compromises.
Console gaming is just complicated enough that I think a simple guide has merits, especially with the advent of the PlayStation 4 Pro, Xbox One X, and even Nintendo Switch. This guide was initially inspired by the confusion created by the various output display modes of some games as part of their support for the PlayStation 4 Pro.
Unsurprisingly, I am heavily leaning on the excellent analysis of the folks at Digital Foundry, who are the best at head-to-head comparisons of games. While graphical and audio comparisons are fairly straightforward, comparing ports on features and overall quality is much trickier and more subjective. I will try my best to be thorough and fair.
I’m launching The List with 100 games to start with. I’m not sure if more than a couple people will find this project of value, but it’s worth a shot. It would be awesome if anyone would like to contribute suggestions or help out, but I am by no means expecting anything. And if anyone has any specific games they would like to learn more about, please do message me on Twitter. Thanks.
This is a great article on a very underappreciated topic — how poorly game developers implement HDR and tone mapping.
It makes me very sad that the video game industry uses ACES.
Permalink
Thanks to the fine folks at Android Central for hosting me on last week's episode of their podcast. We talked about the color accuracy of both Pixel 2 displays, factory panel calibration, software mitigations for burn-in, color management, Google's statement on the Pixel 2 XL, and more. Check it out if you're interested in a more in-depth discussion on the topic.
Permalink
There has been an absurd amount of misinformation circulating about the displays of Google's new Pixel phones. I’ve written hurriedly elsewhere about this, but one point I want to stress is that sRGB has little to do with anything. The devices run Oreo and are thus color managed.
To be completely clear, the 2 XL's panel is really bad, but we've also effectively known this was going to be the case for months based on the LG V30. After discussing with some smart folks, I’ll go ahead and speculate as to what’s going on. These are just some guesses, and none of this is confirmed.
You should never trust your eyes, and this needs to be properly tested and measured, but it does look like the display of the 2 XL undershoots the red and blue sRGB primaries and overshoots green. The panel is also clearly way too cold, which makes the off-axis color balance look really bad at large angles. There does appear to be a green push similar to that that could affect the Samsung Galaxy S4. Green shifting is about the worst result you can have for the color rendering of a display, because people associate it with nausea.
In addition to all of that, there are clearly various other traditional OLED issues and defects affecting the panel. Samsung Display OLED used to suffer from many of these problems, but the company managed to solve almost all of them over the past few years.
None of this can be fixed in software. It’s just a bad panel.
The calibration itself is software. The Pixel 2 XL’s panel is definitely not individually calibrated, and possibly not even batch calibrated. LG Display appears not to have the appropriate equipment and workflow for adequate calibration. Google probably could have paid sufficient money to buy this for the production line and made sure it got done, but it’s also possible the fabs are too immature and this was somehow not feasible.
LG Display was not originally going to ship OLED smartphone displays this year, but seemingly rushed to re-enter the market due to high demand from vendors wanting to better position against the upcoming iPhone X, for all the wrong reasons. (I’m hearing that some vendors now want displays with notches, which is spectacularly depressing.)
The Samsung Display OLED on the 5.0” Pixel 2 meanwhile appears to be pretty reasonable at a distance. These displays are probably batch calibrated. I strongly suspect it’s actually the same panel that the first Pixel phone used, except now calibrated to the Display P3 color space as well as could be managed. Google added some software to Android Oreo to allow for this, which is the same sort of thing vendors like Samsung and Qualcomm have been doing for many years for tons of devices.
On a final note, one thing Google has always done wrong is include optional or even default color profiles that are deliberately inaccurate. The "Vivid" setting should not exist if the product managers truly care about accuracy, and I hope Google doesn't add any more in response to all this overblown controversy. Perception of "dull colors" is not the problem to solve. No consumers complain about the accurate color rendering of an iPhone.
This was yet another brilliant talk by John. Topics include: content and app marketing, focusing on the user, the ninety-ninety rule*, software optimization, the end of Moore’s Law, the Oculus Go, OLED displays vs. LCDs, immersive media, and of course quite a lot on VR.
- (As applied to game development.)
Permalink
I previously did a really poor job of explaining the new iPhones’ wireless charging support, so I would like to rectify that. The new iPhones are Qi 1.2.3 devices, but the iPhone 8 currently only supports Qi 1.1.X. This means inductive-only charging, and no fast charging.
The resonant extension of the Qi standard was introduced with 1.2 in 2014, and compatible chargers have been available for years. Qi 1.2 also supports simultaneously charging multiple devices with optional WP-ID unique identifiers for power receivers.
The resonant specification allows for charging at greater distances (implementation-dependent) and does not require precise device-charger alignment. The medium power extension for 1.2 allows for charging above 5W using Extended Power Profile chargers, and the first devices came to market in 2016.
Apple is using Broadcom’s 2014 BCM59350 which only supports charging up to 7.5W. This limits the many resonant Qi chargers in the market that support up to 15W of power delivery. An upcoming iOS update will add support for Qi 1.2.3 and 7.5W (5V/1.5A) charging on an unknown date.
What I should have originally stated in my iPhone X article is that inductive charging is mostly useless, and resonant charging is only a little better at the moment. Far-field resonant charging will eventually provide a reasonable experience in the future. Of all the available options, Apple has chosen to bet on the future of the Qi specification.
UpdateHere's a good explanation of resonant vs. inductive Qi.
This is an excellent visual overview of smartphone camera imaging. It even covers demosaicing and how vendors perform objective robotic testing.
“There’s a saying in engineering: if you haven’t really tested, it’s broken.”
Uh huh.
Permalink
Here is the video of the talk, and here is the associated paper on arXiv.
Geoffrey Hinton, a pioneer of deep learning who works at Google and the University of Toronto, emailed Tishby after watching his Berlin talk. “It’s extremely interesting,” Hinton wrote. “I have to listen to it another 10,000 times to really understand it, but it’s very rare nowadays to hear a talk with a really original idea in it that may be the answer to a really major puzzle.”
Also worth noting: Tishby’s work wasn’t accepted for NIPS 2017.
I have no clue about the likelihood of proposed explanatory theories, but deep learning is one area of computation that I am optimistic about. It is very easy to improve the efficiency of deep learning workloads right now, and it should be straightforward to gradually improve precision in the future.
Permalink
An accurate article on battery charging in the mainstream press? Kudos to Time.
(I try to avoid charging any of my devices overnight.)
Permalink
There are several errors in this article, one of which is a bad one — the CPU is not 70% more efficient.
Nonetheless, I appreciate when Apple's silicon team provides some information. I think the only technical disclosure in this piece is the redesigned secure element (the Secure Enclave, or SEP). I believe we already knew everything else, if I'm not mistaken.
A three year design lead period for the Neural Engine is to be expected. Apple's "fully custom" GPU is still using tile-based deferred rendering (TBDR), which is Imagination Technologies' IP.
Apple’s silicon team is, for example, obsessed with energy efficiency, but never at the expense of responsiveness.
Meh. I appreciate that pushing single-threaded performance is insanely hard, but I've never been sold on this philosophy. You know what would be a lot more efficient? Getting the software teams to truly care about locked-60fps performance again, as they did until iOS 7. Every Apple device still drops tons of frames, no matter how fast the silicon, and ProMotion isn't a magic bullet solution. That's not the silicon teams' fault.
“We’re thinking ahead, I’ll tell you that, and I don’t think we’ll be limited,” and then he added, almost as a post-script, “It’s getting harder.”
That it is indeed.
Permalink
Yesterday Apple revealed the iPhone X, which had been hyped to an extreme degree for years. The highlight feature of the phone in my opinion is clearly its HDR display. Apple is the first mobile vendor to ship end-to-end support for HDR, while Samsung’s Galaxy S8 was the first device to at least feature full hardware support.
I wrote about this at great length over the past couple months, so for all the details check out these three articles. The first article was mostly wrong about the UI elements, but overall they were hopefully pretty accurate.
Much to my surprise, though, the X’s display really appears to use a diamond PenTile subpixel layout. Apple’s own marketing states that it “uses subpixel anti-aliasing to tune individual pixels for smooth, distortion-free edges.” That’s effectively confirmation, and exactly the same as what Samsung Electronics has always done with OLED. This means there’s definitively no near-term hope that S-Stripe can be scaled up economically to large phone panel sizes at high pixel densities. Samsung hasn’t been holding back.
One thing I will add is that while I appreciate Apple’s intention with advertising contrast of 1,000,000:1 as opposed to an infinite ratio, it’s also ok to say it’s infinite for OLED because of the near perfect blacks. Not shipping ProMotion is probably due to power budgeting, though it’s possibly because of performance constraints.
There are pretty much zero surprises on the silicon side overall. I might write more about the A11 another time, but the IPC improvements are relatively modest. I know many people will raise an eyebrow at this, but I would encourage you to completely ignore Geekbench. And I can’t emphasize enough how almost everything written online about Apple’s CPUs is wrong.
If you subtract out the efficiency gains from removing 32-bit support, you’re left with maybe very roughly a 15% improvement in CPU IPC for the big cores, assuming equivalent clocks to the A10. Apple could have pushed performance and efficiency further, if not for 10FF being really bad. The era of the hyper Moore’s Law curve in mobile is officially over, in my opinion, though maybe the A10 already signaled that. It’s all rough sledding from here on out, based on the state of foundry challenges.
The design of the CPU itself is completely unsurprising. Once it was leaked that it was hexacore, it was obvious that the smaller cores would have been designed for a higher performance target than last year’s Zephyr cores. The design is fully cache coherent but also clearly not ARM SMP.
Regarding “Apple’s first custom GPU,” well… I’ll put it this way: there are probably even people at Apple who don’t really consider it to be fully custom. Personally, I don’t consider it to be custom based on what I know, but I can’t say more. Sorry to be vague, but it’s complicated.
In terms of performance, Apple only claimed a 30% performance improvement, which is not massive. 50% power at iso performance on 10nm is also not necessarily impressive if you think the GPU in the A10 burned too much power in the first place. The reality again is that TSMC’s 10FF is really bad, though, so Apple probably couldn’t achieve more. That Apple went with a tri-core design is interesting. New architectural features are the biggest thing that Apple is touting, but I know nothing about graphics and can’t comment on those.
Apple didn’t say anything else about other interesting things they did in terms of silicon, so I will also not talk about them.
Setting aside the (no doubt really expensive) IR system, the single most impressive advancement might be the new camera color filter. This is a really big deal, because it’s insanely hard to improve on the Bayer RGBG color filter array. We haven’t seen any vendors attempt do this in years in mobile, but it was inevitable some vendor would try again to ship an alternative. Whether Apple’s implementation is RGBW ala Aptina or another subpixel arrangement, I have no idea. I know essentially nothing about image filtering and demosaicing, so there’s nothing more I can say. For the cameras themselves, Apple has finally shipped larger image sensors, though we don’t know the pixel size yet.
The A11’s video encoding performance is also extremely impressive, and it’s fair to say Apple is way ahead of the competition. 4K60 encode simply requires a massive amount of data bandwidth. I’m not sure at this point if deep learning is being applied in this area or not, but Apple is definitely employing its own special techniques to accomplish this.
Face ID seems to be significantly slower than TouchID but more secure. I was skeptical about the latter claim, until Apple explained the sensors and the dot projector. That seems like more than enough data, but keep in mind there will be corner case problems, bugs, and oh right sometimes the deep learning classifier will just miss. Reliability and robustness also matter, in other words. And as Apple tried to lightly dismiss, it won’t work for identical twins. Overall, Face ID should be considered a convenience regression over Touch ID as a necessary concession for the thin bezels of the display, since Apple couldn't yet get fingerprint recognition to work through the display. It works, but Apple is surely unhappy internally about the speed regression.
I almost recently published an article on things I believed Apple needed to improve about the iPhone. The five areas I was going to highlight were: speaker quality, portrait mode quality, shipping a dedicated DLA, camera sensor pixel size, and switching several imaging algorithms to deep learning implementations. As a pleasant surprise, Apple addressed all five of these areas (though I had strong hunches it would do so).
Waterproofing hurts speaker quality, and the primary speaker in the iPhone 7 regressed in some regards despite the improvements to volume and dynamic range. Portrait mode on the iPhone 7 Plus can honestly produce some pretty poor results. The people who work on these algorithms have PhDs in computational photography, though, so I probably shouldn’t criticize what I don't know. I don’t think the previous implementation used deep learning, but I could be wrong. For portrait mode on the front camera of the iPhone X at least, Apple appears to have switched to a deep learning implementation, if I understood Phil Schiller correctly.
Dedicated deep learning ASICs like Apple’s Neural Engine are clearly the direction the industry is moving, so it’s hardly unique or honestly that hard for Apple to do so as well. Inference is too important not to specifically accelerate, so this is something Apple and everyone else clearly need. Implementations should be all over the map and vary quite a bit in terms of results. No one knows how these chips should ideally be designed, so everyone will be experimenting for many years. Apple did disclose a tiny amount of detail on the Neural Engine, such as its being a dual-core design, which was nice of them to do.
There are also a new accelerometer and gyroscope, which is unsurprising if you’re familiar with sensor design standards for VR platforms such as Oculus’s Gear VR or Google’s Daydream. Previously Apple has preferred to source lower power sensor implementations, so perhaps these new sensors are more accurate but draw greater power. The cameras themselves are now also individually calibrated, which is probably really important.
Additionally, there is now hardware codec support for FLAC, which was to be expected given software codec support in iOS 11. ALAC is basically irrelevant now.
The biggest mystery to me going into yesterday’s keynote was the iPhone X’s wireless charging. Since it was revealed to use completely standard Qi charging, I will hazard a guess as to what this is really all about. It’s no secret that Apple wants to push for a completely wireless future, and the Lightning connector’s days are clearly numbered. To get to that point will require far-field wireless charging using resonant technologies.
My understanding, and what I don’t think people realize, is that a resonant specification depends on an inductive specification. If this is the reasoning, then Apple has to help propagate an industry standard to make inductive charging as ubiquitous as possible around the world. Thus, Apple would be pushing for wireless charging now even if it doesn’t see a ton of value in inductive alone. This is just my theory, so I could certainly be wrong.
As an aside, Schiller had to directly contradict his own past comments about the utility of wireless charging (and NFC), which I think is an excellent example of why you should generally avoid saying negative things about other companies or people.
I might write more about the Apple Watch Series 3 in the future, but I at least previously explained all of the cellular details here.
Technical corrections are always appreciated.
This presentation from MIT provides an excellent overview of current techniques and hardware implementations for efficient deep learning computation. There is also an associated paper.
The material requires familiarity with the basics of deep learning and its terminology. Provided you are familiar, though, the presentation is very accessible and easy to follow even if you aren't a machine learning researcher.
Permalink
These are basically the same conclusions I’ve drawn over the past couple days and shared with subscribers. Despite the insecurities of fanboys on both sides, ARKit and ARCore seem pretty comparable overall.
It’s refreshing to see technical blogging that’s fair to all sides, and thus actually accurate. The statements made also match up completely with my limited understanding of AR and hardware product development. I recommend reading the entire article to see what I mean.
I did a little digging yesterday, and there doesn’t seem to be much to ARCore’s Nougat API requirement. Android device support probably boils down to Google’s dedication to validation, and not really hardware or fragmentation.
I’ve also been writing on AR and hardware for subscribers, so check that out if you're interested.
Permalink
As was expected to happen at some point, Intel today introduced its new Xeon-W workstation CPUs.
Intel has had to overhaul its product portfolio due to the massive challenges and delays of its 10nm process. Apple actually pre-announced these new Xeons at WWDC while giving minimal detail. It was clear, however, that the CPUs would essentially be Skylake-X.
Xeon CPUs are not “faster” than Core CPUs, because they use the same microarchitecture. Xeon chipsets, however, come with important features for workstations and servers such as ECC RAM.
To oversimplify, Intel advertises TDPs (thermal design power ratings) of up to 140W for a Xeon-W. AMD’s Vega 56 and 64 GPUs are rated at TBPs (thermal board power ratings) of 210W and 295W, respectively*.
For the iMac Pro, Apple has redesigned the airflow of the chassis and finally added a second fan. Philosophically, though, the company has a low tolerance for fan noise.
Will the iMac Pro significantly throttle? We’ll see.
- I’m only referencing the official vendor thermal ratings, and TDP and TBP mean different things.
Ignoring the dubious value of reviewing a device under extreme time constraints, I was impressed by the production quality of this video.
The Phone itself is extremely interesting.
Permalink
From my perspective, AMD is currently the most fascinating company in tech. Its Zen CPU microarchitecture and Ryzen desktop CPUs met or even exceeded expectations, realizing comparable performance and IPC to Broadwell and providing Intel with real competition in the X86 space for the first time in years. I am increasingly convinced by CEO Lisa Su’s efforts to turn around the company from the dire straits it was in until the launch of Zen.
AMD’s new Vega GPU architecture has been especially interesting to follow in recent months. I will caveat this article by saying I’m not very familiar with desktop parts, especially GPUs, so I don’t really know much beyond the basics.
What I don’t think most people know, though, is that GPUs are process-constrained. Vega is fabbed on GlobalFoundries’ 14LPP process, which is licensed from Samsung Foundry. TSMC’s 16FF+ was a little better than 14LPP in terms of power and performance, though it’s at least possible process maturity may have closed some of the gap over time. Quite how so many people expected Vega 10 to outperform NVIDIA’s GP104 GPU escapes me, then, given that the two GPUs are fabbed on very comparable processes. (HBM2 memory should make a difference, though, on paper.) I think many people simply assumed that if Vega came out later than NVIDIA’s Pascal, then it must be better.
If you are not familiar with the current state of the PC GPU market, NVIDIA has had a significant efficiency advantage since the introduction of its Maxwell architecture in 2014. It was later revealed that NVIDIA had adopted a tile-based rasterizer, which played a major though not exclusive role in eeking out this efficiency advantage.
Beyond that, it was apparent once AMD announced the TBPs (typical board power ratings) for the first Vega cards that the architecture is fairly terrible on power efficiency. This is not good, because power efficiency is pretty much the most important metric for any IC. To speculate on the reasons behind it at this point would be wild guessing, but it does appear that some things went wrong.
Speaking from experience in the mobile space, I’ve seen vendors who are uncompetitive to some degree on efficiency often boost performance to match the competition on benchmarks, by operating their silicon at more inefficient points in the performance/watt curve. That said, Vega’s being able to match Pascal’s performance is not something to be taken for granted either, and is thankfully the case. Vega's clock speeds are also a non-worry.
Software-wise, AMD’s drivers were clearly running very late. Software historically has not been AMD’s strength, though I am optimistic things will be improving from now on. However, one wonders why the drivers and various new features are so delayed.
Everyone knows that Vega was late. While HBM2 yields likely played a role, there’s probably more to it. Someone smart said that AMD did the right thing to delay the products (as opposed to ostensibly doing something stupid).
To me, it looks like AMD probably had enough issues with Vega that it had to rush out a respin. On the one hand, that would clearly not be good. On the other hand, if so, I’m really glad AMD paid to do it and delayed the non-Frontier cards. Respins are really expensive, and in mobile consumers are often not so lucky to get them. That is the extent of my familiarity with these things at least. The situation with Vega is not the end of the world since its performance is still competitive, and Vega will sell out for quite a long time regardless.
For architecture and competitive analysis, I recommend reading AnandTech (and only AnandTech), though of course useful benchmarks are found on many sites. I would also recommend waiting a week or two to see how the AT review gets updated, because it's impossible to actually analyze much of anything before a review embargo.
And as much as this will probably pain gamers to hear, I consider Vega’s performance on deep learning operations to be much more important than its gaming credentials. There is an inordinate amount of money at stake if AMD can manage to move the needle with Radeon Instinct and HIP against NVIDIA’s domination in deep learning.
A former Apple engineer has shared that Apple switched A2DP and HFP over to its own Bluetooth LE audio standard in iOS 9. This blew my mind.
For background, here are some of the basics. There are two “Bluetooths”: Classic and Low Energy (LE). The former is the streaming standard that everyone knows through wireless headsets and speakers, while the latter is basically what every modern peripheral device or hardware accessory, such as a smartwatch, uses to transmit data.
LE is also called Bluetooth Smart. LE is bursty and lower power (though not necessarily inherently more efficient), and was designed to enable devices running on coin cell batteries. You can do crazy things like stream video over it, though, if you so desire. (Don't do that.)
I’ve been vaguely keeping track of progress on BLE audio for a few years. I knew that the Bluetooth SIG was working on an LE audio standard, but am amazed that Apple secretly deployed its own in 2015. But it’s not magic, and is still based on LE. “Configuring the HAs is performed through LE services & characteristics, but the audio streaming channel is secret sauce.”
Bluetooth LEA, as Apple calls it, is not used by the AirPods. I’m not sure why, but it may simply be because LEA’s quality is still inferior to Classic audio streaming. Streaming audio is inherently difficult because of LE’s lower duty cycle, which is what makes LE more efficient in general.
Pairing is the same as for the AirPods, using standard LE protocols, though there may be specific codec features that Apple depends on. To emphasize, this is all still built on top of standard Bluetooth. And I believe the SIG is working on a similar pairing UX feature. (Keep in mind that pairing is not required with LE as it is with Classic. Otherwise, say, Bluetooth beacons wouldn’t exist.)
Aside:
I frequently see people complaining that “Bluetooth sucks” or “Bluetooth is always supposed to get better next year.” Before they were announced, for some reason people even wondered if Apple was going to replace “Bluetooth” for its AirPods. The problem is that people are almost always thinking of the wrong Bluetooth.
I won’t fully explain it here, but basically Classic and LE are different radios. To oversimplify: you can think of Bluetooth 4.0 and later as a completely different spec than 3.0 and earlier. For example, Bluetooth 5 has absolutely nothing to do with the Bluetooth that people normally think of (Classic).
- Thanks to Brendan Sharks for suggesting a correction to the article title.
I can't think of any semiconductor companies with well-designed logos or wordmarks, but at least this one is better than the old one?
I will definitely never get used to writing "Arm."
Permalink
This isn’t really a fix. Sky-high voltages + thermal stress = it’s dead, Jim.
Worth noting: AnandTech got a lot of grief when it didn’t recommend the Nexus 6P or any other Snapdragon 810 or 808 device.
Permalink
I want to write about what should be the highlight feature of the iPhone 8: its HDR display...
This article is available for subscribers on Patreon.
Permalink
I’m going to switch to Firefox for a while to try this out.
Even though Firefox is not my main browser, its “Don't load tabs until selected” option has always been my favorite browser feature. The number of tabs I want to load on first launch is exactly one. In an ideal world, the resource overhead of tabs you’re not currently looking at should be as close to zero as possible.
Permalink
I think you will be thoroughly persuaded by this article.
In short, individual display variance is too significant, and you will probably make things worse. If you want to individually calibrate your TV, don't do it yourself, and certainly not by eye. Have a professional do it.
If you want to learn about TVs and home theater equipment, I recommend following Chris Heinonen and reading his articles. Note that I am only recommending him, specifically.
Permalink
Previously I wrote that iOS should be redesigned for OLED. After seeing iOS 11 debut at WWDC and thinking further, though, I realized Apple might go farther than I originally believed...
This article is available for subscribers on Patreon.
Permalink
Today I am launching subscriptions for Tech Specs, at $10 a month.
I need your support in order make this blog a sustainable effort. I know that $10 is not insignificant, but it's honestly what I think will be necessary to get Tech Specs off the ground. I'm trying to keep my costs as close to zero as humanly possible, and to date have funded everything out of pocket.
Your money will go towards:
- Access to all Tech Specs articles. At least one in-depth piece per week, on average. While it could end being more, I would rather overdeliver than overpromise
- A small number of free articles for everyone, generally introductory educational pieces or minor news commentary
- Maybe even better podcast audio quality. It’s theoretically possible…
Some highlights of recent coverage include:
- Discussing the high-level architecture of Apple’s A10/X
- Describing the current situation with 10nm
- Explaining how ProMotion would work, before it was announced
- Virtually tearing down a Samsung Galaxy S8
I will continue to write about hardware, software, and design. Examples of topics I would like to address include: the iPhone 8, Fuchsia, virtual reality, augmented reality, deep learning, self-driving cars, HDR, displays, color, battery life, smartwatches, Bluetooth LE, how Apple’s AirPods work, how Android works, the Apple Watch, Android Wear, watchOS, benchmarks, understanding the supply chain, eSIMs, real technology economics and financial theory from academia, and the future of computing.
Some of these topics I can write about in great detail. Beyond that, I have many ideas for the future of the blog. There are also certain guests I would like to host on the podcast (eventually).
There will be no ads, ever. I believe in the subscription model.
Advertising's enormous advantage is of course the democratization of content — everyone has access. I am highly sympathetic to this benefit, which is why I will continue to make introductory articles available to everyone from time to time. They will always be a very important part of the blog.
But the advertising model on the internet is often detrimental. When you have reputable technology websites flooded with highly questionable ads and auto-play videos that greatly inhibit the performance and battery life of readers’ devices, the system is broken. This is not an indictment of journalism, but simply economic reality. And inherently all advertising is consequential to the message of its medium.
For all of these reasons, I prefer the subscription model. It also allows me to avoid the temptation of clickbait headlines. Before publishing a piece I can ask myself whether I even have anything of value to add on a topic. If journalists have already covered it well, then that's great, and I’m happy to share those articles.
I also believe there is a great need for tech coverage that provides at least some of the perspective of the industry itself, and it's worth emphasizing how much the average industry observer does not get to see. Talk to an engineer at any tech company, and it's clear that "how things actually work" is often radically different than how it's portrayed online. There is an tremendous amount of work that goes into creating, testing, and manufacturing tech products, and the vast majority of this work goes completely unappreciated in the public record. What goes into making a product is often just as important as the final result.
I genuinely want to do something different. I’ll do this by covering things that are not normally discussed in the press, or often are not on the internet at all. Sometimes I’ll be able to go into much greater depth on technical subjects. Relatedly, nothing is more important to me than accuracy, and I will always correct any identified mistakes.
Lastly, within the realm of independent content I am indebted to several influences, including Jessica Lessin and The Information, Dan Luu, and Chris Pirillo. My thanks to them for the inspiration.
Thank you all for your consideration and your support. It’s deeply appreciated.
Android's open source nature makes it vastly easier to learn about than closed source OSes. As such, I want to address several misconceptions about the platform that constantly come up. This article will be occasionally updated on an ongoing basis as I think of more topics to include.
GMSGMS actually stood for Google Mobile Suite, not Google Mobile Services, at least originally. So many people assumed it stood for the latter that even Google seems to use it now.
Perhaps it was a situation like Qualcomm’s Gobi, its cellular firmware API that so many people confused for a modem brand that eventually Qualcomm gave up and rebranded its modems to Gobi. Or Samsung’s ISOCELL, the deep trench isolation implementation that so many people thought was Samsung’s camera brand, that it also recently gave up and branded its CMOS image sensors as ISOCELL at Mobile World Congress 2017.
Kernel versionI really can’t explain it any better than this thread by Tim Murray.
Force quitting appsIn short: don’t do it. Android manages itself perfectly fine. Unnecessarily closing and re-opening apps causes thrashing, slows down app reloading, and hurts battery life.
If memory serves, with Android X.X (can someone please remind me?), swiping away an app in the multitasking UI no longer force quit even the background services of an app in AOSP. Swiping away apps can actually still force quit them on a specific device, though, depending on the vendor’s chosen implementation.
If, however, an app is actually stalled or causing real problems in the background, you may have to manually force quit it. But in general, avoid doing so. Be nice to your NAND’s endurance, folks.
Project TrebleBased on recent changes to AOSP, Treble appears to be an attempt at a stable driver API. By painfully rewriting its various HALs to conform to a new standardized hardware IDL, the Android team is speeding up Android updates by enabling silicon and device vendor bring-up efforts to be more parallelized, and by making updates a bit more economically viable for the silicon vendors. This does not mean, however, that SoC vendor support no longer matters at all. It’s a huge deal, but it’s not the same thing as having a stable driver ABI. See: Fuchsia.
Update: For much more and up-to-date information, see my recent article.
F2FSF2FS is a file system developed by Samsung LSI and upstreamed into Linux. It was designed for NAND flash memory and is claimed to be faster than ext4, Android’s default file system. The Android team disputes this based on its own testing, and says it sees no significant performance differences between the two. Regardless, there are several issues with F2FS, and more importantly Google cannot hardware accelerate file-based encryption with it. There isn’t a pressing need to replace ext4, though of course a better, more feature-rich file system could always supercede it.
Update: I guess Google has completely changed its mind. F2FS is apparently going to be adopted by Google, seemingly because SQLite 3.21 supports batch atomic writes on F2FS.
SecurityMalware on Android is often portrayed as an ever-growing, constant crisis. While Android does have tons of major security concerns, the overall issue is still hugely overstated.
Firstly, the term malware can mean absolutely anything. The vast majority of stories about mobile security spread FUD and sensationalism, to the detriment of readers. I won’t pretend to be a security expert, but even imperfect sandboxing probably goes a long way compared to the completely unsandboxed traditional PC application environments. It doesn’t seem clear to me whether Android or macOS is more secure overall, for example. As with many things, it probably depends.
There is however an extreme case: the Chinese market. Because Android is out of Google’s control in China, the OS genuinely is a security nightmare in the country. I remember waiting for a flight at the airport in Beijing and watching with amusement as some seemingly low-threat app started downloading itself onto my phone over the air. All I did was merely have Wi-Fi on; I hadn’t attempted to connect to any access points.
Everyone knows the fundamental issue with Android security: the horrible update problem. If devices consistently received timely updates for multiple years, the perception of Android’s security architecture would be radically different. I would personally attribute that to licensing and Linux's deliberately non-stable driver ABI, but there are a few hundred other opinions out there on the matter. And of course the overall topic of security is much, much more complex than what I am addressing here.
Which leads us to...
Android ThingsWhat is Android Things, really? Why is it a distinct platform from Android, in other words? One very important difference is how Google manages the BSPs (board support packages) and drivers for Android Things. It works with the silicon vendors but provides the BSPs itself. Device vendors cannot modify the behavior of kernel drivers or HALs. Developers that need to add drivers for peripheral devices to add to their baseboard are able to write user space drivers, unsurprisingly called user drivers.
Furthermore, the intersection of updates and the Internet of Things would seem to be an obvious disaster, so how does Google address the issue? The company is actually releasing monthly BSP security patches through its Developer Console that soon roll out directly to devices, with the caveat that the direct updates will only be for the same platform version of Android the device is on (such as, say, 7.X Nougat).
Project Brillo is not exactly the same thing as Android Things. Brillo was killed, and the initiative was changed in terms of goals and morphed into Android Things. The Accessory Development Kit (ADK) was also somewhat of a predecessor to Android Things.
Android WearAndroid Wear employs a different, imperfect solution to the update problem: it's closed source. (Some UI view components, though, were open sourced at I/O this year.)
Touch latencyTo oversimplify: Android touch latency was never good, until Android 7.1 essentially finally “solved” the problem. Additional features were added to the new Hardware Composer 2 (HWC2) HAL in 7.1 which can reduce touch latency by up to 20ms, or 1.2 frames, but not always. (It is not correct to say that touch latency is simply 20ms faster in 7.1.) According to the Android team, this was done by staggering some operations on batched input events, not doing everything on the VSync frame boundary, in order to reduce the likelihood of triple buffering and the increased latency that it causes.
The improvement in touch latency is extremely noticeable, and it immediately impressed me on the Nexus 6P after installing 7.1. While the HWC2 improvements make a huge difference, silicon and device vendor implementations still matter! A device still needs to have a quality touch controller, touch stack, and associated software. There are also other parameters that can be tuned by vendors, such as move sensitivity, which should vary based on device size.
It’s also important to understand that touch and general input responsiveness is a function of rendering performance. This is why, say, G-SYNC improves input latency when it manages to improve performance, and especially when exceeding 60fps under VSync. Thus on any device, the higher the realized display refresh rate, the lower the input latency. This is how Adaptive-Sync and proprietary variable refresh implementations will soon benefit Android touch latency.
Graphics renderingFor years people have debated the causes of Android’s infamous “lag” problem. The causes of jank on Android have never been as simple as a binary distinction of whether the OS is hardware-accelerated (running graphics operations on the GPU to some extent) or not. I have no idea what the true reasons are, but I do know that Android’s rendering pipeline is extremely complicated and requires graphics expertise to really understand. At the end of the day, most signs point to initial design decisions made in the early days of Android that are not easily undone.
While many have unrealistically hoped for a single “performance boosting thing,” Android performance does constantly improve in each new release. As previously discussed, one interesting new feature introduced in O is an optional new graphics renderer. (Updated explanation: the option just replaces hwui's GL renderer with Skia's GL renderer.) Skia's renderer will probably be made the default GL renderer in Android P.
Variable refreshOne thing to note is that adaptive sync/refresh will benefit Android more than Apple’s ProMotion benefits iOS.
ColorNo vendor should target anything other than sRGB for a device display until Android O ships. In other words, the display’s software calibration must target sRGB to be correct, because that is the only color space that Android currently supports.
I hope this article at least conveys how almost all engineering decisions involve tradeoffs. There are rarely magic bullets that solve everything.
If anyone spots any errors, corrections are always welcome.
I found Erica's opinions valuable in terms of thinking through iOS 11's multitasking UI redesign, even though I don't agree with her conclusion (that the new design is worse overall). Her concerns about whether it serves all users are commendable. In general it's critical to consider all points of view on subjective decisions. If you don't think about the other side of an argument, you haven't really thought through a problem.
I think the new multitasking UI is awesome and necessary for implementing Spaces, but Erica correctly identifies many of the downsides to the redesign. All of her points are valid, but it's also worth noting that none of these features are strictly necessary to use an iPad. If users never discover the multitasking UI in the first place, they can continue using the iPad exactly the same as always.
All engineering involves making tradeoffs. The cost of not implementing this more power user-friendly redesign would be the iPad continuing to stagnate. Tablets need to continuing evolving to do more than phones, and they've arguably taken far too long to do so. The increase in complexity is a necessary tradeoff in order to make tablets more valuable in their own right. That's not to say there isn't definitely a lot of room for improvement with this new UI, though, of course.
I personally like that a side effect of the new UI is that users can no longer easily swipe away apps, hurting performance and battery life even though they are often actually trying to improve battery life. The previous UI probably would have made it too easy to accidentally swipe away a Space that users had bothered to set up.
iOS 11's new Control Center is also pretty much exactly what I wanted to improve watchOS: an untruncated vertical scrolling list. Having to perform separate swipes to access multitasking and Control Center would be more confusing and time-consuming for users. The unified bottom swipe not only encourages more frequent multitasking, but it also provides a simplification of the previous four-finger gesture shortcut. There are always many aspects to consider regarding accessibility.
Design is hard.
Permalink
Warning: there is a fair amount of swearing in the link above.
Matthewmatosis is my favorite YouTuber, as he makes amazing videos commenting on game design. In this video segment, Matthew talks about leaks in the video game industry, but what he says applies equally to the tech or any other industry. His feelings are obviously not profound, but I could never say it any better myself. The developers deserved their moment of joy after three long years of work.
I also saw the leaks of Mario + Rabbids Kingdom Battle before E3, and I wish I never had. The game's unveiling would genuinely have been an awesome surprise otherwise. Thankfully the game at least looks pretty great. And do check out Matthew's videos if you're interested in game design. I can't recommend them enough.
Permalink
For the second Tech Specs Podcast, I'm joined by Josh Ho, whose writing you may be familiar with from AnandTech's mobile coverage. Topics include: four months of new devices, Apple’s SoCs, benchmarks, technology misconceptions, and explaining things on Twitter.
New 10.5” and 12.9” iPad Pros were widely anticipated going into this year’s WWDC. It had been 19 months since the launch of the original 12.9” iPad Pro, and nine months since the release of the A10. Big updates were due for the iPad line.
Apple first went through the feature and display improvements of the new models as expected. (I had a pretty good hunch ProMotion would be announced going back to last year.) Then it highlighted the usual specs. CPU performance has increased 30%. GPU performance has increased 40%. This is in comparison to… the A9X. Huh?
This is not what silicon people were expecting to hear. Apple pushes performance like crazy, and it was supposed to be releasing a 10nm SoC. These numbers are not impressive by Apple’s standards. Comparing to the A9X and even older SoCs is deliberate marketing framing to make the numbers sound more impressive than they are.
Could Apple actually have pushed purely for efficiency this time? Given its history of CPU designs, this seemed unlikely, though still possible. Apple also neglected to mention any efficiency advances, which it would likely tout in that case. Most surprisingly of all, Apple made no indirect mention of 10nm. This was deeply suspicious. Given the deliberate vagueness of the performance figures, however, it was hard to make much of the situation.
First, though, I need to apologize for an incredibly stupid error in my thinking about this SoC. I expected an A11X, as it looked like Apple had been working on a 10nm design*, and Apple essentially always pushes performance. If you’re familiar with mobile silicon, it was logical to expect that it was waiting to launch a new SoC on the bleeding-edge node. And if you don’t think Apple would be that aggressive on timing or performance, the A9X’s release was successfully brought forward half a year, a massive accomplishment.
I knew to expect that the A11 would use a 64-bit-only CPU, to allow for a more spatially and performance-efficient core design. I didn’t, however, make the rather obvious connection of dropping 32-bit support in iOS 11 to the near-simultaneous release of Apple’s first CPU without 32-bit support. That is to say, any Apple CPU released before iOS 11 would necessarily need to retain 32-bit registers, and therefore it would make little sense to release a new micro-architecture before the fall. This is because iOS 10.3 still needed to run on the 32-bit A6 and A6X. That the new iPad Pros were seemingly delayed a couple months and released just a few months before the A11 is irrelevant, if unfortunate from Apple’s perspective.
Based on what I know about 10nm, though, there seem to be two likely possibilities. The first is that this was Apple’s plan all along. The A10X would deliberately not push the envelope for whatever reasons. There would be no new CPU microarchitecture, aside from maybe some small improvements. The addition of a third CPU cluster and the necessary logic and tuning to make it all work, however, would be far from trivial. And a lot of work would also go into the new GPU, of course.
The second possibility is that the original 10nm A10X design had to be downgraded once it was clear that yields were not going to meet Apple’s targets.
There is a further wrinkle to the picture. If the A10X’s cache sizes really have changed significantly, it would perhaps lend credence to one or the other possibility.
Either way, everything I’ve heard about the A10X points to it still being a 14nm TSMC design, so my best guess is that the original design for the A10X was indeed canceled. This would not be terribly unusual in the world of silicon design, but it would be a rare public setback for Apple’s silicon team. If this is really what happened, the iPad Pro’s intended SoC would have effectively been sacrificed to ensure adequate supply of the A11 for the iPhone 8. For obvious reasons, that would be the correct choice of action.
The bigger implication is that the signs really don’t bode well for 10nm. Final clockspeeds for Snapdragon 835 and Exynos 8895 were really low compared to theoretical expectations. Yields are terrible, and the node is looking as bad as 20nm did. It could even possibly be worse, if TSMC and Samsung Foundry are struggling with issues similar to those that Intel first grappled with during its 14nm ramp-up. Moore’s Law is slowing down because of fundamental physics, not a lack of engineering willpower.
Further details are required, so please consider none of this confirmed. This is merely my best guess as to what happened. Unfortunately the details of these things often never come to light publicly.
And to be absolutely clear: I am not saying that Apple did anything wrong or is trying to be misleading in any way. It still has the fastest mobile CPU by a country mile. 10nm is just not going well.
Updates1) Much to many's surprise, TechInsights today finally publicly provided a die shot of the A10X. Even more surprisingly, it's fabbed on 10nm after all. TechInsights estimates a die shrink of about 45%. Additionally, it echos my previous suspicion that the 10FF ramp was delayed a quarter, and thus the new iPad Pros were indeed delayed from the initial plan of a spring unveiling.
How then do you explain the A10X's poor final clocks and lack of performance increase per-core? From what I can tell, TSMC delivered some fairly awful results this node. 10FF seems to be very leaky and is probably significantly worse than Samsung Foundry's 10LPE. TSMC prioritized the development of its 7nm process, and this is perhaps the consequence: a short-lived, bad 10nm.
Process immaturity aside, whether Apple was originally hoping to design the A10X differently is impossible to guess. I suspect its team working on the A11 is currently killing themselves to somehow make 10FF work.
2) Leakage doesn't actually seem to be the problem with 10nm. Transistor performance improvement has stalled. This is not good.
- Discovered by Ashraf Eassa of Motley Fool.
Apple has some obvious priorities to address this year at its Worldwide Developers Conference (WWDC). Firstly, it needs to significantly redesign iOS for the upcoming OLED iPhone. There are major technical considerations for both hardware and software that Apple has to deal with for its OLED transition. These considerations extend far past a dark mode for apps, which I would also anticipate. Apple has notably already dealt with OLED before for the Apple Watch. I don’t think it will try to match iOS with watchOS aesthetically, but perhaps their overall appearances will be a little closer. And I highly doubt iOS will change from rendering black text on white backgrounds for maximum readability.
iOS’ dominant white and blue are pretty much the worst colors for an OLED user interface, though, so something more akin to watchOS’ use of black, grey, and green is required (to some degree). Apple could even maybe selectively utilize some red tones. For further context on colors and OLED design considerations for energy efficiency, lifetime, and text legibility, Brandon Chester and I discussed some of these topics at length beginning at 55:03 on the first Tech Specs podcast. I will also publish some further thoughts on the reasoning behind the UI changes at a future time.
Secondly, Apple needs to deliver on deep learning. Despite what Apple says externally, internally executive leadership seems to have been caught off-guard by the sudden, massive progress in AI brought about by the deep learning revolution. Backchannel’s exclusive access piece did not instill confidence, but rather marketing desperation, by trying to conflate all areas of machine learning together as one thing. Ignoring the technical errors in the article, it doesn’t matter that Apple has been using machine learning since the 80s; all that people should care about is its current competitiveness in deep learning. In contrast, when Google mentions machine learning, as far as I know it is always referring to deep learning.
Depending on how you prefer to segment technology, many would argue that deep learning is the most important advance in software since either the touchscreen smartphone's user interface or the internet. In contrast to almost every other technology industry buzzword or phrase of the month, having an “AI-first” strategy is actually credibly meaningful. Last year I argued on Twitter that Apple needed to show a suite of deep learning-powered services, or it was going to look really behind. That is thankfully exactly what it did. At this year’s WWDC, Apple needs to demonstrate a continuing wave of progress company-wide on AI. If you want to get a sense of how seriously Google has invested in internal training on deep learning to remain the market leader, see its alternative exclusive access Backchannel piece from a couple months prior to Apple’s.
These articles are sometimes published months after interviews are granted to journalists, but there was one thing in particular I noted at the time of the Apple piece. Craig Federighi was quoted as saying, “We don’t have a single centralized organization that’s the Temple of ML in Apple.” Not many days before the article was published, it was reported that Apple had acquired Turi, which formed the basis of its new machine learning division, an obvious necessity for internal tooling and research. Perhaps I am reading too much into this, but that might suggest Apple’s deep learning strategy was still in flux at the time of the interview.
It may not seem fair, but this year Apple has to continue to prove it can keep up to some extent with the market leaders. If not, its competitive positioning in AI might come to resemble its mastery of the cloud: perpetually years behind. If it sounds like I am being negative about Apple, believe me I’m not. The company is ridiculously competent at almost everything, but it shouldn’t be graded on a curve on server infrastructure or AI. And by “AI,” think of deep learning algorithms, not futuristic images of omniscient assistants from science fiction.
Thirdly, Apple needs to ship a ton of iPad-specific software improvements. I know these are definitely coming, and I suspect Apple will deliver in spades. The Split View multitasking UI is one example that everyone agrees must be replaced; an icon grid with greater information density could help. Adding drag and drop functionality seems really likely. Using the iPad as a drawing tablet or secondary display for the Mac has been rumored multiple times. And while it would require extensive OS-level engineering to bring about securely, multi-user support also seems like a strong possibility.
Brandon mentioned to me last fall that he thought Apple would transition to a 10.5” iPad display size in 2017 in order to switch to using two regular size classes in portrait mode, which would make sense. I eventually saw supply chain rumors that the new iPad would be 10.5”, so that display size with an iPad mini-sized UI seems pretty likely.
Unsurprisingly, I am hoping that iOS 11 will also provide a major performance revamp for the platform. Don’t expect any miracles, but it would be nice if there's at least less Gaussian blur. There's ample opportunity for Apple to continue to tune how iOS works, to better fit its CPU and GPU architectures and make the most of their microarchitectural advantages.
Apple also clearly needs to showcase a significantly improved Siri. (I’ve always been hesitant to describe digital assistants as “AI.”) I’ll leave hardware rumor reporting to the press, but my guesses would be that the A11X iPad Pros, spec-bumped MacBook and MacBook Pros, and Siri speaker all get announced on Monday. We may or may not see the rumored iOS-wide voice command accessibility this year, but the latter product will probably fully depend on a more capable Siri. My one prediction is that the Siri speaker has nothing to do with mesh Wi-Fi. I think Apple would prefer to ship something useful like an 802.11ad-capable router.
Siri never really worked for me personally, and never understood my voice at all on the Apple Watch, until iOS 10 and watchOS 3 were released. Siri is much better now, but its word error rate is still higher than Google's. Where Apple is definitely market-leading is API design, which is criminally under-appreciated as a competitive advantage. SiriKit is probably the best overall voice assistant API, but it’s also the most ambitious in terms of flexibility. Continued expansion of the deliberately limited API surface is required.
I’m also hoping to see broader deployment of differential privacy and similar experimental technologies. Apple is still going to have to pay the efficiency tax and perform deep learning inference on device with non-ideal hardware, until it can ship more appropriate silicon. My sentiments are likewise the same on security, given last year’s political battle between Apple and the FBI.
I’m not sure if Apple will ever make its secret iOS VR framework public, but if it does, it will probably wait until at least the iPhone 8 announcement. Quality VR basically requires OLED.
Apple needs to continue to make writing functional smartwatch apps much, much easier with watchOS’s API, while still preserving energy efficiency. tvOS deserves a better and more performant multitasking implementation, like the original one. iOS 11 will probably drop 32-bit support, to significantly reduce memory usage. Apple at some point should ship improvements for family management of content and media, especially within the Photos app.
From a developer point of view, I wonder if UITableView might be deprecated. Auto Layout seems to be a performance killer, so some sort of magically more efficient way of arranging layouts would be nice, however difficult to conceive. There is also unending room for improvement for macOS security and the Mac App Store, but one shouldn’t hope for too much.
Lastly, this is the last year that I will hold out hope for a swipe keyboard. It would be an enormous improvement for one-handed use and accessibility. Maybe it could even work in a floating window on the iPad? Please, Apple?
One of the most exciting display features in recent years has been variable refresh. While gaming monitors have long offered higher refresh rates, NVIDIA pioneered variable refresh PC monitors with G-SYNC in 2013. By controlling refresh rates from the display’s end of the display chain, G-SYNC significantly improved perceptual smoothness by removing stutter within a certain frame rate range, eliminating tearing entirely, and reducing input lag. Here is a video of the original G-SYNC demo explaining its benefits.
AMD soon followed with its competing FreeSync, and VESA then added a free implementation called Adaptive-Sync to the DisplayPort 1.2a specification. Originally, Adaptive-Sync was actually introduced in 2009 for displays using internal video signaling, in the eDP specification which mobile devices use for their integrated displays.
Following the rollout of panel self-refresh, I have been eagerly anticipating the adoption of Adaptive-Sync in mobile display stacks. There hasn’t been a real reason to include it, however, because everything in mobile OSes targets 60fps.
The reason for this is simple: going past 60Hz increases display energy consumption. Driving higher frame rates also requires greater compute and thus further increased power draw. Additionally, there is an important technological distinction between IGZO and LTPS displays, the former generally being limited to larger mobile panels, while the latter’s greater efficiency has led it dominate the market for small panels for smartphones and tablets.
I won’t explain display backplanes here, but IGZO transistors do provide certain advantages, such as faster switching speed due to higher subthreshold swing. IGZO displays thus make higher refresh rates more easily achievable than they are for LTPS panels in mobile and laptops. And although a display stack running at higher refresh rates will burn more power, variable refresh can significantly mitigate this increase.
You may have noticed that the iPad Pros and 2016 MacBook Pros are advertised as supporting variable refresh rates. (This will be a feature of the iPhone 8, too.) This is actually a different implementation than that of PCs and desktop monitors. What Apple is instead doing is driving display refresh down to a constant 30fps when the screen is displaying static content. This provides a significant improvement to battery life.
Over the past several years mobile displays have been produced with increasingly higher peak brightness and wider color gamuts. Where do you go from here? Higher refresh rates.
Ordinarily, whether you are targeting 90 or 120Hz refresh, this would require quite a improvement in system-wide performance. Android and iOS both target 60fps, but neither actually manages to run at a consistent 60fps (and with smooth frame-pacing) pretty much ever, excepting small numbers of extremely performant apps.
iOS used to run quite consistently at 60fps across all of its system apps prior to iOS 7, but then pervasive Gaussian blur, increasingly complex UI layouts, and other factors led to a persistent and significant decline in performance. The iPhone 5 running iOS 6 was the last mobile device to truly run at 60fps, in other words.
Thus, without a major focus on improving system performance, don’t hold you breath for iOS to achieve a consistent 60Hz frame delivery anytime soon. That would require a serious OS-wide code review effort, and it’s not a given that Apple cares enough.
Even if that were to happen, some things are basically impossible. Assume that Apple wants to target a 120fps device refresh rate, a nice even multiple of 60 and 24fps. There is absolutely no chance that it can magically get every first and third party iOS app to render within the 8.3ms render window (1/120 seconds).
As you might have guessed, there is another way — Adaptive-Sync.
To support the feature, the display controller (part of the SoC) must support the feature, and the display driver (DDIC) and the panel itself must be capable of higher refresh rates. I believe now is the time for mobile implementations to finally arrive. A device like a large tablet particularly has the energy capacity to spare, so much so that battery cell volume in the iPad Pros was replaced with larger speaker cavities partially to save weight.
Note that using Adaptive-Sync to avoid dropping frames is not as good as hitting a target frame rate in the first place. Rendering smoothly at 55fps is still inferior to hitting a steady 90fps delivery target, for example, because you are simply seeing fewer frames. But you do achieve the same benefits as with PC implementations: stutter is eliminated, and input lag is reduced. (There is already no tearing on mobile.)
For Android, I think Google could even in theory eventually disable triple buffering should Adaptive-Sync ever become ubiquitous in mobile. Only a frame of latency would be gained back, but it would be a clean win. There is no reason for this to ever happen, though.
In conclusion, I hope to soon see tablets and other devices that support Adaptive-Sync. It’s a killer feature that would make a huge difference for both input responsiveness and motion image quality.
UpdateI somehow missed that Qualcomm has its own variable refresh implementation called Q-Sync for Snapdragon 835's display controller. Qualcomm didn't mention it to me at CES, so I have no idea about the details. I will try to find out more, but it sounds like it may be a proprietary implementation since it seemingly requires Q-Sync-specific support from display driver vendors. I'm a bit skeptical about adoption. Thanks to Naseer for the heads-up!
There were an enormous number of announcements at this year’s Google I/O. In particular there seemed to be the most advancements in Android development announced since 2014. Rather that attempt to comprehensively summarize the event, here are some scattered thoughts.
AndroidThe biggest Android-related news was clearly the surprise adoption of Kotlin. The main reason it was a surprise was because of the tone the Android team conveyed in response to requests for Kotlin support at last year’s I/O. While many team members used Kotlin, they seemed to suggest that Java was going to remain the platform language for the foreseeable future.
I’m not a programmer so my opinions on languages are invalid, but everything I’ve ever read about Kotlin makes it seem like a pretty good language. I have no idea how well it performs, though. Because Kotlin was designed for seamless Android interoperability from its inception, it can pretty much immediately replace Java for any developer now that it is officially supported in the Android SDK. There will be some gaps in tooling, but it’s as easy a developer transition as they come. The next step will be transitioning APIs to Kotlin. I don’t think many members of the Android team will miss Java.
The app lifecycle has always been a nightmare on Android. The new architecture model proposed by the Android team surely has to be an enormous improvement, but it understandably necessitates new classes. The team is notably proposing more of a reactive-style model instead of a model-view-viewmodel (MVVM) paradigm, to quote Yigit Boyar. We’ll see how that turns out.
Android O is a major performance release, probably the most significant one since 5.0 Lollipop. One of the key efforts by the Android team for O was the elimination of binder lock contention. The result is a “massive improvement in jank immediately after boot as services all start up.” Running the O beta on my Nexus 6P, I was amazed how much of an obvious and appreciable difference this makes. For a thorough description of Binder by Dianne Hackborn, see here.
Significant improvements in OS boot time are also claimed, with the Pixel smartphone starting up in half the time on O. App startup times have also been sped up by different optimizations. I suspect all those app racing videos on the internet played a roll in spurring this effort, though I would strongly caution you not to assume those videos are really representative benchmarks.
Also of major note is that Android O features an optional new renderer, which you can enable from developer settings. Nothing has been said about it, but it is based on Skia. I don’t know anything about graphics libraries, but Android has used Skia from day one so I have no idea what the new renderer entails.
Regarding runtime improvements, ART has switched from using a mark-and-sweep algorithm to a concurrent copying garbage collector. Claimed results are less time spent reclaiming and allocating memory and lower overall usage. I know very little about garbage collection, so I wonder what the tradeoffs are. You should watch the ART presentation if you want to learn more about the new collector and the many other improvements. I do know, however, that Josh's desired simple optimization was unsurprisingly not implemented.
You may be surprised to learn that ART has also at long last added support for automatic vectorization. I won’t explain SIMD utilization here, but I may write an entire article about this topic in the future.
One nice addition that will indirectly improve performance and battery life is that the Play Developer Console will now flag apps that perform poorly in regard to wake locks, wakeups, and jank. Google also said that wake locks will be restricted in future releases of the platform, so developers be warned. These restrictions should be deeply appreciated by users.
Because Android releases are not developed in public, we still know extremely little about Project Treble. Aside from some vague high-level comments, the most specific information given was that "device-specific vendor components" have been sandboxed in some manner. Reducing the bring-up costs of Android updates for higher-end vendors like Qualcomm should be a huge help for the overall ecosystem, but I am skeptical that it will have much impact on a company like MediaTek, which has little financial incentive to provide updates in general. Treble also does not chance the fact that Linux does not have a stable driver ABI. I should point out that it sounds like the transition was a miserable technical slog for many members of the Android team, so thanks to them for their efforts.
With the announcement of Android Go, I immediately wondered if the platform's memory profile had changed. The last time there was a significant increase in RAM requirements for Android was the 5.0 release. There is no Android Compatibility Definition Document for O yet, so it is unclear if the minimum memory requirements will be changing (Section 7.6.1). Based on the ART session, however, overall memory usage should be lower in O.
Much to my surprise, graphics drivers are now updatable through the Play Store. This is not a small detail, and I suspect it was a benefit of Project Treble. Google is also now offering Android developers the equivalent of Apple’s App Slicing. Within security, tamper-resistant secure elements (ala Apple’s “secure enclave”) are now supported in O.
As anyone could predict, deep learning was a huge focus at I/O. Google’s TensorFlow happens to be the most popular deep learning library. While it has been available on Android since launch (and was later made available on iOS), Apple managed to provide a GPU-accelerated mobile framework before Google, with convolutional neural network kernels available in Metal Performance Shaders on iOS 10. The lighter-weight TensorFlow Lite was thus a big (and much needed) announcement, although developers will also at least be able to leverage vendor-specific acceleration libraries through Qualcomm’s Snapdragon Neural Processing Engine SDK. In the near future, TensorFlow Lite will leverage the new Android Neural Network API to allow developers to accelerate their AI algorithms on GPUs or DSPs.
I won’t beat around the bush — the changes in Android 5.0-9.0 have made the platform much more similar to iOS overall. I think the Android team has prioritized the right areas of improvement, even if they’re shipping fewer obvious consumer-facing features.
Everything elseJohnny Lee confirmed that Google’s VPS (Visual Positioning Service) is marketing's branding of the combination of area learning and point clouds stored in the cloud. Standalone Daydream VR headsets use stereo visual odometry and 3D SLAM to provide positional tracking, with drift correction based on area learning. The combination of hardware and algorithms is marketed as WorldSense, which is of course based on a specialized version of Tango.
Google also showed off some amazing VR technology called Seurat, which renders extremely high-fidelity graphics on mobile in real-time via unknown means. The technology could be anything, but it isn’t magic. For similarly impressive demos, check out OTOY’s “eight-dimensional” holographic light field rendering. (Update: Seurat is indeed supposedly some form of light field rendering. This Disney Research paper was released simultaneously.)
Within deep learning, Google stated that it was working on CTC-based sequence models for natural language processing, with “a whole bunch of new implementations" coming soon.
Lastly, I was wrong about the Flutter sessions. There were numerous memes, but none of the animal variety. I apologize for the error.
Romain Guy did not disappoint with his presentation on color at I/O. Color is so complicated that there’s no way to give a succinct introduction to it on Android in 40 minutes, but Romain did about as well as you possibly can.
I would say this is an intermediate level presentation, but if you're interested in learning about color for the first time it will probably feel like an advanced one. Don't worry about all the concepts you don't understand yet. I knew basically nothing two years ago before learning all the material in this talk. Even though color is a massive topic, the individual concepts are relatively easy to grasp. Give it a shot, and you can definitely learn all of the major points over time.
Permalink
To learn more about the silicon content of a Galaxy S8, I decided to identify many of the components in a US AT&T model. The following list is not intended to be particularly comprehensive. If any of the ICs listed below are outdated, there are likely newer revisions of these components in the phone.
Sensors:
ams TMD4906 optical module — ambient light sensor (ALS) + proximity sensor + IR LED
STMicroelectronics LSM6DSL SiP with accelerometer + gyroscope
STMicroelectronics LPS22HB barometer
Asahi Kasei Microdevices AK09916C magnetometer
Maxim Integrated MAX86907E heart rate monitor
Sealed Sensor Connector (SSC) System - maybe from TE Connectivity?
Semtech SX9320 grip sensor (an “ultra-low power capacitive Specific Absorption Rate (SAR) controller” for detection of RF attenuation by the user’s hand)
MSM8998 SoC:
ARM CoreLink MMU-500 system memory management unit
Qualcomm Adreno Venus 3XX video decoder/encoder (annoyingly marketed as a “VPU”)
Qualcomm WCD9341/9340 Aqstic (Tavil?) audio codec
Miscellaneous:
Broadcom BCM4361 Wi-Fi combo chip in a Murata module (Yes, Broadcom is still making mobile Wi-Fi combos.)
Toshiba THGBF7G9L4LBATRA 64 GB UFS 2.0 NAND + controller
Sony IMX333 and IMX320 BSI CMOS image sensors
Synaptics Namsan fingerprint sensor
Silicon Mitus SM5720 PMIC?
Maxim Integrated MAX98506 digital audio codec (DAC) + headphone amplifier (Hilariously, there are people selling this online as a USB charging controller for some reason.)
NXP PN553 NFC controller
Texas Instruments DRV2624 haptic driver
RichWave RTC6213N single-chip broadcast FM radio tuner
Xilinx XC4000 FPGA? (not sure)
Xilinx XC5000 FPGA? (not sure)
NXP PCAL6524-GPIO GPIO expander
Trustonic Trusted Execution Environment (TEE)
Possibly a Microchip USB controller?
There might be an Xceive XC2028 TV tuner in Korean GS8 models.
Used in system bring-up:
ARM CoreSight STM-500 System Trace Macrocell
Though they’re not terribly useful to publish, here are also some web benchmark results for reference:
JetStream 1.1 (Samsung browser): 75.710 +/- 0.26588
JetStream 1.1 (Chrome): 67.077 +/- 0.56466
Kraken 1.1 (Samsung browser): 2,342.0ms +/- 0.9%
Kraken 1.1 (Chrome): 2,837.6ms +/- 0.5%
Octane 2.0 (Samsung browser): 12,541
Octane 2.0 (Chrome): 11,322
WebXPRT 2015 (Samsung browser): 166 +/- 4
WebXPRT 2015 (Chrome): 158 +/- 3
Note that the beta of the Samsung browser is testing faster at the moment.
If anyone who works with silicon has any corrections, please let me know. I will have more to say on the GS8 in the future. For now I recommend following AnandTech’s technical hardware coverage.
The following notes are not intended to be comprehensive or predictive, but are merely some stray thoughts:
I have far fewer expectations this year than in previous years, as most of what I was anticipating was already introduced in the Android O Preview. Namely, this included finally making JobScheduler mandatory, and no longer allowing apps to register broadcast receivers for implicit broadcasts in their manifests.
I’m expecting the following topics to be discussed during the opening keynote: deep learning, deep learning, and deep learning. The machine learning sessions and office hours should be comically over capacity.
Regarding Project Treble, we have basically no details right now. I’m hoping the Android team will elaborate on it, at least during the Fireside Chat.
I was expecting the usual ART optimization improvements, and this year seems fruitful. The new garbage collector should be exciting. Josh Ho also suggested an obvious, albeit minor win to me last year: performing full ahead-of-time (AOT) compilation instead of profile-guided optimization (itself a form of AOT compilation) while on power for apps that have not run yet. The purpose would be strictly to improve first run performance and energy usage. I'm not expecting this to be mentioned, if it was even added.
Yigit Boyar is going to introduce some improvements for Android app architecture. As part of this, the new approach to managing app lifecycles was previously teased and will now be announced in detail.
Romain Guy is going to present on color. You would not believe how complicated the topic is. Color management on Android is still a work in progress, so even though not everything will ship this year, I have faith that everything (such as HDR) will fundamentally be addressed eventually.
I have no idea if anything will be announced at I/O, but at some point UHD content will be launched on Google Play. I’ve seen that the Chromium team is working on HDR support, which will be ready before color management is added to the browser. We may thus see HDR content limited to sRGB before UHD content is launched. I expect Google to push Dolby Vision. Either way, I don’t expect any Android devices will fully support HDR this year.
For anyone interested in Fuchsia, there are two talks on Flutter. I expect at least 50% of the slides to feature animal memes of some kind. Wise developers should embrace Flutter as the future of mobile development on Google’s platforms.
Lastly, I’m personally curious to learn more Android Things.
This makes me very happy.
It would be interesting if Google ended up buying Lyft one day.
Permalink
Yesterday Microsoft held its Education event, where it unveiled Windows 10 S and the Surface Laptop. While I will have more to say another time, I first want to discuss the intriguing new hardware.
With the new Surface Laptop, Microsoft seems to have substantially improved the overall quality of its hardware. I would argue that its hardware to date hasn’t been truly competitive at the high end of the market, but I would still recommend Microsoft’s devices over those from every other Windows OEM due to their superior hardware-software integration. Based on my experience with the Surface Pro 4, you are going to get a much smoother, more polished experience for criteria like trackpad performance by running software tailored by Microsoft.
The company is comparing its new laptop directly to the 13" MacBook Pro, particularly emphasizing how the Laptop weighs 0.26 pounds less than the Pro. Part of the weight difference is due to the Laptop's Alcantara surface, which I find to be the most interesting engineering decision. This material choice trades off structural rigidity and thermal dissipation efficiency for lower weight and greater comfort.
It is critical to note, though, that the Laptop only offers 15W U-series Core CPUs from Intel, while the 13" MacBook Pro also offers 28W CPUs for its more expensive configurations. In other words, the Surface Laptop has been aimed at a lower TDP, and thus lower performance, target than the 13" Pro. An eventual 15” Surface Laptop with H-series CPUs now seems likely, and many would be excited by such a product. Microsoft’s concession to its OEM partners is that it is once again only competing at the very high end of the market.
First, the bad news. The Laptop features one “full-size” USB Type-A port and one Mini DisplayPort, but no Type-C ports. At this point, Microsoft’s affinity for legacy ports and eschewing of any and all progress in connector standards is comical. Enterprise usage isn’t even a real concern, so there’s really no excuse.
I also strongly recommend not buying the base configuration with only 4GB of RAM. That makes the real starting price $1,299, in my opinion.
To be frank, there have been plenty of issues with Microsoft's hardware in the past. While it hasn't skimped on battery capacity in its recent products, the company has never shipped particularly spatially efficient computers with its Surface line. For the Surface tablets this philosophy went so far as to deliberately utilize empty space internally in order to optimize weight distribution, a design decision I found bizarre.
Previously questionable design tenets seem to have been abandoned with the Surface Laptop, however, and I think Microsoft's willingness to somewhat normalize on design has resulted in its most compelling device to date, at least on paper. And as it acknowledged on stage, that's exactly what its fans have wanted it do - just make a normal laptop.
In the grand tradition of PC vendors harmlessly creating new marketing terms for laptop audio, Microsoft has branded its speaker implementation as Omnisonics, not to be confused with ambisonics. Since it could not cut holes into the Alcantara surface, it has instead placed its two speakers behind the keyboard, radiating sound out through the keycap edges. The result is going to be mediocre audio quality from the speakers. On the plus side, the Surface Laptop uses Dolby Audio Premium DSP algorithms and meets whatever hardware requirements the license entails.
Since the Laptop is not a tablet, I can’t think of a single good reason why the aspect ratio of the display should be 3:2 instead of 16:9 or 16:10. Panos Panay stated that Microsoft wanted to maximize display area (optimizing closer to 1:1), but this strikes me as absurd for a product that never changes orientation. I’m particularly not a fan of the aspect ratio because it makes the laptop lid more likely to wobble. (This is one of the main reasons the Surface Book form factor did not work.)
Microsoft ships the most accurate displays of any Windows OEM, but while its panels have been very bright, they haven’t been competitive on efficiency. (Seriously, please never source Panasonic panels.) Based on the rest of the system design, though, I’m hopeful that the Laptop features an efficient display. Although there are not yet many Win32 apps in the Windows Store, keep in mind that there is effectively no High DPI support for Win32 apps.
Now for the good news. One of the best things about the Surface Laptop is that Microsoft has learned from the Surface Studio color calibration debacle. Unlike with the Studio, the Laptop’s display is correctly calibrated to sRGB, because there is no system-wide color management in Windows. It’s great to see Microsoft improving like this. The company continues to individually calibrate its displays, a practice going back to the Surface 3 which is a big deal and should result in great color and greyscale accuracy.
Microsoft didn’t exactly ship Kaby Lake-U early, but it can tout it as a significant advantage over Apple, as Kaby Lake provides the rough equivalent of a 300MHz CPU frequency boost over Skylake. The one battery life claim made, up to 14.5 hours of video playback, also emphasizes the advantage of Kaby Lake's addition of HEVC hardware decode, which Skylake was sorely lacking.
The entire rest of the laptop basically looks great, as do the four available colors. One thing that many people have missed is that, though, is that the GPU and DRAM frequencies for the priciest SKUs are lower than those for the 13" MacBook Pro.
(Sidenote to Microsoft: please make the technical specifications listed in your Fact Sheets more directly accessible to prospective buyers. They are important. Compare to Apple. I would also criticize Apple’s minimal consumer disclosure, mind you.)
The combination of lower clock speeds, the Alcantara, and the size of its singular fan makes me somewhat skeptical about the energy and thermal efficiency of the case design. I would expect conservative DVFS tuning. Public testing will have to wait on a review by AnandTech. Panay did weirdly seem to suggest that the keyboard feels warm during normal use.
Even though much of this article has been criticism and concerns, overall I have a very positive impression of the product. Microsoft is clearly on a roll, and the Surface Laptop appears to be its best hardware to date. While the device is particularly aimed at college students, especially since MacBooks have traditionally done well at US universities, I think it will sell well to a broad audience.
The Electronic Times recently reported that Google wants to invest ~$880 million in LG Display for future production of OLED displays. If this rumor is true, I suspect the potential strategic investment would not just be for securing displays for future Pixel devices, but for helping LG Display to seriously re-enter the mobile OLED market. The company previously sold flexible OLEDs to LG Electronics for its G Flex and G Flex 2 smartphones in 2013 and 2015, respectively, but I am not aware of any other smartphones ever using LGD OLED.
To date Samsung Display has been far ahead of everyone else in mobile OLED due to its vastly greater investment in the segment. LG Display could probably make reasonable OLED displays, though, if it had the financial incentive to make major investments in smaller panels. It has already proven its OLED capabilities with its Apple Watch displays, however difficult they were to make, and of course its leading W-OLED panels for the TV market.
This rumored change in strategy would probably be more about industrial design demands than display quality considerations. Several vendors, including Google, want to be able to compete with Samsung and Apple’s (upcoming) bleeding-edge smartphones that strongly associate OLED displays with high-end industrial design. While OLED is not necessary at all for creating a design with minimal bezels, some or even most of these vendors likely require OLED because they want to bring curved displays to market, sacrificing some image quality in the process.
To be clear, there are many advantages (and some disadvantages) to working with OLED displays over traditional LCDs from an industrial design point of view, which I won't fully enumerate here. One of the major differences, while it sounds obvious, is that OLEDs do not have LCMs (liquid crystal modules).
No matter what, it's not at all clear that investing in OLED over LCD long term would be a smart move, and most display suppliers remain skeptical of the former. OLED is better overall now, but it has strong downsides in terms of lifetime, costs (due to lower yields), and various quality deficiencies such as severe off-angle color shifting and chromatic aliasing. LCD meanwhile constitutes the lion's share of the market. microLED won't come to market for years, but it has greater potential than OLED should its production become economically feasible.
If vendors bothered to pay for high quality displays, we would see smartphones other than those from Samsung with correctly calibrated OLEDs with leading-edge quality. Perhaps a vendor or two other than Apple may one day do that. For now, given Samsung Display’s massive lead I remain skeptical that anyone can compete with it on quality over the next few years.
A couple weeks ago I had caught wind of some web benchmark being killed. The first thing I did was check Octane, but the test harness was still unchanged.
Yesterday Google announced that Octane has been retired. It claims the deprecation is due to Octane being over-optimized against for years, sometimes to the detriment of real world application performance. You can still run the benchmark, but the page notes that it is no longer being maintained.
This looks really bad. Octane was far from the worst benchmark, and it had been optimized against for years by pretty much everyone anyway. It did not suddenly become outdated overnight. (This does not in any way mean that benchmarks are somehow useless or unnecessary.)
If Google is working on a new browser benchmark, great. But it's hard to believe that Google didn't actually kill Octane because Edge now beats Chrome on it, which Microsoft immediately promotes upon launching Edge in the Creators Update for Windows 10.
Accurate information on color is rare to come by on the internet. This is an excellent introductory article by Chris Heinonen on color volume and HDR that I can't recommend enough.
Permalink
After Apple's rather surprising admission of fault yesterday about not updating the Mac Pro, I would like to address another area where the company happens to be blame-free. By now I have read every possible conspiracy theory under the sun about why Apple hasn't shipped [insert_your_desired_device_here]. Some of the most recent speculation is that Apple didn't announce new high-end iPads because some upcoming iPad-specific software is not ready yet. This is probably nonsense.
Anyone who follows mobile silicon knows how simple the current situation is in all likelihood: the iPads are delayed because Apple can't ship the A11X (Fusion) in sufficient volumes yet to its desired quality metrics (final clockspeeds, et al.). More succintly, Apple has not yet introduced an A11X iPad because 10nm is a bit of a disaster. And despite what you may read, a 10nm tablet SoC would be an A11X, not an A10X.
Everything I have heard points to both Samsung Foundry and TSMC suffering very poor yields currently, in the realm of 30-40%. 10nm is just a shrink node, but it turns out that shrinking transistors is excruciatingly challenging these days because of pesky physics. And if 10nm ends up being an outright bad node, we've seen this leaky transistor nightmare before.
If you're not familiar, 10nm from Samsung Foundry and TSMC is not at all the same as Intel's 10nm. Their 10nm is actually very comparable to Intel's 14nm, with nearly equivalent density. All node names are marketing nonsense these days anyway. 14nm was particularly egregious given the reuse of 20nm's BEOL, and TSMC didn't even call it 14nm simply because "four" sounds like "death" in Mandarin; "16nm" doesn't really exist.
Internal delays are still real delays. With yesterday as an extreme exception, Apple doesn't like to talk about products until just before they're ready to ship. When it does talk in advance, even off-the-record, things can go wrong, and forward-looking statements can go unfulfilled. It doesn't suffer a negative marketing impact by keeping internal delays internal, but much more importantly it also doesn't realize the greater profits it would have if it had been able to ship on time. The A11X delay hurts its bottom line.
The situation really is probably that simple. It's not that Apple suddenly feels like it can and should wait longer between iPad refreshes (19 months now for the 12.9" iPad Pro). And despite Intel's newly constant delays, it is often not actually to blame for Your Theoretical New Mac of Choice not being released. This is a broader topic I may address another time.
While ARM's big.LITTLE has evolved from its initial simplistic CPU migration and cluster migration iterations to simultaneous multi-processing (global task scheduling) to energy aware scheduling, the strict segmentation between big and LITTLE clusters has remained non-ideal. For example, the efficiency of shared memory access among CPUs and the speed of task migration have been significant downsides. Yesterday ARM announced the future of multi-core CPU design for its IP ecosystem, a series of technologies collectively branded DynamIQ which has major implications for SoC design.
I believe the reason DynamIQ took so long to announce, and was probably a ton of work to bring about, was how many interconnected systems were required to be redesigned in concert. And it was probably harder still to get them all working together well and efficiently. New interconnect designs and features are necessary to make the newly possible cluster designs work, and there is a new memory subsystem design for which no details are yet being provided. IP blocks such as accelerators can also now be plugged into these fabrics through a new dedicated low-latency port.
One of the biggest takeaways is that it will finally be possible to combine different core designs together within a cluster. If all of the enabling cache design and scheduling are well-implemented, such CPU designs could theoretically realize significant performance and efficiency improvements. Hopefully a DynamIQ system will not be too much harder to implement for silicon vendors, but I wouldn't assume it will be easy. ARM will really have to make it workable with its stock IP at the very least.
It's hard to say much more about DynamIQ, as ARM is still holding back most of the important details, which I would not really be qualified talk about anyway. There are other announcements such as new CPU instructions for deep learning, which I personally care less about but are still very important for many designs, such as IoT systems without a DSP or GPU. Since Cortex-A53's successor is likely coming at some point, depending on ARM's design goals, I wonder if the first DynamIQ systems will be based on that new core.
It's hard to imagine launching a new mobile OS without support for existing apps. In regards to Fuchsia, I initially thought the Magenta kernel might be compatible with the Android user space. While the potential implementation details for Android support have not been clear, another possibility has become more apparent. I'm not saying anything specific will or will not happen, so please take the following with a grain of salt.
Before starting this blog I spotted some initial comments in Fuschia source about a hypervisor. There was extremely little mentioned, it seemed almost tangential or experimental, and honestly I thought, "nah, that wouldn't make any sense."
Google has since been developing a hypervisor as part of Magenta though. (The first commits were seemingly on the same day as my first blog post.) I have been hesitant to write anything, because you can virtualize anything.
This one difference could imply a ton. If this is what Google is doing, then Fuchsia really is a fully new OS, and I can understand why some people would take offense to the idea of it being a mashup of Android and Chrome OS. Significant amounts of code (Mojo) are derived from Chromium, however, and the ability to run Android apps in VMs could look roughly the same as user space compatibility would to users, at least superficially. Fuchsia will still end up gradually replacing Android and Chrome OS for consumers, while Chrome OS will live on for education.
In this scenario, Google would not be swapping kernels outright, but instead technically would be running two different kernels on a Fuchsia device. (I will again stress that the consumer definition of an OS is not just the kernel either.) The kernel underlying both Fuchsia and Android/Linux would be Magenta. Perhaps using a microkernel would make this a far more manageable or performant approach.
It is entirely possible to run virtual machines like containers, and many companies offer such solutions. Given such an implementation, it is possible to virtualize an individual app without providing an entire second desktop environment for users to manage, despite the app being run on a guest VM.
Hypervisors and containers are not new ideas whatsoever. Fuchsia’s implementation is a Type-1 hypervisor (bare metal) and is currently being built for x86 and MSM8998, which offer hardware-assisted virtualization commonly featured by CPUs. I have basically zero knowledge about virtualization, so there's not much more I can say beyond that.
Running Android on top of a hypervisor would make Android more of a legacy environment than a legacy API for Fuchsia, per se. It would also make Google’s statements that Chrome OS is not going away and that it wouldn’t make any sense for Android and Chrome OS to merge strictly true on a technical level. Again, this still means Fuchsia would crucially provide a stable driver ABI that Linux does not offer.
The massive upside to this approach would be that Magenta would be a clean slate free of the constraints of Linux, Unix, and possibly POSIX (to some degree?). I’m not a kernel expert, but I understand why this would be a huge deal resulting in numerous important technical differences vs. a *nix OS. I’m sure the Fuchsia team would stress the advantages of its decisions. Performance, efficiency, and a million other things could potentially be improved over Linux.
As for the downsides, wouldn’t a hypervisor significantly hurt mobile battery life? Performance and other functional tradeoffs also seem inevitable; virtualization is certainly not costless. But if the downsides are moderate, Fuchsia’s native performance is hopefully unaffected, and the need for virtualization is eliminated long term by Fuchsia replacing Android entirely, these seem like potentially reasonable costs. Without benchmarking, though, there is no hard data upon which to base an opinion.
Relatively seamless Android compatibility is a must in my opinion, because I don’t see how Fuchsia could be so much radically better on its own that consumers and developers would pounce on it otherwise. I’m sure there will be all sorts of carrots and sticks to incentivize writing Flutter apps, but it’s hard to picture Android developers re-writing all of their apps anytime soon. This is the same developer base that Google has struggled to even get to adopt something as crucial as the JobScheduler API to not waste users' battery life, which should have been the OS’ responsibility in the first place.
Google will probably first market Flutter heavily at I/O 2017 as a better, reactive, and cross-platform way of making apps, and then at some point add on “and they’ll run on Fuchsia as well!” It’s hard to imagine wild developer and device vendor enthusiasm for such an approach without making it clear that Android will eventually become legacy. This is due to the network effects implicit with technological platforms. (Unfortunately “network effects” is used as a hand-wavey phrase with little substance behind it, and no one in the tech industry seems familiar with the technical academic details from economics, but that is a discussion for another day.)
There are other considerations at play such as the AOSP vendor ecosystem, and I’m sure Google performed all due diligence in assessing its strategic technical options. There’s also still a lot in flux technically. Mojo IPC was changed, for example, and Mojo itself was seemingly absorbed into Magenta — Fuchsia’s API could end up being called anything regardless. Previous tricky questions such as “how will Fuchsia maintain compatibility with libraries like bionic?” would become irrelevant, though, if Android is simply virtualized. One remaining key question is how Fuchsia and Android apps would interact. This challenge seems extremely important and far from trivial.
As with any of my articles, technical corrections are both encouraged and highly appreciated.
I am very, very against instant takes on these sorts of things in general, but it's hard not to instinctively think this is a mistake. Mobileye's tech is going to be ancient history.
Permalink
The Geneva International Motor Show is going on this week. It’s generally regarded as the greatest auto show in the world, and I would kill to attend it to see the new Italian exotics alone.
One of the biggest headlines this year is the 2017 Porsche GT3. The GT3 has regained a manual option, so all is right again in this world. The GT3 in my opinion sets the standard for all other drivers’ cars, as it vies for Car of the Year awards with alarming consistency.
The above link is a walkthrough of the car by Andreas Preuninger, the head of GT cars at Porsche and one of the world’s leading experts on engineering drivers’ cars. This is unfortunately a rather short interview with Andreas, but here are some gems from the past, describing two phenomenal recent Porsches.
The launch I was most looking forward to, though, was that of Ferrari’s successor to the F12berlinetta, the 812 Superfast. As with the rest of its current line, the 812 was penned in-house by Ferrari, unlike the F12 which was of course co-designed with Pininfarina. The 812’s styling is controversial, though I actually like it in the metal, much to my surprise. I didn’t care for the surface detailing of the F12, so Ferrari has “fixed” its front mid-engined GT in my mind, at least if you’re looking at the rear from above. Aerodynamics is not doing any favors for taillight design these days…
Nintendo's newest system launches tomorrow. By now, there is very little to discuss about the Switch's hardware that has not already been covered in detail online, though I would like to highlight the excellent technical overviews of both Digital Foundry and AnandTech. Ryan Smith also figured out immediately that the Switch actively converts DisplayPort to HDMI. (I don't like Nintendo's docking implementation, for what it's worth.)
If you're familiar with mobile silicon, it wasn't very hard to understand the Switch's hardware. The key points are that it uses a revised NVIDIA Tegra X1 SoC with a Maxwell GPU, still fabbed on TSMC's 20SoC process. 20nm was unequivocally a bad node. Transistor leakage was a massive challenge on the process, which came just before the transition to FinFETs. Essentially this means that the X1 in the Switch is far from competitive in terms of computational efficiency.
Furthermore, in order to fit its power and thermal budgets, Nintendo had to downclock the X1's CPU, GPU, and memory quite considerably to provide reasonable handheld battery life. Resulting performance is not very impressive to say the least. There wasn't much that Nintendo could do, however, since NVIDIA had nothing newer to sell it that could ship within Nintendo's target deadlines.
Some, namely Apple, were able to wrangle 20nm well enough to take advantage of its benefits, but many silicon vendors stumbled severely with it. To see a game console utilize 20SoC is frankly a bit depressing. A lesser problem is that ARM's Cortex-A57 is not exactly a terribly efficient CPU architecture by 2017 standards. The Maxwell GPU, however, did feature a new, more efficient tiling architecture that was perfectly competitive upon its initial release.
Less known to many is that the original X1 shipped in a rather sad state. NVIDIA failed to make a working interconnect with cache coherency, and ended up shipping broken silicon. The end result was that the four LITTLE CPUs had to be disabled, so only the four big CPUs were actually active in the original SHIELD TV and Google's Pixel C. This did not stop NVIDIA from advertising eight functional CPU cores, however.
New to the Switch is a revised X1 chipset, for which NVIDIA probably removed the four LITTLE cores entirely, and likely replaced the broken interconnect with something simpler and hopefully fully functional. This would be the minimal level of fixes that I hope Nintendo demanded. Beyond that, the revised X1 in the Switch and the 2017 SHIELD TV likely features fairly minor improvements. It's possible that there are differences between the two new chipset revisions, but there is no public information available either way.
Updates1) To be clear, even if NVIDIA is calling both chipsets (2015 and 2017) "X1," if it really has removed the LITTLE cores and replaced the interconnect of the latter, it would actually be a new chipset. It's also worth noting that both SHIELD TVs come with 3 GB of LPDDR4 RAM, while the Switch features 4 GB, an important difference.
2) I was wrong: the A53s are still there, and the logic is still broken. Unbelievable. There's no public evidence of what has changed then, though if I had to guess, there might be some semi-custom tweaks to the GPU and not much else. (Those could be important differences, mind you.) Not making any claims!
There is no substitute for color management. Unfortunately, Android has a major color management problem.
Ultra HD (UHD) is a generational advance in consumer display standards. UHD marketing generally refers to the combination of high dynamic range (HDR) output, an expanded color space beyond sRGB, and a minimum resolution of 2160p (4K). While I intend to write an introductory HDR article in the future, today I want to focus on the many technologies required in order to correctly render UHD content.
There are only two color gamuts permitted for devices and content to be certified by the UHD Alliance: DCI-P3 and Rec. 2020. I am specifically referring to the gamuts, not the color spaces. In order to support an HDR-capable panel, a display vendor also has to utilize an HDR-capable DDIC (display driver). And in software, there will be a separate ICC profile specifically for HDR. (HDR for mobile may also require local tone mapping, but I’m not knowledgeable enough to know the details.)
Android gained "HDR" support in Nougat for Android TV. While strictly speaking this is true, I put HDR in quotes because there is no color management. What Google means is that Nougat supports the HDR10, Dolby Vision, and HLG electro-optical transfer functions (EOTFs), aka non-linear gamma. This almost works.
UHD content currently requires a display that targets the P3 color space. If you watch UHD content on a P3 display from, say, Google Play on Android TV, what actually happens is that desaturated frames are presented to the hardware because sRGB is the assumed color gamut, but the TV then oversaturates the image back to the correct P3 color space. The wrong color values get reversed, in other words. The net result will be a correct image, but only on P3 displays. All of the UI controls overlaid on top of the content will, however, be oversaturated.
While LG and Samsung — if LG even manages to correctly calibrate a panel to a target gamut for once — will claim that they are shipping HDR devices with the G6 and Galaxy S8, and the hardware will be technically capable, Android still has no color management. Want to watch a UHD movie in a separate window while multi-tasking with other apps at the same time? Too bad, all of your color and gamma will be wrong. Color (or HDR) modes are not a real solution, despite what Microsoft will tell you.
Extended sRGB, or e-sRGB, is a linear extension of the sRGB color space. This is an available option for some kinds of applications, such as OpenGL apps. There is no substitute for color management, and all apps will require it.
It will be interesting to see whether the LG G6 is calibrated for sRGB or P3, because targeting one inherently compromises on the other. Neither strategy will work perfectly regardless, because Android is stuck on sRGB. sRGB remains the de-facto standard for most content, and this critically includes web content. Without color management, there is simply no way to simultaneously support multiple color gamuts. This is why Samsung calibrates to sRGB and includes a separate P3 color mode, even though almost no consumers will ever know or bother to change color modes. At best, this is a very annoying necessity.
The LG G6 will also suffer from color banding, due to insufficient color gradation steps. Even though its display hardware should be capable of 10-bit output, Snapdragon 821 only has an 8-bit display controller pipeline. Sadly, the iPhone 8 will probably be the first mobile device that can support UHD end-to-end, despite other hardware being HDR-capable dating back to 2016’s Note7.
It is not clear to me if adding color management to Android is even technically possible at this point. But for Fuchsia’s new rendering pipeline, there is probably no excuse not to design it with color management in mind from the outset. Mobile UHD devices require it, if they are ever going to actually work. Don’t let consumers down, Google.
Updates1) I'm extremely happy to say that Romain Guy himself says "there is no reason why color management cannot be added to Android." Awesome! Hopefully these problems will be addressed in Android O then :)
2) Color management is indeed a feature of Android O.
"Deep3DBox is among the best, yet it spots only 74 percent of bikes in the benchmarking test. And though it can orient over 88 percent of the cars in the test images, it scores just 59 percent for the bikes."
As will be a long-running theme on this blog, self-driving cars are much, much more difficult to realize than many people in tech think they are.
Permalink
This is an article worth reading in its entirety. I hope this is enough to cement Dolby Vision's success over the HDR alternatives.
Permalink
In case you weren't already familiar with Milan Milanovic's excellent RF analysis.
Permalink
There are probably several important caveats to this research, but I'm guessing Smash Bros. deep learning papers are going to be extremely common on arXiv from now on.
Permalink
For the inaugural Tech Specs Podcast, I'm joined by Brandon Chester, former Mobile Editor at AnandTech. Topics include: Google's Fuchsia, the Samsung Galaxy S8, the LG G6, a tiny bit about the iPhone 8, OLED design considerations, CES, and Apple going UHD.
1) License Dolby Vision for HDR. (This is optional, but I imagine Apple will push DV when all is said and done if it cares about quality.)
2) Re-enable HEVC decode in future shipping silicon (likely in an existing SoC).*
3) Support HDCP 2.2 output.
4) Launch UHD iTunes. Apple would certainly have been working on encoding a UHD library and providing the necessary infrastructure around that for a long time.
It's unclear if Apple has gotten any further with its licensing negotiations around the HEVC Advance issue. Perhaps there is finally licensing progress? Otherwise, AVC it is. I'm not sure, but I don't believe the UHD Alliance mandates HEVC encoding. AVC encoding would certainly be quite burdensome on bandwidth, and Apple has* waited.
UpdateAs became clear at WWDC 2017, Apple did resolve its HEVC licensing issues.
Since writing my last piece, some Googlers reached out and said my assessment was largely correct. The Andromeda part was clearly wrong, though, so I will try to rectify that.
Based on code like this that was pointed out to me, it looks like Andromeda might be tied to Android's free-form window mode for laptops/2-in-1s and possibly tablets. Though my initial article incorrectly conflated the two independent efforts, Andromeda and Fuchsia will still eventually combine together as a laptop platform. I think the app “chrome” will probably look like Android (because they will be Android or eventually Flutter apps), but with floating windows and elements of or even an overall UI similar to that of Chrome OS. Supported inputs would be mouse or trackpad and keyboard, and possibly touch. Not necessarily wildly different visually from Chrome OS today, in other words, but using the Android API. Andromeda may also explain the “Chrome OS will merge into Android” claim originally made by the Wall Street Journal. The underlying OS will still eventually be Fuchsia, though.
For those wondering about ARC, it was kind of a failed experiment. It sort of worked, but it didn’t really prove to be a workable solution. I really don’t want to get into this any further here. And as stated before, NaCl looks dead.
I think the Android API and runtime will continue to function as before on all Fuchsia devices, except now the underlying OS will be Fuchsia, and the kernel will be Magenta, not Linux. And then there would also be Mojo, Flutter, etc. at least starting on Andromeda devices. It’s hard to imagine pushing both Flutter and the Android API forever, though. Android will likely gradually have to fade away (over many, many years).
Back to the more important point: yes, Fuchsia will be Google’s new OS underpinning all its consumer devices eventually. I think both a “Pixel 3” laptop (or whatever this hypothetical product would be called) and the Pixel 2 smartphone will probably eventually run on top of Fuchsia, but I make zero promises as to anything shipping at any particular point in time, because I have no idea. Still, that is my suspicion based on public commits.
And again, if Google doesn’t break compatibility with the Linux user space, yes, it really can swap out the Android kernel (Linux) for Magenta/Fuchsia, and leave the Android API in its place. Standards like POSIX do exist, after all. Here is some code pointing to exactly that. (Updated thoughts.)
And here as well is some “proof” about Fuchsia replacing Linux (in extremely deliberate scare quotes):
Pink + Purple == Fuchsia. “Pink” is a reference to Apple’s Taligent operating system (which ran legacy Mac OS apps on top of a microkernel). “Purple” is Project Purple, the original iPhone project.
Sure sounds like Fuchsia will run Android apps on smartphones to me! (Yes, engineers working on ambitious special projects love to make witty references, and I think these Apple-related ones are really classy and ace.) Andromeda also happens to be a type of Fuchsia plant, which may be a coincidence. I remember the Purple reference from last year, but didn’t know what Pink referred to at the time. And I never thought to connect these references to Fuchsia being able to run Android apps on a modern smartphone platform. Major kudos to Zargh for connecting the dots! (Click the links for his explanation of the references in greater detail, but I don't really want to draw even more attention to the engineers for the sake of their privacy.)
And as someone on Hacker News once again commented: “ANDROid + chROME + DArt = ANDROMEDA?” I would doubt the Dart part though.
Lastly, I will again note efforts such as A/B (seamless) updates that Google added in Nougat, which may help out with the Fuchsia transition. Crazier things have happened. Samsung once migrated Galaxy Gear owners from Android to Tizen, for one.
To clarify: I unfortunately used Andromeda equivalently with Fuchsia in this article. They are independent things, but this article is entirely about Fuchsia. Since it is an Android effort, however, Andromeda could eventually be built on top of the same OS (kernel) for phones, laptops, etc. - Fuchsia. I make no claims about release timing or what the final marketing names will be. Please replace Andromeda with Fuchsia in your head while reading this article. For a more recent and much more accurate summary of Fuchsia, please see this article.
I decided to dig through open source to examine the state of Google’s upcoming Andromeda OS. For anyone unfamiliar, Andromeda seems to be the replacement for both Android and Chrome OS (cue endless debates over the semantics of that, and what it all entails). Fuchsia is the actual name of the operating system, while Magenta is the name of the kernel, which specifically is a microkernel. Many of the architectural design decisions appear to have unsurprisingly been focused on creating a highly scalable platform.
It goes without saying that Google isn’t trying to hide Fuchsia. People have clearly discovered that Google is replacing Android’s Linux kernel. Still, I thought it would be interesting for people to get a better sense of what the OS actually is. This article is only intended to be an overview of the basics, as far as I can comment reasonably competently. (I certainly never took an operating systems class!)
To my naive eyes, rather than saying Chrome OS is being merged into Android, it looks more like Android and Chrome OS are both being merged into Fuchsia. It’s worth noting that these operating systems had previously already begun to merge together to an extent, such as when the Android team worked with the Chrome OS team in order to bring Update Engine to Nougat, which introduced A/B updates to the platform.
Google is unsurprisingly bringing up Andromeda on a number of platforms, including the humble Intel NUC. ARM, x86, and MIPS bring-up is exactly what you would expect for an Android successor, and it also seems clear that this platform will run on Intel laptops. More on this later.
My best guess is that Android as an API and runtime will live on as a legacy environment within Andromeda. That’s not to say that all development of Android would immediately stop, which seems extremely unlikely. But Google can’t push two UI APIs as equal app frameworks over the long term: Mojo is clearly the future.
Ah, but what is Mojo? Well it’s the new API for writing Andromeda apps, and it comes from Chromium. Mojo was originally created to “extract a common platform out of Chrome's renderer and plugin processes that can support multiple types of sandboxed content.” It seems to have enabled Android apps in Chrome OS, and now it will serve an even more extensive role as the developer API for Andromeda. (Sidenote: as far as I can tell, Native Client is well and truly dead. Pour one out for the valiant effort that was NaCl.)
Mojo in Fuchsia features intriguingly extensive language support. C/C++, Dart, Go, Java, Python, and Rust all have bindings to Mojo. I am guessing that C/C++ is for native development, Go is for networking, Java is for Android(?), Python is for scripting, and Rust is for writing portions of the kernel. (Or perhaps Rust's usage is minimal, suggests a commenter on Hacker News.) Mixing and matching languages aside, the main UI API is based on, yes, Dart.
Flutter is an existing Google widget framework for apps written in Dart, and it has been repurposed to become the UI framework for Andromeda. Flutter includes a series of Material Design widgets and was engineered to render apps up to 120fps.* I imagine Andromeda’s standard UI components will look similar if not identical to those of Android. A physically based renderer, Escher, is apparently being used to render, well, material elements and shadows in a high quality manner.
I have very strong reservations about Dart as the language of the primary platform API, but it’s best to wait for the fully revealed details before forming an opinion. The reason for Dart is obvious, however: to enable a cross-platform app framework. The pitch will clearly be that developers can write a Flutter app once and have it run on Andromeda, Android, and iOS with minimal extra work, in theory. (Flutter does not target the web.) Even if Andromeda is its full replacement, Android will be a separate developer target for many years given its gigantic installed base.
Andromeda's actual app runtime is called Modular, “a post-API programming model that allows applications to cooperate in a shared context without the need to call each other's APIs directly." To do this it uses Mojo inter-process communication (IPC) messages, which are exchanged via low-level primitives in the form of message pipes (small amounts of data), data pipes (large amounts of data), and shared buffers.
I'm not technically knowledgeable enough to understand how the various languages interface over these IPC calls, and what exactly that enables. The IDL used is the Fuchsia Interface Description Language (FIDL), “an encoding format used to describe [Mojo] interfaces to be used on top of Magenta message pipes. They are the standard way applications talk to each other in Fuchsia.” Right now at least, only C/C++, Dart, and Go have supported bindings. Dart is thus the main platform language. (My understanding is that Go is not exactly ideal for writing UI apps.)
Why Andromeda?Observers have naturally wondered for years if Google would ever unify Android and Chrome OS, in line with Sundar's obvious desires. Despite some questionable half-measures along the way, this will now finally take place. Andromeda will immensely help Google bring together and unify a broad array of in-house technologies, beyond just its two consumer OSes.
Andromeda clearly serves Google’s own purposes. The only platforms that the company really supports are the web, Android, and iOS, in that order. I think Windows support is effectively limited to Chrome at this point. Flutter exactly exemplifies this strategy. Google will no longer have to field separate Android and iOS apps and teams, and can now greatly focus its app development efforts. "More wood behind fewer arrows," in other words.
ObservationsAs you would expect, Google submitted patches for Fuchsia to both LLVM and GCC.
Interestingly web rendering is currently based on WebKit, but this is simply “meant as a stopgap until Chromium is ported to Fuchsia.”
There is a machine learning/context framework baked into Fuchsia, which seems conceptually similar to the existing Google Awareness API.
QuestionsWhat happens to the Android Open Source Project and its huge ecosystem of partners?
Does this platform only ship on laptops for its initial release?
Will Android Studio be the basis for Andromeda’s IDE? If so, ouch. IDEs written in Java are wildly slow…
How do you replace the enormous contributions of the Linux kernel and Linaro? It seems like Google might preserve compatibility with the Linux user space to make its new OS a tractable effort in the first place, such as by supporting ELF binaries. But what about existing massive projects, like an Energy Aware Scheduler? Or will Google continue to utilize much of its existing Linux code for now?
Progressing the PCI can think of more than a few (read: one hundred) reasons why you would want to replace Android. I will highlight only one: to completely rewrite the rendering pipeline. The market for Chrome OS meanwhile is of course mostly limited to education, not to diminish it. Andromeda, however, will provide a laptop OS with native apps and backwards compatibility with Android. The general UI could very well look much the same visually as Chrome OS does now. I also have to imagine the Android update problem (a symptom of Linux's lack of a stable driver ABI) will at last be solved by Andromeda (minus the carriers), but one can never be too sure.
The promise of a laptop platform that can bring all the advances of mobile, bereft of the vestiges of PC legacy, while also embracing proven input and interface paradigms is extremely appealing. And since Apple has only inched macOS along in recent years despite its decades of cruft and legacy, I welcome Google’s efforts wholeheartedly. Hopefully 2017 will finally be the beginning of the new PC.
UpdateSince I've gotten a ton of questions about Fuchsia being a mobile platform, here is some clear evidence:
Via the Magenta repository: "Magenta targets modern phones and modern personal computers with fast processors, non-trivial amounts of ram (sic) with arbitrary peripherals doing open ended computation."
And here is Google working on Fuchsia bring-up on Snapdragon 835 (MSM8998), which doesn't really fit the IoT segment since mobile SoCs stipulate virtual memory and a memory management unit (MMU). If this is for tablets, why not just use Android?
A commenter on Hacker News has also pointed out an Atheros Wi-Fi driver and an Intel GPU driver. There are tons of similar examples in the Fuchsia source.
Notes
-
I am not a programmer, so if anything stated above is incorrect, please, please send me corrections. I would also greatly appreciate any clarifying comments, which were one of my primary motivations for writing this piece.
-
For anyone interested, I intend to write quite often about consumer technology on this blog. Topics will include hardware, software, design, and more. You can follow via RSS or Twitter, and possibly through other platforms soon.
-
At least in trivial cases. I don’t see the average garbage-collected language using a virtual machine allowing for a target higher than 60fps realistically. (Yes, it is JIT'ed, and the Dart team is working on Ahead-of-Time compilation.)