Hello, dear readers! Today's article describes Ark, a newJavaScript-based mobile development platform. If you haven't read themyet, you might want to start by having a look at my past articles onCapacitor,ReactNative,NativeScript,andFlutter;having a common understanding of the design space will help usunderstand where Ark is similar and where it differs.
If I had to bet, I would guess that you have not heard of Ark. (Icertainly hadn't either, when commissioned to do this research series.)To a first approximation, Ark—or rather, what I am calling Ark; I don'tactually know the name for the whole architecture—is a looselyFlutter-like UI library implemented on top of a dialect of JavaScript,with build-time compilation to bytecode (like Hermes) but also withsupport for just-in-time and ahead-of-time compilation of bytecode tonative code. It is made by Huawei.
At this point if you are already interested in this research series, Iam sure this description raises more questions than it answers.Flutter-like? A dialect? Native compilation? Targetting whatplatforms? From Huawei? We'll get to all of these, but I think weneed to start with the last question.
In my last article on Flutter, I told a kind of just-so history of howDart and Flutter came to their point in the design space. Thanks tocorrections from a kind reader, it happened to also be more or lesscorrect. In this article, though, I haven't talked with Ark developersat all; I don't have the benefit of a true claim on history. And yet,the only way I can understand Ark is by inventing a narrative, so herewe go. It might even be true!
Recall that in 2018, Huawei was a dominant presence in the smartphonemarket. They were shipping excellent hardware at good prices both tothe Chinese and to the global markets. Like most non-Apple, non-Googlemanufacturers, they shipped Android, and like most Android OEMs, theyshipped Google's proprietary apps (mail, maps,etc.).
But then, over the next couple years, the US decided that allowingHuawei to continue on as before was, like, against national securityinterests or something. Huawei was barred from American markets, anumber of suppliers were forbidden from selling hardware components toHuawei, and even Google was prohibited from shipping its mobile apps onHuawei devices. The effect on Huawei's market share for mobile deviceswas enormous: its revenue was cut in half over a period of a coupleyears.
In this position, as Huawei, what do you do? I can't even imagine, butspecifically looking at smartphones, I think I would probably do aboutwhat they did. I'd fork Android, for starters, because that's what youalready know and ship, and Android is mostly open source. I'd probablyplan on continuing to use its lower-level operating system piecesindefinitely (kernel and so on) because that's not a valuedifferentiator. I'd probably ship the same apps on top at first,because otherwise you slip all the release schedules and lose revenueentirely.
But, gosh, there is the risk that your product will be perceived as justa worse version of Android: that's not a good position to be in. Youneed to be different, and ideally better. That will take time. In themeantime, you claim that you're different, without actually beingdifferent yet. It's a somewhat ridiculous position to be in, but I canunderstand how you get here; Ars Technica published a scathingreviewpoking fun at the situation. But, you are big enough to ride it out,knowing that somehow eventually you will be different.
Up to now, this part of the story is relatively well-known; the partthat follows is more speculative on my part. Firstly, I would note thatHuawei had been working for a while on a compiler and language run-timecalled ArkCompiler,with the goal of getting better performance out of Android applications.If I understand correctly, this compiler took the Java / Dalvik /Android Run Time bytecodes as its input, and outputted native binariesalong with a new run-time implementation.
As I can attest from personal experience, having a compiler leads tohubris: you start to consider source languages like a hungry personlooks at a restaurant menu. "Wouldn't it be nice to ingest that?"That's what we say at restaurants, right, fellow humans? So in 2019 and2020 when the Android rug was pulled out from underneath Huawei, I thinkhaving in-house compiler expertise allowed them to consider whether theywanted to stick with Java at all, or whether it might be better tochoose a more fashionable language.
Like black, JavaScript is always in fashion. What would it mean,then, to retool Huawei's operating system -- by then known by the name"HarmonyOS" -- to expose a JavaScript-based API as its primary appdevelopment framework? You could use your Ark compiler somehow toimplement JavaScript (hubris!) and then you need a UI framework. Havingditched Java, it is now thinkable to ditch all the other Androidstandard libraries, including the UI toolkit: you start anew, in a way.So are you going to build a Capacitor, a React Native, a NativeScript, aFlutter? Surely not precisely any of these, but what will it be like,and how will it differ?
Incidentally, I don't know the origin story for the name Ark, but to meit brings to mind tragedy and rebuilding: in the midst of being cut offfrom your rich Android ecosystem, you launch a boat into the sea,holding a promise of a new future built differently. Hope and hubris inone vessel.
In the end, Huawei builds two things: something web-like and somethinglike Flutter. (I don't mean to suggest copying or degeneracy here; it'srather that I can only understand things in relation to other things,and these are my closest points of comparison for what they built.)
The web-like programming interface specifies UIs using an XML dialect,HML,and styles the resulting node tree with CSS. You augment these nodeswith JavaScript behavior; the main app is a set of DOM-like eventhandlers.There is an API to dynamically create DOMnodes,but unlike the other systems we have examined, the HarmonyOSdocumentation doesn't really sell you on using a high-level frameworklike Angular.
If this were it, I think Ark would not be so compelling: the programmingmodel is more like what was available back in the DHTMLdays. I wouldn't expectpeople to be able to make rich applications that delight users, giventhese primitives, though CSS animation and the HML loop and conditionalrenderingfrom the template system might be just expressive enough for simpleapplications.
The more interesting side is the so-called "declarative" UI programmingmodel which exposes a Flutter/React-like interface. The programmerdescribes the "what" of the UI by providing a tree of UI nodes in itsbuild function, and the framework takes care of calling build whennecessary and of rendering that tree to the screen.
Here I need to show some example code, because it is... weird. Well, Ifind it weird, but it's not too far fromSwiftUI in flavor. Asmall example from the finemanual:
@Entry@Componentstruct MyComponent { build() { Stack() { Image($rawfile('Tomato.png')) Text('Tomato') .fontSize(26) .fontWeight(500) } }}The @Entry decorator (*) marks this struct (**) as being the mainentry point for the app. @Component marks it as being a component,like a React functional component. Components conform to an interface(***) which defines them as having a build method which takes noarguments and returns no values: it creates the tree in a somewhatimperative way.
But as you see the flavor is somewhat declarative, so how does thatwork? Also, build() { ... } looks syntactically a lot like Stack() { ... }; what's the deal, are they the same?
Before going on to answer this, note my asterisks above: these areconcepts that aren't in JavaScript. Indeed, programs written forHarmonyOS's declarative framework aren't JavaScript; they are in adialect of TypeScript that Huawei calls ArkTS. In this case, aninterface is a TypeScriptconcept.Decorators would appear to correspond to an experimental TypeScriptfeature,looking at the source code.
But struct is an ArkTS-specificextension,and Huawei has actually extended the TypeScript compiler to specificallyrecognize the @Component decorator, such that when you "call" astruct, for example as above in Stack() { ... }, TypeScript will parsethat as a new expression typeEtsComponentExpression,which may optionally be followed by a block. When Stack() is invoked,its children (instances of Image and Text, in this case) will bepopulated via running the block.
Now, though TypeScript isn't everyone's bag, it's quite normalized in theJavaScript community and not a hard sell. Language extensions like the handling of @Componentpose a more challenging problem. Still, Facebook managed to sell peopleon JSX, so perhaps Huawei can do the same for their dialect. More onthat later.
Under the hood, it would seem that we have a similar architecture toFlutter: invoking the components creates a corresponding tree ofelements (as with React Native's shadow tree), which then are loweredto render nodes, which draw themselves onto layers using Skia, in amulti-threaded rendering pipeline. Underneath, the UI code actuallyre-uses some parts of Flutter, though from what I can tellHarmonyOS developers are replacing those over time.
So we see that the source language for the declarative UI framework isTypeScript, but with some extensions. It also has its restrictions, andto explain these, we have to talk about implementation.
Of the JavaScript mobile application development frameworks wediscussed, Capacitor and NativeScript used "normal" JS engines from web browsers, whileReact Native built their own Hermes implementation. Hermes is alsorestricted, in a way, but mostly inasmuch as it lags the browser JSimplementations; it relies on source-to-source transpilers to get accessto new language features. ArkTS—that's the name of HarmonyOS's"extended TypeScript" implementation—has more fundamental restrictions.
Recall that the Ark compiler was originally built for Android apps.There you don't really have the ability to load new Java or Kotlinsource code at run-time; in Java you have class loaders, but those loadbytecode. On an Android device, you don't have to deal with the Javasource language. If we use a similar architecture for JavaScript,though, what do we do about eval?
ArkTS's answer is: don't. As in, eval is not supported on HarmonyOS.In this way the implementation of ArkTS can be divided into two parts, afrontend that produces bytecode and a runtime that runs the bytecode,and you never have to deal with the source language on the device wherethe runtime is running. Like Hermes, the developer produces bytecodewhen building the application and ships it to the device for the runtimeto handle.
Incidentally, before we move on to discuss the runtime, there areactually two front-ends that generate ArkTS bytecode: one written inC++ that seems to only handle standard TypeScript andJavaScript,and one written in TypeScript that also handles "extendedTypeScript".The former has a test262 runner with about 10k skippedtests,and the latter doesn't appear to have a test262 runner. Note, I haven'tactually built either one of these (or any of the other frameworks, forthat matter).
The ArkTSruntime isitself built on a non-language-specific common Arkruntime, andthe set of supported instructions is the union of the coreISAand the JavaScript-specificinstructions.Bytecode can beinterpreted,JIT-compiled, or AOT-compiled.
On the side of design documentation, it's somewhat sparse. There aresome core designdocs;readers may be interested in the rationale to use a bytecodeinterfacefor Ark as a whole, or the optimizationoverview.
Indeed ArkTS as a whole has a surfeit of optimizations, to an extentthat makes me wonder which ones are actually needed. There aresource-to-source optimizations onbytecode,which I expect are useful if you are generating ArkTS bytecode fromJavaScript, where you probably don't have a full compilerimplementation. There is a completely separateoptimizerin the eTS part of the run-time, based on what would appear to be anovel "circuit-based"IRthat bears some similarity to sea-of-nodes. Finally the whole thingappears to bottom out inLLVM,which of course has its own optimizer. I can only assume that thissituation is somewhat transitory. Also, ArkTS does appear to generateits own native code sometimes, notably for inline cache stubs.
Of course, when it comes to JavaScript, there are some fundamentallanguage semantics and there is also a large and growing standardlibrary. In the case of ArkTS, this standard library is part of therun-time,like the interpreter, compilers, and the garbage collector(generational concurrent mark-sweep with optionalcompaction).
All in all, when I step back from it, it's a huge undertaking.Implementing JavaScript is no joke. It appears that ArkTS has done thefirst 90% though; the proverbial second 90% should only take a few moreyears :)
If you told a younger me that a major smartphone vendor switched fromJava to JavaScript for their UI, you would probably hear me react interms of the relative virtues of the programming languages in question.At this point in my career, though, the only thing that comes to mind iswhat an expensive proposition it is to change everything about anapplication development framework. 200 people over 5 years would be myestimate, though of course teams are variable. So what is it that wecan imagine that Huawei bought with a thousand person-years ofinvestment? Towards what other local maximum are we heading?
I didn't mention it before, but it would seem that one of the goals ofHarmonyOS is in the name: Huawei wants to harmonize development acrossthe different range of deployment targets. To the extent possible, itwould be nice to be able to write the same kinds of programs for IoTdevices as you do for feature-rich smartphones and tablets and the like.In that regard one can see through all the source code how there is aculture of doing work ahead-of-time and preventing work at run-time; forexample see the design doc for theinterpreter,or for the fileformat,or indeed the lack of JavaScript eval.
Of course, this wide range of targets also means that the HarmonyOSplatform bears the burden of a high degree of abstraction; not only canyou change the kernel, but also the JavaScript engine (usingJerryScript on "lite" targets).
I mention this background because sometimes in news articles and indeedofficial communication from recent years there would seem to be someconfusion that HarmonyOS is just for IoT, or aimed to be super-small, orsomething. In this evaluation I am mostly focussed on the feature-richside of things, and there my understanding is that the developer willgenerate bytecode ahead-of-time. When an app is installed on-device,the AOT compiler will turn it into a single ELF image. This shouldgenerally lead to fast start-up.
However it would seem that the renderinglibrarythat paints UI nodes into layers and then composits those layers usesSkia in the way that Flutter did pre-Impeller, which to be fair is aquite recent change to Flutter. I expect therefore that Ark (ArkTS +ArkUI) applications also experience shader compilation jank at startup,and that they may be well-served by tesellating their shapes intoprimitives like Impeller does so that they can precompile a fixed,smaller set of shaders.
Maybe it's just that apparently I think Flutter is great, but ArkUI'sfundamental architectural similarity to Flutter makes me think that jankwill not be a big issue. There is a render thread that is separate fromthe ArkTS thread, so like with Flutter, async communication withmain-thread interfaces is the main jank worry. And on the ArkTS side,ArkTS even has a number of extensions to be able to share objectsbetween threads without copying, should that be needed. I am not surehow well-developed and well-supported these extensions are, though.
I am hedging my words, of course, because I am missing a bit of socialproof; HarmonyOS is still in infant days, and it doesn't have much inthe way of a user base outside China, from what I can tell, and myability to read Chinese is limited to what Google Translate can do forme :) Unlike other frameworks, therefore, I haven't been as able tocatch a feel of the pulse of the ArkUI user community: what people arehappy about, what the pain points are.
It's also interesting that unlike iOS or Android, HarmonyOS is onlyexposing these "web-like" and "declarative" UI frameworks for appdevelopment. This makes it so that the same organization is responsiblefor the software from top to bottom, which can lead to interestingcross-cutting optimizations: functional reactive programming isn't justa developer-experience narrative, but it can directly affect the shapeof the rendering pipeline. If there is jank, someone in the building isresponsible for it and should be able to fix it, whether it is in theGPU driver, the kernel, the ArkTS compiler, or the application itself.
I don't know how to evaluate ArkTS for peak performance. Although thereis a JIT compiler, I don't have the feeling that it is as tuned foradaptive optimization as V8 is.
At the same time, I find it interesting that HarmonyOS has chosen tomodify JavaScript. While it is doing that, could they switch to a soundtype system, to allow the kinds of AOT optimizations that Dart can do?It would be an interesting experiment.
As it is, though, if I had to guess, I would say that ArkTS iswell-positioned for predictably good performance with AOT compilation,although I would be interested in seeing the results of actually runningit.
In this series I have tried to be charitable towards the frameworks thatI review, to give credit to what they are trying to do, even whilenoting where they aren't currently there yet. That's part of why I needa plausible narrative for how the frameworks got where they are, becausethat lets me have an idea of where they are going.
In that sense I think that Ark is at an interesting inflection point.When I started reading documentation about ArkUI and HarmonyOS and allthat, I bounced out—there were too many architectural boxdiagrams, too many generic descriptions of components, too many promiseswith buzzwords. It felt to me like the project was trying to justifyitself to a kind of clueless management chain. Was there actuallyanything here?
But now when I see the adoption of a modern rendering architecture and abold new implementation of JavaScript, along with the willingness toexperiment with the language, I think that there is an interesting storyto be told, but this time not to management but to app developers.
Of course you wouldn't want to market to app developers when yoursystem is still a mess because you haven't finished rebuilding an MVPyet. Retaking my charitable approach, then, I can only think that allthe architectural box diagrams were a clever blind to avoid piquing outsideinterest while the app development kit wasn't readyyet :) As and when the system starts working well, presumably over thenext year or so, I would expect HarmonyOS to invest much more heavily inmarketing and developer advocacy; the story is interesting, but you haveto actually tell it.
All of the previous app development frameworks that we looked at werecross-platform; Ark is not. It could be, of course: it does appear tobe thoroughly open source. But HarmonyOS devices are the main target.What implications does this have?
A similar question arises in perhaps a more concrete way if we startwith the mature Flutter framework: what would it mean to make a Flutterphone?
The first thought that comes to mind is that having a Flutter OS wouldallow for the potential for more cross-cutting optimizations that crossabstraction layers. But then I think, what does Flutter really need?It has the GPU drivers, and we aren't going to re-implement those. Ithas the bridge to the platform-native SDK, which is not such a large andimportant part of the app. You get input from the platform, but that'salso not so specific. So maybe optimization is not the answer.
On the other hand, a Flutter OS would not have to solve themake-it-look-native problem; because there would be no other "native"toolkit, your apps won't look out of place. That's nice. It's notsomething that would make the platform compelling, though.
HarmonyOS does have this embryonic concept of app mobility, where likeyou could put an app from your phone on your fridge, or something.Clearly I am not doing it justice here, but let's assume it's acompelling use case. In that situation it would be nice for all devicesto present similar abstractions, so you could somehow install the sameapp on two different kinds of devices, and they could communicate totransfer data. As you can see here though, I am straying far from mydomain of expertise.
One reasonable way to "move" an app is to have it stay running on yourphone and the phone just communicates pixels with your fridge (orwhatever); this is the low-level solution. I think HarmonyOS appears tobe going for the higher-level solution where the app actually runs logicon the device. In that case it would make sense to ship UI assets andJavaScript / extended TypeScript bytecode to the device, which would runthe app with an interpreter (for low-powered devices) or use JIT/AOTcompilation. The Ark runtime itself would live on all devices,specialized to their capabilities.
In a way this is the Apple WatchOS solution (as I understand it);developers publish their apps as LLVM bitcode, and Apple compiles it forthe specific devices. A FlutterOS with a Flutter run-time on alldevices could do something similar. As with WatchOS you wouldn't haveto ship the framework itself in the app bundle; it would be on thedevice already.
Finally, publishing apps as some kind of intermediate representationalso has security benefits: as the OS developer, you can ensure someinvariants via the toolchain that you control. Of course, you would have to ensurethat the Flutter API is sufficiently expressive for high-performanceapplications, while also not having arbitrary machine code executionvulnerabilities; there is a question of language and framework design aswell as toolchain and runtime quality of implementation. HarmonyOScould be headed in this direction.
Ark is a fascinating effort that holds much promise. It's also still inmotion; where will it be when it anneals to its own local optimum? Itwould appear that the system is approaching usability, but I expect adegree of churn in the near-term as Ark designers decide which languageabstractions work for them and how to, well, harmonize them with therest of JavaScript.
For me, the biggest open question is whether developers will love Ark inthe way they love, say, React. In a market where Huawei is still adominant vendor, I think the material conditions are there for a gooddeveloper experience: people tend to like Flutter and React, and Ark issimilar. Huawei "just" needs to explain their framework well (and whereit's hard to explain, to go back and change it so that it isexplainable).
But in a more heterogeneous market, to succeed Ark would need to make across-platform runtime like the one Flutter has and engage in someserious marketing efforts, so that developers don't have to limitthemselves to targetting the currently-marginal HarmonyOS. Sellingextensions to JavaScript will be much more difficult in a context wherethe competition is already established, but perhaps Ark will be able toproductively engage with TypeScript maintainers to move the language so itcaptures some of the benefits of Dart that facilitate ahead-of-timecompilation.
Well, that's it for my review round-up; hope you have enjoyed theseries. I have one more pending article, speculating about some futuretechnologies. Until then, happy hacking, and see you next time.