David Rousset: Recent Episodes

View Details

Saviez-vous que la quasi-totalité des constructeurs (Denon, Onkyo, Yamaha & co) vous mentent sur la puissance réelle de leur ampli intégré audio/vidéo ? Allez, je prends le risque de me lancer pour tenter d’expliquer pourquoi et nous enchainerons ensuite sur l’intérêt des blocs de puissance. Cela fait suite à mon précédent article : Différences de qualité d’image ou de son entre 2 câbles HDMI ? où je m’amuse à partager les choses rigolotes que j’ai apprises pendant de nombreuses années à réaliser mes systèmes home cinéma.

Donc nous allons voir pourquoi quasiment toutes les marques vous mentent sur la puissance annoncée de leurs intégrés. Je vais également tâcher de partager les caractéristiques importantes à surveiller pour faire la différence entre les différents blocs de puissance. Merci aux pro et aux passionnés avertis de commenter et de corriger les éventuelles erreurs ou imprécisions que je pourrais faire, à l’aide d’informations sourcées bien sûr.

Les intégrésCommençons par les intégrés. Comme leur nom l’indique, ils contiennent tout : une section pré-ampli, un écran d’affichage en façade, la carte HDMI, la carte réseau ainsi que des étages d’amplification. L’unique alimentation va donc devoir servir tout ce beau monde.

Par ailleurs, il faut revenir aux chiffres importants sur la manière de mesurer la puissance d’un ampli. La norme admise est de mesurer 2 canaux actifs sous 8 ohms, les gauche / droite, le tout soit sur une plage audio complète audible par l’être humain (20 Hz – 20 kHz) soit sur une fréquence donnée de medium (1 kHz), le tout avec une distorsion contenue.

Déjà, il faut savoir qu’il est beaucoup plus facile de donner de la puissance à 1 kHz que sur la plage complète utilisée en conditions normales. Sauf que beaucoup de marques (comme Denon, Yamaha, Marantz & co) trichent déjà à ce niveau en annonçant des produits soi-disant 7x160W (voir plus) mais ensuite donne la valeur de 160W sous 2 canaux actifs (voir parfois 1 seul !) à une fréquence particulière de 1 kHz avec un taux de distorsion important.

Alors que ce qui vous intéresse, c’est sa capacité à fournir suffisamment de puissance réelle (RMS), à faible distorsion sur l’ensemble des enceintes connectées, soit moins de 1% pour les moins bons et moins de 0.1% pour les meilleurs.

Démonstration par l’exemple avec un Denon « moyen de gamme », le 3800h : https://www.denon.com/fr-fr/product/av-receivers/avc-x3800h mais vous pouvez faire le même exercice avec votre ampli HC. Le Denon annonce fièrement 9 canaux de 180W. On peut légitimement penser que c’est énorme et largement suffisant pour driver la plupart des enceintes. Creusons un peu les spécifications.

Les 180W annoncés sont en fait sous 6 ohms (et donc pas 8 comme normalement attendu), à 1 kHz avec un seul canal actif le tout avec un taux de distorsion énorme de 10% ! Bref, une autre façon de le dire : il est peut-être capable d’envoyer un son horrible jusqu’à 180W si une seule enceinte est branchée, et c’est même potentiellement dangereux pour votre enceinte. Cette information est donc débile et inutile. Une annonce plus réaliste partagée dans les specs est qu’il affiche en fait 105W avec 2 canaux avec un taux de distorsion correct sous 8 ohms et une plage de fréquence complète. Continuons de creuser les spécifications.

On découvre que l’appareil consomme jusqu’à 660W. Je vous rappelle que c’est un intégré. Donc ces 660W vont être partagées entre les différentes sections de l’intégré ainsi qu’avec les étages de puissance. Pour finir, une partie de cette énergie est perdue en chaleur. Soyons super généreux et imaginons que seulement 30% de l’énergie soit perdue en chaleur (c’est en fait plus), il reste 462W utilisable. Imaginons que 20% soit dédié aux autres sections de l’intégré, il reste 370W pour les enceintes. Si les 9 canaux sont actifs, il faut potentiellement diviser ces 370W par 9 soit… 41W par enceintes. On est bien loin des 9x180W affichés par la marque ! Sachant que c’est plus compliqué que cela car il faudrait mesurer sur un signal propre. Pousser l’ampli à fond va générer de la distorsion et faire un signal écrêté qui peut détruire une enceinte.

Prenons maintenant un bon élève : Arcam. J’ai eu moi-même un Arcam AVR10 par le passé (mais j’aurai pu faire la même démonstration avec d’autres marques tout aussi sérieuses comme Nad ou Rotel) bien plus cher que le Denon 3800. Voici sa fiche technique : ARCAM AVR10 Spec.pdf.

On y découvre que non seulement la puissance sous 8 ohms, à faible distorsion et sur la plage complète de fréquence n’est « que » de 80W mais qu’avec 7 canaux actifs (avec un signal 1 kHz et un taux de distorsion plus élevé), nous sommes désormais à 65W. C’est déjà nettement plus honnête et réaliste même si ces chiffres semblent moins flatteurs que les chiffres bidons annoncés par Denon. Pour finir, la consommation maximum de l’Arcam est de… 1500W ! Au lieu des 660W affichés sur le Denon pour soi-disant 9x180W ! Arrivé à ce stade, j’imagine que vous aurez compris que l’on a des « vrais Watts » d’un côté et un vrai mytho de l’autre

On imagine aussi que l’on doit être sûrement encore en dessous de 41W pour le Denon tous canaux actifs.

Déjà à ce niveau, vous voyez que l’on s’est déjà tous fait avoir par le marketing. Je trouve même que nous sommes très proches de la publicité mensongère. Et encore, je n’ai même par parlé de puissance en crête (max) ou puissance RMS… Pour résumé, faites attention aux valeurs et allez creuser le reste des specs !

En revanche, pour contre balancer, j’aimerai ajouter 2 choses importantes pour équilibrer ce constat :

– En fonction des enceintes utilisées (de petites biblio par exemple) et de leur nombre (un petit 5.1 par exemple), un intégré peut déjà être largement suffisant. Encore plus dans une petite pièce. Cependant, n’espérez pas correctement alimenter un système Atmos 7.1.4 avec un intégré.

– La puissance totale consommée est rarement la puissance max potentielle mais une puissance un peu « standard » ce qui nous empêche d’avoir une conclusion précise sur la vraie puissance au final de l’intégré. Pour cela, il faut du matériel spécial pour mesurer la sortie des amplis. Je vous invite à lire cet article de Audioholics comme toujours excellent à ce sujet : Receiver Power Consumption Rating vs Output Power Is Not Watt You Think! | Audioholics

Quelques articles et sources en complément :

– Puissance des amplificateurs et des enceintes : comment ne pas risquer l’endommagement (2020) – Le blog de Son-Vidéo.com (son-video.com)
– AV Receiver & Amplifier Power Ratings Explained | Home Cinema Guide (the-home-cinema-guide.com)
– Understanding Amplifier Power Output Specifications (lifewire.com)

Pour continuer, il est communément admis qu’il faut en général une puissance environ 2 fois supérieure à la puissance annoncées des enceintes. C’est effectivement moins dangereux d’avoir un ampli plus puissant que ses enceintes que l’inverse. Je sais cela peut paraitre contre intuitif mais cela est expliqué dans l’article de Son Video au-dessus.

Or, malheureusement, nous sommes beaucoup à commettre l’erreur de prendre des enceintes bien trop grosses (souvent des colonnes en LR et une centrale rikiki…), surdimensionnées par rapport à nos besoins et surtout à l’ampli. D’ailleurs, j’en profite pour rappeler que l’idéal en HC est d’avoir 3 fois la même enceinte en LCR et que des 2 voies sont souvent largement suffisantes, les basses étant gérées par le ou les caissons. Malheureusement, je vois bien trop souvent des systèmes avec des colonnes énormes et une centrale toute ridicule à côté. Les gens se plaignent alors de ne pas entendre correctement les dialogues. L’enceinte centrale est pourtant l’enceinte la plus importante en home cinéma, c’est sans aucun doute l’enceinte la plus sollicitée.

J’en profite aussi pour faire un autre rappel : plus l’on descend dans le bas du spectre des fréquences, plus cela consomme de l’énergie. Pour la faire courte, générer des basses demande bien plus d’énergie que des aigus. C’est donc surtout là que les blocs de puissance peuvent aider. Et c’est aussi pour ça que c’est dramatique de demander à un pauvre intégré de piloter vos colonnes configurées en « Large » car il n’a déjà pas beaucoup de puissance en réserve comme nous l’avons vu plus haut, vous l’essoufflez encore davantage plutôt que de laisser le caisson de basses faire son travail.

Les blocs de puissanceC’est pour toutes ces raisons que les blocs de puissance sont assez populaires et justifiés lorsque les enceintes commencent à être gourmandes ou trop nombreuses. L’intérêt du bloc de puissance est d’avoir une alimentation bien plus costaud permettant de soulager l’alimentation de l’intégré. Il semble admis qu’il faut au moins alimenter LR voir le LCR (Left Center Right) en bloc, surtout si des colonnes sont présentes. En effet, 80% de l’information sonore en HC vient de la scène avant, particulièrement de la centrale. Il faut donc les bichonner.

Petite pause. Tout cela va bien entendu dépendre de votre budget et vos attentes. Vous pouvez parfaitement vous satisfaire d’un intégré, ce fut mon cas pendant des années. Ayez juste conscience de sa « vraie puissance » (radicalement différente du mensonge marketing) et de ses limites si un jour vous souhaitez améliorer la qualité du son. Attention également à ne pas prendre des enceintes surdimensionnées. Tout est question d’équilibre.

Depuis le début, nous parlons de puissance, de Watts mais outre le taux de distorsion harmonique (THD) qui doit être le plus bas possible, il y a 3 autres facteurs ultra importants à regarder quand on prend un bloc de puissance :

– Le rapport signal / bruit, exprimé en dB (décibels). Plus cette valeur sera grande, meilleure sera la conception de l’appareil. Si cette valeur est trop faible, vous pouvez entendre un souffle / bruit de fond dans vos enceintes même lorsqu’aucun signal n’est joué. Inversement, un rapport SNR (Signal to Noise Ratio) élevé vous garantira l’absence de souffle sur les scènes calmes. Cela va également jouer en partie sur la dynamique perçue du signal, car le bas de la gamme risque d’être noyée dans le bruit de fond si le SND est mauvais. Les valeurs les moins bonnes des amplis bas de gamme tournent autour de 90/100 dB, les meilleurs peuvent monter jusqu’à plus 120 dB. Je vous rappelle d’ailleurs sur les décibels sont sur une échelle dite logarithmique et l’on considère que l’intensité sonore double tous les 3 dB. Donc entre le moins bon et le meilleur des blocs, l’intensité du bruit de fond peut être jusqu’à 10 fois supérieure.

– Le damping factor ou facteur de charge. Plus la valeur est élevée, meilleur sera le son. C’est la capacité de l’ampli à être capable de contrôler précisément les mouvements / oscillations du HP, surtout dans le registre du grave (ou les débattements du HP sont évidemment les plus importants). Pour tenter de faire simple, un bloc avec un DF faible va perdre le contrôle sur l’oscillation du HP là où un bloc avec un DF suffisamment haut pourra en quelque sorte l’activer et le stopper de manière nette le HP. Les moins bonnes valeurs de DF commencent autour de 100 pour aller jusqu’à plus de 1000.

– Slew rate représente la capacité d’un ampli à envoyer le courant le plus vite possible pour reproduire le plus fidèlement possible la courbe du signal que l’on lui demande de jouer. Le plus simple est de regarder le graphique et les explications (en anglais) de cet article : https://mynewmicrophone.com/what-is-amplifier-slew-rate…/. Les valeurs oscillent de 10 V/µs à plus de 60. Alors que le Damping Factor jouent plus sur l’amélioration des basses, le slew rate lui agit plus sur la reproduction fidèle des hautes fréquences.

Vous voyez donc que même si un bloc n’est pas censé « colorer » le son, ses caractéristiques intrinsèques vont malgré tout jouer sur la qualité globale perçue de la restitution de la bande sonore. Certains auront l’oreille pour faire la différence entre 2 blocs, d’autres non. Par ailleurs, meilleure sera le traitement acoustique de votre pièce, plus vous serez en mesure d’entendre la différence.

Vous comprendrez maintenant pourquoi certains vous soulent parfois avec des blocs comme les Emotiva, Rotel ou ATI. C’est en se basant sur les specs.

Par exemple, prenons le bloc XPA-5 Gen 3 : https://emotiva.com/en-france/products/xpa-5-gen3. Il annonce 5 x 250W RMS (5 canaux actifs) avec un taux de distorsion inférieur à 0.1% sous 8 ohms, un SNR de 117 dB et un Damping Factor qui commence à être généreux de 500. On voit que c’est un excellent appareil sur le papier et confirmé par de nombreux testeurs. C’est également à des années-lumière que ce que peut vous restituer même le meilleur des intégrés. Est-ce que la différence de prix est justifié par rapport à d’autres blocs nettement moins chers ? C’est uniquement à vous de voir en fonction de vos finances, exigences, matos et pièce d’écoute. Le niveau d’exigence de chacun étant bien entendu par nature différent. Les finances aussi !

En effet, meilleures seront les specs, plus cher sera le bloc en général. Donc le budget reste le maitre principal de votre décision. Fixez-vous un budget max et cherchez le bloc aux specs les meilleures qui rentre dedans. N’oubliez pas d’aller voir du côté du marché de l’occasion car un bloc de puissance est quasi increvable. D’autres facteurs peuvent rentrer en ligne de compte comme l’esthétique de certains blocs (comme les superbes McIntosh ;)) ou tout simplement l’attachement à une marque particulière.

Sources supplémentaires :

– Tableau des décibels – niveaux de décibels des sons courants – Pulsar Instruments
– Big Six : rapport signal-bruit, silence s’il vous plait ! – Les Numériques (lesnumeriques.com)
– Qu’est-ce qu’un amplificateur audio ? – Sono Matériel, Le Blog (sonomateriel.com)
– Qu’est-ce que le rapport signal/bruit ? Peut-on l’améliorer et si oui, comment ? – Le blog de Son-Vidéo.com (son-video.com)
– En quoi consiste le facteur d’amortissement ? | Rotel
– Damping Factor Explained | KEF USA
– PS AUDIO: Damping factor – what it is and why it’s important – Audiophile News & Music Review (hifianswers.com)
– Slew Rate | Audio Science Review (ASR) Forum

J’espère que cela vous donnera les outils pour mieux comprendre votre matos et comment le choisir à l’avenir. Fouillez les specs, lisez les review pour aller plus loin que la tentative d’arnaque marketing. J’aurai aimé que l’on m’explique tout cela quand j’étais plus jeune et que je ne comprenais pas l’intérêt des blocs tout en mettant des enceintes énormes sur mon petit Denon

View Details

Cette question revient souvent chez les passionnées de home cinéma : y-a-t-il un intérêt à dépenser plus dans un câble HDMI ? Si j’achète un câble plus cher, aurais-je une meilleure qualité en échange ? Quelles sont les différences entre les différents câbles HDMI ?

Cela fait plusieurs années que je traîne dans les forums de Home Cinéma, ayant moi-même une salle privée assez haut de gamme. J’ai même été modérateur d’un gros groupe Facebook et ces questions revenaient en boucle.

Pour donner suite à ces débats houleux sur l’intérêt d’un câble HDMI plus cher pour améliorer (ou non) la qualité de l’image et du son, j’aimerai partager avec vous ce que je sais et ce que j’ai pu trouver comme sources sur le sujet.

Commençons par être direct avec un spoiler alert : non un câble HDMI, même très cher, ne peut améliorer ni la qualité vidéo, ni la qualité audio. Essayons maintenant de comprendre pourquoi.

Pour commencer, il faut savoir que, contrairement aux transferts vidéo & audio analogiques d’antan, le protocole HDMI met en place le transfert d’information numérique, donc basé sur du binaire. C’est un domaine que je connais bien étant moi ingénieur en informatique. Vous pouvez retrouver les spécifications officielles sur le site suivant : https://www.hdmi.org/spec/index. Je vous conseille également un bref passage sur la fiche Wikipédia HDMI : https://en.wikipedia.org/wiki/HDMI qui explique l’encodage TMDS.

Le protocole s’occupe donc se s’assurer de l’arrivée de l’ensemble de l’information au bout du tuyau. Pour cela, il indique la manière d’encoder l’information, de gérer en partie les erreurs avec un protocole de correction d’erreurs. Les spécifications sont différentes en fonction de l’évolution de la norme : HDMI 1.3, 1.4, 2.0 et 2.1. Vu que les débits et fonctions supplémentaires vont augmenter, les normes sur la conception des câbles évoluent logiquement.

Les grosses différences furent avec l’arrivée du HDMI 2.0 (4K, HDR, débit audio supérieur) et HDMI 2.1 (8K principalement). Elles ont pour but principal d’améliorer la capacité des câbles à tenir la qualité du signal malgré l’augmentation de la bande passante. Par ailleurs, sauf dans les câbles dit optiques, les câbles HDMI en cuivre sont dit passifs. Donc, il n’y aucun composant traitant ou améliorant le signal dans un câble HDMI. Ainsi, un câble certifié HDMI 2.0 a de très fortes chances de laisser passer un signal HDMI 2.1 avec par exemple du 4K 120 Hz ou du VRR (Variable Refresh Rate).

Par ailleurs, il est également rappelé que le HDMI peut supporter des pistes audio compressées. C’est le cas d’ailleurs le plus fréquent avec des pistes comme le Dolby Digital, Dolby Digital+ ou DTS des Blu-Ray ou plateformes de streaming. Si un câble modifiait le signal pour « l’améliorer », il ruinerait donc sa structure même et le rendrait inutilisable. Essayez par exemple de modifier un seul bit d’un fichier ZIP puis de l’ouvrir. Il y a de fortes chances que vous cassiez le fichier (sauf si la correction d’erreurs arrive à gérer votre modification aléatoire).

Alors cela veut-il dire que tous les câbles HDMI sont égaux ? Non, bien sûr. La différence va justement se faire sur la qualité de conception du câble en lui-même et des connectiques. Le but du jeu étant d’être capable de tenir la bande passante nécessaire pour envoyer le signal (numérique rappelons-le à nouveau) le plus loin possible. Donc les différences entre 2 câbles vont se faire principalement sur leurs capacités à garder le signal de qualité suffisante sur une longueur importante.

Que se passe-t-il si le signal est trop altéré ? Soit vous n’avez plus du tout d’image (le plus fréquent), soit l’image peut sauter (en fonction de la bande passante associée), soit vous pouvez avoir un phénomène de plein de petits points à l’image.

Que se passe-t-il si le signal arrive bien à domicile ? Vous aurez une copie parfaite de l’image d’origine. C’est le principe du numérique. Donc 2 câbles HDMI capables d’amener le signal de bout en bout auront exactement la même image. D’ailleurs, vous noterez à nouveau sur le site officiel HDMI dans la section câble : HDMI Cables – Different Cable Types que jamais il n’est fait mention de différence de qualité possible entre 2 câbles. Les certifications ne parlent que de la capacité à gérer certaines fonctionnalités ou bandes passantes.

En résumé, un câble mal conçu et peu cher risque de ne pas être capable de tenir la charge d’un signal 4K HDR avec la piste TrueHD dedans ou alors si vous le débranchez / rebranchez souvent, les connectiques peuvent lâcher. Par contre, cela ne veut pas dire pour autant qu’il faut dépenser une fortune pour avoir un câble qui marche. Les câbles Amazon Basic sont réputés pour être super fiables et très bien conçus. Si vous avez besoin de tirer un câble plutôt long vers votre projecteur (plus de 7m), je vous conseille vivement un câble optique. Ils sont plus fins et permettent de transférer le signal sur 15 ou 20m sans aucun souci. Ils sont actifs par contre mais alimentés par le port HDMI. Pour ma part, j’ai un câble HDMI Moushou 8K de 12m dont je suis ravi mais il existe plein d’autres marques.

Oui mais alors 2 questions peuvent venir à votre esprit :

• Pourquoi alors des marques comme AudioQuest vendent des câbles HDMI hors de prix ? Quel est l’intérêt ?
• Pourquoi mon copain spécialiste du home cinéma ou mon vendeur préféré me soutient qu’il y a une différence ?

Bon commençons par le vendeur, c’est le plus simple : car c’est bien pratique de vous mentir pour augmenter ses marges. Des fois, certains vendeurs sont assez idiots pour finir par croire leur mensonge. C’est donc soit de la malhonnêteté, soit de l’incompétence, soit un subtil mélange des deux. C’est malheureusement très fréquent dans l’audio ou le home cinéma, l’ésotérisme est roi alors que c’est pourtant purement basée sur la science à l’origine.

Et pour votre pote ? Cela s’appelle l’effet placébo ou le biais de confirmation : https://fr.wikipedia.org/wiki/Biais_de_confirmation. Je vous rassure, nous en avons tous été victimes dans notre vie je pense. Imaginez dépenser 20 fois plus cher dans un câble qui vous donne exactement le même résultat qu’un autre à 15€. Vous avez 2 possibilités : soit vous vous rendez compte que vous vous êtes fait avoir, soit vous niez la réalité en bloc. Cela est également bien documenté sous l’appellation de dissonance cognitive.

D’ailleurs, demandez à votre pote qui vous vante les mérites d’un câble HDMI hors de prix de vous expliquer pourquoi magiquement l’image serait meilleure ? La réponse sera systématiquement la même : « je la vois » sans autre tentative d’explication technique. Malheureusement, la perception humaine est rarement juste.

Alors à moi désormais de prouver le contraire. Bon déjà, si vous savez lire les spécifications et les interpréter, la démonstration est censée s’arrêter là. Mais voici quelques ressources crédibles l’indiquant également :

• https://www.audioholics.com/…/truth-about-expensive…: “In sum, there are quality differences between cables, but if the cable appears to be working properly (i.e. no visible artifacts), then spending more for an expensive cable won’t yield better picture or sound” / “En résumé, il y a des différences de qualité entre les câbles, mais si le câble semble fonctionner correctement (i.e pas de défauts / points visibles), alors dépenser plus pour un câble hors de prix ne vous apportera pas une meilleure image ou un meilleur son”
• https://www.youtube.com/watch?v=Zgy5fX-VPCs où Linus teste un câble HDMI de 1000$ et ne voit aucune différence avec un autre de 10$. Bon, certains pourront alors dire que sa vision ou son protocole de test n’est pas bon, du coup…
• https://www.youtube.com/watch?v=XFbJD6RE4EY où Linus a cette fois-ci un appareil permettant de mesurer le signal qui passe dans un câble HDMI et teste une vingtaine de câbles. Conclusion : il conseille également de ne pas dépenser plus qu’un câble Amazon Basic ou équivalent.

Pour finir, parlons d’AudioQuest. Ils font d’excellents câbles, aucun doute là-dessus. Le site https://www.audioquest.com/cables/digital-cables ne parle jamais d’amélioration d’image ou de son, et pour cause ! Cela ne peut pas améliorer un signal numérique.

Vous pensez vraiment que si leur câble améliorait vraiment la qualité de l’image ou du son, ils n’en feraient pas la pub directement sur leur site ? Cela serait sans aucun doute un avantage concurrentiel à mettre en avant. Alors pourquoi ils ne le font pas ? Car cela serait de la publicité mensongère. Du coup, l’astuce consiste à dire que la qualité de leur câble et de la performance HDMI est meilleure. Ce qui est vrai ! Ils résistent sans aucun doute mieux aux interférences et permettent d’avoir des longueurs supplémentaires sans coupure d’image.

Mais au final, cela ne change pas le problème de la qualité d’image : si un autre câble 10x moins cher permet de véhiculer la même image, cela sera strictement la même.

Pour terminer, j’ai même trouvé ce document super intéressant rédigé par AudioQuest eux-mêmes : https://www.fpga4fun.com/files/HDMI_Demystified_rev_1_02.pdf où vous pourrez voir l’intérêt de la qualité de leur conception. Mais une des conclusions est évidente et sans appel : “There’s a wide spread myth about digital video cables like HDMI. Due to the cliff effect, all the cables have the same perfect picture when they work, even those of lowest cost” / “Il y a un mythe très répandu sur les cables vidéo numérique comme le HDMI. Due à l’effet de seuil, tous les câbles ont la même image parfaite quand ils fonctionnent, même ceux qui coûtent le moins cher”

Et ce n’est pas moi qui le dit mais… « Xiaozheng Lu, Senior Vice President, Product Development, AudioQuest »

Bref, ne vous laissez pas « mystifier » et n’oubliez pas que notre passion est basée sur de la science, pas de la croyance.

View Details

From VR audio experiments to casual gaming in VR on an arcade machine up to more serious usage to create new ways of collaboration using either AR or VR, you should have a pretty good understanding of what you can do today after reading this.

Indeed, in this article, I’ll share lot of fun experiments I’ve been working on to build either immersive or augmented reality WebXR experiences using Babylon.js as well as more serious / business scenarios. You’ll be able to learn by experimenting & reading the source code of each demo. If you don’t have yet a compatible device, I’ll share some fallback as well as videos using the Valve Index, Oculus Quest 2 or HoloLens 2.

If you prefer to discover what’s possible to build with WebXR today watching a video, you can go to this 20 min conference I’ve done: Building Fun Experiments with WebXR & Babylon.js by David Rousset – GitNation that will cover what’s explained below.

Babylon.js is a free & open-source 3D engine built on top of WebGL and WebGPU. It supports Web Audio & WebXR out of the box. You just have to focus on the experience or game you’d like to build, we’ll manage the complexity of many Web APIs for you.

It’s being used at Microsoft in many of our products such as Microsoft Teams, PowerPoint, Bing, SharePoint, our Xbox Streaming service (xCloud) and others. It’s also used by some of our big partners like Adobe for instance.

WebXR?WebXR is a Web API enabling both Virtual Reality (VR) and Augmented Reality (AR) scenarios. I’m expecting it to be more and more popular as it will be one of the major building blocks of the metaverses on the web. You should have a look!

To know more about this specification:

• WebXR Device API (immersive-web.github.io)
• The Immersive Web Working Group/Community Group | immersive-web.github.io
• Using WebXR with Windows Mixed Reality – Mixed Reality | Microsoft Docs
• WebXR | ARCore | Google Developers
• Fundamentals of WebXR – Web APIs | MDN (mozilla.org)

To be able to use it, you need a compatible headset such as the Valve Index, Oculus Quest 2, Windows Mixed Reality headsets, HoloLens 2 or any SteamVR compatible headset for VR and an Android smartphone or HoloLens 2 for AR. For the browser, you need a Chromium-based one such as Google Chrome, Microsoft Edge, Opera, Samsung Internet, Chrome for Android or the Oculus Browser.

For Babylon.js, our documentation entry point is there: WebXR | Babylon.js Documentation (babylonjs.com)

The XR Experience HelperBabylon.js offers a full complete VR experience via a single line of code. It will transform an existing scene VR compatible, will offer teleportation (you must provide the name of the mesh acting as the floor) and will display the right models of the current controllers being used.

For instance, to immerse yourself into a famous sequence of Back To The Future, navigate to this URL: https://playground.babylonjs.com/#TJIGQ1#3 and have a look to the code. You’ll notice the magic happens thanks to this line of code:

var xr = await scene.createDefaultXRExperienceAsync({floorMeshes: [scene.getMeshByName("Road1"), scene.getMeshByName("Herbe1")]});

Where 2 objects can support the teleportation target: ‘Road1’ and ‘Herbe1’.

If you have a compatible browser and a WebXR compatible device connected, you’ll see a VR icon displayed on the bottom right:

If you don’t have a compatible device, you can try to install this Chrome Extension: WebXR API Emulator – Chrome Web Store (google.com) that will emulate a WebXR device. Open the developer tools and you’ll be able to simulate someone using a VR headset:

Here’s a video showing the full experience using a Valve Index in Microsoft Edge on Windows 11:

You can see that you can move using the teleportation target like inside any classical VR game and you can even choose where to face when landing.

Please note that the laser enabled on the controller will generate a Pointer down even when the trigger button will be pressed. This means you can design your 3D scene to work with mouse or touch during the conception and it will be automatically compatible with our VR Experience Helper.

By the way, if you love this scene like I do, have a look to the same version but compatible with WebGPU. If you run it inside a WebGPU compatible browser, you’ll have up to 10x performance gain!

The VR Arcade machineI love building small video games, I love arcade machines and I love VR. So, I’ve naturally mixed all of that into a single demo.

First, you can have a look to the original 2D Canvas game I’ve ported years ago:

I then simply took this 2D canvas to render it on a 2D plane in the 3D canvas in Babylon.js. Indeed, you must render everything in the WebGL canvas to be able to view & interact with elements once in VR. Classical HTML elements are not projected in the 3D canvas in VR.

Babylon.js supports the 2D canvas via the Dynamic Texture: Dynamic Textures | Babylon.js Documentation (babylonjs.com).

I then just had to position the plane on top of the arcade machine model. I downloaded the model from Sketchfab.

If you need help positioning objects in the scene, I’m strongly encourage you to use our inspector tool:

You can find the source code and the demo ready to be played there: https://playground.babylonjs.com/#LDB9FM#7. You can play to the platformer game using the keyboard, a gamepad controller or, of course, your VR controllers once in entered in immersive mode.

Here are 2 videos showing the experience, the first one using the Valve Index, the second one using the Oculus Quest 2.

Valve IndexMeta Quest 2Virtual 360 VR PianoI love composing music and I still love VR. So once again, I had to combine both! By the way, you can listen to my music composition on my Sound Cloud: Stream David Rousset music | Listen to songs, albums, playlists for free on SoundCloud.

I’ve then decided to create 360 piano and associate notes to each key using our Web Audio support. I’ve also used the Web Audio analyser feature to display the wave of each notes on a ribbon.

You can try it even with a flat 2D screen there: https://playground.babylonjs.com/#4ZG2UY#37

To make it work in VR, you simply need to uncomment line 12 and comment line 11. The camera will then be at the center of 360 piano.

Meta Quest 2Here’s a video (of a slightly previous version of my code) inside the Meta Quest 2:

HoloLens 2The same demo works also inside a HoloLens 2. I’ve slightly modified the code to make it better for HoloLens by removing the skydome and replacing it with a black texture. On the HoloLens 2, all pure black textures will become transparent so you can see through. You can try it on your HoloLens 2 if you’ve got one: https://playground.babylonjs.com/#4ZG2UY#42

Here’s a video of me trying it in my HoloLens 2:

You can see that we have hands tracking support available in Babylon.js. I’m then simply taping my fingers to generate a pointer down on the keys to trigger the notes. Once again, you simply have to make your demo work with mouse & touch, and it will work in VR.

AR Magical Orb DemoI haven’t done this demo but it’s really super cool. This sample code will use the WebXR AR feature to analyze your environment and let you place an orb on a flat surface.

You can read the code & try the demo on a AR compatible device (HoloLens 2 or an Android smartphone) from there: https://aka.ms/OrbDemoAR

Here’s a video of myself testing it with my Samsung S8:

More serious scenariosOf course, WebXR is not just made for fun & gaming experiences. To be honest, it’s probably even more used today for “serious” scenarios (even if for me, gaming is a serious business).

eCommerceFirst, WebXR, and its AR feature, is interesting for eCommerce scenarios. I would encourage you to read this great article on the Babylon.js blog: WebXR, AR and e-commerce: a Guide for Beginners | by Babylon.js | Medium. It contains a demo you can try on your Android smartphone (or HoloLens 2): https://aka.ms/AREcommerce

Watch the result:

Metaverse / Virtual VisitI’ve also been working recently on building a “Metaverse” demo where I was able to call someone using Microsoft Teams from a VR scene using Azure Communication Services (a CPaaS running on top of WebRTC) aka ACS. The idea was to experiment a concept where you could visit a house for instance helped by a sales representative connecting from Microsoft Teams.

Here’s a video of the demo running on my Valve Index:

I’ve first built a prototype in the Babylon.js Playground: https://playground.babylonjs.com/#JA1ND3#539 where you can navigate into the scene, press the “Call” button and experience a fake video. You can click on the “A” button of an Oculus controller to put the video on top of the left controller.

Then I’ve integrated the ACS JavaScript SDK to get the video streamed by the ACS infrastructure from Microsoft Teams. You can try the sample & read the code from my Github repo: GitHub – davrous/acsauth: Deploy in less than 10 min an Azure Communication Services sample to be shared & tested with your colleagues & friends. You’ll find the complete instructions to setup it in the readme.

Metaverse integrated inside Microsoft TeamsI’ve also been working with the awesome people from FrameVR who are using Babylon.js to build their custom Metaverse solution. They have recently published their app in the Microsoft Teams apps store and shared about it: Frame Blog: Bringing the Metaverse to Microsoft Teams (framevr.io).

Have a look to the result:

Other Microsoft productsBabylon.js and its WebXR support is also used inside SharePoint Spaces or Power Apps: Use the 3D object control in Power Apps – Power Apps | Microsoft Docs.

You then see that WebXR is ready for prime time!

I hope this article and the various demos with their associated code will help you to better understand what can and could be done using WebXR. I hope it will inspire you to build great immersive or augmented reality experiences!

If you’ve got questions, please ping me on Twitter: https://twitter.com/davrous or ask them directly to the Babylon.js community: https://forum.babylonjs.com/

View Details

In this tutorial, we’ll see how to use Azure Communication Services (ACS) to enable real-time audio/video conferences in your existing web apps, including doing calls directly to Microsoft Teams from your app!

In this first article, we’ll start by discovering some basics of the service. You don’t need to be a developer nor to install anything specific on your machine. You’ll be able to try it right in your browser in a couple of minutes. This will show you how to do calls between various users using your app or by directly call a user using Microsoft Teams.

If you’re new to Microsoft Azure, you can start evaluating it for free using this link: Create Your Azure Free Account Today | Microsoft Azure. You’ll be able to setup the various samples shared below thanks to that. If you’ve got a MSDN subscription, this is also including some Azure credits up to $150/130€: Monthly Azure Credit for Visual Studio Subscribers | Microsoft Azure.

Ready? Let’s jump into it!

Azure Communication Services?Azure Communication Services (ACS) is a set of rich communication APIs, video APIs, and SMS APIs for deploying your applications across any device, on any platform. If you’re looking on enabling chat, audio/video conferencing, phone calls or SMS inside an existing app, you should have a look to this service. You can view it as your building blocks to create your own custom version of Microsoft Teams, it uses the same underlying infrastructure. This CPaaS (Communications Platform As A Service) will manage scalability, quality & availability of this service for you. This platform is also built on top of our secure and compliant cloud.

Of course, this service comes as a price, and you can find the cost of each services there: Azure Communication Services pricing | Microsoft Azure.

ACS exposes its services via various SDKs: Azure/Communication: Azure Communication Services – Samples and Tools (github.com) available for JavaScript, .NET, Java, Android, iOS & Python developers.

You can also optionally use on top the ACS UI library: Overview – Page ⋅ Storybook (azure.github.io) which is made of React based components implementing the Microsoft Fluent Design to help you building visually engaging web apps. The best sample mixing all those concepts is probably: Group calling hero sample – An Azure Communication Services sample overview | Microsoft Docs.

In our case, we will use regular vanilla HTML & CSS to focus more on the SDK itself.

Create an instance of ACS in your Azure Portal & do a quick testOf course, the very first step is to create an Azure Communication Services instance in your Azure Portal. The steps are described in our documentation: Quickstart – Create and manage resources in Azure Communication Services – An Azure Communication Services quickstart | Microsoft Docs or you can watch this YouTube video if you prefer: Create an Azure Communication Services resource in the Azure Portal – YouTube.

To verify that your ACS resource is ready to be used, you can then quick test it by navigating on this page: https://aka.ms/acsquicktest.

It’s is a slightly improved version of this sample: Quickstart – Add video calling to your app (JavaScript) – An Azure Communication Services quickstart | Microsoft Docs.

You need to enter a valid ACS token by following this small documentation: Quickstart – Quickly create Azure Communication Services identities for testing – An Azure Communication Services quickstart | Microsoft Docs and press the “Initialize ACS agent” button. If successful, the UI should be updated like:

Choose the right camera, microphone & output speakers, check the “Call echobot” checkbox, press the “Start Call” button and verify that your setup works as expected.

Next you have 2 possible options.

First is to share the link to this page to a colleague/friend, generate a second ACS token, grab the ACS ID from the Azure Portal:

You’ll be able to call someone else though the infrastructure of your ACS resource. If you’ve got 2 webcams on your desktop machine, you can try it locally like demonstrated in this video:

Or you can use your smartphone to simulate a second client.

The second option is to test the Microsoft Teams interop feature of ACS by copy/pasting a Teams meeting link instead of the “8:acs…” id like demonstrated in this video:

The code source of this page can be found there:

• acsauth/testacs.js at main · davrous/acsauth (github.com)
• acsauth/testacs.html at main · davrous/acsauth (github.com)

At this stage, you should have understood how easy it is to enable audio/video in your web app and how to connect to a Microsoft Teams meeting. If you’d like to achieve the same in your Android, iOS or Windows UWP app, please choose a different platform in our doc: Quickstart – Add video calling to your app (JavaScript) – An Azure Communication Services quickstart | Microsoft Docs.

In the next article, we’ll see how do create an authentication layer to use a social account (Github, Twitter, Google or Microsoft) to authenticate against Azure Communication Services. You’ll be able to deploy a ready-to-use sample shareable with your colleagues in less than 10 min.

At last, the cherry on the cake will be to see how to call someone in Teams from a metaverse thanks to Azure Communication Services!

View Details

You’ve probably already heard the magic word: “shader” before. But what exactly is it? How does it help to draw beautiful & fast pixels?

This article is following: Frame & (variable) refresh rates or why Tesla is responsible for the 60 fps war where you can learn why we’re targeting 60 fps on most monitors, what is Vsync, what are the physiological limits of the human eyes to process motion and why variable refresh rate was the solution we were waiting for.

Most of the time, I’ll try to share live samples that will run in your browser to help you better understand, whatever the device, thanks to the power of WebGL and Babylon.js. We will even have a small tutorial to play with.

Before the GPU era (pre-3DFX) Before we all had a powerful GPU to render 3D, everything was computed by the CPU. I’ve dedicated a whole series on how to create a 3D software engine from scratch: Tutorial series: learning how to write a 3D soft engine from scratch in C#, TypeScript or JavaScript that many cool people have ported to C++, WPF, Java, etc. If you’re a developer, it should help you understand the basics of 3D and will help you understand the following parts. But even if you are not a developer, it might help you help you understanding the various passes needed to render 3D by just looking at the different rendering images.

In a nutshell, 3D engines are generating triangles (polygons) that will surface the geometry of the 3D objects to be rendered. By the way, we’re calling a 3D object a mesh. Each of the triangles are composed of 3 points having 3 coordinates (x,y,z). A 3D point is named a “vertex” and several of them are named “vertices”. To project this 3D world onto your 2D flat screen, we’re using several matrix computations described in the first article of the series. Then, we’re filling those triangles by drawing lines inside it. This process is known as the rasterization.

During this process, we can apply several refinements such as computing the lightning of each face (using a normal vector), the shadow applied on it, the potential environment reflection and so on. Some of the most basics ones are described in the tutorial such as the flat shading, gouraud shading or texture mapping.

Alone in the Dark was one of the first real-time 3D engine ever released in a game in 1992 and was using flat shading in a pre-computed environment:

In 1993, Strike Commander was using textured gouraud shading and was killing almost any 80486 CPU machine at that time

ID Software with Quake, as well as the original release of the very first Tomb Raider in 1996 were the top of the art 3D engines running only on a CPU.

But you’d quickly understand why this approach became too heavy and complex for the CPU, especially on higher resolutions. That’s why, we needed the GPU to go further. The first generations were basically boosting the triangles generation. But quickly an awesome and powerful new toy arrived: the “Shader”.

GPU and their Shaders Historically, there’s been 2 important types of Shaders:

• the Vertex Shader acting on each vertex (3D point) will have an impact on each triangle / geometry.
• the Pixel Shader, also known as Fragment Shader, is dedicated to the rasterization process, choosing precisely how each pixel should be drawn (color & transparency) based on your specific lightning algorithm.

More recently, geometry and tessellation shaders were available, but we won’t cover them in this article.

A shader is a small piece of program, written in a low-level language that looks like C, compiled into a native binary executed by the GPU. Two languages are used today GLSL (for OpenGL Shader Language) and HLSL (for High Level Shader Language used by DirectX).

To have a better overview of how those 2 shaders work, I’d recommend you reading those great introductions:

• Our Babylon.js documentation: Introduction to Shaders
• The awesome Learn OpenGL site: https://learnopengl.com/Getting-started/Shaders
• Building Shaders With Babylon.js on Smashing Magazine by David Catuhe
• Writing HLSL Shaders in Direct3D 9

The order of execution is the following one:

A more detailed schema found in GPU Performance for Game Artists will help you better understanding the complete process:

Still, this is important to understand that in 3D engines, a lot of the job is also done on the CPU side. That’s why, as discussed in the previous article, the low performance of rendering is often due to CPU-Bound scenarios. Alongside the game logic (AI, input, animations), the CPU is also doing the matrix computation to update the 3D positions of every objects as well as preparing the textures to send to the GPU memory. Again, GPU Performance for Game Artists gives us a simple diagram to understand what’s going on:

I highly recommend you reading the complete article. It explains in great details the various optimization tricks we’re using in today modern video games.

Draw calls are often a very important metric to pay attention to. It’s the number of orders the CPU is sending to the GPU to update the screen. Too many draw calls will put a high pressure on the CPU and will lower down the frame rate. There are various optimizations to be done to reduce the draw calls explained in the article. A 3D artist must understand this while creating his assets. But let’s go back to the GPU world and do a small tutorial.

Small tutorial to let you modify your first shader You’ll need to run this on a desktop machine rather than on a smartphone (even if it should work ).

Let’s try to play with it using a tool named CYOS (Create Your Own Shader) written by my friend David Catuhe. It’s going to help us understanding those magic shaders.

• Navigate to: http://cyos.babylonjs.com/ and change the template to “Basic” and meshes to “Ground”

• You see that the Pixel Shader only takes the color of the texture as-is without any other computation. To have a more realistic rendering, you need to take into account the light and its position. Switch the template to “Phong”

• You now see that the lightning is considered to compute the color of each pixel to know how much light we should apply (between 0 and 1) as well as computing the specular highlight.

• Change the line “gl_FragColor = vec4(color * ndl + vec3(specComp), 1.);” by “gl_FragColor = vec4(color * ndl, 1.);” and press the play/run button. You have removed the specular effect.

• So far, we’ve played with the pixel shader, acting on the color of each pixel. Now, let’s play with the vertex shader to understand what it could do. Switch the template to “Wave”.

• You can now see that the mesh (3D object) is being deformed. In the vertex shader, change the line “v.x += sin(2.0 * position.y + (time)) * 0.5;” by “v.x += sin(2.0 * position.z + (time)) * 0.5;” and will see a different deformation applied

Congrats! You should now have a basic understanding on how a pixel and vertex shader work. The Vertex Shader is called first to act on the geometry and do the rasterization job. Then, the Pixel Shader takes those data as input and handle each pixel to find the right color, lighting & shadows.

If you’d like to study shaders with a graphical tool rather than with code, we recently added a very powerful tool in Babylon.js named the Node Material Editor (NME). This tool will let you create shaders via a drag’n’drop visual editor. You still need to understand the basics of 3D shared in my previous tutorial series, but it makes stuff so much simpler. A couple of cool YouTube videos to watch to learn more:

• Node Material Editor: Vertex Shader which greatly explains how to build the wave vertex shader demonstrated just before.
• Node Material Editor: Lights and Textures

Let’s study how you can simulate water Let’s try to go a step further by illustrating what could do a pixel shader via a concrete example: simulating water. Think about it. Simulating water is not something that easy. You need to compute the reflection, refraction, transparency, deformation of the light going through it and potentially interactions with other objects! The idea is then to approximate that based on 2 constraints: the performance we’d like to achieve and the complexity of the shader we’d like to build.

Open this sample in another tab:

You’ll see that the waves are animated, and you’ve got the feeling that there is some volume. It’s based on the water properties selected in the text editor on the left in the screenshot. Those properties will configure the pixel shader used behind for you. Fun fact, there is no 3D geometry generated by this water material. It’s just a dead simple quad (a flat rectangle) and the pixel shader gives you this illusion. You can check it by opening the inspector as seen in the previous article and display the wireframing. Remember, I told you 3D engines were cheating with optical tricks? It’s the case here.

As said in Matrix: “There is no spoon”. The spoon deformation was probably done by changing the properties of the associated pixel shader rather than really bending it! Cheater.

This cool and simple Babylon.js water shader runs on every platform without any issue. It’s because it’s using simple enough equations for the performance as well as a set of instructions available everywhere for cross-devices compatibility (we’ll come back to these notions later). But as I told you, you can build more complex shader to achieve even better photorealistic results.

Open this sample from Shadertoy in another tab for instance.

This beautiful water shader is again a pure optical illusion. It doesn’t use any texture, it’s fully procedural (generated by an algorithm). It’s “just” a pixel shader applied on a plane. But it’s much more complex & realistic than the previous one we used. That’s why, it only runs at 15 fps (not even full screen!) on my current machine whereas the previous water shader is running at a solid 60+ fps.

Shaders complexity? Complexity is about the time needed to generate each pixel based on the equations you’re using. Obviously, the more mathematical complex & realistic computation you’ll do per pixel, the more time the pixel shader will need to be executed. This will then have a direct impact on the frame generation time.

But complexity is also about the number of instructions (lines of code if you wish) you’re using and the type of operations available. Thus, shaders have a limit on the number of instructions you can use. Those limits depend mainly on the GPU you’re using as well as the OpenGL or DirectX version you’re using. For instance, in DirectX 12, we’ve got up to version 6.0 of the shader model supported: HLSL Shader Model 6.0. If you’re using one of the new instructions of the shader model 6.0, you will lock your 3D engine to DirectX 12 compatible platforms & hardware. That’s why, 3D engines are also providing different shaders based on the platforms & hardware support. Writing cross-platforms games is definitely not a simple job. Hopefully, some middleware such as Unity or Unreal help a lot. Babylon.js is also hiding the shader complexity for you.

Water simulated with both a Vertex & Pixel shader So far, only the pixel shader part was used for our water use cases. But if we need physical interaction between on object being on the water or entering the water, you need to play with the geometry. Some water effects are then also using the vertex shader for that, such as in this incredible interactive demo:

If you’re curious and would like to know more about this WebGL sample, or any other WebGL demos, you can use our Spector.js tool: https://spector.babylonjs.com/. It’s a browser extension that will inject itself into the page to trace the various WebGL commands sent to the GPU. It will let you view the various frame construction steps as well as the code of the shaders.

For instance, here’s part of the vertex shader code extracted by the Spector.js extension:

In today’s modern games, the most impressive water shaders I’ve seen where done for the Assassin’s Creed series, Uncharted 4, Batman Arkham Knight and more recently, Sea of Thieves. Look at those animated GIFs:

The Batman’s one is really impressive but isn’t interactive. This means that it’s probably “only” a pixel shader as it doesn’t have to physically interact with other objects. You should now have understood that in an opposite way, the Sea of Thieves water effect is using both pixel & vertex shaders as it’s completely interactive. It also uses an advanced technic named: Subsurface scattering. Should we then conclude it’s the most impressive one?

Performance & PBR Let’s recap. You want to render images in native 1080p at 60fps? You then have: 1920×1080=2073600, 2 million pixels to generate every 16ms. You want native 4K? It’s now more than 8 million of pixels to handle! 4x more pixels to draw during each loop.

This means that the Pixel Shader will have to go through those 2 or 8 million of pixels to calculate its color based on the 3D environment, lights, shadows and various effects.

That’s why, you can choose to lower the rendering resolution and use upscaling approaches to compensate that. If you can’t maintain a native 4K rendering at 30 or 60 fps, then just lower the resolution until you can reach the frame rate target. Upscaling is a process that doesn’t cost a lot of hardware power compared to the power required to process each pixel for the rendering. There is even some super smart new approach for upscaling based on AI / Deep Learning such as DLSS from nVidia. Thanks to that, you can compute a frame in 1440p and upscale it to 4K (2160p). The AI algorithm will create a 4K image almost identical to a 4K native rendering. This AI upscaling technology comes at a just the right time as real-time RayTracing will consume a lot of the GPU power.

As you may have guessed, writing shaders requires very specific skills. There are used for special effects such as water as we seen together. But today, the most important usage is for the lightning process. In the past, 3D artists were trying to create realistic rendering with 3D tool more or less manually. They were choosing how much of light a specific surface will reflect or absorb for instance to try to create a realistic rendering. They were controlling the various properties exposed by the shader as seen in the Babylon.js water demo. Today, everybody is using PBR. PBR stands for Physically Based Rendering. PBR is responsible for the look of all modern video games. As its name suggests, this approach aims to simulate real life lighting (using rasterization not raytracing). You can read our documentation to know more: Start with Physically Based Rendering (PBR). Disney was one of the pioneers working on this. The idea is to have a set of predefined equations & parameters to help artists building photorealistic rendering.

Here are a couple of demos you can execute on your device to see it live (click on the image to open it):

PBR helps the 3D artist to better control the rendering and could also helps several artists to work together to have a consistent look. Even better, as almost all 3D engines support PBR today, it’s easier to switch an asset from one engine to another while keeping a very similar rendering. For instance, this page is containing links to various assets displayed in Babylon.js as well as all our WebGL competitors engines.

Again, our documentation is covering a lot of the latest effects done in PBR such as sheen, clear coat, irradiance, subsurface, energy conservation: Master Physically Based Rendering (PBR) with demos in the browser most of the time.

As you can see, most of the graphic quality and performance of video games you like comes from the shaders. They can do super complex computing to generate photorealistic rendering. Hopefully, the GPU is highly optimized for that, through huge parallel processing, to deal with millions of operations in an extremely fast way. For instance, the AMD GPU of the Xbox One X has 2560 shading units, a nVidia GTX1080ti got 3584 and the brand new Xbox Series X has 3328 units. A shading unit could act as a pixel shader or vertex shader unit. Those shader processing units are used since the release of the unified shaders architecture.

I hope this article will help some you better understand what this mysterious word “Shader” means. I also hope you will now look at video games in a different way, wondering how the magicians behind it all managed to create such amazing virtual worlds!

View Details

I’ve decided to do a new small new series to share my knowledge on 3D engines & games. This first one will be dedicated to a general introduction to 3D engines. More specifically, why and how we try to target 60 fps. I’ll also explain why 30 fps on consoles, and 90+ fps in VR. We’ll try to understand what are the physiological limits of the human eye to process motion. At last, we’ll see what VSync is and why variable refresh rate (or VRR) was the solution we were waiting for!

The goal of 3D engines is to create realistic real-time rendering of 3D environments, objects and characters. The idea is basically to try to imitate the “real world” as realistically as possible in terms of graphic quality and smoothness. To reach 60 fps, we’re using lot of smart tricks to simulate materials, as well as lighting & shadows for instance to try to use our available power in the most efficient way.

But do you know why we often target 60 fps as the ultimate goal for video games enthusiasts?

Tesla, at the origin of 60 fps We need to go back in history to the choice made by Tesla a long time ago (the genius engineer not the car manufacturer). What? Tesla is at the origin of 60fps in video games? Yes!

As explained in this article: Why Is the US Standard 60 Hz?, when he was researching on how to build and distribute electricity, Tesla “figured out that 60 Hz at 220 Vac was the most efficient”. Still, because of tradeoff made with Edison, the US ended up having 110V 60Hz AC. In Europe, we decided to go for 220V 50Hz.

Tesla’s induction motor But why am I sharing that with you? Because, this has driven the choice for the refresh rates of CRT-based TVs and monitors! This is perfectly explained in this article: What so special about the 60Hz refresh rate? :

“When electronic displays became common, with the dawn of commercial television, they were all CRT-based – and CRTs are very sensitive to external magnetic fields. To minimize possible problems from this source, the vertical frequency of CRT televisions was chosen to match that of the power lines: 60 Hz in North America, and 50 Hz in Europe. (Due to bandwidth limitations in the broadcast TV system, interlaced scanning was used, so these were actually the field rates rather than the frame rates.) Later, as personal computer displays started to show up on the market in the late 1970s and early 1980s, these were also of course CRT-based and used TV-like timings for the same reasons. The original 60 Hz “VGA” timing standard (640 x 480 pixels at 60 Hz) was actually a progressive-scan version of the standard North American TV timing <…>*

Today, of course, LCDs and other flat-panel technologies don’t have the same concerns as the CRT did, but the huge amount of existing 50/60 Hz video has caused these rates to be carried on in modern TV and computer timing standards.”

As explained, due to bandwidth limitation in the broadcasting, we weren’t viewing 50 or 60 fps videos on our CRT TV but half of it (interlacing was using 1 line out of 2) which results in 25 fps in Europe and 30 fps in the US. You now know why videos are displayed in 30 fps in US and 25 fps in Europe. I won’t explain why movies / cinemas are using 24 fps, you can easily find the historical reason for that in your favorite search engine. Spoiler: it was purely for economic considerations; it’s unfortunately not linked to our physiological capacities.

Today, most of our LCD-panels on our laptops or TVs have a refresh rate of 60 Hz. That’s why, we try to generate 60 frames per second to be perfectly aligned with this refresh rate.

By the way, please note that refresh rate and frame rate are different. Refresh rate has just been explained before. It’s the speed at which your monitor / TV will update the screen every second. Generally, it does refresh by displaying line by line from the top of the screen down to the bottom. The frame rate is the speed at which your computer will generate new frames every second. Obviously, those 2 rates won’t always be synchronized. The graphic card can generate less frames per second than your monitor can refresh and vice-versa. We will see later the various strategies to manage that by explaining what’s Vsync.

But before, let’s briefly try to understand the hard job of a game developer.

16ms to do all your tasks Real-time means you have less than 16ms to compute a new frame if you’d like to target 60fps on PC or 33ms for 30fps on console. Targeting VR? You have now only 11ms for the 90fps target. We will see why those specific number below. But trust me, this is a very constraint world. It’s super complex to achieve. Indeed, as a game developer, inside the game loop, you should also manage the AI of enemies, the physics engine, the collision engine, sounds effects, inputs, loading resources from your hard drive or SSD, managing the network and so on. If one of those components takes too much time and raise the render loop time just by a couple of ms above your target, the framerate will go down and the experience will suffer. You’ll miss the 16ms target and thus, the framerate will be lower than 60 fps. And here comes the drama in social networks.

The game loop A 3D engine is then all about clever optimizations to approximate photorealistic rendering with the fewer resources possible. It’s a battle, or a game for some, against the machine to use its power in the most efficient way. We can also consider that 3D engines are cheating. They try to make you think it’s real by using sometimes some very smart optical tricks. We’ll see that in the next article.

The two main components used are the CPU (or we could even say multi-CPUs as all CPUs are multicores today) and the GPU which embeds hundreds (if not thousands) of dedicated cores. Obviously, the GPU will do most of the job of the 3D rendering task but still, the CPU as well as the data sent between the CPU and GPU are really import parts of the performance. The complete architecture has a huge impact on the global performance and quality of the rendering: the quantity of memory and its bandwidth, the various bus between each component, dedicated hardware, the type and speed of the drive (SSD or not?). Today, we’re often talk about TeraFLOPS or TFLOPS to evaluate the power of a machine. It’s indeed an interesting indicator of the performance of a system but we can’t fully rely on it. 2 systems with similar TFLOPS can still show significant performance differences based on their architecture. Still, if one of them got a truly higher TFLOPS, it will have faster or better rendering for sure.

Going back to the CPU. If it takes too much time to do some of its tasks (AI, Physics or load data from the hard drive for instance), the GPU will have to wait and won’t be able to target the ideal FPS. You will be in a CPU-bound scenario. The GPU won’t be used at its maximum capacity. Indeed, never forget than 16ms/60fps is the time allocated for the complete machine to deliver its tasks, CPU, GPU, memory, I/O, inputs. And to be honest, it’s often on the CPU side where the bottleneck is. That’s why, a very big GPU could be useless if the underlying CPU or architecture is not serving it properly.

In the other way, there could be cases where the 3D scene is too complex to be rendered: too highly detailed models’ geometry (polygons), too complex shaders, etc. Then the GPU will struggle and will need more than the 16/33ms to be rendered. You’ll be GPU-bound.

Writing a performant 3D game is a true art of balancing all those constraints. As you could expect, the right balance is easier to find on a fix hardware configuration (such as the consoles) rather on PC with the heterogenous and rich diversities of components. That’s why, you have more options on PC to tune this balance yourself in order to push the cursor rather on graphics quality or performance, based on your preferences.

But what if the game can’t reach the target frame rate (30 or 60 fps)? Using profilers, you will have first to find where the bottleneck is and make a choice. If it’s the CPU, maybe there are too many objects to update on screen, let’s lower this number. If it’s the GPU, maybe the 3D models are too detailed, let’s simplify them. You’d like to keep the current rendering quality for artistical reasons but can’t reach 60 fps? Target 30 fps if it doesn’t hurt the gameplay. We even have tools now that are super-efficient such as Simplygon to help us doing this task. The 3D artist and the level design job are also super critical. They have to understand the architecture of a gaming machine to build their assets in the most optimized way. It’s really about balancing a complex equation.

To illustrate that, let’s play with a real interactive demo in your browser. We’re going to play with Babylon.js, an open-source WebGL engine built and used by Microsoft for the web. Open the following Babylon.js demo: https://www.babylonjs.com/demos/sponzadynamicshadows/ and click on “Control panel” and “Debug”:

This will display our inspector pane on the right. Then, search for the stats tab. Our engine is showing you what takes the most time for each rendering pass. This specific demo doesn’t run at 60 fps on the machine where I’m writing this article. It’s between 50 and 60 fps. Still, as you can see in the screenshot, the “Frame total” time is way below 16ms, around 6ms. This means the GPU is not the bottleneck. If the CPU was strong enough and the screen was allowing it, the GPU could generate up to 166 fps for this specific scene. Using the Edge / Chrome developer tools (pressing F12 on a PC), you can even further review the cause of this thanks to the Performance tab:

In this screenshot, we see that the game loop (manage by the requestAnimationFrame API in JavaScript) took 19 ms to fully execute which translates to 52 fps. As the GPU only need 6 ms to render the frame, we’re under the 60 fps threshold because of the CPU. We’re CPU-Bound. You’ll find plenty of other demos to play with on our playground: https://playground.babylonjs.com/ if you’re interested in hacking more around this topic.

What’s the limit of the human eye and why 60 fps is better than 30? We need to solve a common question people have around frame rate. Can we really see and feel the difference between 30 and 60 fps or even more? What is the physiological limit of the human eyes and our brain?

By far the best article I’ve found dealing with this question is: Why movies look weird at 48fps, and games are better at 60fps, and the uncanny valley…. You should really read this article to better understand how our eye is working. To my point of view, this is the key part is: “Ocular microtremor is a phenomenon where the muscles in your eye gently vibrate a tiny amount at roughly 83.68Hz (on average, for most people). (Dominant Frequency Content of Ocular Microtremor From Normal Subjects, 1999, Bolger, Bojanic, Sheahan, Coakley & Malone, Vision Research). It actually ranges from 70-103Hz.”. It then explains why the minimum target is around 41 fps** to have the best results to extract spatial information.

This awesome article also explains the importance choice between higher resolution and better frame rate: “Higher resolution vs. frame rate is always going to be a tradeoff. That said, if you can do >~38-43 fps, with good simulated grain, temporal antialiasing or jitter, you’re going to get better results than without. Otherwise jaggies are going to be even more visible, because they’re always the same and in the same place for over half of the ocular microtremor period. You’ll be seeing the pixel grid more than its contents. The eye can’t temporally alias across this gap – because the image doesn’t change frequently enough.

Sure, you can change things up – add a simple noise/film grain at lower frame rates to mask the jaggies – but you may get better results in some circumstances at > 43fps with 720p than at 30fps with 1080p with jittering or temporal antialiasing – although past a certain point, the extra resolution should paper over the cracks a bit. At least, that is, as long as you’re dealing with scenes with a lot of motion – if you’re showing mostly static scenes, have a fixed camera, or are in 2D? Use more pixels.”

Thanks to the researches done in this article, we have factual and scientific explanations on why 60 fps is better than 30.

There are plenty of other articles that I’ve found with less factual references. This one is a good example: What is the highest frame rate (fps) that can be recognized by human perception? At what rate do we essentially stop noticing the difference?. I often read that you can see a 1000Hz frequency flash in a dark room, which is totally true. But does this mean you can process a moving object at 1000fps? Based on the previous article, I don’t think so. There seems to be a huge difference between stimulating your eye with super-fast image and your capacity to process and extract spatial information out of it.

This is confirmed by this other great article I would recommend reading: How many frames per second can the human eye really see? And here are the main parts:

“The first thing to understand is that we perceive different aspects of vision differently. Detecting motion is not the same as detecting light. Another thing is that different parts of the eye perform differently. The centre of your vision is good at different stuff than the periphery. And another thing is that there are natural, physical limits to what we can perceive. It takes time for the light that passes through your cornea to become information on which your brain can act, and our brains can only process that information at a certain speed”

By the way, it partly explains why we render at 90 fps in VR: “But out in the periphery of our eyes we detect motion incredibly well. With a screen filling their peripheral vision that’s updating at 60 Hz or more, many people will report that they have the strong feeling that they’re physically moving. That’s partly why VR headsets, which can operate in the peripheral vision, update so fast (90 Hz).”. It’s also because some studies show that 90 fps is reducing motion sickness in VR.

The conclusion is interesting:

• Some people can perceive the flicker in a 50 or 60 Hz light source. Higher refresh rates reduce perceptible flicker.
• We detect motion better at the periphery of our vision.
• The way we perceive the flash of an image is different than how we perceive constant motion.
• Gamers are more likely to have some of the most sensitive, trained eyes when it comes to perceiving changes in imagery.
• Just because we can perceive the difference between framerates doesn’t necessarily mean that perception impacts our reaction time.

Combining those 2 articles would indicate that going higher than 120 fps would be useless for most people but there are still passionate debates on this. But for sure, we know you can feel the difference between 30 and 60 fps.

This explains also why the new Variable Rate Shading (VRS) is really interesting. The center of our vision is very sensitive to details where as the periphery is less sensitive to detail but more to motion. Let’s benefit from that by lowering the quality of the rendering on the periphery (edges of the images) in order to boost the FPS to satisfy better motion perception.

The new Xbox Series X will support this technology.

Additional resources on this topic from Wikipedia:

• Frame rate: “The temporal sensitivity and resolution of human vision varies depending on the type and characteristics of visual stimulus, and it differs between individuals. The human visual system can process 10 to 12 images per second and perceive them individually, while higher rates are perceived as motion.[1] Modulated light (such as a computer display) is perceived as stable by the majority of participants in studies when the rate is higher than 50 Hz. This perception of modulated light as steady is known as the flicker fusion threshold. However, when the modulated light is non-uniform and contains an image, the flicker fusion threshold can be much higher, in the hundreds of hertz.[2] With regard to image recognition, people have been found to recognize a specific image in an unbroken series of different images, each of which lasts as little as 13 milliseconds.[3] Persistence of vision sometimes accounts for very short single-millisecond visual stimulus having a perceived duration of between 100 ms and 400 ms. Multiple stimuli that are very short are sometimes perceived as a single stimulus, such as a 10 ms green flash of light immediately followed by a 10 ms red flash of light perceived as a single yellow flash of light.[4]”
• Motion perception

So, we’ve seen how sensitive we are to FPS as human being but what about the way the hardware and rendering are working?

Strategies to adapt the frame rate to the refresh rate As explained before, the frame rate is the raw capacity of your machine/GPU to generate a certain number of frames per second (FPS) whereas the refresh rate is the speed at which the hardware (the monitor) clean and draw a new image on screen. This is expressed in Hz. The refresh rate of a monitor is fixed. Most of the time, it will be at 60 Hz. Which means that whatever you’ll feed with, it will update itself every 1/60s = 16,66ms. The FPS can of course vary depending on the current load of the GPU or if the CPU is struggling managing the game logic. In conclusion, most of the time, you have to fit a variable frame rate inside a fixed refresh rate. So how to deal with that?

V-Sync on or off? Stuttering or tearing? What’s V-Sync? V-Sync stands for Vertical synchronization or synchronizing with the vertical refresh rate of the monitor. As explained before, almost all monitors run at a fixed refresh rate of 60 Hz. By default, your monitor will ask to your GPU to provide a new frame every 16.6ms. Then, it will draw it on screen line by line from the top to the bottom.

But then you have 2 choices: either you’re synchronizing the frame rate to the vertical refresh rate, or you simply don’t synchronize with the refresh rate and send the frames as-is.

Let’s first review the non-synchronized approach (Vsync off).

V-Sync off

If you’re generating more frames per second than the refresh rate of your monitor, a new frame will be served to the monitor before it will have finished drawing the previous one. Remember, the monitor is drawing on the screen line by line from the top to the bottom. This means the monitor will draw the beginning of an image from the top and will continue with a new image in between. It could be for instance 40% of a frame followed by 60% of the next frame. This will generate a tearing effect in the image on screen:

Of course, you will have the same phenomena if you’re generating less frames per second than the refresh rate with V-Sync off. If the frame takes more than 16ms to be rendered, the monitor will start to redraw the same frame and will receive a new one in between a couple of ms later. So, it will continue by drawing this new frame on top of the previous one which will result as a tearing effect again. This tearing effect can be almost unseen during motion if it appears at the top or bottom of the screen. But if it occurs in the middle of the image, you’ll probably see it and it’s not great.

However, the benefit of this approach is that you’ll have the best response time to your inputs.

Let’s review V-Sync on now.

V-Sync on There will be 2 cases again. Let’s start by the best one: your GPU is faster than the refresh rate.

As you can see with the green boxes, the GPU will wait for the monitor signal before sending a new frame. This will remove completely the tearing as the image buffer will be synchronized with the refresh rate. The GPU will be on hold waiting for the monitor to finish its refresh job. It will then be under-utilized.

Oh, by the way, If you’d like to understand the frame buffer concept, the 2 best articles I’ve found are:

• Explaining VSync and other things…
• How Many FPS Do You Need?

In a nutshell, the graphic card is generating a new frame and stores it inside a buffer. It’s the frame buffer. It’s been a long time since we’re often using a double buffer rendering approach. While one buffer is read and sent to the monitor to be displayed, a second buffer is used to compute the next frame. Once the first buffer has been sent to the monitor, the second buffer became the primary buffer to be used by the monitor and the first one become the cache buffer for the next frame. You can even use a triple buffering solution to digest potential high GPU peaks, but the trade-off is that you will generate higher input latency. We’re then not going further than triple buffering as the delta between the frame rendering and what you’re doing with your gamepad / mouse would be too high (input lag).

The Techspot article got a full amazing explanation of the various strategies and this graph perfectly explains what happens with V-Sync on when the framerate is below 60 fps:

This diagram is super interesting. If you’re missing the 16.6ms target, the monitor HAS to do something anyway while the GPU finishes computing the next frame. Then, the simplest idea is to redraw the previous frame from the frame buffer. But this will generate some stutter.

Wait.

This is really important to understand. If you can’t maintain a solid 60 fps rendering, the same frame will be displayed twice. This means that you will show the same frame during 33ms. This means that you will render at 30 fps! To be exact you will alternate between either a 30 fps or 60 fps rendering on some frames which is far from being a nice experience. Double or triple buffering can absorb small temporal loss of a frame rate but if the average framerate is below 60 fps, you will still end up having some stuttering.

This is why consoles are usually targeting 30 fps rather 60 fps. The cost to switch from 30 to 60 fps is super high. It’s not interesting to be in-between. Let’s say that your GPU can render your game at 45 fps, it will be rendered on a fixed refresh rate 60 hz monitor at 30 fps because of the above explanation. You can’t be in-between (well, except on some monitors that can be set to run at 45 Hz). On a console, as you’ve got a fixed hardware, if you’re running at 45 fps, it’s a better idea to use this 15 fps additional budget on something else: more advanced graphic rendering, more detailed objects, better AI and so on. To run a game at 60 fps, you need a very stable over 60 fps rendering. A single frame below 16ms and you’ll have a drop to 30 fps.

Still, 30 fps is already a good enough experience for the human eye. We’ve seen before it’s better to have around 40fps to start to have something really interesting for our motion perception but having a stable 30 fps is way better than having some stuttering trying to reach 60 fps.

The solution: Variable Refresh Rate Up to now, the GPU was forced to align itself with the refresh rate of the monitor which generates so many complex issues we’ve seen before. What if we invert the master? What if the GPU would now decide when the monitor should refresh the image on screen?

This is the concept behind a variable refresh rate. The GPU will generate its frames as fast as it can and will notify in real-time the monitor rather than letting the monitor calling the GPU for a new frame like seen before. Indeed, remember than the fixed refresh rate of 60 Hz comes from the CRT era. LCD or OLED panels shouldn’t be forced to follow those rules.

Thanks to that, if the GPU is rendering at 42 fps, the monitor will refresh at 42 Hz. Even better, if the rendering varies from 30 to 60 fps, the monitor will also adapt from 30 to 60 Hz in real-time. This is then eliminating all issues: no stutter, no drop to 30 fps, best input lag and the overall smoothness perception will be better. Going higher than 60 Hz will depend on the underlying hardware and technology. Some monitors can go as high as 144 Hz and even more.

On PC, this technology exists either with the nVidia G-Sync implementation or AMD FreeSync. On console, the Xbox One S and Xbox One X are already supporting FreeSync. The next generation of console PS5 and Xbox Series X will also support VRR over HDMI 2.1 up to 120 Hz. Very recently, some Samsung QLED and LG OLED TV have added some support for VRR.

We can at last free ourselves from the original Tesla decision to limit ourselves to fixed 60 Hz refresh rate!

Well, you should now better understand the difference between frame rate and refresh rate. You should also better understand why it’s better targeting 30 fps if 60 fps can’t be render at a very stable rate but why 60+ fps is much better to have when possible.

The next article will be dedicated on explaining what pixel and vertex shaders are: Understanding Shaders, the secret sauce of 3D engines.

View Details

During our last PWA Builder release, we recently announced a new feature to generate from your PWA a signed APK. It’s fully generated from the cloud, no need to install anything on your machine such as Android Studio and it’s ready to be published directly to the Play Store!

Let me show how easy it is. In a couple of minutes, you will be able to submit your PWA to the Google Play Store.

But before that, first question you may ask: why would I publish my PWA in a Store? Isn’t the beauty of PWAs to potentially replace native applications from apps stores by being installed directly from the browser? Yes, indeed! But as an app developer, you should try to reach all possible users, wherever they are and go.

Users often search for a specific application through the store of their device. Publishing a PWA in apps’ stores, either in the Microsoft Store, the Samsung Galaxy Store or the Play Store, offers the following advantages: discoverability, trustworthiness, easy install and business insights as you will be able to get some analytics from the stores dashboards. View it as another way to distribute your PWA. Your users will be able to find your app directly from a browser via a link, from a search engine or from the apps stores! You will target all those channels with a unique code base and unique hosting of your PWA.

Before jumping into the steps of generating your package for the Play Store, let me first explain you what a Trusted Web Activities (TWA) is. What the heck?!? Yet, another acronym! Don’t panic, the concept is simple. It’s a way to publish your PWA as an APK to the Play Store. Once you’ll click the “Install” button from the store, you will have an identical behavior as if you would have installed your PWA directly from the browser. In order to prove you’re the owner of the PWA / website published in the Play Store, you must create a digital asset link associated to the submitted APK at the root of your website, under a specific folder. You can read more about this: Using Trusted Web Activities. PWA Builder is going to simplify a lot this process!

On my side, I’ve followed those steps to publish my small PWA game Apples Crusher into the Play Store. Don’t hesitate to install it on your Android phone to check how this works. By the way, I’ve already blogged about how I’ve made this game: Using WebGL and PWA to build an adaptive game for touch, mouse and VR devices. But any PWA will work.

Step 1: submit your PWA URL to PWA Builder Navigate to https://www.pwabuilder.com, enter the URL of your PWA and click “Start”:

We will then analyze your PWA and suggest possible enhancement if needed for your manifest or service worker.

If your score is already high enough, you can click on “Build My PWA”.

Then choose the Android platform and click “Download”. It will start the building process of our signed APK on our servers and once done, you’ll be able to explore the downloaded ZIP archive.

Step 2: test your APK on your Android phone First, you will have to enable the developer mode on your phone and also allow the installation of apps from unknown sources in your settings.

Then, you have 2 ways to proceed. If you’ve generated the signed APK from a desktop machine, copy your downloaded signed APK on your Android phone via USB and install it.

But you’ve got an even better solution than USB… Simply do everything on your Android device via the browser and PWA Builder! Look at this video to see how I’m packaging our own PWA as an APK and install it directly from my phone:

Generating and installing a signed APK from your PWA in less than 45s! You can even submit this APK to the Store still from your mobile. How cool is that?

Step 3: create the digital assets link and put in on your webserver As you may have noticed in the previous video, once our generated APK is launched, the PWA is shown with the address bar. Obviously, this is something you’d like to remove in order to have a full app like experience. For that, you need to prove you’re the owner of the PWA and associate it to your signed APK.

When you will download our ZIP archive from our site containing the signed APK, you’ll find also the next steps in the “Next-steps.md” file pointing to: Creating your asset link file. In a nutshell, you need to:

1. Download & launch this Android app from the Play Store: Peter’s Asset Link Tool
2. Select your installed signed APK containing your PWA
3. Generate the Digital Asset Link and copy the content into a file named assetslinks.json under a folder named .well-known at the root of your PWA.

Generating the digital asset link from your signed APK In the case of my PWA Apples Crusher game, it’s living here: https://applescrusher.azurewebsites.net/.well-known/assetlinks.json

Step 4: publish to the Play Store You’re ready to publish your APK to the Play Store! For that simply follow the Google documentation: Upload an app.

You will have a couple of screenshots to provide, to answer some questions about your app category, policy and so on. Please note you will have a warning after having uploaded our signed APK. It will say it’s not optimized. You can discard this warning; it won’t prevent you to publish.

A couple of hours – days later, once your PWA will have been validated by Google, it will be in the Play Store, congrats!

Please reach out either via our official Twitter account or on our github for any feedback.

Feel free to have a look to other platforms we’re building for as well as to the various components that could improve your PWA.

View Details

By busting 9 Myths on PWAs, we’ll see that PWAs are stronger than ever. It’s an approach that more and more developers are now considering in order to target multi-devices using a unique code base. As the New Edge is now based on the Chromium open-source project, this leverages new possibilities for PWA developers as well as for people using the WebView in their native apps. Indeed, the new WebView2 control will also offer the Chromium engine as explained in this article: Microsoft Edge WebView2 (developer preview). Both Win32 and UWP developers will be able to benefit from this new control.

Talking about native developers, they sometimes tend to underestimate the full potential of the latest web platform to build apps. That’s why, during the Insider Dev Tour 2019, we’ve done a fun session named: “Myst Busting PWAs – The New Edge Edition”. As I had a lot of pleasure creating the session, I thought it could be useful to create an article from it. I would also advise you to read a complementary article I wrote: 4 ways to create cross-platform apps using web technologies. We will exclusively talk about PWAs in this one.

To help you go through all the myths and associated demos, I’ve created this small PWA: https://mbp.azurewebsites.net/. Feel free to install it on your machine to play with the demos on your side. You’ll need the latest version of Edge Beta or Chrome installed as I’ll be using some specific APIs such as CSS backdrop filter and Service Worker Push Messaging, which is now enabled by default in those browsers.

You should then be able to view my beautiful PWA as in this screenshot:

The concept is simple: let’s review a myth together and bust it using the dedicated demo(s). Ready? Let’s go!

Myth 1 – PWAs need to be built from scratch Let’s start by a quick reminder of what a PWA is. It’s a web app served over a secure channel (HTTPS), exposing a web manifest to describe it (it’s a JSON file) and implementing a service worker to better manage offline or even boost the performance of the website. Those 3 components are required as the minimum set of features to create a PWA. Still, a PWA is a web app. It’s not a completely new technology that would force you to recreate your existing web app from scratch.

So, if you’ve got a recent and modern web app, already implementing a responsive design, you’re not very far from making it a PWA. Even better, we’ve got a tool to help you making it a PWA in a very easy way!

This open-source tool is named PWA Builder: https://www.pwabuilder.com. Let’s review it briefly together. Navigate to PWA Builder and you’ll see you can enter the URL of your website to have a review of it. Let’s do an inception analysis by entering the URL of PWA Builder into PWA Builder:

PWA Builder will check the 3 minimum requirements. It will check if you’ve got a manifest in place and can help you filling some missing properties or create a new one from scratch. It will check if you’ve got a service worker and finally check if you’ve a valid SSL connection. It will give you a score out of 100.

If you don’t have yet a service worker, we’re providing some samples ready to be used. Still, as I was explaining in this article Using WebGL and PWA to build an adaptive game for touch, mouse and VR devices in the Service Worker section, you’ll have to tune the code to really match your needs.

So, do you need to build a PWA from scratch?

• All the PWA core technology is designed to be layered on top of your current web app
• PWAs do not need to use any particular framework, or even be a single page application
• Don’t start from scratch if you already have a high-quality web app!

Then, we can safely say that this first myth is…

Myth 2 – PWA performance is limited First, as a reminder, don’t forget that the JavaScript engines powering the PWAs are blazing fast today! The performance optimizations done in the background to analyze your JS code and make it a compiled version of it to reach near native performance are truly impressive. I had the chance to view some sessions done by the Chakra team at Microsoft and from the V8 team by Google, and the algorithms used by the JS Virtual Machines (VMs) to JIT your code are spectacular. And I’ll talk about the GPU access later on.

Still, the heuristics used to guess the types through your code tree are not perfect and asm.js was the first attempt to help the VM doing a better job to produce a better native code from the JS one. Later one, an even more powerful approach was introduced with Web Assembly (WASM): “WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications”.

Both asm.js and WASM are used to produce a very optimized code for the browser from an existing C or C++ library. They won’t boost the performance of your regular JavaScript code. They are designed to run alongside JavaScript, allowing both to work together.

Launch this demo from my PWA launcher by clicking on the “Myth 2” button or directly from this site.

WebSight demonstrates a comparison of performance between JavaScript, asm.js, and WebAssembly. A user uploaded static image or live video is displayed for each target. Performance is measured by the length of time it takes to detect face(s) or eyes in the image or video.

Each target is run in its own web worker. The popular open source computer vision library, OpenCV, was compiled using Emscripten to asm.js and wasm (WebAssembly module) to utilize the Viola-Jones algorithm for object detection.

On a Surface Laptop 2, I’m running at 2 fps in JS, 20-25 fps in ASM.js and 40-45 fps in WASM.

As you can see, the performance boost between pure JS, asm.js and WASM are significant. So, if you have an existing C++ library, without any UI dependency and well-known to be CPU intensive, think about WASM as a possible path to reach the browser.

As a second demo of the potential of those technologies combined, open this link.

In Babylon v4.0, we introduced the support of the Ammo.js physics engine which is a port from a C++ library to WASM. Except for the physics engine, all the code of Babylon.js is written in TypeScript and compiled to regular JavaScript code. This sample is then a perfect demonstration of both technologies working together to re-use a C++ component and gain an important performance boost.

The last performance frontier for JavaScript will be multi-threading. Web Workers was a tentative project to better use multi-cores but they are by far much limited than true multi-threading.

Still, as you have seen during these 2 demonstrations, PWAs can benefit from really great performance.

So, PWA performance is limited?

Myth 3: Creating UIs is hard We often hear this from native developers that don’t know very well the web platform and its latest powerful features: creating UI and even more, beautiful UI, is hard. I’ve been doing XAML in the past to build Silverlight, WPF and UWP applications and really loved it, even if the learning curve could be high. I truly also love the web platform offering very efficient and simple to use UI components. It’s just a matter of spending a bit of time learning CSS correctly.

First, if you’re coming from a XAML world or a similar UI XML layouts like the one on Android, you’ll be glad to know that CSS Grid (inspired from XAML) and CSS Flexbox are now fully supported in all modern browsers and can be used safely in production. These are the new the core basics to describe your UI and to implement responsive design in an easy way. You’ll find some interesting articles explaining how to combine those 2 CSS features:

• How to combine Flexbox and CSS grids for efficient layouts
• Using CSS grid and flexbox together
• Relationship of grid layout to other layout methods

But what about the ready-to-use components to inject into those flexible boxes and grid elements? During the last years, we’ve seen many approaches inspired by the web components principles to create UI controls in a very clean & reusable way. One of the most popular one is probably React. For instance, if you’re wondering how we’re creating the UI/UX on our big popular web office apps, have a look to Fabric UI. Fabric UI is built on top of React to provide you sophisticated ready-to-use components such as the People picker, a Nav menu and many others accessible from only a few lines of code. You’ll find plenty of similar other frameworks available such as Ionic by searching on the web. I already talked about some of them in my previous article: 4 ways to create cross-platforms apps using web technologies.

Going a step further is by directly using the standard Web Components built in recent modern browsers. Web Components enable component-based markup across frameworks. It’s a family of web standards used to build components that act, look and perform like native page elements. It encapsulates your markup, CSS, and JavaScript to make sure they work on any pag or with any framework. And you now have a strong support across browsers, and great libraries are available like stencil, lit-element, and x-tag to manage a fallback via polyfilling.

Let’s see a small simple sample of a basic web component. Here’s the script to declare your new web component:

Once defined, you can use in your HTML page as any other standard HTML tag:

Insider dev tour

And you’ll obtain this:

I encourage you to read more about this on MDN: Web Components.

We decided to use this technology, supported by lit-element for the polyfilling, to create our new Microsoft Graph Toolkit. It’s a collection of web components to easily connect to the Microsoft Graph using a few lines of markup. Look at this small JSFiddle sample to better understand how this works:

I really love this simple but efficient approach. Feel free to have a look to the project by the way and don’t hesitate to submit feedback and/or contributions.

Well, again, it looks like that this myth sometimes believed by native developers is…

Myth 4: you can’t do high GPU intensive apps First, lot of people are often very surprised to discover the level of quality of the rendering as well as the performance of WebGL engines such as Three.js, Babylon.js, PlayCanvas, Sketchfab, Pixi.js and many others.

For instance, open those 2 samples:

Those 2 samples will work and probably render at 60fps on your smartphones, desktops, tablets and possibly even your console or Smart TV (performance less guaranteed there )! Without even talking about the WebVR support of some of those frameworks.

We’re using advanced rendering approaches like Physically Based Rendering or PBR, advanced shadows, etc. usually found in modern video games. Lot of the magic happens on the GPU side thanks to shaders. It’s a C like programming language built to discuss with the GPU. If you’re curious about how a shader works, you should try this tool: http://cyos.babylonjs.com/ built by my friend David Catuhe or check the infinite possibilities demonstrated on ShaderToy. If you’re interested in building a shader without writing GLSL code, have a look to the Node Material Editor. For instance, this demo is fun: https://nme.babylonjs.com/#NAI5U9 and you can even export the visually built shader to check the generated code for you.

WebGL 1.0 enabled already a lot of features and great GPU usage but WebGL 2.0 allows you to go a step further. If you know DirectX or OpenGL in the native world, you probably know that each new version has always exposed new APIs to better take advantage of the new hardware features implemented in the GPU. In the same way WebGL 2.0 is giving you access to special GPU features that weren’t accessible in WebGL 1.0 even if the hardware was capable of more. In certain cases, this could allow a performance boost and a better GPU usage. When you’re doing 3D or any GPU intensive apps, you need to use as much as possible the GPU as almost all the time, your web app / PWA will be limited by the CPU. This is even more true in JavaScript apps as we’re still limited in some cases because of its mono-threaded nature. Let’s then use the CPU in the most efficient way.

Note: Babylon.js now also supports OffScreen canvas to be able to free the UI thread while rendering 3D. Check the demo to see how it could improve the user experience.

Let’s go back to WebGL 2. To give you a simple example, let’s use the same demo, one using WebGL 1, one using WebGL 2.

Launch my Demo PWA and click on Myth 4.

It will run a Babylon.js demo using particles using a default WebGL 1 backend. Particles are often used in video games to achieve special FX such as fire, explosions or fluid simulation.

In you’re running on one of the latest Windows 10 updates, try opening the task manager while running this Myth 4 demo.

You can see that one of the 4 CPU cores of my Surface Laptop 2 is used at 100%, which logically roughly translated to a global usage around 25% of the whole CPU. JavaScript usually can’t use more of the CPU, except when using web workers makes sense. We’re then definitely into a CPU-bound scenario. Your framerate will be limited because of the CPU usage even if the GPU isn’t being used at 100%.

Let’s now ask Babylon.js to move some of the particles update logic to the GPU by using WebGL 2 which enables GPU particles. For that, simply check the “Use GPU particles” in the UI. First, you’ll notice that this enables much more particles to be potentially displayed as we’re switching from 50K particles max using WebGL 1 to 200K as a starting point in WebGL 2. Now, let’s have a look to the Windows 10 task manager details:

As you can see, even if we’re now displaying 4x more particles, the CPU usage has fallen to 2.0% while increasing the GPU usage to almost 100%. You’ll notice also that the CPU is not running any more at 3.0 GHz but only at 0.86 GHz. Switching to WebGL 2.0, in this specific case, has helped to better use the global machine resources.

We’re also working with Google to go even further in the GPU usage from a web app by collaborating on WebGPU. It will enable us to be closer to the metal, in the same way DirectX12 could help us being faster than DirectX11 or Vulkan compared to OpenGL. It will also provide some cool goodness like Compute Shaders. The Babylon.js team has worked on a preview version of the engine running on top of it: WebGPU is coming to Babylon.js. A sneak preview of it was done during a Google I/O conference Next-generation 3D Graphics on the Web. In this video:

You can see for instance an interesting boost in the framerate while lowering down the CPU usage a lot!

Well, I hope you’ll agree with me on the fact that this myth is…

Myth 5: PWAs can’t reach all my users

Lot of companies are interested in the stores because the different channel and engagement models compared to the web. Sometime, this could be an important driver to decide to use a native stack rather than web technologies to create an app as it will be easier to target the apps stores. Well, good news, today it’s also very easy to package your PWA to ship on an app store if you’d like to. PWA Builder can also help you generating the packages for you.

Let’s review an interesting application which took this approach. It’s Urza Gatherer, a PWA created to help you managing your Magic The Gathering cards collection: https://www.urzagatherer.app/

This application has been packaged to be published to the Play Store and the iOS store.

This idea is pretty simple. Take your existing PWA, host it inside a native app with the WebView full screen and you’re ready to go into stores. This has also interesting benefits. For instance, you will probably have to validate your app only once, as it’s the native package that will be validated. But as you’re getting the content from your web server to get the PWA content, any update on the PWA will be immediately available in the app installed on the user’s device. You won’t be forced to publish a new package to get the bug fixes or new feature exposed in the PWA, except if you’re adding a new native feature at the package level.

We’ve been working with other big partners to use this approach such as Twitter or Pinterest.

But PWAs are not just about stores obviously. You can also install a PWA via the browser’s installation mechanism on Android or on a desktop.

For instance, in the new Edge, when it detects that the PWA has a valid manifest, it will offer you to install it on your machine using the “+” button in the omnibar on the right:

Then, the PWA will be launched without the Chrome of the browser, if you’ve asked for that in your Manifest, making it a native app-like experience:

You’ll be even able to search for the app on your desktop:

And find all the PWAs you’ve installed by going to the edge://apps or chrome://apps tab:

You then see that PWAs can reach all your users:

• Go to PWABuilder.com to get store packages for every platform. Next version of Edge coming in the future.
• Web discovery helps you convert your existing Web users to more highly engaged PWA users.

Myth 6: PWAs look like web pages

To my point of view, creating a beautiful app is a matter of skills, not technologies. But of course, having great rendering features definitely help to raise the bar. Let’s review what you can do with the latest Web standard in this area.

Launch my installed PWA demo app and choose the Myth 6 to open my Fluent demo. Click on the hamburger button to open the menu:

To have a full idea of what this demonstration app is implementing, I’ve done this small video:

I’ve been able to mimic, using pure web standards, some Fluent design such as the Acrylic and the Reveal effects. To see in action the Reveal effect, simply move your mouse cursor around and you’ll find the borders of some element being revealed smoothly. The Acrylic blurring effect is being done thanks to the freshly available CSS backdrop filter. Try to scroll the page to see how the Acrylic effect is impacting the background.

Note: for curious people, this lib is using CSS backdrop filters, CSS variables, DOM Mutation Observers, Pointer Events, CSS Gradients & a couple of optimizations tricks for great performance.

Web pages have evolved a lot during the last months:

• Latest CSS & JavaScript specifications enable you to create fast, beautiful & responsive UI/UX: Grid Layout, Flexbox, CSS Filters, CSS Paint API (aka Houdini), WebGL, and so on
• You can embrace the design language of the targeted platform if needed

Myth 7: PWAs can’t run in the background Thanks to the push notification feature available inside the Service Worker, you can register to a push notification service and get notified of an event even if your PWA is closed. To know more about this feature:

• Official Edge Web Push Notification sample
• The very useful web-push node package
• Push API
• Notifications API

This is a very useful way to activate your PWA on a client machine via a server-pushed trigger. Let’s take a sample app I’ve been working on: https://northwindpwa.azurewebsites.net/ supposed to simulate a fake customers relationship management app.

Click on the chat button:

It will open a chat app which is the PWA we’re going to install. For that, click on “Install Northwind Customer Chat” in the omnibox and this will open the installed PWA on your desktop:

Close it. To generate a server-push notification, go back to the first screen and click on the “Contact Customer” green button. On Windows 10, you’ll have this notification generated:

And you can find it also in the notification center in case you’ve missed it:

Clicking on the notification will open back the PWA. You could even pass some data (such as the last message typed) to restore the PWA in the right context.

In conclusion, using Service Workers, you can subscribe to push notification services that will call you back, using WNS in Edge (Windows Notification Services) as well as other push services for different browsers.

Surprisingly, once again, this myth is…

Myth 8: Desktop hardware still requires a native app If accessing to any hardware from a web page was impossible in the past, it’s not the case anymore. As the browsers evolved, they now can cover a lot of different scenarios around hardware access using:

• Web BlueTooth to access BT LE devices
• Web USB for USB
• Web Midi for MIDI keyboards
• Web VR/XR to connect to all VR headsets & controllers

And more on the way!

In this article, we’re going to play with Web BlueTooth. If you’ve got a BT LE compatible devices, you can navigate to edge://bluetooth-internals or chrome://bluetooth-internals to have a useful debugging tool. You can enumerate the devices available near you and connect to one of them to check the available services exposed, try to read/write values, etc:

Let do a quick demo using this great Nordic Thingy 52 device and its associated demo page: https://developer.nordicsemi.com/thingy/52.

And as I think it’s easier to understand how it works using a video, I’ve done this one:

You see that we can already cover a lot of various hardware and platform access. But with Google, via the Chromium open-source project, we’re working on going even further through what known as the Project FuGu: https://developers.google.com/web/updates/capabilities. Web Share Target, Native File access, Badging, Contact Picker are some of the great features in the pipeline.

We’re also working on building new APIs to help PWAs developer targeting foldable / dual screens devices via the proposal of the Window Segments Enumeration API.

To summarize:

• Many hardware interfaces are available today for the Web.
• Future collaborations between Microsoft and Google in project Fugu will continue to enable more options within a PWA

Myth 9: Real brands aren’t using PWAs for consumer apps We’re reaching our last myth! I can often hear that also from people that tend to underestimate the importance of PWA’s usage as a way to build well-known apps. Let’s briefly answer this myth with a single picture:

Twitter, Pinterest, Starbucks and Trivago. You probably already heard of some of them. Some being even shipped as a PWA in our Microsoft Store. You’ll find more well-known PWAs on this site: https://www.pwastats.com/.

I hope by reviewing all those 9 myths you’ll have a better understanding of the tremendous potential of PWA’s technologies to build high quality apps for the web but also for your current & futures devices!

View Details

Vous n’avez aucune idée de l’accueil chaleureux que l’administration française réserve aux talents étrangers. J’ai eu la chance de vivre et d’accompagner une personne ayant vécu une histoire totalement hallucinante et irrespectueuse. Voici une petite histoire d’une gestion absurde de process en 2019 à l’ère du numérique. Accrochez-vous, même un scénariste d’Hollywood n’aurait pu être si créatif.

Note : c’est la version « director un-cut » dite version longue d’un thread Twitter qui a été populaire : https://twitter.com/davrous/status/1157224994436452352. On m’a conseillé d’en faire un article complet pour plus facilement partager ce cauchemar et peut-être éveiller les consciences. Le voici.

Février 2018, ma chérie, après plusieurs années de présence, de hautes études et de travail / impôts en France, aimant ce pays, se sentant en harmonie avec ses valeurs demande la naturalisation. Ayant travaillé dans plusieurs start-ups françaises, dans de grandes entreprises internationales ainsi que dans une fondation au service des jeunes français, elle souhaite sauter le pas. Quelle fierté d’attirer de nouveaux talents!

Elle se renseigne sur Internet pour connaître la procédure. Mais n’arrivant jamais à joindre le numéro censé gérer la prise de rendez-vous pour la dépose du dossier, elle regarde le site officiel, réuni elle-même l’ensemble des pièces. Elle vérifie les prérequis et me demande de l’amener à Évry.

Évry. La préfecture la plus incroyable et douce de France sans doute. Une sorte de paradis de l’étranger, terre d’accueil onirique où les valeurs de la République française brillent de milles feux.

Alors si vous ne connaissez pas le principe, c’est assez amusant. Vous êtes dehors, tôt, voire très tôt (mais vraiment hein) à faire la queue contre des barrières que nous, chanceux français, avons plutôt l’habitude de balancer sur les méchants pendant les manifestations. Déjà, j’étais en stress car le temps d’attente moyen dehors est de 3h et il y a 2 queues. Comment savoir si vous êtes dans la bonne queue ?

Heureusement, une dame arrive avec un mégaphone pour nous expliquer les règles du jeu (mieux vaut le prendre pour un jeu). La dame avec un ton des plus aimables : « Retcheu peutcheu neutcheu GRETEUCHEU sur la gauche ! Teuteucheu reutche patanoutcheu DEUTEUCHE sur la droite ! ». Merci au son saturé et inaudible de son appareil de guerre.

Je suis en panique totale. Moi, le français natif, n’a absolument rien compris de ce qu’elle vient de gueuler dans le mégaphone. Alors je n’ose imaginer un étranger maîtrisant peu la langue. Tout le monde sur place semble pourtant détendu et être rompu à cet exercice d’humiliation. Bref, coup de bol, on est dans la bonne queue et nous arrivons au dispatch.

– Un monsieur : “Vous venez pour quoi ?” de manière aussi accueillante qu’une porte de prison (et le bonjour semble optionnel)
– Elle, avec un ton joyeux tout droit sorti du monde des bisounours : “Bonjour, pour une dépose de dossier de demande de naturalisation !“
– Le gentil monsieur me regarde alors avec affection : “c’est lui le mari ?” (ah non, je me trompe, en me regardant en fait avec dédain, je me souviens mieux)
– Elle : “euh, comment ça ?“
– Lui, pas content : “C’est pour une demande de naturalisation par mariage et c’est le futur mari ?“
– Elle : “Ah non ! C’est pour une demande par décret“
– Lui : “dans ces cas-là, il faut appeler le numéro !” puis il nous tourne le dos.
– Elle, toute gentille (elle restera gentille tout le long de cette histoire, ce qui a le don de m’énerver) : “oui mais personne ne répond jamais du coup je me suis dit que j’amènerai…“
– “Non ! Il FAUT appeler le n-u-m-é-r-o ! C’est comme ça“, il se tourne alors vers l’organigramme et dit “il y a 300 personnes ici, il y en aura bien une pour s’occuper de vous” puis il nous tend le numéro de téléphone magique que l’on connaissait déjà et nous prive définitivement de sa bonne humeur contagieuse.

Le numéro, le fameux numéro, que dis-je, la blague du numéro ! Disponible uniquement le mardi de 9h à 12h et de 14h à 16h30. Bien sûr, il est donc quasi impossible d’avoir l’une des 2 malheureuses personnes censées vous aider à prendre… Un simple rendez-vous ! Apparemment Internet, ils ne connaissent pas encore. Le standard téléphonique ne tient pas la charge non plus. Il date peut-être de l’ère soviétique.

Bon, on ne se démonte pas. 1er mardi, 200 appels le matin et autant l’après-midi. Échec.

Je vais sur les forums, certains disent qu’il leur a fallu… 1 an ! Pour avoir quelqu’un au téléphone ! J’hallucine grave.

198, c’est le nombre de tentatives du matin. Je me suis permis d’arrondir à 200, j’espère que vous ne m’en tiendrez pas rigueur.

Certains dans les forums me conseillent d’éviter certaines préfectures, dont celle d’Évry et indiquent qu’il vaut mieux viser d’autres bien plus efficaces. Ah oui ? Et comment fait-on pour être reçu dans une autre préfecture demandé-je innocemment ? Bah en déménageant tout simplement ! Non, je ne rigole pas. Certains m’indiquent qu’ils choisissent sciemment l’endroit où ils vont vivre en fonction de la réputation de la préfecture. Totalement dingue.

3ème mardi, on arrive à avoir quelqu’un ! Ah non en fait on découvre que la phase 2 c’est un répondeur qui demande d’attendre. Et après 40 min d’attente de musique, cela raccroche. Dépression. Mais jour de chance, nouvelle tentative et on a quelqu’un après 30 min. Et vous savez quoi ? 1er rendez-vous décroché ! FUCK YEAH !!!!

Petite blague pour détendre l’atmosphère : 2 semaines après que nous ayons réussi cette épreuve du feu, ils ont mis en place une demande de rendez-vous via un site Web. Fini le numéro magique ! J’étais presque dégoûté que les autres derrières allaient avoir un process plus simple. Ça rend vraiment con ces histoires ! Nous découvrirons plus tard que cela n’est pas forcément mieux…

Avril 2018. Le 1er rendez-vous. Il dure en fait 2 min. Ils vérifient 2 ou 3 trucs puis indiquent qu’ils vont envoyer une convocation par courrier et donnent l’ensemble des pièces à réunir pour monter le dossier. Bon ok, why not. Il y a peut-être moyen de faire cela en numérique non ?

Mai 2018. La convocation arrive par courrier que 2 mois plus tard ! Finalement, c’est fastoche 🙂 Je suis en déplacement, elle m’envoie une photo de la convocation. Je regarde vite fait la date du prochain rendez-vous, écrite à la main (à la main ???) et je vois Août. Je lui réponds : “Top ! C’est super cool, cela va assez vite en fait !“. Elle me répond : “Regardes bien la date…“.

Stupeur. La date du prochain rendez-vous est dans…1 an et 3 mois. C’est Août 2019 en fait ! On a relu plusieurs fois pour être sûrs. Je me suis dit qu’ils s’étaient forcément plantés vu que c’est écrit à la main, cela peut arriver. Sauf que pour vérifier, il faut appeler le numéro injoignable. Tristesse absolue…

1 an et 3 mois plus tard…

Août 2019. Le jour J tant attendu ! On attend derrière les portes roses. Les portes du bonheur ! Elle est préparée à bloc, a révisé les rois de France, les fleuves, la Marseillaise, les membres du gouvernement, parle parfaitement le français (bien mieux que la majorité des français dont je fais partie). Elle rentre…

Et ressort 2 min après. Ça pue grave. On a fait une grosse connerie. Que dis-je, une énorme bêtise ! Pendant les 1 an et 3 mois d’attente, on a été assez cons pour… vivre et… déménager. Il y a 1 mois. Putain… à un mois près…

Or, il faut changer l’adresse dans les 8 jours sur la carte de séjour. C’est la loi ! La carte n’est donc pas valide. Il faut remplir un formulaire pour faire le changement et reprendre rendez-vous. 1 an supplémentaire dans les dents du coup ? On verra… La personne qui l’a reçu lui a dit de reprendre un rendez-vous en ligne et qu’elle reviendra dans 2 mois refaire l’entretien. Pas de panique soi-disant ! Mais avant de revenir sur ce point, commençons par résoudre le problème du changement d’adresse.

Pour faire un pauvre changement d’adresse, vous vous attendriez à aller quelque part sur un site officiel, fournir un justificatif EDF ou équivalent et c’est tout non ? Ouh là là, non, pauvre fou ! Pas pour les étrangers ! Il faut remplir un formulaire et aller le déposer à un guichet. Allez ! Soyons des dingos, allons voir un aimable guichet.

La suite à la sous-préfecture de Palaiseau. Première journée, elle arrive vers 8h15 pour une ouverture à 9h. Une soixantaine de personnes devant. Grave erreur. Il faut absolument faire partie des 30 premiers pour avoir le droit d’avoir audience auprès de notre administration. Quel que soit le motif. Oui vous avez bien lu. Quel-que-soit le motif. Cela parait absurde non ? Attendez, vous n’allez pas être déçu de la suite.

Elle arrive jusqu’à un fonctionnaire de l’administration qui lui donne ce précieux conseil : il faut poser sa journée, venir à 6h du mat avec une chaise pliante, à manger et à boire. À 9h, espérer faire partie des 30 chanceux quotidiens. Ensuite, il faut s’attendre à passer la journée dans la sous-préfecture pour, rappelons-le… Indiquer un changement d’adresse. On est passé dans la quatrième dimension.

Petite aparté. Apparemment, d’autres administrations françaises semblent nettement plus efficaces pour gérer un changement d’adresse et ce, même pour un étranger. Un exemple : les impôts ! C’est amusant mais eux, ils ont effectué le changement d’adresse immédiatement. Pas besoin de vous rendre où que ce soit, tout est fait en ligne et très rapidement. Mais pourquoi une telle différence ? C’est rapport à l’argent me dit-on. Malgré tout, je suggère ardemment aux préfectures et sous-préfectures d’organiser des rencontres avec l’administration fiscale afin d’échanger sur ces « best practices » comme on dit chez les startupers.

Là commences la deuxième partie de la saga que j’ai intitulée : « douces nuits d’été sur un trottoir de Palaiseau ».

Elle a tenté les 2 journées suivantes de venir à 6h30 puis ensuite à 6h mais à chaque fois, plus de 30 personnes. Je prends les choses en main. Je lui dis que nous allons descendre de 1h tous les jours jusqu’à trouver l’heure optimale pour être pris. Du coup, le lendemain, nous sommes arrivés à 5h, motivés, devant la sous-préfecture. Et devinez quoi ? Il n’y avait personne dans la rue ! Victoire ! Ah, ah, quelle bande de fainéants ces étrangers. Il suffisait de venir à 5h du mat. Ils me font doucement marrer.

Ah mais non en fait ! Une voiture fait des appels de phares et donne un coup de klaxon. Des personnes sortent de voitures, tapies dans l’ombre. Elles nous indiquent l’existence d’une liste non officielle sur laquelle il faut s’inscrire. Il n’y a que 4 personnes présentes dans la rue et pourtant, on est 11ème sur la liste. Les autres se sont inscrits puis se sont barrés dormir chez eux. Ambiance.

Bon bien sûr, vous avez envie de dire que sa liste non officielle, il peut se la mettre où je pense et faire l’avion avec. Mais bon. Il est 5h du mat, vous n’avez pas envie de vous battre car d’une part, vous ne savez pas bien vous battre et en théorie théorique, 11ème, ça passe dans le wagon des 30 galériens quotidiens.

A noter bien sûr que c’est illégal de faire ça. Ce sont simplement les gens qui essaient de s’organiser par eux-mêmes tout en grugeant finalement les autres qui viennent vraiment 4 ou 5h avant camper. Le trottoir appartenant à la ville, la préfecture ne peut rien faire. Enfin, c’est ce qu’elle raconte, c’est toujours pratique de rejeter la faute sur l’autre hein.

En discutant avec les autres galériens, j’apprends au passage qu’à certaines préfectures, une véritable mafia s’est installée pour vous faire payer votre place sur cette liste. Comme manifestement les autorités s’en foutent, certains ont flairé le business de la détresse. On m’a parlé de tarifs jusqu’à 150€ !!! Nous, au moins, c’était gratos. Certains étrangers sur place me disaient qu’ils préféraient d’ailleurs payer plutôt que de prendre le risque de poser une journée de travail pour rien. Chères têtes pensantes de l’administration française, vous entendez cela ? Ils seraient prêts à payer plutôt que de vivre cela ! Montez votre startup les gars !

Bon au fur et à mesure, la tension monte sur place car les premiers sur la liste arrivent plus ou moins au dernier moment. Du coup, certains commencent à s’auto-proclamer chef de la liste pour la réorganiser à leur avantage. On vous rappelle que c’est quand même un pauvre bout de papier qui a été déclaré liste officielle de journée. Certains m’ont dit qu’ils voyaient même des mecs débouler avec… bah… leur propre liste forcement Bah oui ! D’où ta liste pourrie est la vraie ? C’est la mienne mec !

Bon, je ne vous raconte pas d’ailleurs la tension sur place quand numéro 1 se pointe comme une fleur à 8h/8h30 pour gentiment vous passer devant.

D’après une discussion que j’ai eu avec la sous-préfecture, il faut normalement appeler la police en cas de liste mais après, vous vous retrouvez seuls avec les 40 autres à 5h du mat… C’est vous qui voyez !

Malgré tout, je fais quand même connaissance avec plein de gens super sympas, Sénégalais, Algériens, Tunisiens, Ivoiriens. Tous avec de bonnes grosses galères alors qu’ils sont tous en règles. Mais ils le prennent avec beaucoup de calme et philosophie étonnamment.

Après 4h d’attente et beaucoup de fatigue, tout va se passer en 10 min. La police arrivant 5 min avant l’ouverture pour gérer une éventuelle émeute. On est donc 11ème à cause de la liste magique mais nos chances d’être pris sont maximum. On passe la barrière !!!

– Le responsable de la barrière : “C’est pour quoi ?” (en perpétuant cette longue tradition d’absence de bonjour)
– Nous : “Bonjour, c’est pour un changement d’adresse !“
– Lui : “Non.”
– Nous : “Quoi non ?“
– Lui : “C’est le lundi, mercredi ou vendredi et là, on est MARDI.“
– Nous : “Quoi ?!? Mais on est venus tous les autres jours et on n’a pas réussi donc vous pouvez peut-être…“
– Lui : “NON ! Le changement d’adresse, c’est le lundi, mercredi ou vendredi ! Et là… on est MARDI ! Faut lire Monsieur, c’est écrit !”

(Là, je retire la lame de rasoir des mains de ma chérie et je l’empêche de se tailler les veines).

Eh oui ! Nous sommes de gros abrutis ! Nous n’avions pas vu que le changement d’adresse, ça ne marche pas ni le mardi ni le jeudi (bah oui, c’est tout de même évident). Heureusement que j’aime les trottoirs de Palaiseau et que nous avions nos chaises pliantes. Vraiment charmante cette ville, je vous la recommande chaudement.

Seule lumière de la matinée, je rencontre l’une des responsables qui me reconnaît grâce au fil Twitter (truc de fou !!!). Elle est charmante, concernée, s’excuse de la situation et compatis réellement de tout cela. Elle m’explique l’envers du décor.

Il n’y a qu’une seule personne au guichet pour gérer TOUTES les demandes, peu importe le sujet, pour la journée. Pas toujours la même heureusement. Mais pour éviter le pétage de câble du fonctionnaire winner du jour, ils limitent entre 15 et 30 personnes l’accès à la préfecture pour tous les dossiers. Souvent cette personne ne mange pas de la journée. Je ne sais pas comment ils choisissent cette personne pour cette journée de l’enfer. À la courte paille peut-être. Ou en représailles d’un comportement passé peu apprécié de ses camarades.

Elle m’explique que le changement d’adresse ne peut se faire via le Web pour éviter la fraude, afin de vérifier le passeport et carte de séjour. J’avoue être moyennement convaincu car un changement d’adresse ne devrait pas tout remettre en cause les vérifications précédentes. J’apprendrai plus tard qu’il y a un projet pour faire le changement d’adresse un jour en ligne.

Mais on la sent vraiment humaine et en manque de moyen. Leur job est super difficile, beaucoup craquent sous la pression et l’agressivité des gens puis partent en arrêt… Ce qui forcément ajoute du délai et de la pression sur ceux qui restent. Cercle vicieux. Job bien pourri apparemment.

A noter que le changement d’adresse se fait en 10min. Il suffit donc d’être assez malins pour camper un jour où le guichet dédié sera ouvert. Bonne nouvelle, ce changement est immédiatement opposable à la procédure de naturalisation. À se demander pourquoi l’agent d’Évry ne l’a pas fait au lieu de nous jeter…

Malgré tout, je tiens à remercier chaleureusement cette dame pour avoir pris le temps de nous parler, de nous expliquer “l’autre côté” qui est loin d’être simple aussi. Manque de moyens et de volonté politique pour changer quoi que ce soit.

Au passage, cette situation est loin d’être récente et semble donc se dégrader :

– Palaiseau : les étrangers font la queue la nuit devant la sous-préfecture
– Le service des étrangers en sous-préfecture de Palaiseau est totalement saturé

Entre temps, dans les bonnes nouvelles, nous avons réussi à choper un nouveau rendez-vous pour déposer à nouveau le dossier de naturalisation d’ici 2 mois, via le site en ligne qui est une blague monumentale. Il ouvre tous les dimanches à minuit pile et à 00h01, il pète avec une erreur 500. Traduction : le site se vautre comme une crêpe. Bref, à l’heure du Cloud, il ne tient pas la charge et propose des formulaires conçus par des malades mentaux comme celui-ci :

Dès la première question, on se fout de votre gueule et on vous manque de respect. Faut le faire quand même, non ? Si je croise les développeurs web ayant commis ce crime, je serai dans l’obligation de les taper. À moins que la connerie fait partie du cahier des charges (franchement c’est possible). Le site est peut-être hébergé chez la mère de l’un des stagiaires aussi. Du coup, forcément, il pète rapidement. Je ne vois vraiment pas d’autres explications logiques.

Certains galèrent énormément pour réussir à prendre le rendez-vous via le site et peuvent mettre jusqu’à plusieurs mois.

Restait à cracker le code du process de changement d’adresse… Et je vous la fais courte, nous n’avons jamais réussi, malgré plusieurs autres matins à 5h du mat, à faire partie des 30 glorieux. Certains arrivent carrément la veille et dorment avec un duvet sur le trottoir. Heureusement, notre histoire a fini par arriver aux oreilles d’un très gentil Sénateur qui nous a aidé à obtenir un rendez-vous pour faire le changement d’adresse. Merci énormément à lui. Mais j’étais à la fois super content et super triste. Content bien sûr d’avoir réussi à faire ce changement d’adresse maudit. Triste de devoir by-passer tout le système qui devrait pouvoir offrir une expérience un peu plus humaine à n’importe qui.

Septembre 2019. Aujourd’hui, nous sommes donc retournés à Evry la merveilleuse pour en finir, nous l’espérions, avec tout cela. Elle répète la Marseillaise dans la voiture avec Mireille Mathieu, révise à nouveau les membres du gouvernement. Bah oui, à chaque fois ils sont différents, vu les délais entre chaque rendez-vous.

Vous vous souvenez de la dame qui avait dit pendant l’entretien de naturalisation : “ne vous inquiétez pas, cela va prendre deux mois et on se revoit” ?

Et bah on prend le ticket et… Ça pue à nouveau. Direct. On n’est plus dans le même endroit que la dernière fois. Nous ne sommes plus devant les fameuses portes roses donnant accès aux pièces sacrées et enviées de tout étranger qui se respecte et où se déroulent les entretiens pour la naturalisation.

Nous sommes apparemment repartis à zéro ! Arrivée au guichet, on explique qu’il y a sûrement eu une erreur. Une très gentille dame est également surprise du process et nous ramène devant les portes roses, et nous demande d’attendre. Espoir…

Elle sort et nous demande de la suivre. Une autre dame sort avec un grand sourire. Bon signe ! (Spoiler : non). Elle nous explique que suite au changement d’adresse, on repart bien de zéro. Je suis halluciné. Je montre mon mécontentement.

En retour, cette petite remarque assassine : “nul n’est censé ignorer la loi !“.

Sauf que c’est totalement faux. Cette expression est réservée au pénal. Je te garantis ma grande que je peux te trouver plein de lois dont tu n’as même pas idée de l’existence. Cette phrase est d’une agressivité inouïe. D’autant qu’ils sont nombreux à la préfecture à ne même pas connaître leurs propres procédures et lois. Et franchement, vu le bordel, je ne leur en veux pas forcément mais qu’ils ne viennent pas faire la morale en retour !

Ensuite elle nous explique qu’un étranger signe un contrat avec l’Etat et que ce contrat n’a pas été respecté puisqu’elle a changé d’adresse et s’était engagée à le signaler. C’est vrai qu’elle a acheté une maison, paie désormais des impôts fonciers et des frais de notaire, une vraie trahison pour l’Etat. C’est assez logique de se faire punir en retour.

Donc voilà la conclusion après presque 2 ans : retour à la case départ sans toucher les 10000 francs. J’ai une image encore plus fortement dégradée de mon pays.

Conseil aux étrangers : ne changez surtout jamais d’adresse une fois le processus de naturalisation lancé. Sauf si vous voulez jouer au même jeu débile que le nôtre.

Allez, je vous en ai quand même gardé une bonne pour la fin.

Du coup, le guichet l’a quand même reçu pour faire à nouveau le premier entretien, celui censé valider rapidement que vous êtes éligibles pour le futur vrai entretien. Vous vous souvenez ? Celui qui avait duré 2 min et où nous avions reçu une convocation 1 an et 3 mois après, suite à cela. Bon, moi, j’attendais super énervé mais serein sur cet entretien malgré tout. L’évidence même : vu qu’elle avait déjà tous les papiers pour son entretien final, elle est forcément en règle pour le tout premier.

Quand surgit l’ultime blague. En fait, son dossier n’est pas recevable car l’une des pièces n’est pas conforme. Il faut une version originale de son extrait d’acte de naissance tamponné par son pays d’origine et là, c’est une copie. Elle doit donc retourner au pays pour récupérer ce papier avant de faire à nouveau sa demande. Les bras m’en tombent et je perds tout espoir.

L’ultime question du coup : comment avons-nous pu passer la première fois ??? Erreur humaine d’après la dame. On devrait même être content car de toutes façons le dossier n’aurait pas été validé par le Ministère ainsi. Ah bon, bah ça va alors, je suis content puisqu’on me dit de l’être. Par contre, elle se dit admirative de voir une femme tenter la demande de naturalisation par elle-même, en dehors du mariage. Elle dit que c’est assez rare. Elle semble sincère mais franchement, vous avez l’impression que le pays aide ce genre de femme courageuse ?

On ne sait pas comment tout cela est possible. Elle pense peut-être avoir donné une fois ce fameux original mais ne s’en souvient pas vu la tonne de justificatifs et rendez-vous qu’elle a eu. Ils l’ont peut-être tout simplement paumé. C’est possible vu que tout est géré sur papier et que l’on écrit les dates de rendez-vous à la main…

Certains sur Twitter m’ont dit vivre sur le territoire depuis 40 ans et avoir abandonné la procédure de naturalisation, dégoutés. D’autres m’ont dit être partis dans d’autres pays capables les accueillir plus dignement. Vous êtes nombreux à m’avoir partagé vos histoires bien tristes et révoltantes.

A l’heure de la fameuse Start-up nation, des Next40, de Station F, des défis du numérique dans lesquels la France est en retard sur presque tout, nous ne sommes pas capables d’accueillir de jeunes talents diplômés étrangers dans des conditions descentes. Et d’accueillir des étrangers tout court en règle.

Donc, si un jour vous entendez à nouveau cette hérésie de dire que l’on donne tout et facilement aux étrangers, vous pourrez dire que c’est totalement faux. C’est même une immense connerie.

Devenir français, ça se mérite. On ne donne pas la naturalisation à n’importe qui !

Mais la France mérite-t-elle vraiment ces talents ?

View Details

We’re going to review in this article, 4 major ways to use web technologies to create cross-platforms applications. The idea is to help you decide when and how to re-use your web skills to have the highest reach possible, decide of the UI richness level you need and try to share your JavaScript code cross-devices as much as possible. It’s based on my latest customers’ meetings, internal discussions & projects I’m working on at Microsoft as well as my own researches on the web. Thanks to that, I’ve identified 4 possible paths: PWA, Electron, Hybrid apps and JavaScript Native driven approaches. We’re going to cover each of them and I will share their pros & cons.

Note: I’ve been working on preparing this content for 2 conferences I’ve delivered on this topic. One was in Ukraine for the Item 2018 conference (video here) and one was during the French MS Experiences 2018 event. As the feedback was positive and as people asked me for written content to share, I’ve decided to write this article. It’s also my very personal point of view I’m sharing. I’d love to discuss about your own point of view, feedback and experiences in the comments section!

Let’s start by the life of a front web developer today. It looks like this (mess?):

And it’s just a slice of what exists today. Obviously, you can’t know all those frameworks & tools. There isn’t a specific combination to rule them all neither. You will have to make a choice based on your preferences, needs, projects requirements and existing skills set of your team members. But the good news is whatever the stack you may have chosen, it’s a web stack. This will provide you additional benefits for cross-platforms support.

PWA Let’s start by the obvious and main solution you should think of: Progressive Web App or PWA. If you still don’t know what a PWA is, here are good resources to read/watch:

– Our main entry point containing tons of good resources: https://developer.microsoft.com/en-us/windows/pwa

– Building Progressive Web Apps during BUILD 2018 by Jeff Burtoft

– Progressive Web Apps on Google Developers

– Progressive web apps on MDN

In a nutshell, a PWA is a web app using the latest available features to match as close as possible a native app. For instance, it can be installed on a desktop/mobile, will work perfectly offline and should use a modern UI approach. As it’s based on standards, it works cross-platforms on Windows, Mac, Android, iOS and could even fallback in any web browser on your Smart TV for instance!

The minimum viable you need to implement to start claiming you’re building a PWA is:

– HTTPS. A PWA must be served over a secure channel.

– Web App Manifest. A small piece of JSON describing how your PWA should behave or how to install it.

– Service Worker. The core part of your PWA enabling a true offline scenario as well as boosting the performance of your web app.

I already covered in a previous article how PWA Builder can help you building this manifest and how I’ve been implementing it in my own PWA: Service worker, manifest and icons to make it a PWA.

By the way, my PWA, which turns out to be a WebGL/WebVR game, is available there: https://aka.ms/applescrusher. You can load it in any recent browser or device.

Note: you can read the full article: Using WebGL and PWA to build an adaptive game for touch, mouse and VR devices if you’d like to know how to build a similar PWA game.

If you’re opening my PWA on an Android device, you will have a similar experience as shown in this video:

Thanks to the Web App Manifest built by PWA Builder, the user can install the web app on his home screen. You can review the manifest on Github as well as the complete source code of this PWA game.

You’ll see I’ve decided to use a display property set to standalone. This will remove the border UI (the “chrome”) of the browser when the app will be launched from the home screen. The experience will then be different than the classical one you have inside the browser. I’m also forcing the orientation to be set to landscape. This is very useful in the case of games as the gameplay is often optimized for a specific orientation. Today, the usual trick is to use a media query to ask for the user to rotate the screen. You’ve probably already seen that. Thanks to the Web App Manifest, you can be sure that your app is using the right orientation directly.

Now that we covered the beauty of the manifest, let’s talk about the Service Workers. As a reminder, the Service Worker is a piece of JavaScript acting as a client proxy between your web page and the server. You first need to register it and once installed, it will be able to catch every fetch request sent by your web page to the network.

Based on your application’s logic and targeted experience, you have several possible strategies to manage the service worker’s cache. PWA Builder is providing various service workers ready to be used. You will probably have to adapt their code to perfectly match your needs, but it’s a good starting point.

As explained in my article, in my case, I’ve coupled a service worker with the usage of IndexedDB. The idea is to provide an offline experience that should be available just after the first complete load of the web page. Thus, if you manage to load my game on your mobile just once, even by switching to airplane mode immediately after, you will still be able to reload it. Just like a normal native app you would have installed from a store. In my case, the first load could be considered as the downloading process of an app from a store.

To highlight the result, I’ve done this second video to show you that this is working as expected:

As you can see, even without any network connectivity, the game can still be loaded from the browser. The service worker is getting the application files (JS, HTML, CSS and images) from its cache while the game engine is taking the game’s 3d assets from IndexedDB. I’m pretty proud that the IndexedDB layer I’ve written years ago for Babylon.js is still totally relevant for PWA

We just covered some of the main benefits provided by PWA via this use case. But there’s now a lot of PWAs out there on the web, and they have interesting feedback to share. For instance, if you go on: https://www.pwastats.com, you’ll learn that the PWA version of Twitter has managed to create a better user engagement.

And you’ll see several other great testimonials like this one.

However, even if the web platform has done impressive progresses in term of features exposed, there are still some limitations compared to native apps. This web site is interesting to give you an idea: https://whatwebcando.today/

As you can see, you can’t have access to NFC for instance. To go beyond this barrier, there are several possible solutions, the cross-platforms ones are exposed after.

In our case, at Microsoft, we offer you this step further on Windows 10. You can submit your PWA into the Windows Store for a better discoverability, possible monetization and to better integrate the OS paradigms by using some of its design language. You can also call the native WinRT APIs from JavaScript to go beyond of what the standard web specifications offer you today and break the barrier described just before.

Our main documentation entry point on PWA for Windows 10 is: https://developer.microsoft.com/en-us/windows/pwa. Please read our documentation explaining how this works: https://docs.microsoft.com/en-us/microsoft-edge/progressive-web-apps/get-started.

PWA Builder can package your PWA for the Windows Store for you and we’re sharing some possible native integrations on it. But if you’d like to know what are the APIs you can call from JavaScript once hosted in a Windows 10 native UWP container, please read: Windows Runtime (WinRT) for JavaScript.

Some of our partners have published their app using this approach such as Twitter.

And to conclude on my own experiment, my game is also available in the Windows Store. I’ve explained in the dedicated section of my article the customization I’ve made for the Windows Store.

PROs & CONs So, what are the PROs & CONs of a PWA you need to have in mind when you’ll think of your new app.

PROs

• It’s just a web site!
• Easy updates on the web server
• Same code everywhere

• Use your favorite framework (Angular, Vue, React) or language (TypeScript, JavaScript)

• High reach / Any device

• Indexable

CONs

• Same UI/UX on all platforms
• Don’t (always) have a full access to the platform / hardware

Indeed, even if you’re submitting your app to the Windows Store, the updates will be done on the web server hosting the PWA and will be immediately reflected on the user’s device. You will have to go through the validation process the first time, of course, but future updates will be much faster to push to the device than an update of a classic native app where a new package has to be pushed every time. If you don’t publish on a store, well, it’s just a web site again. Web is offering by far the highest reach possible today as every device has a web browser. It’s also easy to be indexed by search engines. Deep linking is part of the web whereas it could be something difficult to implement in a native app.

Of course, not everything is perfect using a PWA approach. You will have the very same UI/UX on all platforms. If you need to differentiate the UX on a specific platform to embrace its specific design language, this is not something easy. Also, as discussed before, you won’t always have a full access to the platform. If your app must discuss with a USB device for instance, this could be a blocker.

So, what are the options to resolve those cons?

Electron You probably already used an Electron app without knowing it.

Visual Studio Code, Microsoft Teams, Slack, Skype or Discord are all based on Electron for instance, to name a few. You can view an exhaustive list of other apps using Electron here: https://electronjs.org/apps.

So why Electron is so popular and what does it offer? Electron can package your existing web app, which could already be a PWA by the way, and extends it on the desktop to break the barriers discussed in the previous section. It targets desktops only: Windows, MacOS & Linux. Electron can’t target the iOS or Android devices.

Electron is packaging your web app alongside with the full Chromium and Node.js binaries. It means that every time you’re installing an Electron app, it installs a full copy of Chromium and Node.js with it.

Note: Electron is a project created and maintained by Github. As Github has been acquired by Microsoft, it turns out Electron is a Microsoft project in a way now

There used to be some important delta between the version of Chromium shipped with the Electron project and the official Chrome release. This has been reduced recently. For instance, the Electron 3.0 stable release has shipped in September 2018 with Chromium 66. You can find more recent releases of Electron shipping Chromium 68 or 69. Still, Electron is usually a couple of months behind the latest public builds of Chrome/Chromium. It means that you won’t be able to ship a code using the very latest features shipped with Chrome inside an Electron app. Keep that in mind. But you will just have to wait for a couple of weeks/months before it becomes available in the Chromium version shipped in the Electron branch.

An electron app can better integrate the desktop environment. As explained in their doc, you can set the recent documents, manage notifications or drag files out of the window for instance. You can also have access to the file system via Node.js as explained in this doc: Electron Application Architecture.

As shown in the diagram above, Electron is providing an abstraction layer via a set of APIs. You can then have a common JavaScript code targeting several platforms, the bridge is done by those APIs. But you can also use or write a native Node.js plug-in to make specific calls to the platform if needed. In the case of Windows 10 for instance, if you’d like to make native calls to the WinRT API, you can use this Node.js module: https://github.com/NodeRT/NodeRT. This is a project written & used by Slack to better integrate the Windows 10 platform.

On my side, I’ve decided to check, for fun, how far I could use Electron to create a copy of the default installed Calculator app which is written using UWP/XAML. First of all, to achieve the Fluent Acrylic effect / blurry background, I’ve found this NPM package: windows10-fluently-vibrancy for Electron apps.

Using the latest CSS features such as CSS Grid and Flexbox combined with some other tricks, I’ve managed to create something relatively closed to the design of the Calculator app. Look at this video comparing both versions:

So, who’s who?

The first one is the UWP version and the second one is the Electron prototype I’ve built. You’ll see I’ve been able to do the compositing effect with the background to create the acrylic effect and even managed to partly implement the reveal effect on some of the buttons. The background compositing effect is done thanks to a call to a native Windows API. This was to illustrate you how Electron could embrace some of the platform’s specifics.

So as you can see, you can use Electron to share your code as much as possible cross-desktops, but you can still use the platform’s API to do a great job integrating its design language.

PROs & CONs Again, to help you deciding whether Electron could be a good choice for you, here are the PROs & CONs:

PROs

• Use your favorite web stack (front + node.js)
• Controlled browser’s engine
• Can interact with the system via native calls
• Linux / MacOS / Windows easy targeted

CONs

• Embed a full Chromium & Node.js
• Size

• Security updates of Chromium

• Desktop only

• Web UI

You can then take an existing website or PWA to embed it into an Electron app. You don’t have to learn anything specific on the front development side as it’s based on a very common stack all web developers know and love. As it’s embedding Chromium, you’re guaranteed to have a unique rendering & JavaScript engine to validate your code against. Compared to a standard PWA, you can go a step further by making native API calls through the Node.js layer.

However, there are some drawbacks to these obviously. First, the size of your installation package. Indeed, you won’t rely on the already installed browser of the platform as you’re shipping yours inside your package. This also means that you’ll be responsible for updating the embedded browser. If a security issue is discovered inside the Chromium version you’re shipping, your app could be a security breach for the user’s system. You must pay attention to this whereas with a classical PWA, this is the responsibility of the browser’s vendor to update & maintain his product you’re running on top of. At last, Electron is not targeting iOS / Android mobile platforms, it’s desktop only. Still, nothing prevents you to ship it as a PWA for desktops & mobiles and package it as an Electron app to offer a more advanced version for the desktops. At last, most of the time you’ll implement the same Web UI/UX of a regular web app even if I’ve demonstrated that nothing prevents you from embracing the platform design language.

Hybrid web app Another way to push the actual PWA barriers is to use a hybrid approach. Hybrid means you’ll create a “native app” with a WebView component that will host your web app. Then, you’ll use a bridge between this WebView component and the native app to do native calls to the platform from your JavaScript code. The WebView could either take 100% of your app’s layout or part of it. In both cases, your app will be considered hybrid. You could for instance use only a small part of your app’s layout to embed a web map service. In our case, we will only talk about full layout WebView, we won’t talk about native components in this section.

There are 2 options to build a hybrid app:

• Apache Cordova
• Your own native app with a WebView inside it
• XAML/C# or C++
• Swift / Objective C
• Java, etc

But if you use the second option, you will have to handle yourself the communication between the WebView and your JS code with the native code.

That’s why, the most obvious choice to consider is Cordova. You can learn more about Cordova: https://cordova.apache.org/ if you don’t know it yet. You’ll find some apps based on Cordova here: https://phonegap.com/app/.

Cordova is offering you a framework that will simplify the communication between your JS code living in the WebView and the native platform through plugins. Those plugins expose JS endpoints you can call from your code and then, the endpoint is making native API calls for you.

There’s a bunch of plugins available by default in Cordova providing various services like battery status, file access, vibration and so on. Features often not available in browsers. You can also search for existing plugins or write your own. The plugin must be written in the native language of each targeted platform: Objective C for iOS, Java for Android, C#/C++ for Windows, etc. So, writing a plugin requires specific skills.

Your web app code (HTML/CSS/JS), like Electron, can either be directly embedded in the app package or hosted on your web server.

Cordova then helps you to access the platform’s API from JavaScript. But what about the UX/UI you’ll build?

You have 2 choices for the UI:

• Offer the same UI across all platforms as a regular web app or PWA
• Embrace the UI of the targeted platform

As we’re running inside a WebView, embracing the mobile’s UI experience using pure CSS/HTML & JS is not something simple. Even a simple iOS or Android slider could need a lot of a work to be built with HTML/CSS and to mimic as close as possible the native control.

That’s why, you can use another framework on top of Cordova to help you on this part too. A current very popular one is Ionic: https://ionicframework.com/.

It offers a set of controls that will change their look/behavior dynamically based on the platform they’re running on. It uses a modern web component approach via Stencil.js. For instance, here’s the sample code running on 3 different mobile platforms (iOS/Android/Windows):

You can see the UI is adjusted to match the design language of each platform. Still, you have a unique HTML / JS code to maintain on your side. Even if the UX/UI proposed by Ionic is impressive, it’s still not completely identical to the native controls. But honestly, it will do the job most of the time for your users.

You can find a list of apps built on top of Ionic: https://showcase.ionicframework.com/apps/top.

Please note that Ionic is also currently working on a new approach named Capacitor to move away from Cordova. It uses their own hybrid solution instead. It can target iOS/Android, Electron & PWA.

PROs & CONs Hybrid apps is then another interesting approach for you to consider but what are their PROs & CONs?

PROs

• Can better integrate with the hardware / platforms via plugins than a PWA
• Can have a look & feel very similar to the native controls thanks to Ionic…

CONs

• … but it’s still not a pure native experience
• Performance
• UX

Cordova is often used to target mobile platforms, even if it can perfectly address also desktops one. You can check their interesting platform status: https://cordova.apache.org/docs/en/latest/guide/support/. Compared to Electron, it doesn’t ship a unique browser engine (Chromium) but rely on the already installed platform’s WebView (Safari based on iOS, Chrome on Android, Edge on Windows, etc.). It can also make calls to the native platform and break some of the barriers of a regular PWA. Regarding UX, you can offer to your users a native-like UI thanks to Ionic.

But it’s still not a true native experience. The performance will vary based on the rendering / JavaScript engine of the target platform. A slow device could clearly show some differences versus the native UI controls. At last, even if the job done by Ionic is impressive, the UX will never be perfectly matching the native UX.

Still, thanks to this approach, we can perfectly imagine having a PWA shared & extended across-platforms. A lot of the code, if not all, will be shared. Having a non-perfect mobile UX could totally make sense for your targeted users.

JavaScript-driven Native Up to now, we’ve found various ways to do calls to the platform APIs (Windows 10 PWAs, Electron or Cordova) but never managed to find a perfect solution for the UI part. Still, those solutions could be used to share almost 100% of the JS code as well as the HTML/CSS for the UI.

The JavaScript-driven Native is using the native controls of the platforms and will offer this perfect UI solution you may looking for. Its approach consists of keeping only the JavaScript code in common with other platforms (it could be Electron or PWA) and targets mobile platforms only. You will have to rewrite the views to use a new language targeting both iOS and Android native controls at the same time. Those views, as well as the business layer, will be driven by JavaScript.

Today, 2 frameworks are using this approach: Telerik NativeScript and Facebook React Native. Both are very similar in their approach. The choice between one of them will be driven by your existing skills. You’re already using Angular or Vue? NativeScript will probably be a natural path. You’re a master of a React.js? React Native is the way to go. But again, there’s no absolute choice. You should evaluate deeply this approach as well as both frameworks implementing it before making your final decision.

Telerik NativeScript On my side, even if I have basic React.js knowledge, I decided to first play with NativeScript. Based on their FAQ: “NativeScript framework enable developers to use pure JavaScript language to build native mobile applications running on the major mobile platforms – Apple iOS, Google Android and Windows Universal. The applications UI stack is built using native UI components and because of that no compromises with the User Experience with the applications is done.”.

If you’re a developer like myself, you probably already wonder how this magic works? Well, check out their doc: NativeScript – a Technical Overview.

NativeScript is not only providing an XML-based UI declaration to abstract the instantiation of native controls of each platform, it also abstracts the platform API via a JavaScript SDK. For instance, look at this file access done from a single JavaScript line of code:

You see that it will be translated to the appropriate Java or Objective C code. Something Electron is also offering, but in the desktops’ world.

Here are cool diagrams illustrating the execution flow in detail:

Let’s now see a video I’ve made illustrating their XML-based language to dynamically create native UIs on both an iPhone and Android phone at the same time.

The layout is described using an XML language you’ll have to learn. But it’s obvious, even more if you’ve already built native applications (XAML or Android are using XML approaches). The controls (Label, StackLayout or GridLayout shown in the video) are mapped to their corresponding native controls on iOS/Android. The structure of the project looks like an Angular one with .HTML, .CSS and .TS files. The styling of the native controls can be achieved via CSS. In the demo, I’m showing an Angular approach. This means that an Angular developer will be able to reuse his habits / skills building a NativeScript application.

Both phones are connecting to the NativeScript playground via websockets thanks to the QR code generated. Then, you can live edit the UI code in the browser and it will be reflected by instantiating native controls on the connected phones. As you can see, the sliders, switches, datepicker & calendar controls are offering the native & perfect UX of both platforms. There won’t be any difference from a native app on this side for your user.

React Native React Native is very similar to NativeScript, offering the same set of tools to remotely test/debug for instance. The idea is to write your business and view logic in JavaScript and remove the major need for native UI expertise. The business logic will be shared across all platforms in JavaScript: iOS, Android, Web and macOS. The philosophy of Facebook behind is “Learn Once, Write Anywhere”. React is not something that easy to learn. I often discuss about this with the developers I’m meeting during conferences. But once you understand how it works, it’s a beautiful approach. Bonus: spending time learning React.js will offer you an easy path to React Native.

React Web components are written in JSX. The props & states are bound to the DOM. Here’s a sample React Web component:

React Native components are also written in JSX. But this time, the props & states are bound to the native views. Look at the same component written for React Native:

It means that you’ll be able to re-use your React.js skills, but you’ll obviously have to rewrite to views for React Native to use the appropriate language for native components.

Some famous apps are built using React Native:

At Microsoft, we’re also using React Native for Microsoft Teams on iOS and Android. There’s an interesting article to read about Microsoft Teams architecture: Microsoft Ignite Live Blog – BRK3118 – Microsoft Teams Architecture: “The Teams client architecture shares a code base between Windows and Mac, which allows Microsoft to ship updates to both platforms at the same time. Also interesting is that Microsoft is moving from Angular to React for the desktop client. Angular was their choice back in the days, because that seemed the right choice then, but moving forward a shift to React provides code share possibilities between the desktop and mobile clients.”

This architecture diagram is interesting. It shows a possible solution to create a complete cross-platforms application using web technologies. MS Teams decided to use React.js for the Web version of the app, to package and extend it on Windows & Mac using Electron and to ship it on iOS & Android using React Native. Thanks to that, we’re able to share almost a unique JavaScript code base.

PROs & CONs So, is JavaScript Native the ultimate solution? Of course not! But there are great positive points.

PROs

• Rich Native UI & perfect platform integration
• Re-use your JavaScript business layer / logic
• Target iOS & Android with a unique UI code

CONs

• Need to learn the native controls / syntax
• Debugging can be much more complex than just hitting F12
• Less reach than PWA

This approach is currently very popular and has great positive aspects. But it still fairly new and feedback from teams / products using it should be read before going into this direction. For instance, the blog posts series of Airbnb is a must read: React Native at Airbnb. They’ve ended up moving away from React Native and they’re explaining why. Of course, you’ll find also very positive testimonials on the official React Native page such as the one from Pinterest: Supporting React Native at Pinterest. Our own Microsoft Teams group seems very happy with their choice.

JavaScript Native is very promising and offer a perfect solution for reaching native UX using web technologies in background. But it’s at a cost: a possible complexity added by the required layers on top of your JS code to manage the communication with the native platform. It means that you will probably have sometimes to extend the chosen framework to build your own native component. Or you will fight with debugging issues. Still, it seems to fulfill his promises most of the time: targeting the 2 mobile platforms, thanks to a single UI code, it uses the native UX / UI controls of each, powered by a single JS business layer code shared across all platforms.

Conclusion I’d like to start by this sentence: “The Web is for audience reach and native apps are for rich experiences. Both are strategic. Both are valuable. So when it comes to mobile, it’s not Web vs. Native. It’s both.”.

It comes from Mobile Web vs. Native Apps or Why You Want Both written by Luke Wroblewski in 2016.

Today, you can target both reach and rich experiences using web technologies. The web can now build high quality apps on any device.

To my point of view, PWA should be considered first. This is what I’m recommending to partners & customers I’m meeting during my Microsoft life. It has the highest reach, the highest portability and it’s now mature enough to be used in production without issue. Its full support is widely available, and the Progressive part will make it compatible with older browsers anyway. At Microsoft, we’re going to invest a lot into PWA in the future.

PWA has of course 2 major limitations: you won’t have a native UX and you won’t have a full access to the hardware / platform (except on Windows 10). If it’s a blocker, and if you’re an Angular/Vue or React developer, there are some interesting combos to investigate. Based on your specific needs, I would recommend one of those:

• PWA + JavaScript Native for mobiles
• PWA + Electron for desktop
• PWA + Electron + JavaScript Native

There are the common patterns I start to frequently see. But again, there’s no absolute answer. You’ll have to first think about the UX you’d like to provide to your user, the level of portability you may need, the specific platform/native integration needs you may have. Finally, you will have to consider the existing skills of your team and the potential learning curve associated with a new technology introduced. But I’ve met several teams made by former C# or Java developers having switched to those approaches without specific difficulties, just after a couple of months of ramp up.

I hope this will help some you having a better understanding of what you can and what you can’t do today with web technologies. Feel free to share your own experience and feedback in the comments’ section!

View Details

We’ll see via this detailed tutorial how to build a cross-platforms game using an adaptive gameplay. This will allow the same web app to be played on touch screens, using a mouse or pen on desktops, up to the ultimate immersive experience thanks to WebVR! By using PWA, it will run totally offline, can be pinned on your mobile home screen or even be pushed to the Microsoft Store.

You can play to the game by navigating to: https://aka.ms/applescrusher and the source code is available on my github.

The gameplay is dead simple. If you’re using a mouse or a pen, you can draw single lines to cut the Apples. If you’re using multi-touches screens, you can draw several lines simultaneously to cut them. Finally, if you’re using WebVR with 6-DOF (six degrees of freedom) controllers such as the Mixed Reality controllers, HTC Vive or Oculus Touch, you can destroy them using a light sword or with banana guns!

Here’s a video showing all possible gameplays:

It all started during the Junction 2017 hackathon. I was participating as a Microsoft coach and co-creator of Babylon.js to help people building cool stuff on top of WebVR. One of the participant asked me to help her building a small Fruit Ninja like game using free glTF assets from Remix3D. I’ve then shown her the complete workflow to create such a game. After building a small prototype, it turned out that people were having fun playing with a stupid cylinder mesh attached to the VR controllers crushing some apples moving in front of them!

I’ve then decided to continue working from time to time on this small game. It was even one of the main topics of our WebVR session at BUILD 2018: WebVR, not just Holograms in the web but powerful platform with Deltakosh.

Let’s now see the various parts of it to let you build a similar 3D experiences on your side. This is the agenda of this tutorial:

– Building the scene using glTF
– Building a loading screen using CSS animations and CSS Grid
– Drawing lines with touch/mouse/pen using pointer events and ribbons
– Testing if a line has cut an apple
– Various gameplays parameters for touch, mouse, pen & VR
– Detecting a connected VR headset
– How to crush the apples using the light sword?
– How to fire ammos from a banana gun?
– Service worker, manifest and icons to make it a PWA
– Pushing to the Microsoft Store

Building the scene using glTF If you’re a developer without a 3D artist working near you or with poor 3D modeling skills, I’d advised you to get free or paid assets for your project. By the way, if you’re a 3D artist, we (developers) love you! And we really need your magical skills. Check out the cool stuff you can build for the web below.

There are 3 interesting repositories of 3D assets working great with WebGL engines: Sketchfab, Remix3D and Google Poly. My advice is to use the glTF standard 3D asset file format when you can. Indeed, using the glTF standard will let you easily move from an 3d engine to another or use various tools in your production chain. All models from Remix3D can be easily downloaded as glTF via Paint 3D on Windows 10 for instance. Sketchfab will auto-convert the downloadable assets to glTF if you need also and some models can also be downloaded in glTF on Google Poly.

To build the scene, you can use various free tools such as Paint 3D on Windows 10, Blender or Unity 3D. Babylon.js has a native support for .glTF/.glb files for files exported by Paint 3D (or any other 3d tools exporting glTF) but has also exporters for Blender, the Babylon.js EditorToolkit for Unity3D, Maya to Babylon, Maya to glTF, 3DS Max to Babylon and 3DS Max to glTF.

On my side, I’m often using Paint 3D as it’s so simple for what I’d like to build. For instance, I’ve imported the following low poly assets: island, sunflower, daisy, rocks, cloud, tree 1, tree 2, tree 3.

To create this:

I’ve then exported the scene to .GLB. You can download it: ApplesCrusherBackground.glb and drag’n’drop it in our Sandbox tool: https://sandbox.babylonjs.com.

Display the inspector (pressing the middle button on the bottom right) and find the 4 cloud meshes to display their bounding boxes. Note their ids.

We can now load this glTF asset in Babylon.js, find the 4 cloud meshes and animate them to bring a bit of life to our background scene.

Go to our playground: https://playground.babylonjs.com/, click on “Examples” and search for glTF:

This is the basic code to load a glTF file into the current scene. Now we need to get the reference to the 4 meshes thanks to their ID and animate them. Check our documentation to discover how to animate meshes: http://doc.babylonjs.com/babylon101/animations or search in the playground in the “Examples” section using the “animation” keyword. In our case, we’re just going to animate the position.x property.

Mixing all those samples together, you should obtain this solution / sample code: http://playground.babylonjs.com/#694VT1#1

Using a similar approach, I’m loading those 2 models: a light sword and a banana gun to replace the default VR controllers while in-game and this Apple to act as the terrible enemies to be destroyed!

Building a loading screen using CSS animations and CSS Grid By default, Babylon.js is providing his loading screen with its logo being rotated via CSS animations. It can be customized but I wanted something better. Indeed, it’s important to provide a nice loading screen for the user while you’re downloading the assets. It’s also important to use as much as possible CSS for this job as the transition/animation are very often hardware accelerated and managed by a separate thread. If you’re relying only on JavaScript to create a custom animation, it could be interrupted by other loading tasks as JavaScript is still mono-threaded. Your animation could stop for a couple of ms and continued several times during the loading process which doesn’t provide a great experience.

You can find plenty of cool loading screens based on CSS animations with your favorite search engine. On my side, I really loved this one: How To Create A Custom Preloading Screen by Petr Tichy. I’ve then slightly customized to change the colors to fit my Apple’s one. I also wanted to center the logo and the animations vertically and horizontally. People used to CSS know that this simple task was quickly becoming a nightmare in the past. Hopefully, the CSS Grid specification is making this task a real simple job (which should have been this way for years honestly…). The level of support is now excellent: https://caniuse.com/#feat=css-grid

This first animation sequence is working on all devices:

Drawing lines with touch/mouse/pen using pointer events and ribbons The initial gameplay was clearly made for a VR only usage. I had integrated a small gameplay for the mouse where you were able to click on an apple to destroy it. But it was mainly to help me debugging the game to avoid spending time inside the headset.

You can test this version here: https://playground.babylonjs.com/#22KIIK and you’ll see that it’s far from being optimal to play with mouse and even less using touch. While discussing about this with Jeff Burtoft, he advised me to rather draw lines to cut the fruits. I’ve then started working on that.

First, let’s talk briefly about pointer events. We’re using it intensively inside Babylon.js. If all our demos on our main site and samples on the playground are multi-touches compatible, this is thanks to pointer events. This specification is, to my point of view, the best one to address mouse, touch and pen across devices. And it’s not because Microsoft initially made it

It’s really well made and starts to be fairly well supported by browsers: https://caniuse.com/#search=pointer%20events.

If you don’t know this great API yet, I’d suggest you to start reading one of my previous article on it: Unifying touch and mouse: how Pointer Events will make cross-browsers touch support easy.

As you can see on “Can I Use”, some browsers haven’t implemented it yet. That’s why, we’re using the jQuery PEP polyfill to cover those. Using it is quite simple. Simply reference the library from their CDN and add the “touch-action” HTML property on the HTML element that will receive the multi-touches event (the HTML canvas in our case). This property is a way to replace the “touch-action” CSS property as polyfilling CSS is complex.

Now that we know how to manage the various types of input, we need to find a way to draw lines/curves in our 3D canvas. Those lines will also have to generate hit testing against the targets to check if we manage to cut them or not.

First, we need to find a way to draw those lines. I’ve decided to use the ribbon feature of Babylon.js. It’s a way to build parametric shapes which can build very complex meshes if needed such as illustrated by this sample. But in our case, we’ll build simple flat shape that will act as our lines.

To understand how it works, I’ve built this full commented sample: https://playground.babylonjs.com/#LRGL8M.

Simply read the code and you should better understand the approach.

You’ll note that in this sample, we’re not limiting the size of the lines we’re drawing. This would mean that the player could press the canvas and keep drawing like mad to destroy the apples. In the final code, I’ve added a distance computation to limit the size of the lines. You can check that by reviewing the code of the next section.

Testing if a line has cut an apple We need to know if under the line you’re drawing, there’s some apples. For that, we need to send rays in this direction and check if one of those rays is intersecting the mesh of an apple. If so, you’ve touched an apple and it should be destroyed.

Let’s start by reviewing the initial mouse gameplay where you were able only to click on an apple to destroy it. As a reminder, the code is there: https://playground.babylonjs.com/#22KIIK

Go to line 471 of this first sample where we’re registering some code to handle the pointerdown event. We’re using the picking feature of Babylon.js described in our documentation: Babylon101 – Picking Collisions.

We’re using a predicate to filter the meshes tested to only the collection of apples. If we’ve got a hit, we’re calling our collision handler function giving it the picked mesh as a parameter.

This collision handler is doing the magic by playing a sound to indicate you’ve crushed the apple. Then, it takes the position of the apple crushed to put instead a particles emitter and activate it for 250 ms to achieve the awesome special effect. Check our documentation to know more about how particles work: http://doc.babylonjs.com/babylon101/particles. Finally, the apple mesh is hidden and put back in the collection, ready to be destroyed again!

Now, we can use this click logic to extend it to the lines used to cut the apples. Review this updated gameplay in this new sample: https://playground.babylonjs.com/#PKQ6JV#1

First, as you have seen, the lines have a vertical thickness. So, during each pointerdown / move event raised, we need to do a hit testing at the exact coordinates where you’ve clicked/touched but also above and under this point on the Y axis to consider the thickness. This is the job of the testApplesCollisionAround() function. You can review its code at line 679.

But doing that on each pointer move is not enough. Indeed, based on the speed of your pointer move, you could miss your target between 2 events. The coordinates of the first pointer move could be just on the left of the apple you’re targeting and the coordinates of the second pointer just on its right. Visually, the line would cover the apple, but our hit testing approach would miss it which could frustrate the user.

To try to avoid that, I’m then computing the distance between two pointer moves. If it’s bigger that a specific threshold, I’m creating intermediate linear pointers coordinates to achieve hit testing on it. This is the job of the testApplesCollisionBetween() function available for review at line 693.

Various gameplays parameters for touch, mouse, pen & VR To be able to dynamically adapt the gameplay for the various types of inputs, I’m simply using a JSON object to configure a couple of properties as you can see in the code: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L78

Basically, this is defining:

– the size of the apple. For instance, in VR, the size needs to be smaller, specially against the laser saber to have a better experience. It’s bigger with the banana guns as it’s too difficult to target them otherwise. At least, the size is bigger in touch mode rather than in mouse mode.
– The deltaX, Y, Z properties indicate where the apple could potentially pop in front of you. This really impact the VR mode as for instance, if the apple pop too high, you won’t be able to touch it using the light sword.
– At last, there’s some properties for the ribbons drawn: maximum distance per ribbon and its thickness.

To switch between each of them, I’m registered a detectPointerType function on the pointerDown event. This function simply checks the type of input which has triggered a pointer down on the canvas DOM element: mouse, touch or pen. It then changes the gameplay accordingly. In VR, this gameplay switch is done when you click on the one of the options just before starting the game: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L870

Detecting a connected VR headset If your browser doesn’t support WebVR or if you don’t have a VR headset connected, the starting screen will be this one:

However, as soon as you’ll connect a VR headset, it will now display this:

Clicking on this “Play in VR” button will not start the game immediately but will first ask WebVR to enter the VR/immersive mode. To do that, rather than using the default Babylon.js VR button of the VRHelper, we’re just asking in the options to use a custom button instead: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L642

Then, inside your VR headset, you’ll be able to click on the “Play” button using one of your controllers. It will enter VR thanks to this line: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L857

And if you’re disconnecting the VR headset, it will switch back to the first screen, and it will switch the gameplay options to the mouse one. How can you manage that in your code?

First, you need to register your code to the onVRDisplayChangedObservable like that: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L740. This event is triggered by WebVR when you’re plugging/unplugging a VR headset and exposed by the Babylon.js engine. If a VR display is connected, we’re updating the text of the button from “Play” to “Play in VR”.

To be sure that the game is ready to be played in VR, we’re also monitoring the onVRRequestPresentComplete event triggered by WebVR: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L768. If the rendering inside the headset worked, we’re good to go.

How to crush the apples using the light sword? To know that the light sword has cut an apple, we need to do a collision test. We need to know that the mesh of the light sword is intersecting the mesh of one of the apples. We could do that with some custom code in the game rendering loop. On each frame rendered, we could test ourselves if there’s an intersection between on of the two light swords and one of the 10 apples. Doing this could be a boring task and potentially complex if you’re not a guru in math.

Hopefully, there’s a very simple way to do this in Babylon.js using the actions and the actions manager. In the documentation: http://doc.babylonjs.com/how_to/how_to_use_actions, there’s a trigger which is interesting for our task:

“***BABYLON.ActionManager.OnIntersectionEnterTrigger***: Raised when the mesh is in intersection with a specific mesh. Raised just once.”

We then need to use this trigger and register an action on the light sword for each apple. The action is to execute code if the condition is fulfilled and this code is simply the same collision handler we’ve used before when we were cutting apples by drawing lines. It’s the same logic to be used: get the position of the apple touched, display the particles system at this very same position and remove the apple from the screen.

You can check how these actions registrations are done in the createAndSetupVRHelper() function: https://github.com/davrous/applescrusher/blob/master/js/applecrushervr.js#L671

How to fire ammos from a banana gun? First, let’s see how to fire some bullets/ammos from the banana guns. I’ve decided to use this Remix3d model to act as the ammo: the Plunger of DEATH!!!

To simplify the job of getting the position from where it will be launched, I’ve imported the model in Paint3D to place it exactly where I wanted to on top of the banana pistol:

This simplify a lot the job for our code. When loading the model, I then hide the plunger. The mesh will only act as a reference be used later.

Now, we need to monitor the trigger button of the VR controller. If it’s pressed enough, we’re activating our firing logic. This logic consists of getting the current position of the hidden ammo on the banana pistol, clone the object and its rotation, detach it from its parent (the banana) and moving it forward.

The animation is done by code in the game render loop. It checks if there’s some ammos in a specific dedicated collection and move them forward. I must confess that the current trajectory is very simple and not very realistic, I was too lazy to do better. Well, destroying some apples with a plunger launched from a banana pistol is not THAT realistic anyway . We could have used the animations engine of Babylon.js to use an easing function instead to have something more natural. Even better: if you’re really motivated, you can search on the web ballistic equations and try to implement them. In the meantime, my awesome equation will do the job.

Now, we need to understand how to check the collisions between those bullets and the apples. This one is a bit trickier. We could first think to reuse the same approach as the light sword. But this time, testing the intersection on each frame will probably fail. To be honest, it could also fail with the light sword even if it has few chance to do so.

The reason behind is because we’re living in a quantum world with real-time 3d. We’re updating all our objects every tick, provided by requestAnimationFrame in JavaScript/WebGL games. In non-VR, the optimal tick will happen every 16ms to achieve 60fps (as most of the screens have a refresh rate of 60hz). In VR, it should be every 11ms to achieve 90fps.

There’s a high chance that between 2 ticks, the ammo will be just before the apple, just about to go through it and then, just after the apple. It means that the ammo trajectory will cut the apple but using a tick approach to test the intersection, we could miss it.

To better understand, look at those 2 screenshots:

If the animation is slow enough, the ammo would collide with the apple and the logic being used with the light swords would work. But let’s imagine that the animation is too fast, and the first frame looks like the first screenshot followed by a new frame 11ms later that looks like the second screenshot. The arrow has gone through the apple, but the action’s trigger wouldn’t be raised.

The solution is to cast rays in front of the ammo to check if this ray is intersecting with the apple, this would mean that the ammo is about to cut the apple.

I’ve built the sample code for you: https://playground.babylonjs.com/#WDLE60. To make it works, launch this playground link, click on the VR button to enter VR and press the second button of your VR controller (menu button on Mixed Reality headset) to switch from the default controller model to the banana pistol. Press the trigger to launch an ammo. If you’re correctly targeting of the boxes, the ammo will make it disappear if it’s on its trajectory.

Check the castRay() function to understand how this works which is the same one as used in the game.

Service worker, manifest and icons to make it a PWA Progressive Web App or PWA enables web developers to create web apps that can’t be distinguished from native apps. The web platform is now offering so many features and great performances that a lot of scenarios / apps could be done using web technologies without any issue.

Regarding games, we have a direct access to the GPU using WebGL 1.0/2.0, 3D spatial audio using Web Audio, Gamepad API, WebVR, touch support and so on. We’re then covered to create great cross-platforms games! On the 3D rendering side, we can really achieve great performance with high quality rendering. People are often surprised to see the quality & the performance of our Babylon.js demos but most of them just don’t know that WebGL is simply a subset of OpenGL exposed in the browser. We’re discussing with the GPU directly via shaders, a C like programming language compiled to native code for the GPU.

To be honest, the only remaining bottleneck is JavaScript. JS engines are really super impressive on the job they’re doing to jit the code on the fly. ASM.js & Web Assembly could even sometime go a step further. But at the end, JavaScript is mono-threaded and unfortunately the web workers won’t help much on this. For games, with our game loop that must do all its job in 16ms, this could be quickly an issue. For instance, a lot of features are CPU based like physics engines, collisions & picking. If we had access to threads, we could do much more in 16ms. Still, even with those constraints, we can push the browser to the limits and create awesome experiences! Moreover, those issues are very specifics to a gaming approach. Business apps will have much less risks to hit those walls. And even for games, as you see with this one or other available on the web using various WebGL engines, you can already create very performant experiences.

Let’s go back to PWA. If you don’t know yet what it is, here are some resources I’d recommend you reviewing first:

– Building Progressive Web Apps during BUILD 2018 by Jeff Burtoft
– Progressive Web Apps on Google Developers
– Progressive web apps on MDN

A PWA has to be served using HTTPS, expose a manifest and use a service worker.

The web app manifest is a JSON file describing how the web app should behave when pinned on your home screen or exposed via a apps store. Store apps developers won’t be surprised to discover properties such as the orientation you’d like to force (or not), the background color, a splash screen, etc. You also have to provide various icons resolutions.

To help you building quickly this, we have done PWA Builder. It allows you to edit / generate the manifest and will save you a lot of time generating the various icons by automatically converting a provided reference image.

Navigate to PWA Builder: https://preview.pwabuilder.com/ and enter the URL of the game: https://david.azureedge.net/applescrusher/index.html

You can see I’m forcing a landscape orientation, setting a specific background color (for the splash screen) and asking the remove the chrome of the browser if the web app is pinned by using the standalone value for the display property. This was the easy part.

Now, to enable a full PWA, you need to provide a meaningful service worker. It can either boost the performance of the loading phase and enable offline. PWA Builder is providing various service workers ready to be used. But to be honest, it’s difficult to provide a service worker generically working for any purpose. You will probably have to tune the service worker code to perfectly match your needs.

My goal for my PWA game was to be ready to go offline as soon as the game has been loaded at least once. Indeed, there’s no dynamic part in my game. As long as you’ve downloaded all the assets such as glTF files, sounds & music and of course the classical CSS, HTML & JS file, you shouldn’t have to download them again from the web server to execute the game a second time. The objective was then to navigate to the PWA, wait for the game to be ready and go to airplane mode immediately.

To illustrate that, let me show you how the “cache-first network” wasn’t good for my scenario.

I’ve first tested to put all my resources inside the precacheFiles array:

which sounded as a good idea. But using Chrome F12 to look at network requests simulating a Fast 3G connection, it clearly shows it’s far from being a good idea:

We can see that my resources are downloaded twice! The XHR request is done by Babylon.js to query the various .GLB assets specified in our code. In background, the service worker is querying via a fetch the very same resources specified in the precacheFiles array. Indeed, Babylon.js is not aware that a service worker is being installed to precache the files. Anyway, the service worker wouldn’t be ready to catch the network requests of Babylon.js. That’s why, we’re double requesting the files. The first load takes the equivalent of loading twice the site.

After this process, my goal was in a way reached as I can immediately go offline. All the files will be in the service worker cache and future XHR requests from Babylon.js will go through the SW cache first. But downloading two times the files is not a good approach. A possible solution would be to wait for the service worker to finish its precaching installation process and notify the main JS code to start the downloading after that. However, this would break my progressing downloading % bar as I wouldn’t know the current download status of the SW. I also wanted something simpler and more transparent.

I’ve then decided to have a look to this service worker provided by Google. I liked the approach of having 2 caches: PRECACHE and RUNTIME. My idea was then to put in the PRECACHE the minimum needed files for my web app to load: the HTML page, CSS and the various JS code & libs. Then, as soon as the SW would be installed, it would catch the XHR requests of Babylon.js to put it in the RUNTIME cache. Problem: to enable offline, you need to load at least 2 times the site. The first time, the PRECACHE cache will be built and the service worker will be registered. However, it won’t have time to catch the XHR request of Babylon.js. The second time you’ll load it, the SW will get the files from the PRECACHE and will start to populate the RUNTIME cache by catching the XHR requests. Again, a possible solution would be to wait for the SW to be installed and ready to catch XHR requests. It would mean to send a message to my game logic via a postMessage from the SW as soon as it will be ready to do its job.

Fortunately, we have another last option that I decided to implement. Some years ago, I’ve built an IndexedDB layer in Babylon.js to enable offline scenarios. I even wrote an article about it: Using IndexedDB to handle your 3D WebGL assets: sharing feedbacks & tips of Babylon.JS.

It was done before the service workers were created, but you’ll see it’s still perfectly valid and useful!

We’re going to continue the second solution using the SW with the 2 caches. However, this time, we’re going to ask to Babylon.js to download the resources and store them in IndexedDB. As explained in the documentation, it’s quite easy. You just have to create .manifest files associated to your resources. Those .manifest files must be part of the PRECACHE collections as they are requested by Babylon.js to check the offline options. Indeed, if we want to go immediately offline after the first load, those files must be in the cache.

As a conclusion, here’s the final workflow:

– The service worker preinstalls in the PRECACHE cache all minimum files (HTML, CSS, JS and .manifest)
– Babylon.js is querying via XHR the glTF assets and store them in IndexedDB
– The SW is ready to cache non-vital files for a first offline usage such as the VR assets: controllers’ models, light sword and banana pistol.

If you go immediately offline after the first load, Babylon.js will query the .manifest files and knows it has to get the glTF files from its IndexedDB table. We’ve reached our goal. The game can be played in offline immediately after the first load!

You can check the service worker code on github: https://github.com/davrous/applescrusher/blob/master/pwabuilder-sw.js

Note: I had to slightly add a final tune by using the {ignoreSearch: true} option. Babylon.js is generating timestamps after the URL to bypass the browser cache to get the .manifest files. Each request was considered different by the service worker if this option is not enabled and would break the offline mode then.

Pushing to the Microsoft Store On Windows 10, PWA is a first class citizen to publish in the Microsoft Store. The submission process is described in this article.

Once hosted in the Microsoft Store, running on Windows 10, your PWA can call WinRT APIs to embrace the philosophy of the OS and the design languages of UWP. You can for instance create tiles, toast notifications, accessing Bluetooth devices, etc.

On my side, I’ve simply use of one the Windows sample provided by PWA Builder: https://www.pwabuilder.com/windows to change the app title bar color. By default, my app was hosted in a container using the default grey app bar color. This was breaking the launch sequence to my point of view and it wasn’t using the 3 mains colors of my theme (taken from the apple 3d object):

I’ve then copy/pasted the code provided by PWA Builder. Then, I added a feature detection in my web app to check if my code was executed inside a Windows 10 store app by checking the window.Windows object. If it’s true, I’m calling the configureWindows10StoreApp() function which calls some Windows APIs to change the colors of the app bar. Now, the app bar uses the same color as the splash screen and the foreground color is the same one as the yellow used for the circle and % text. The color for the button hovering is also one of my colors’ theme.

Finally, I also decided to use PWA Builder to generate the Visual Studio solution for me. I could have downloaded the APPX file to submit directly to the store, but I wanted to do a final customization.

As my game can be launched in the Windows Mixed Reality Portal for people having VR headsets, I wanted to better integrate this environment. For that, you can create a 3D launcher icon to be positioned in your VR world. The full process is explained in the documentation: Create 3D models for use in the home. You also check this cool video from #ifdef which explains the steps in a fun way.

I’ve then used the glTF Toolkit to process my Apple 3d asset to make it ready for the MR portal. Then, I’ve updated by Visual Studio package.appxmanifest file to tell it to load my converted model:

Thanks to that, you’ll be able to place my apple in your cliff house. Clicking on it will launch my app.

If you’ve running on Windows 10, please try download and play with the game from the store: https://www.microsoft.com/p/apples-crucher/9pl4cf3hx9dg

I had fun building this experiment which combines lot of great features of the web platforms and I’ve learned a lot. I hope this will help you building similar cross-platforms experiences.

David

View Details

Nous n’en avons pas conscience tous les jours mais l’industrie informatique est à l’aube d’un changement majeur : la loi de Moore, telle que nous la connaissons aujourd’hui, va s’arrêter d’ici 3 à 4 ans maximum. L’ère de la puce Silicium vit ses derniers moments de gloire. Les processeurs, les ordinateurs, la manière de programmer vont sûrement radicalement changer dans très peu de temps.

Mise à jour le 22/11/2018 suite à mes 2 conférences de 45 min à Paris Web 2018 et Microsoft Expériences 18 et notre session de 20 min à la Experiences TV 18 avec Fanny Bouton et Olivier Ezratty.

Note : cela fait suite aux 2 podcasts que Deltakosh, Meulta et moi-même avons en partie consacrés à l’ordinateur quantique et à la physique quantique. Vu le succès de ces épisodes et vos retours, j’ai décidé d’en faire un article. Attention : pour ceux qui ne nous connaissent pas, ces podcasts sont souvent à prendre au second degré ! Nous serons heureusement plus sérieux ici.

Si vous préférez regardez ou écouter, j’ai 3 contenus basés sur cet article à vous proposer :

– Si vous n’avez que 20 min de disponible, je vous conseille de regarder la vidéo du keynote de Devoxx France 2018 que j’ai eu la chance d’animer en partie : https://youtu.be/ciM6xK05t2o

– En 20 min toujours, notre interview de la MS Experiences TV avec Fanny et Olivier qui vous donne un aperçu : https://youtu.be/aLIz9I58FxY

– En 45 min, ma session de Paris Web 2018 sur laquelle j’ai eu d’excellents retours et qui est quasiment la même que celle que j’ai jouée à MS Experiences 18 : https://vimeo.com/296245907. C’est certainement celle qui vous permettra de comprendre au mieux.

Ce sont des condensés de cet article. Sinon, attachez votre ceinture car nous allons passer en vitesse lumière !

L’Intelligence Artificielle Mais avant de tenter de vous expliquer pourquoi tout va bientôt changer, revenons sur ce qui agite l’actualité des derniers mois. Nous parlons beaucoup de plusieurs grandes thématiques qui émergent ou vont bientôt émerger. Et la plus emblématique d’entre elles est sans nul doute l’IA ou Intelligence Artificielle. Tout le monde en parle avec plus ou moins de bonheur et plus ou moins de fantasmes. C’est également devenu un fourretout à tous les trucs cools du moment.

Une des applications intéressantes que vous utilisez déjà presque tous les jours est par exemple la reconnaissance vocale. On la retrouve dans les serveurs téléphoniques vocaux, dans vos smartphones avec Cortana, Siri et autres Alexa. Mais ce que nous n’avez peut-être pas noté, c’est que la qualité de cette reconnaissance par une machine a atteint très récemment celle d’un être humain : Historic Achievement: Microsoft researchers reach human parity in conversational speech recognition et a donc très fortement progressée ces 5 dernières années.

Nous exposons également des services dans le fameux « Cloud » pour reconnaitre des images automatiquement et en faire des transcriptions. A chaque fois que j’y pense, cela me retourne la tête. N’importe quel développeur est forcément subjugué par les algorithmes mis en œuvre. C’est le cas chez Microsoft avec Cognitive Services Computer Vision API où vous pouvez découvrir ses capacités en jouant avec cette démo : https://www.captionbot.ai. Tous les principaux acteurs du marché se positionnent sur ces nouvelles offres. Facebook utilise des algorithmes similaires. Vous les avez d’ailleurs déjà forcément vu en œuvre lorsque l’on vous propose de tagguer automatiquement vos amis en reconnaissant leurs visages sur les photos que vous postez. Il y a également IBM avec Watson Visual Recognition et Google Cloud Vision API.

Bien sûr, il y a un côté terrifiant à ces nouvelles technologies, notamment lié à l’avenir de nos vies privées mais également sur des sujets plus fondamentaux encore. Pour vous donner une idée, je vous invite à regarder cette excellente vidéo de Laurent Alexandre : Impact de l’Intelligence Artificielle sur l’économie – Laurent ALEXANDRE au Senat (HD) sur laquelle il faut également savoir prendre du recul. Mais, comme il le dit justement, aujourd’hui, nous les français, « en matière d’Intelligence Artificielle, nous sommes un pays du tiers monde ».

Mais l’IA dispose également d’un potentiel inouï pour améliorer le quotidien de millions de personnes. Nous en avons fait la démonstration lors de la //BUILD 2016 avec Saqib Shaikh utilisant ces services couplés à une paire de lunettes « intelligentes » pour lui permettre de « voir » le monde qui l’entoure : Microsoft Cognitive Services: Introducing the Seeing AI project. Facebook vous propose également un texte alternatif à la photo que vous allez poster pour en améliorer son accessibilité pour les personnes aveugles et mal voyantes : Under the hood: Building accessibility tools for the visually impaired on Facebook. Tout cela m’a inspiré pour créer une extension permettant de reconnaitre l’image d’une page web et de la décrire à une personne potentiellement aveugle : Creating an extension for all browsers: Edge, Chrome, Firefox, Opera, Brave and Vivaldi. On peut aujourd’hui mettre en place de telles solutions en une dizaine de lignes de code !

Ces dernières années, dans les sujets plus typés science-fiction (mais qui sont pourtant déjà là), nous avons pu également voir successivement un ordinateur battre un joueur humain aux échecs, au jeu de Go et plus récemment au poker, ce qui inquiète nettement plus les experts. On parle également fréquemment de la voiture autonome de Tesla et de Google. Plus récemment, Microsoft a commencé à développer une IA, DeepCoder, capable d’écrire du code !

Alors pourquoi l’IA émerge à ce point à cette période précise ? En effet, d’après Wikipedia, le concept d’intelligence artificielle a démarré avec Alan Turing et son fameux test en 1950. Pourquoi a-t-il fallu autant de temps pour en voir les premiers résultats tangibles et concrets ?

C’est qu’il fallait la convergence de 3 domaines clés : la donnée, l’infrastructure pour la capter/gérer/traiter et la puissance de calcul des puces pour la digérer. Soit en traduction technologique : le big data, le Cloud/Machine Learning (Apprentissage automatique)/Deep Learning (Apprentissage profond) et les CPU/GPU. Bref, l’équation magique semble être : IA = Big Data x Machine Learning x CPU.

En effet, pour pouvoir créer une intelligence artificielle qualifiée de « faible » par certains, donc dans un premier temps spécialisée (reconnaissance vocale, d’image, sémantique, de visage, etc.), il faut d’abord entrainer un modèle avec un algorithme particulier. La première pièce est la donnée, ou plutôt la quantité de données. En effet, il faut avoir de gigantesques volumes de données pour les injecter dans une boîte noire que l’on appelle le Machine Learning. Ces données peuvent être vous et vos habitudes de navigations dans le cas des réseaux sociaux ou ciblages publicitaires. Mais également les données de votre smartphone ou de votre véhicule pour le trafic routier/voitures autonomes, les données médicales comme votre rythme cardiaque avec les objets connectés, etc. Dans le domaine du jeu vidéo, une autre application que vous avez peut-être connue est Kinect. Pour pouvoir reconnaître vos gestes et votre « squelette », du Machine Learning a été utilisé à partir d’enregistrements vidéos de plusieurs milliers d’êtres humains comme l’explique cette vidéo : Real-World Machine Learning: How Kinect Gesture Recognition Works.

En termes d’architecture, il faut des moyens colossaux pour aspirer ces données et pour mettre en place ce Machine Learning ou Deep Learning en termes de matériel, réseaux et logiciels. Ils sont actuellement déployés dans d’énormes Data Center que l’on appelle le Cloud, vendus par des sociétés comme Microsoft, Amazon ou Google.

Voilà pour une partir des explications des raisons de son émergence. En complément de tout cela, je vous invite également à lire l’article de mon collègue Pierre-Louis Xech : Cyclogenèse de l’IA qui propose un angle intéressant.

De mon côté, je m’intéresse à cette transformation pour 2 raisons principales. L’une est purement professionnelle. Cette transformation profonde à venir de l’industrie informatique me fascine autant qu’elle m’inquiète. Vais-je être capable de m’adapter aux nouvelles connaissances nécessaires ? Il est certain que l’IA va remplacer de nombreux postes/métiers « facilement » automatisables. Mais cela va-t-il également remplacer le métier de développeur et si oui à quelle échéance ?

Bref, en fonction du point de vue, il semblerait que nous allons devenir les acteurs de notre propre destruction ou au contraire les bâtisseurs d’un monde meilleur. J’aime par exemple cette phrase : « La puissance des algorithmes n’est ni un rêve ni un cauchemar : elle est les deux en même temps, et s’orientera selon ce que nous en ferons. La plupart des grandes innovations ont ainsi questionné la société » issue de cet article : Algorithmes, voitures autonomes, big data : bienvenue dans le pire des mondes digitaux

Pour ma part, je suis un éternel optimiste jusque dans mes modestes essaies de science-fiction : Les H-Men : une anomalie génétique couplée à la technologie pour de supers pouvoirs. Même si cela relève bien sûr du délire de geek que je suis, je pense sincèrement que l’IA pourra aider des enfants handicapés comme mon fils Nathan. C’est donc, vous l’aurez compris, la deuxième raison de mon fort intérêt pour l’IA et de l’informatique quantique. Ils pourraient grandement aider les projets actuels gérés par Microsoft autour de la génomique : MSR Genomics & Microsoft Genomics. Malgré tout, il se trouve que même la plus puissance des IA actuelle ou le plus puissant des superordinateurs actuel sont incapables de bien modéliser et comprendre la nature, la chimie et donc de nous aider à faire de meilleurs médicaments ou soigner nos enfants.

Mais pourquoi cette longue introduction autour de l’IA alors que je vous ai promis de parler d’ordinateur quantique ?

Rappelez-vous de ma fausse équation magique : IA = Big Data x Machine Learning x CPU.

Il y a un truc qui va clocher. Le CPU. On avait pris l’habitude depuis les années 70 de voir nos chers CPUs doubler la quantité de leurs transistors tous les 2 ans et donc voir leurs performances virtuellement augmenter de manière exponentielle. Bah désolé, la fête est bientôt finie.

La loi de Moore est morte Le transistor. Une des inventions les plus importantes du 20ème siècle. Le composant de base des processeurs. Pour augmenter la puissance de nos CPUs, on a d’abord eu l’idée d’augmenter à la fois leur nombre et la fréquence d’horloge les animant en cadence.

Gordon Moore, l’un des fondateurs d’Intel, a alors créé une sorte de règle, feuille de route pour l’industrie, prophétie auto-réalisatrice : la loi de Moore. L’idée est de doubler le nombre de transistors tous les 2 ans en les miniaturisant de plus en plus. De 1971 à aujourd’hui, nous sommes ainsi passés des 2300 transistors du premier microprocesseur d’Intel, le 4004, à des puces embarquant plusieurs milliards de transistors !

Cependant, comme l’explique très bien cet article (en Anglais) : Moore’s law really is dead this time, il y a eu plusieurs barrières qui furent particulièrement difficiles dans la course à la miniaturisation. En 2005 d’abord, autour des 90nm, on se posait sérieusement la question pour repousser les limites de la miniaturisation. Cela est d’ailleurs rappelé par cet article plus récent : Processeurs : la fin de la loi de Moore… et le début de l’incertitude qui indique la complexité rencontrée : « au début des années 2000, quand l’effet conjugué de la hausse des fréquences d’horloge et de la diminution de la taille des circuits avait fait bondir la chaleur générée par les circuits. Un obstacle que les industriels ont contourné en optant pour des architectures multicoeurs, qui ont mis fin à la course à l’augmentation des fréquences d’horloge ». Enfin, plus récemment, les 22nm ne furent envisageables que grâce à l’invention du transistor tri-gate.

Aujourd’hui, les CPU que vous achetez/utilisez dans vos PC, Mac et autres smartphones sont souvent gravés à une finesse de 14nm. Plus fort encore, le dernier Snapdragon 835 annoncé au CES 2017 arrive lui à être gravé en 10nm. J’ai également pu voir qu’une chaine de production allait bientôt démarrée à 7nm cette année (2018). Les derniers iPhone Xs utilisent d’ailleurs des puces en 7nm.

Pour vous donner une idée de la prouesse que cela représente, regardons les ordres de grandeur de longueur. On s’aperçoit ainsi que 1nm correspond à la taille d’une molécule ou du rayon de la double hélice de notre ADN. Plus surprenant encore, 10nm correspondent environ à 100 atomes mis les uns à côté des autres ! 100 atomes, vous imaginez !

Le palier des 7nm semble donc être le premier candidat sérieux à l’atteinte du mur de la miniaturisation. Cependant, les dernières recherches précisées dans cet article : le plus petit transistor du monde ressuscite la loi de Moore indiquent que des nanotubes de carbone pourraient nous permettre d’envisager d’aller plus loin, vers les 5nm peut-être. Malgré tout, à force de nous rapprocher de la taille critique d’un atome, nous allons finir par être confrontés au phénomène inéluctable d’effet tunnel illustré par cette vidéo :

Ainsi, 2020 semble être une échéance couramment partagée. Bien sûr, on pourrait se dire que ce n’est pas pour autant la fin du monde numérique. Voir même la fin du monde digital comme on aime le dire, à tort, dans les sphères branchées du marketing !

En effet, on pourrait envisager d’augmenter la taille des puces à défaut de pouvoir continuer à miniaturiser. Cela nous permettrait de continuer à augmenter le nombre de transistors.

Comme je vous le disais dans ma conférence Paris Web 2018, cela a d’ailleurs déjà commencé. D’une part, on constate 18 mois après entre les cartes nVidia les plus puissantes d’une génération à l’autre (GTX 1080ti / RTX 2080ti) une simple augmentation de performance de 25% en performance brute sur de la rasterization. La loi de Moore est déjà mise à mal et est sauvée en 4K grâce à l’usage du DLSS permettant une mise à l’échelle basée sur le deep learning. Par ailleurs, malgré la miniaturisation, on constate une augmentation de la taille des puces (die) :

On peut ainsi imaginer augmenter le nombre de serveurs et leur capacité de calculs distribués. Oui, mais plusieurs problèmes pointent rapidement le bout de leur nez :

– L’évolution ne pourra certainement plus être exponentielle comme avant

– On va se retrouver confrontés à nouveau aux problèmes de consommation, dissipation de chaleur que nous avions eu au début des années 2000, qui furent résolus en partie avec les multi-cœurs.

On pourrait également se dire que cela va nous forcer à nouveau à bien penser nos codes et nos algorithmes. Comme il y a quelques dizaines d’années finalement, avant que nous codions un peu tous comme « des porcs », profitant avec une certaine insouciance de l’augmentation régulière de la puissance des processeurs afin de cacher notre fainéantise intellectuelle.

Malgré tout, cela ne va pas suffire à combler notre addiction vis-à-vis de la loi de Moore sur laquelle beaucoup comptent toujours. Tout cela malgré le mur atomique que le silicium est sur le point de rencontrer. En effet, l’IA a besoin de toujours plus et l’industrie ne se satisfera pas de cette contrainte physique.

Par ailleurs, il existe des problèmes dans certains domaines comme la chimie, la création de médicaments, l’étude de la matière qui seront à jamais insolvables pour un ordinateur classique. Il faut donc changer de paradigme si l’on veut continuer à progresser et à innover. Certains problèmes nécessitent en effet aujourd’hui plusieurs milliards d’années de calcul pour être résolus ! Par exemple, même le plus puissant des supercalculateurs actuel n’est capable de modéliser une molécule plus complexe que celle de la caféine.

La mécanique quantique à la rescousse Arrivée à une échelle si petite, les règles changent et pas qu’un peu. En effet, il faut savoir que les lois de la physique classique, celle que nous connaissons bien et que nous subissons tous les jours, ne s’appliquent plus à partir d’une certaine échelle.

Nous ne savons toujours pas exactement pourquoi mais des éminents physiciens ont fini par le constater. Nous n’avons toujours pas non plus trouvé une règle générale permettant de réunir le monde tangible dans lequel nous nous sommes habitués à vivre avec le monde étrange qu’est celui des particules quantiques. Car oui, avant d’envisager de comprendre en quoi l’ordinateur quantique pourra continuer de faire vivre la loi de Moore, il va falloir tenter de comprendre un minimum le fonctionnement de la mécanique quantique.

Bon, déjà, commençons par tous nous détendre. Il est manifestement extrêmement compliqué de vulgariser la mécanique quantique. Il faut également s’ouvrir l’esprit et lutter contre nos barrières ancrées en nous depuis des siècles car les principes apparaissent comme totalement contre intuitifs pour l’esprit humain. Certains se demandent même si le cerveau humain ne serait tout simplement pas trop limité pour en comprendre le fonctionnement !

Pour vous rassurer, j’ai découvert en écoutant l’émission de radio « la tête au carré » de France Inter dédiée à l’ordinateur quantique que Pascale Senellart, directrice de recherche CNRS au laboratoire de photonique et nanostructure, nous raconte à 32’58 que: « quelqu’un qui vous dirait qu’il a vraiment compris la physique quantique, c’est quelqu’un qui ment un peu ». Par ailleurs, je me permets d’ajouter une citation célèbre de Richard Feynman : “Je pense pouvoir dire sans trop me tromper que personne ne comprend la mécanique quantique”. Les bases sont posées. Si les experts ont déjà du mal à comprendre, n’espérez pas prétendre à un PhD de physique quantique à la fin de cet article.

Bref ce que j’essaie de vous dire est qu’il est franchement difficile de trouver de bons articles expliquant simplement ce qu’est la mécanique quantique et que par conséquent je risque moi-même d’être approximatif. Malgré tout, j’ai essayé de vous digérer tout cela du mieux que j’ai pu et de vous partager les vidéos et articles qui m’ont semblés les mieux conçus et les plus pédagogiques.

Au passage, si un physicien spécialisé en physique quantique passe par là, il serait gentil de sortir son pif de son microscope à effet tunnel, de sortir de son labo et de venir essayer de nous aider à comprendre ce qu’il se passe de si chouette dans ce monde de l’infiniment petit.

D’ailleurs, avant de se lancer, commençons par prendre une petite pause comique en regardant cet excellentissime sketch d’Alexandre Astier : Alexandre Astier – La Physique Quantique parfaitement dans le ton de notre podcast.

8 principes clés La meilleure source de vulgarisation que j’ai pu trouver sur la mécanique quantique est de loin celle de la Science Etonnante. De très loin même. Le travail effectué par David Louapre est tout à fait remarquable !

Du coup, la bonne nouvelle est que j’ai simplement besoin de vous partager son contenu. Il y a d’abord 2 vidéos à regarder.

La première explique la mécanique quantique en 7 idées :

Que vous devriez faire suivre immédiatement par celle expliquant l’intrication quantique :

Et pour couronner le tout, David a écrit un excellent billet de blog complétant sa première vidéo : La mécanique quantique où il détaille et illustre très bien les 7 idées exposées.

Au final, vous devriez alors avoir de vagues notions de ce que sont :

– Le principe de superposition : vous savez, le coup des plusieurs états en même temps.

– L’indéterminisme de la mesure : les mesures dépendent, en partie, du hasard !

– La réduction des états quantiques : l’indéterminisme est cassé après la première mesure. Le fait de mesurer force la particule à choisir son état. C’est bizarre mais c’est comme ça. Par contre, cela énerve passablement Meulta. Mais cela ne change rien au phénomène heureusement.

– La dualité onde – corpuscule : des objets que l’on croit ponctuel se comporte en fait comme des ondes. D’ailleurs la mécanique quantique fut longtemps appelée mécanique ondulatoire.

– L’effet tunnel : l’effet ondulatoire de la matière, qui lui permet, entre autres, de passer à travers un obstacle.

– La quantification des propriétés physiques : un électron ne peut être qu’à des orbites précises autour du noyau, certaines lui sont “interdites” ! Cela avait retourné la tête de Deltakosh dans notre épisode 4.

– Le principe d’incertitude de Heisenberg : on ne peut pas parfaitement définir un objet à la fois grâce à sa position ET sa vitesse. C’est un peu soit l’un, soit l’autre.

– Le spin et l’intrication quantique : le spin n’a que 2 valeurs possibles, le + ou le -… ainsi que toutes les superpositions de ces 2 états bien entendu. Plus surprenant encore, certaines particules sont liées. Cela veut dire que le changement d’état d’une particule va avoir une répercussion immédiate sur une autre pouvant se trouver à des kilomètres d’elle.

Pour finir, après la première publication de cette article, David a également sorti une vidéo sur l’ordinateur quantique : Les Ordinateurs Quantiques — Science étonnante #40

Bon, déjà, avec tout ça, je vous promets un méchant carton à votre prochaine soirée pour épater la galerie !

Pour revenir à la loi de Moore et aux processeurs actuels, le principal problème vient de l’effet tunnel. En effet, c’est l’effet qui va se produire avec la miniaturisation des transistors. Les électrons (le courant faisant les 0 et les 1 de notre bien aimé binaire) ne resteront plus dans le chemin que nous avons défini/dessiné. Ils passeront à travers les barrières que nous aurons fixées. Bref, un monumental bazar ingérable pour conserver la stabilité de la logique du processeur.

L’ordinateur quantique Le qubit Le fameux ordinateur quantique repose donc sur les principes balayés précédemment afin de mettre au point le qubit, pour quantum binary digit, l’analogue quantique du bit.

Bon, gros problème : j’ai été incapable de trouver une explication totalement claire de son fonctionnement et, plus embêtant, de son utilisation.

Malgré tout, commençons par l’article Wikipedia qui lui est dédié : https://fr.wikipedia.org/wiki/Qubit

On nous explique que « le qubit se compose d’une superposition de deux états de base, par convention nommés |0> et |1> » et qu’« un état qubit est constitué d’une superposition quantique linéaire de ces deux états ». Jusque-là, rien de bien surprenant puisque cela reprend le principe de superposition et de spin vu précédemment.

Par ailleurs, on apprend toujours aussi logiquement, d’après les principes vus avant, que l’on ne peut copier la valeur d’un qubit vers un autre puisque cela impliquerait de lire sa valeur, et donc de réduire son état quantique. On peut cependant transporter son état sur un autre qubit grâce à la téléportation quantique basée sur le principe d’intrication quantique toujours décrit également dans le chapitre précédent.

Alors tout ça me permet de comprendre à peu près la structure d’un qubit mais cela ne m’aide pas pour autant à avoir la moindre idée de comment on exploite cela pour créer de l’information ou pour calculer quoi que ce soit !

Je vais quand même vous partager quelques articles qui tentent d’expliquer et qui m’ont un peu aidé :

– La révolution quantique de demain. Un petit extrait : « Ce qubit peut ainsi faire 2^4=16 calculs en un seul coup. Le gain est exponentiel car avec 4 bits classiques, on ne pourrait faire qu’un seul calcul! Dans un ordinateur classique, on est obligé d’utiliser des milliards de transistors pour réaliser de nombreuses opérations rapidement. Avec un ordinateur quantique, il suffirait de 300 qubits pour obtenir 2^300 résultats, ce qui est plus que le nombre de gouttes d’eau dans nos océans, ou encore, plus que le nombre d’atomes dans notre univers observable !** ». Superbe « punch line » mais aucune explication réelle du pourquoi du comment.

– Qu’est-ce qu’un ordinateur quantique ? qui tente à nouveau d’expliquer un peu mieux mais qui nous sort aussi de grandes envolées cosmiques du genre : « Or, il pourrait bien être possible d’avoir des ordinateurs avec une architecture de 50, 100, 300 qubits, et donc d’avoir un ordinateur quantique 2^50, 2^100 et donc 2^300​​ fois plus rapide qu’un ordinateur normal ! » ou « le facteur de rapidité correspondant à 2^300 est plus grand que le nombre d’atomes dans l’univers. C’est un nombre immense et un ordinateur quantique de ce calibre serait donc immensément plus rapide qu’un ordinateur normal ». A nouveau, on ne sait pas trop pourquoi mais comme on dit par chez moi, ça claque sévère !

Ce que j’en retiens, sans vraiment comprendre totalement pourquoi :

– La puissance d’un ordinateur quantique augmente potentiellement de manière exponentielle en fonction du nombre de qubits. En effet, sa puissance de calcul double à chaque fois que l’on lui adjoint un nouveau qubit.

– Le principe du qubit permet d’envisager des calculs parallèles de manière incroyablement plus rapide que les architectures informatiques actuelles.

Par ailleurs, il faut savoir qu’il est extrêmement difficile de stabiliser ces fameux qubits. Certains laboratoires ont réussi à fabriquer des processeurs à base de quelques-uns d’entre eux. Cependant, leur durée de vie n’est souvent que de quelques secondes. En effet, les particules quantiques sont très sensibles au monde qui nous entoure comme le magnétisme et la lumière.

En termes de fabrication « concrète » d’un qubit, j’ai pu trouver ce dossier particulièrement intéressant : Fabrication d’un ordinateur quantique qui décrit les différentes approches actuelles. Ce dossier est d’ailleurs l’un des meilleurs que j’ai pu lire sur l’ordinateur quantique d’une manière générale. Je vous en conseille la lecture complète. On y apprend ainsi que la difficulté physique principale est de gérer la décohérence des états quantiques.

Allez, essayons maintenant d’en savoir davantage sur l’ordinateur quantique en lui-même.

D-Wave Aujourd’hui, il semblerait qu’un seul modèle abouti et commercialisé de l’ordinateur quantique existe. Il est fabriqué par la société D-Wave Systems qui aurait réussi à fabriquer une machine stable alignant jusqu’à 2000 qubits ! C’est d’ailleurs une source de débats et d’interrogations fréquentes chez les experts dans le domaine car la quasi-totalité des labos peinent à aligner plus qu’une dizaine de qubits de manière stable.

Pour continuer sur le travail de vulgarisation, je vous invite à regarder la vidéo suivante : 10 CHOSES à SAVOIR sur l’INFORMATIQUE QUANTIQUE. Bien que souvent très approximative, elle balaie, en moins de 20 minutes, l’ensemble des thématiques abordés dans cet article. On y apprend des choses rigolotes sur le D-Wave :

– L’ordinateur D-Wave se trouve dans un caisson blindé qui diminue de 50000 fois le champ magnétique terrestre.

– Il fonctionne à une température de 0,015° Kelvin, soit -273,135° Celsius. Soit plus froid que l’espace lui-même qui se trouve à 3°K ! Donc très proche du 0 absolu.

– Comparons maintenant la consommation énergétique du D-Wave 2X face au supercalculateur Pangea de Total qui était l’un des plus puissants au monde début 2016 avec 6,7 pétaFLOPS au compteur. Le Pangea nécessite 4,5 MégaWatts là où le D-Wave 2X de 1000 qubits n’a besoin que de… 25000 Watts, soit 180 fois moins.

Certains clients connus de D-Wave sont Google, la NSA et Lockheed Martin. La NSA a d’ailleurs annoncé qu’elle avait stoppé tous ses investissements sur les technologies non quantiques. On va comprendre pourquoi juste derrière.

Algorithmes et programmation Vous l’aurez compris, un ordinateur quantique étant radicalement différent d’un ordinateur classique, la manière de concevoir un algorithme l’est tout autant.

Si je reprends l’émission de France Inter, Pascale Senellart s’est tentée à une explication que j’ai trouvée très intéressante mais perturbante (comme toujours en quantique). Elle pose le problème d’un algorithme chargé de trouver le moyen de sortir d’un labyrinthe. Si un être humain essaie de sortir du labyrinthe, ou si vous essayez d’envisager un algorithme de sortie pour un CPU classique, vous allez surement tenter de parcourir le labyrinthe en profondeur jusqu’à trouver le chemin, unique, vers la sortie. En d’autres termes, vous allez essayer d’aller à gauche, de voir que cela ne marche pas, puis d’aller à droite tout en prenant soin de noter vos réussites et erreurs. Bref, au final, vous allez sortir du labyrinthe en ayant la capacité d’expliquer le chemin que vous avez parcouru. Un ordinateur quantique va lui calculer en parallèle toutes les possibilités, il va ensuite vous dire qu’il est sorti du labyrinthe mais ne sera pas capable de vous dire comment il l’a fait ! Cela est également relativement logique. En effet, pour savoir par où il est passé, il faudrait stopper en cours de route l’algorithme quantique, faire des mesures et donc… réduire les états quantiques ! Cela aurait pour conséquence de casser l’algorithme quantique.

D’après ce que j’ai compris, cela est lié au fait que les algorithmes quantiques sont probabilistes. Ils donnent la réponse correcte avec une haute probabilité et la probabilité d’échec peut être diminuée en répétant l’algorithme.

Au final, l’article le plus complet que j’ai pu trouver sur un calculateur quantique est sans conteste celui de Wikipedia : calculateur quantique. Il couvre absolument tout, de manière très détaillée. Il n’est pas particulièrement facile d’accès mais je permets d’extraire 2 citations intéressantes :

« Les calculateurs quantiques demandent des techniques de calcul différentes de la programmation, mais utilisant beaucoup l’algèbre linéaire classique pour conditionner et traiter simultanément des ensembles de données liées. » … « Il ne se prête donc a priori qu’aux calculs dont la complexité réside dans la combinatoire. On trouve ces problèmes dans l’ordonnancement et les autres calculs de recherche opérationnelle, en bio-informatique, et bien entendu en cryptographie. Le faible volume des entrées-sorties par rapport à celui du traitement semble rendre toutefois plausible leur usage à distance un jour à travers le réseau Internet. »

On peut déjà comprendre ici que le domaine d’application des ordinateurs quantiques va être différent de ceux des ordinateurs classiques.

Les algorithmes quantiques Je n’ai entendu parler que de 2 algorithmes quantiques principaux :

– Algorithme de Shor permettant de factoriser rapidement un entier naturel en nombres premiers. Pratique pour casser RSA, le moyen de chiffrement considéré le plus sûr aujourd’hui.

– Algorithme de Grover permettant de chercher dans une liste.

C’est le premier qui est le plus célèbre bien évidemment et qui intéresse au plus haut point la NSA. En effet, le niveau de protection du chiffrement actuel repose sur le fait qu’il est virtuellement temporellement impossible pour un ordinateur classique de casser la clé. Cela demande en effet de pouvoir factoriser la clé publique en produit de nombres premiers. Avec les clés actuellement utilisées, il faudrait des milliers d’années pour la casser. Avec l’algorithme de Shor exécuté sur un ordinateur quantique comme le D-Wave, cela est censé être fait en quelques minutes. Vous comprenez maintenant mieux pourquoi la NSA s’y intéresse de près. Heureusement, il y existe la cryptographie quantique pour palier à cela, mais c’est encore très loin d’être exploitable.

Ce qu’il faut retenir de tout cela, c’est que les algorithmes applicables à l’ordinateur quantique sont très différents de ceux que nous avons l’habitude de voir et surtout pour l’instant très peu nombreux. Leur conception repose sur l’usage de portes quantiques qui vont jouer sur les probabilités des qubits tel que cela est expliqué en partie dans les ressources Q# que je partage ci-dessous.

Par ailleurs, je trouve parfois absurde de tenter de comparer la vitesse d’exécution d’un ordinateur classique avec un ordinateur quantique de manière absolue. Lorsque vous lisez qu’un ordinateur quantique est 2^300 plus puissant qu’un ordinateur classique, c’est en grande partie faux. Sur l’algorithme de Shor, cela apparaît évident. Mais aujourd’hui, un ordinateur quantique est incapable d’utiliser la quasi-totalité des algorithmes fonctionnant parfaitement sur un ordinateur classique. Peut-être qu’un jour, nous arriverons à créer un ordinateur quantique « générique » capable de fédérer les 2 mondes actuels mais il n’y a aucune certitude. L’autre possibilité, actuellement plus probable, est que les 2 seront complémentaires. Nous continuerons à avoir des ordinateurs portables et des smartphones pour les tâches classiques et nous aurons probablement des ordinateurs quantiques pour des tâches impossibles à réaliser pour un ordinateur classique. Mais l’inverse sera sûrement vrai, un ordinateur quantique ne pourra se substituer à un ordinateur classique sur certaines opérations.

Pour résumer, voici une citation (dont je ne trouve malheureusement plus la source) qui m’a paru bien résumer la situation : « L’ordinateur classique est déterministe, l’ordinateur quantique est probabiliste »

Je vous invite également à nouveau à regarder ma vidéo de ma session Paris Web 2018 où j’essaie d’expliquer en quoi un algorithme quantique est bien plus rapide qu’un algorithme classique sur des problèmes de combinatoires grâce à la superposition des qubits et l’utilisation de portes quantiques. Je l’ai enfin mieux compris pour ma part grâce à cette vidéo fantastique : The Mathematics of Quantum Computers | Infinite Series

Allez pour vous remercier d’être allés aussi loin dans l’article, je vous offre la meilleure vidéo que j’ai pu trouver (en anglais) expliquant l’ordinateur quantique et les qubits, le tout en moins de 8 min ! La voici : Quantum Computers Explained – Limits of Human Technology. Franchement, regardez, ce seront les 8 min les mieux dépensées de votre journée, je vous le garantis (satisfait ou remboursé).

Domaines d’applications

Comme je vous l’explique dans mes conférences, la nature est quantique. L’idée d’un ordinateur quantique est donc de se comporter comme la nature pour mieux la comprendre. Les domaines d’applications théoriquement envisagés pour l’ordinateur quantique sont donc extrêmement prometteurs :

– Être capable de nettement mieux maitriser la chimie pour, par exemple, améliorer la fixation de l’azote ou la capture du CO2.

– Le Machine Learning et plus spécifiquement le Deep Learning se prêtant parfaitement au côté probabiliste de l’informatique quantique.

– Pouvoir prévoir la météo de manière exacte.

– La Génomique

– Maîtriser la matière en réussissant, par exemple, la mise au point de supraconducteur à température ambiante.

Bref tout ce qui relève de problèmes à très fortes combinatoires.

Expérimenter l’algorithmique quantique Si vous souhaitez tenter de comprendre l’algorithmique quantique, Microsoft, Google et IBM fournissent 3 projets pour « simuler » des qubits sur des ordinateurs actuels ou à travers le cloud. En effet, tout reste à faire dans ce domaine et les différents acteurs ont besoin de votre créativité !

Côté Microsoft, nous avons récemment sorti un Quantum Development Kit ainsi qu’un langage dédié à la programmation quantique s’appelant Q#. Il a pour but d’aider au développement et à la compréhension de protocoles quantiques, d’algorithmes quantiques, de correction d’erreurs quantiques et de périphériques quantiques. Grâce à ce kit, vous pouvez simuler un système de 30 qubits sur un ordinateur équipé de 16 Go de RAM et nous offrons également un simulateur dans Azure de 40 qubits. Tout cela est disponible à travers une extension Visual Studio sous Windows ou VS code pour Linux et MacOS. Vous trouverez l’ensemble à l’adresse suivante : http://www.microsoft.com/quantum. Enfin, si vous souhaitez faire votre premier “hello world” quantique (soit l’algorithme de téléportation quantique), suivez cette excellente série de vidéo : MS Quantum Playlist.

Vous en voulez encore plus ? Vous voulez comprendre plus en détail ? Voici 2 excellents articles permettant de vous mettre en pied à l’étrier quantique :

– A Beginner’s Guide to Quantum Computing and Q#

– Quantum Teleportation in Q#

– Pour apprendre le fonctionnement des portes quantiques en Q#: https://aka.ms/quantum-katas ainsi que quelques algorithmes.

Attention, c’est quand même assez complexe malgré tout.

Côté Google, il y a un projet nommé « Quantum Computing Playground » : http://www.quantumplayground.net qui se trouve être une expérimentation Chrome basée sur WebGL. L’idée est de simuler, à l’aide du GPU, un ordinateur quantique de 22 qubits.

Enfin côté IBM, le projet s’appelle IBM Quantum Experience : https://www.research.ibm.com/ibm-q/qx. Il vous permet d’exécuter, à travers le Cloud, des algorithmes quantiques et d’expérimenter ce qu’il est possible de faire avec l’informatique quantique. J’ai également particulièrement apprécié la vidéo d’une ingénieure de chez IBM que je vous recommande chaudement : A Beginner’s Guide To Quantum Computing

L’approche de Microsoft L’approche de D-Wave est intéressante mais pour certains « la machine actuelle n’est pas un calculateur quantique général, mais optimisé pour un type de calcul nommé le recuit simulé ».

Les grandes entreprises comme Google, IBM ou Microsoft planchent donc toutes à trouver un moyen de créer un calculateur quantique général.

Microsoft a choisi une approche différente des autres suite à la découverte, en 2012, d’une particule appelée fermion de Majorana qui avait été théorisée en 1937. Cette particule permettrait de mettre au point un qubit dit topologique, plus stable car moins sensibles à la décohérence et plus prometteurs pour une fabrication industrielle de l’ordinateur quantique. En effet, tous les qubits ne sont pas équivalents. Les particules quantiques sont très sensibles à la chaleur, à la lumière ou au magnétisme. Du coup, en fonction de l’approche retenue pour construire un qubit, le taux d’erreur associé ou sa durée de vie peuvent fortement variés. N’oublions pas que l’informatique quantique est probabiliste. Donc si le fait d’ajouter de nombreux qubits les uns derrière les autres augmentent fortement le taux d’erreur associé, cela réduit d’autant l’intérêt. Contrairement aux processeurs classiques, comparer 2 processeurs quantiques en fonction du nombre de qubits pour en déduire sa puissance n’a pas de sens. Globalement, bien entendu, plus le nombre de qubits sera important plus la puissance de calcul associée est censée également être importante. Mais si, et seulement si, le taux d’erreur des qubits utilisés reste maitrisé.

Pour en savoir davantage, je vous invite à lire ces 2 articles :

– Ordinateur quantique : Microsoft et Alcatel-Lucent sont-ils sur la bonne piste ?

– Des ordinateurs quantiques topologiques avec des fermions de Majorana ?

Mais surtout, pour comprendre notre approche et son potentiel, je vous invite à regarder cette excellente vidéo (en anglais) faite par les chercheurs travaillant sur notre ordinateur quantique chez Microsoft :

Elle est, je trouve, particulièrement claire et explique très bien l’avantage de notre approche du qubit topologique.

Pour finir, sachez que Microsoft a tout simplement une division complète dédiée à la réalisation d’un ordinateur quantique qui se nomme Station Q. Vous pouvez en connaître davantage sur ces différents sites :

– https://stationq.microsoft.com/

– https://www.microsoft.com/en-us/research/group/station-q/

Vous y trouverez pléthores d’articles et vidéos passionnantes à découvrir.

D’autres articles / formations complémentaires Je tenterai de mettre à jour régulièrement cette section en fonction de mes découvertes et apprentissages. Voici une liste complémentaire de ressources susceptibles de vous aider :

– La ressource ultime en français : l’ebook fabuleux de plus de 300 pages d’Olivier Ezratty qui couvre presque tous les aspects de l’informatique quantique.

– Lecture Collection | The Theoretical Minimum: Quantum Mechanics

– Q is for Quantum par Terry Rudolph qui vous donnera certainement envie d’acheter son livre

– Les cours quantiques du MIT

– Introduction à la physique quantique – partie 1, un cours relativement simple et accessible à tous.

– Les cours passionnants d’Etienne Klein: Cours n°1 – La naissance de la physique quantique

– Quantum Computing for Computer Scientists qui est excellent mais qui nécessite d’avoir bien compris de nombreuses notions avant. Retrouvez les slides ici.

J’espère que ce long et détaillé article vous aura aidé à mieux appréhender ce futur qui se précipite depuis quelques mois. L’informatique quantique semble être incontournable pour continuer notre course à la puissance de calcul et pour, je l’espère, améliorer notre monde. Il existe d’autres approches moins médiatiques comme les processeurs neuromorphiques. Par exemple, IBM a réussi à reconstituer un cerveau numérique de rongeur avec 48 puces neuromorphiques. Mais cela est une autre histoire…

N’hésitez pas à partager vos commentaires sur ce blog ou sur twitter : @davrous.

View Details

A couple of months ago, Facebook has introduced a very cool new feature: Richer 3D Posts on Facebook and New Ways to Share. You can now share 3D content in your news feed. However, creating this 3D content was reserved to 3D experts. Hopefully, using the latest version of Paint 3D on Windows 10, anyone can now create its own content!

In this tutorial, we’ll see how to create this great HoloRobot:

In less than 5 minutes!

Assets requirements To share 3D content on Facebook, you must follow their requirements. The 3D assets must be saved using the .GLB file extension (GLB is the binary version of the glTF 2.0 file format). The GLB file must be under 3MB. Facebook is providing this tool to check if your content is ready to be shared: https://developers.facebook.com/tools/3d/validation which seems to be a fork of this one.

Tutorial 1: creating 3D content using just your awesome design skills As we’re using Paint 3D in this tutorial, you first need to learn how to use this simple but powerful tool. Couple of great short videos to watch:

– Overview of the tool
– Windows 10 Paint 3D: The Magic select tool
– Windows 10 Paint 3D: How to make a penguin
– Windows 10 Paint 3D: Creating the perfect pigeon
– Official videos playlist

You can also read the official documentation associated to Paint 3D.

Once we’ll have created your desired 3D content, click on the top left button to expand the menu.

Click on “Export file” then choose “3D – GLB”.

Save it on the disk. Go to your Facebook wall and click to create a new post. Then, you simply need to drag’n’drop your saved .GLB file into the right area.

Say what’s on your mind and you’re done!

People will be able to see your 3D creation on PC/Mac in any recent browser and on Android and iPhone devices. On smartphones, rotating the device will move the camera around the object and you’ll be also able to move using touch to rotate around the Y axis or to pinch to zoom. On desktop, use the mouse or your touch screen if you’ve got one.

Tutorial 2: creating 3D content using free existing assets If you’re as bad as I am in designing 3D objects, you can use and mix existing content in Paint 3D. There are several great 3D assets platforms available:

– Remix 3D from Microsoft, also natively incorporated in Paint 3D
– Poly from Google, which was mainly designed for AR / VR experiences with low-poly (light) 3D objects.
– Sketchfab

Please keep in mind 2 important points before using some of the assets to share on Facebook:

– The total size of your 3D mix shouldn’t be bigger than 3MB
– Pay attention to the copyright / license to re-use or credit the author

For the HoloRobot I’ve shared at the beginning, I’ve been using 2 assets from Remix 3D and Poly:

– MS Gundam RX-78-2 on Google Poly
– HoloLens on Remix 3D

You can see how to create it from scratch in this short video:

Thanks for reading. Now, feel free to share your awesome 3D creations you’ll be able to build for Facebook in the comments section!

View Details

A few months ago, I’ve shared the work we’ve done in Babylon.js v3.0 to create WebVR experiences: from zero to hero, creating WebVR experiences with Babylon.js on all platforms. I was quite happy of the job we’ve done but it wasn’t as simple and powerful as I first imagined. That’s why, I’ve quickly started to work on a new feature for v3.1 I’ve named VRExperienceHelper to make the life of WebVR content creators much simpler. Babylon.js v3.1 has just shipped, to know more about this release, check this blog post: announcing Babylon.js v3.1.

Basically, I’ve integrated most of the code & ideas shared in the previous article directly in the engine. Thanks to that, it’s now very straightforward to enable VR on all platforms, add interactions with controllers and/or teleportation. Let me show that through this small tutorial. You will even learn how to create a WebVR scene using Paint3D!

Prerequisites We will use our Mansion scene as a starting point. Open this sample in your browser: https://playground.babylonjs.com/#HARNA9 that will show you how to load a Babylon.js scene.

This scene has been entirely done with our 3DS Max exporter, including integration of 3D spatial sounds using Web Audio. It also embeds the usage of our Actions Manager.

We’re now going to use the VRExperienceHelper. You can check our documentation on this new feature.

Swiching to VR In Babylon.js, enabling VR on all platforms is just about this line:

var VRHelper = scene.createDefaultVRExperience();

The helper will then:

– add a HTML button on the bottom right of the WebGL rendering canvas

– check if WebVR is supported on your platform

– by default, replace the current active camera by a Device Orientation camera, using the active camera position. This can be changed in the options passed to the constructor by setting {createDeviceOrientationCamera:false}

Using the previous sample, if you had this magic line, you’ll ended up with this code sample: https://playground.babylonjs.com/#HARNA9#1

If you open it on a smartphone or a device with motion sensors, you’ll see you can already rotate the camera by moving the device around you.

Clicking on the VR button will:

– Enter VR in your headset if you’ve got one and render in stereo. If you’ve got controllers, it will download their associated models from our CDN. In v3.1, we’ve got a default support for Windows Mixed Reality spatial controllers, Oculus Touch and HTC Vive. If your device is not part of this list, we will display a generic model for it.

– If your browser doesn’t support WebVR, or if it does but no headset is connected, we will fallback to our VRDeviceOrientation camera for a carboard like experience.

Note: Windows Mixed Reality spatial controllers are using glTF models. Don’t forget to reference the glTF extension after referencing babylon.js. This extension can be found here and its documentation there.

Enabling interactions Enabling interactions is just about 1 line of code:

VRHelper.enableInteractions();

The VRExperienceHelper will then create a cursor that will follow your gaze if you’ve got no VR controller, like using an Xbox gamepad on desktop or using a mobile VR solution. The experience will be identical as the one we’ve created in the Windows Mixed Reality Portal, also known as the Cliff House.

Adding this line to the previous sample will create this demo: https://playground.babylonjs.com/#HARNA9#2

You can watch the result in this video:

First, you’ll see the usage using a regular Xbox gamepad. You can notice that the gaze cursor is precisely following every facet of the mesh looked at. Try for instance looking at the floor and at one of the walls of the mansion and you’ll notice it slightly changes (moving to horizontal to vertical) to give you the right feedback, exactly as inside the Windows 10 MR Portal. As I really liked the UX of the Cliff House, I’ve decided to mimic it as much as possible.

When looking at the circle mesh on top of the door of the mansion, the gaze cursor will change from white to blue and will become bigger. This is to indicate to the user he can launch an action associated to this mesh. To trigger it, simply press the A button of the gamepad while looking at it.

In the second part of the video, I’m powering one of my VR controllers. You’ll see that immediately after loading the associated model, a pointing laser will go out of the controller and will use the gaze cursor as the new pointing target. If you’ve got 2 controllers and you’d like to display the laser pointer to the other one (if you’re left handed for instance), simply press the main button of your controller to activate/deactivate it on the desired controller. On Windows Mixed Controller and HTC Vive, the main button is the grab button and on Oculus Touch, it’s the A button. To trigger an action, use the trigger button of the controller.

Let’s see another cool sample. Please load this Playground sample: https://playground.babylonjs.com/#PVM80T and review quickly the code. It shows how to use our GUI layer to build UI directly in the 3D space, which is mandatory to build UI in WebVR.

You can click on the button, radio buttons and the color picker using the mouse or touch. You’ll see also that changing the value of the color picker will change the color of the small sphere near it. We’re using the Pointer Events specification to support all type of inputs. In VR, we’re simulating those same pointer events inside the VRExperienceHelper to build interactive UIs in a seamless way. In a nutshell, this means that once your UI works in 2D using mouse or touch, in will work in VR.

Let’s transform this second sample in VR with interactions enabled by adding the same 2 lines of code we’ve seen previously.

You’ll ended up with the same code as: https://playground.babylonjs.com/#PVM80T#1

Have a look to the result in this video in VR:

At the beginning, I’m only using the Xbox gamepad again to click on the button & radio button. You see also that we’re simulating pointer move if you keep pressing the A button while moving your gaze on the color picker. In the second part, I’m either using the right or left controller to play with the UI using the laser pointers.

If you’d like to see how to implement a gaze selection based on a timing approach for mobile VR solutions for instance, please have a look to this sample: https://playground.babylonjs.com/#5P51YL

Building a fruit ninja like game I’ve made a complete gaming sample to show you how to build a Fruit Ninja like game in WebVR. You can see the result in this video:

During //BUILD 2018, I’ve shown you how to create this game using the following samples code:

– 2D (non-VR) version of the game: https://playground.babylonjs.com/#22KIIK
– How to switch VR controllers from their default models to laser saber: https://playground.babylonjs.com/#IY99P5#4 and activate glow on a button

Play to the game: https://playground.babylonjs.com/#22KIIK#4 which mixes all those samples together. You can even play it using mouse or touch even if it’s much less funny than in VR!

I’ve tried to comment the code as much as possible to make it clear, so please read it.

Features:

– GUI in VR like seen just before

– how to enable/disable the gaze/laser pointers while in game

– how to switch the models of the controller from the default one loaded from our CDN to a laser saber found on Remix3d and vice versa.

– how to manage collisions between the laser saber and the fruits using the Action Manager.

– Web Audio, nice particles effects, etc.

At last, I’ve built a PWA version of this game available there: Apple Crusher PWA

Enabling teleportation Finally, enabling teleportation is again just about 1 line of code:

VRHelper.enableTeleportation({ floorMeshName: "NameOfTheMesh" });

Once used on the previous Mansion scene we’ve used, you’ll have this code: https://playground.babylonjs.com/#HARNA9#3

Again, the VRExperienceHelper is providing for free a lot of cool UX features, inspired from the Cliff House experience. To teleport, on Mixed Reality/Oculus Touch/Xbox gamepad, move the thumbstick forward or press up the touchpad of the Vive to display the teleportation target. Move it where you’d like to go using gaze or the laser pointer of the controller. On release, you will be teleported on the point you’ve selected using a specific animation timing. You’ll notice also we’re reducing the FOV (Field of View) during the animation to avoid motion sickness using our vignetting post-process.

You can also move a step backward using the thumbstick/touchpad down and rotate left/right the camera by 45 degrees.

Bonus – build VR scenes with Paint3D and Remix3D If you’ve got a PC running Windows 10 Fall Creators Update, you can use the great 3D models available on http://www.remix3d.com, build a scene inside Paint 3D and export it to glTF (.GLB) format. Then making a WebVR version out of it using Babylon.js is very simple. Let me show you how.

First, select your favorite assets from Remix3D.com. On my side, I’ve chosen those characters: Gabe (Yeti), Saul the Snowman, Grey wolf – low poly, Unicorn, Smiling Sun, Fireworks 1, Fireworks 2 and I’ve decided to use this cool island to put the characters on it: Snow Patch. It will also be used for the teleportation area.

Start by opening “Snow Patch” in Paint 3D, scale it bigger and then add the various other assets. You should obtain something similar as:

Now, we’re going to paint a floor on top of the island that will act as the teleportation mesh for the VRExperienceHelper. Move the camera to view your scene from above, like you’re looking from the sky vertically:

Select 3D doodle – Sharp edge to draw the shape where you’d like to teleport onto:

You will then have a mesh model on top of the Island. Reduce the height and try to set the position as close as possible to the ground.

Now, export your work to .GLB :

Next step is to check the name of the red 3D doodle mesh. For that, take the .glb file you’ve exported and drag-n-drop it inside our Babylon.js Sandbox tool: https://sandbox.babylonjs.com/ :

Using the babylon.js debug inspector (middle button available on the lower right of the screen), find the 3D doodle mesh you’ve created. In my case, it’s named “mesh_id81”.

We’re now ready to use this Paint3D scene in WebVR. Babylon.js WebVR sample using the VRExperienceHelper and this Paint3D scene: https://playground.babylonjs.com/#513Z88.

Reading the code, you’ll see I’m just scaling the world to have a better ratio for VR and enabling teleportation on mesh named “mesh_id81” as explained just before. I’m also making it non visible.

Check this video that shows the experience inside the headset:

I hope you’ll be able to create your own Mixed Reality world or experiences in an easy way thanks to Babylon.js v3.1. Feel free to share your creations and/or questions & feedbacks to the team on our forum.

David