Mozilla Gfx Team Blog

View Details

Hi there! This is a guest post from Robert Mader, who contributed enormous improvements to Firefox’s graphics stack on Linux. TL;DR In the upcoming Firefox 94 release we will enable the EGL backend for a big group of our Linux users. This will increase WebGL performance, reduce resource consumption and make our life as developers … Continue reading Switching the Linux graphics stack from GLX to EGL →

View Details

Hi there! This is a guest post from Robert Mader, who contributed enormous improvements to Firefox’s graphics stack on Linux.

TL;DR In the upcoming Firefox 94 release we will enable the EGL backend for a big group of our Linux users. This will increase WebGL performance, reduce resource consumption and make our life as developers easier going forward.

Background In order to use hardware accelerated APIs like OpenGL with windowing systems like X11 or Wayland there needs to be an interface bringing them together. For OpenGL on X11 most programs use GLX, while its successor, EGL, gets used on Wayland, Android and in the embedded space. While EGL has some major advantages compared to GLX and, in theory, can be used on X11 just as well, its adoption there has been very slow.

I can only speculate why exactly that is, but I think it comes down to the following reasons:

Games and similar applications barely benefit from the switch
Applications and toolkits that would benefit from it often don’t enable hardware accelerated rendering on X11 in the first place. Likely because of the bad and complex driver situation in the past etc.
Because of the slow adoption, X11 EGL implementations remained buggy and incomplete → back to 2.

What changed? Firefox is an application that benefits heavily from hardware acceleration in many areas. However, until recently, software rendering remained the default. It was only this year that finally Webrender, Firefox’s new rendering engine, got enabled for most Linux users.
There is a very long list of developments that made this step easier and thus possible. To name a few:

OpenGL drivers got better
Xorg DDX drivers got better (e.g. the “modesetting” driver becoming the standard for Intel)
Composited desktops became more common
Plugin support (Flash Player) was dropped from Firefox
Webrender made hardware acceleration much more desirable compared the old OpenGL layers backend
New technologies such as Wayland and DMABUF emerged

The last point was crucial for the topic of the post. When Martin Stránský implemented Wayland hardware acceleration support in Firefox, he could not reuse GLX code, but instead used the Android EGL one. From there, an interesting dynamic started.

Improving the EGL backend and sharing code Step by step, a number of improvements were made to the EGL/Wayland backend which had effects on other platforms as well:

In order to improve WebGL performance and allow efficient hardware video decoding, Martin implemented zero-copy GPU buffer sharing via DMABUF. This is much easier on EGL than on GLX. And while Firefox did have a similar buffer sharing implementation for X11 (using Xrender), that one was never stable enough to get turned on by default.
I improved the EGL backend to not only support OpenGL ES but also “desktop” OpenGL, making sure it’s not lacking behind the GLX backend.
I went on an made it possible to use the EGL backend on X11 as well.
Martin extended the DMABUF and VAAPI support to X11.
Greg, an independent Wayland contributor, wrote an initial implementation for partial damage on EGL.
Jamie Nicol extended the partial damage support to properly work on Android – and thus on X11 as well.
Greg made sure our GPU detection (and smoke test from days when drivers would often crash) works on Wayland without requiring Xwayland to be present, making it not require GLX any more.

This is just a very small extraction of examples and maybe it gives you an idea of what I’m trying to say: more and more code gets shared between Wayland, X11/EGL and Android. This improves code quality, increases available time to spend on features and bugs, reduces the maintenance burden – you name it.

Making EGL the default Over the last year, more and more user found out about the possibility to use EGL on X11 – likely because it’s a prerequisite for hardware video decoding. Lots of bugs got fixed in Firefox but also other components. Now we finally feel ready to let it ride the trains. As of Firefox 94, users using Mesa driver >= 21 will get it by default. Users of the proprietary Nvidia driver will need to wait a little bit longer as the currently released drivers lack an important extension. However, most likely we’ll be able to enable EGL on the 470 series onwards. DMABUF support (and thus better WebGL performance) requires GBM support and will be limited to the 495 series onwards.

Benefits for users So what exactly can you expect, and why? Mainly:

Improved WebGL performance. Thanks to DMABUF zero-copy buffer sharing, WebGL can be done both sandboxed and without round-trip to system ram. WebGL is not only used in obvious places such as games, but also in more subtle ways, e.g. on Google Maps.
Reduced power consumption. With partial damage we don’t need to redraw the whole window any more if only a small part of the content changed. Common examples here are small animations on websites or when loading tabs.
Less bugs. EGL is more modern, much better suited for complex hardware accelerated desktop applications and used on more platforms, compared to GLX.
Hardware video decoding by default is another crucial step closer – in fact for most users it should now be only one preference away (but beware, it still has a couple of bugs).

Special thanks There is long list of people who have contributed to this step. To name a few: Martin Stránský, Andrew Osmond, Jamie Nicol, Grep V, Jan Ikenmeyer (Darkspirit), Michel Dänzer, the Firefox GFX-Team, the Mesa project and contributors, the Nvidia drivers team, the GTK team.

Finally: thanks a lot to all users who filed bugs and helped us fix them!

About the author Hi, I’m Robert Mader, a free time FOSS contributor, mainly working on Firefox and Mutter/Gnome-Shell.

View Details

WebGPU is a new standard for graphics and computing on the Web. Our team is actively involved in the design and specification process, while developing an implementation in Gecko. We’ve made a lot of progress since the last public update in Mozilla Hacks blog, and we’d like to share!

WebGPU textured+lit cube in Firefox Nightly with WGSL shaders.
See full code in the fork. API Tracing Trouble-shooting graphics issues can be tough without proper tools. In WebRender, we have the capture infrastructure that allows us to save the state of the rendering pipeline at any given moment to disk, and replayed independently in a standalone environment. In WebGPU, we integrated something similar, called API tracing. Instead of slicing through the state at any given time, it records every command executed by WebGPU implementation from the start. The produced traces are ultimately portable, they can be replayed in a standalone environment on a different system. This infrastructure helps us breeze through the issues, fixing them quickly and not letting them stall the progress.

Rust Serialization Gecko implementation of WebGPU has to talk in multiple languages: WebIDL, in which the specification is written, C++ – the main language of Gecko, IPDL – the description of inter-process communication (IPC), and Rust, in which wgpu library (the core of WebGPU) is implemented. This variety caused a lot of friction when updating the WebIDL API to latest, it was easy to introduce bugs, which were hard to find later. This architectural problem has been mostly solved by making our IPC rely on Rust serde+bincode. This allows Rust logic on the content process side to communicate with Rust logic on the GPU process side with minimal friction. It was made possible by the change to Rust structures to use Cow types aggressively, which are flexible and efficient, even though we don’t use the “write” part of the copy-on-write semantics.

API Coverage * The W3C group has agreed on the CPU data transfers API of writeBuffer/writeTexture, as well as the new asynchronous buffer mapping semantics with mappedAtCreation flag. We implemented these in Gecko, using a bit of shared memory. * The group introduced a new simplified way of creating pipelines, using implicit bind group layouts. We also implemented this in Gecko, while keeping some of the concerns on the table. * There were major rewrites of the render pipeline API and bind group layouts, both of which landed in Gecko before they became available in other browsers. * Some of the pieces of the API are still not implemented, such as queries and render bundles.

Validation The API on the Web is required to be safe and portable, which is enforced by the validation logic. We’ve made a lot of progress in this area: wgpu now has a first-class concept of “error” objects, which is what normal objects become if their creation fails on the server side (the GPU process). We allow these error objects to be used by the content side, and at the same time it returns the errors to the GPU process C++ code, which routes them back to the content side. There, we are now properly triggering the “uncaptured error” events with actual error messages:

In a draw command, indexed:false indirect:false, caused by: vertex buffer 0 must be set

GPUValidationError

What this means for us, as well as the brave experimental users, is better robustness and safety, less annoying panics/crashes, and less time wasted on investigating issues. The validation logic is not yet comprehensive, there is a lot yet to be done, but the basic infrastructure is mostly in place. We validate the creation of buffers, textures, bind group layouts, pipelines, and we validate the encoded commands, including the compute and render pass operations. We also validate the shader interface, and we validate the basic properties of the shader (e.g. the types making sense). We even implement the logic to check the uniformity requirements of the control flow, ahead of the specification, although it’s new and fragile at the moment.

Shading Language WebGPU Shading Language, or WGSL for short, is a new secure shading language for the Web, targeting SPIR-V, HLSL, and MSL on the native platforms. It’s exceptionally hard to support right now because of how young it is. The screenshot above was rendered with WGSL shaders in Firefox Nightly, you can get a feel of it by looking at the code.

Our recent update carried basic support for WGSL, using Naga library. The main code path in Gecko right now involves the following stages:

When a shader module is created:
1. Parsing WGSL with Naga and building an intermediate representation (IR).
2. Validating and analyzing the IR (with Naga) for the usage of global handles (such as texture-sample pairs) and the uniformity of control flow.
3. Producing a SPIR-V module by Naga.
When a pipeline is created:
1. Naga reflection information of the entry points is validated against the pipeline and each other input/output requirements.
2. The SPIR-V module is then passed down to gfx-hal, where the backends invoke SPIRV-Cross to generate the platform-specific shaders.

Next Steps One of the areas of improvement here is related to SPIR-V. In the future, we don’t want to unconditionally route the shader translation through SPIR-V, and we don’t want to rely on SPIRV-Cross, which is currently a giant C++ dependency that is hard to secure. Instead, we want to generate the platform-specific shaders straight from Naga IR ourselves. This will drastically reduce the amount of code involved, cut down the dependencies, and make the shader generation faster and more robust, but it requires more work.

Another missing bit is shader sanitation. In order to allow shaders to execute safely on GPU, it’s not enough to enable the safety features of the underlying APIs. We also need to insert bound checks in the shader code, where we aren’t sure about resource bounds being respected. These changes will be very sensitive to performance of some of the heaviest GPU users, such as TFjs.

Most importantly, we need to start testing Gecko’s implementation on the conformance test suite (CTS) that is developed by WebGPU group. This would uncover most of the missing bits in the implementation, and will make it easier to avoid regressions in the near future. Hopefully, the API has stabilized enough today that we can all use the same tests.

Contributing There is a large community around the Rust projects involved in our implementation. We welcome anyone to join the fun, and are willing to mentor them. Please hop into a relevant Matrix room to chat:

Gecko side (C++, JS) – #gfx-firefox:mozilla.org
Core implementation (Rust) – #wgpu:matrix.org
Shader translation (Rust) – #naga:matrix.org
API Specification – #WebGPU:matrix.org

View Details

WebGPU progress update in Gecko: API tracing, Rust serialization, API coverage, validation, and the Shading language.

View Details

This is going to be a rather technical dive into a recent improvement that went into WebRender.

Texture atlas allocation In order to submit work to the GPU efficiently, WebRender groups as many drawing primitives as it can into what we call batches. A batch is submitted to the GPU as a single drawing command and has a few constraints. for example a batch can only reference a fixed set of resources (such as GPU buffers and textures). So in order to group as many drawing primitives as possible in a single batch we need to place as many drawing parameters as possible in few resources. When rendering text, WebRender pre-renders the glyphs before compositing them on the screen so this means packing as many pre-rendered glyphs as possible into a single texture, and the same applies for rendering images and various other things.

For a moment let’s simplify the case of images and text and assume that it is the same problem: input images (rectangles) of various rectangular sizes that we need to pack into a larger textures. This is the job of the texture atlas allocator. Another common name for this is rectangle bin packing.

Many in game and web development are used to packing many images into fewer assets. In most cases this can be achieved at build time Which means that the texture atlas allocator isn’t constrained by allocation performance and only needs to find a good layout for a fixed set of rectangles without supporting dynamic allocation/deallocation within the atlas at run time. I call this “static” atlas allocation as opposed to “dynamic” atlas allocation.

There’s a lot more literature out there about static than dynamic atlas allocation. I recommend reading A thousand ways to pack the bin which is a very good survey of various static packing algorithms. Dynamic atlas allocation is unfortunately more difficult to implement while keeping good run-time performance. WebRender needs to maintain texture atlases into which items are added and removed over time. In other words we don’t have a way around needing dynamic atlas allocation.

A while back A while back, WebRender used a simple implementation of the guillotine algorithm (explained in A thousand ways to pack the bin). This algorithm strikes a good compromise between packing quality and implementation complexity.
The main idea behind it can be explained simply: “Maintain a list of free rectangles, find one that can hold your allocation, split the requested allocation size out of it, creating up to two additional rectangles that are added back to the free list.”. There is subtlety in which free rectangle to choose and how to split it, but the overall, the algorithm is built upon reassuringly understandable concepts.

Deallocation could simply consist of adding the deallocated rectangle back to the free list, but without some way to merge back neighbor free rectangles, the atlas would quickly get into a fragmented stated with a lot of small free rectangles and can’t allocate larger ones anymore.

Lots of free space, but too fragmented to host large-ish allocations. To address that, WebRender’s implementation would regularly do a O(n²) complexity search to find and merge neighbor free rectangles, which was very slow when dealing with thousands of items. Eventually we stopped using the guillotine allocator in systems that needed support for deallocation, replacing it with a very simple slab allocator which I’ll get back to further down this post.

Moving to a worse allocator because of the run-time defragmentation issue was rubbing me the wrong way, so as a side project I wrote a guillotine allocator that tracks rectangle splits in a tree in order to find and merge neighbor free rectangle in constant instead of quadratic time. I published it in the guillotiere crate. I wrote about how it works in details in the documentation so I won’t go over it here. I’m quite happy about how it turned out, although I haven’t pushed to use it in WebRender, mostly because I wanted to first see evidence that this type of change was needed and I already had evidence for many other things that needed to be worked on.

The slab allocator What replaced WebRender’s guillotine allocator in the texture cache was a very simple one based on fixed power-of-two square slabs, with a few special-cased rectangular slab sizes for tall and narrow items to avoid wasting too much space. The texture is split into 512 by 512 regions, each region is split into a grid of slabs with a fixed slab size per region.

The slab allocator in action. This is a debugging view generated from a real browsing session. This is a very simple scheme with very fast allocation and deallocation, however it tends to waste a lot of texture memory. For example allocating an 8×10 pixels glyph occupies a 16×16 slot, wasting more than twice the requested space. Ouch!
In addition, since regions can allocate a single slab size, space can be wasted by having a region with few allocations because the slab size happens to be uncommon.

Improvements to the slab allocator Images and glyphs used to be cached in the same textures. However we render images and glyphs with different shaders, so currently they can never be used in the same rendering batches. I changed image and glyphs to be cached into a separate set of textures which provided with a few opportunities.

Not mixing images and glyphs means the glyph textures get more room for glyphs which reduces the number of textures containing glyphs overall. In other words, less chances to break batches. The same naturally applies to images. This is of course at the expense of allocating more textures on average, but it is a good trade-off for us and we are about to compensate the memory increase by using tighter packing.

In addition, glyphs and images are different types of workloads: we usually have a few hundred images of all sizes in the cache, while we have thousands of glyphs most of which have similar small sizes. Separating them allows us to introduce some simple workload-specific optimizations.

The first optimization came from noticing that glyphs are almost never larger than 128px. Having more and smaller regions, reduces the amount of atlas space that is wasted by partially empty regions, and allows us to hold more slab sizes at a given time so I reduced the region size from 512×512 to 128×128 in the glyph atlases. In the unlikely event that a glyph is larger than 128×128, it will go into the image atlas.

Next, I recorded the allocations and deallocations browsing different pages, gathered some statistics about most common glyph sizes and noticed that on a low-dpi screen, a quarter of the glyphs would land in a 16×16 slab but would have fit in a 8×16 slab. In latin scripts at least, glyphs are usually taller than wide. Adding 8×16 and 16×32 slab sizes that take advantage of this helps a lot.
I could have further optimized specific slab sizes by looking at the data I had collected, but the more slab sizes I would add, the higher the risk of regressing different workloads. This problem is called over-fitting. I don’t know enough about the many non-latin scripts used around the world to trust that my testing workloads were representative enough, so I decided that I should stick to safe bets (such as “glyphs are usually small”) and avoid piling up optimizations that might penalize some languages. Adding two slab sizes was fine (and worth it!) but I wouldn’t add ten more of them.

The original slab allocator needed two textures to store a workload that the improved allocator can fit into a single one. At this point, I had nice improvements to glyph allocation using the slab allocator, but I had a clear picture of the ceiling I would hit from the fixed slab allocation approach.

Shelf packing allocators I already had guillotiere in my toolbelt, in addition to which I experimented with two algorithms derived from the shelf packing allocation strategy, both of them released in the Rust crate etagere. The general idea behind shelf packing is to separate the 2-dimensional allocation problem into a 1D vertical allocator for the shelves and within each shelf, 1D horizontal allocation for the items.

The atlas is initialized with no shelf. When allocating an item, we first find the shelf that is the best fit for the item vertically, if there is none or the best fit wastes too much vertical space, we add a shelf. Once we have found or added a suitable shelf, an horizontal slice of it is used to host the allocation.

At a glance we can see that this scheme is likely to provide much better packing than the slab allocator. For one, items are tightly packed horizontally within the shelves. That alone saves a lot of space compared to the power-of-two slab widths. A bit of waste happens vertically, between an item and the top of its shelf. How much the shelf allocator wastes vertically depends on how the shelve heights are chosen. Since we aren’t constrained to power-of-two size, we can also do much better than the slab allocator vertically.

The bucketed shelf allocator The first shelf allocator I implemented was inspired from Mapbox’s shelf-pack allocator written in JavaScript. It has an interesting bucketing strategy: items are accumulated into fixed size “buckets” that behave like a small bump allocators. Shelves are divided into a certain number of buckets and buckets are only freed when all elements are freed. The trade-off here is to keep atlas space occupied for longer in order to reduce the CPU cost of allocating and deallocating. Only the top-most shelf is removed when empty so consecutive empty shelves in the middle aren’t merged until they become the top-most shelves, which can cause a bit of vertical fragmentation for long running sessions. When the atlas is full of (potentially empty) shelves the chance that a new item is too tall to fit into one of the existing shelves depends on how common the item height is. Glyphs tend to be of similar (small) heights so it works out well enough.

I added very limited support for merging neighbor empty shelves. When an allocation fails, the atlas iterates over the shelves and checks if there is a sequence of empty shelves that in total would be able to fit the requested allocation. If so, the first shelf of the sequence becomes the size of the sum, and the other shelves are squashed to zero height. It sounds like a band aid (it is) but the code is simple and it is working within the constraints that make the rest of the allocator very simple and fast. It’s only a limited form of support for merging empty shelves but it was an improvement for workloads that contain both small and large items.

Image generated from the glyph cache in a real borwsing session via a debugging tool. We see fewer/wider boxes rather than many thin boxes because the allocator internally doesn’t keep track of each item rectangle individually, so we only see buckets filling up instead. This allocator worked quite well for the glyph texture (unsurprisingly, as Mapbox’s implementation it was inspired from is used with their glyph cache). The bucketing strategy was problematic, however, with large images. The relative cost of keeping allocated space longer was higher with larger items. Especially with long running sessions, this allocator was good candidate for the glyph cache but not for the image cache.

The simple shelf allocator The guillotine allocator was working rather well with images. I was close to just using it for the image cache and move on. However, having spent a lot of time looking at various allocations patterns, my intuition was that we could do better. This is largely thanks to being able to visualize the algorithms via our integrated debugging tool that could generate nice SVG visualizations.

It motivated experimenting with a second shelf allocator. This one is conceptually even simpler: A basic vertical 1D allocator for shelves with a basic horizontal 1D allocator per shelf. Since all items are managed individually, they are deallocated eagerly which is the main advantage over the bucketed implementation. It is also why it is slower than the bucketed allocator, especially when the number of items is high. This allocator also has full support for merging/splitting empty shelves wherever they are in the atlas.

This was generated from the same glyph cache wokrload as the previous image. Unlike the Bucketed allocator, this one worked very well for the image workloads. For short sessions (visiting only a handful of web pages) it was not packing as tightly as the guillotine allocator, but after browsing for longer period of time, it had a tendency to better deal with fragmentation.

The simple shelf allocator used on the image cache. Notice how different the image workloads look (using the same texture size), with much fewer items and a mix of large and small items sizes. The implementation is very simple, scanning shelves linearly and then within the selected shelf another linear scan to find a spot for the allocation. I expected performance to scale somewhat poorly with high number of glyphs (we are dealing in the thousands of glyphs which arguably isn’t that high), but the performance hit wasn’t as bad I had anticipated, probably helped by mostly cache friendly underlying data-structure.

A few other experiments For both allocators I implemented the ability to split the atlas into a fixed number of columns. Adding columns means more (smaller) shelves in the atlas, which further reduces vertical fragmentation issues, at the cost of wasting some space at the end of the shelves. Good results were obtained on 2048×2048 atlases with two columns. You can see in the previous two images that the shelf allocator was configured to use two columns.

The shelf allocators support arranging items in vertical shelves instead of horizontal ones. It can have an impact depending on the type of workload, for example if there is more variation in width than height for the requested allocations. As far as my testing went, it did not make a significant difference with workloads recorded in Firefox so I kept the default horizontal shelves.

The allocators also support enforcing specific alignments in x and y (effectively, rounding up the size of allocated items to a multiple of the x and y alignment). This introduces a bit of wasted space but avoids leaving tiny holes in some cases. Some platforms also require a certain alignment for various texture transfer operations so it is useful to have this knob to tweak at our disposal. In the Firefox integration, we use different alignments for each type of atlas, favoring small alignments for atlases that mostly contain small items to keep the relative wasted space small.

Conclusion Various visualizations generated while I was working on this. It’s been really fun to be able “look” at the algorithms at each step of the process. The guillotine allocator is the best at keeping track of all available space and can provide the best packing of the algorithms I tried. The shelf allocators waste a bit of space by simplifying the arrangement into shelves, and the slab allocator wastes a lot of space for the sake of simplicity. On the other hand the guillotine allocator is the slowest when dealing with multiple thousands of items and can suffer from fragmentations in some of the workloads I recorded. Overall the best compromise was the simple shelf allocator which I ended up integrating in Firefox for both glyph and image textures in the cache (in both cases configured to have two columns per texture). The bucketed allocator is still a very reasonable option for glyphs and we could switch to it in the future if we feel we should trade some packing efficiency for allocation/deallocation performance. In other parts of WebRender, for short lived atlases (a single frame), the guillotine allocation algorithm is used.

These observations are mostly workload-dependent, though. Workloads are rarely completely random so results may vary.

There are other algorithms I could have explored (and maybe will someday, who knows), but I had found a satisfying compromise between simplicity, packing efficiency, and performance. I wasn’t aiming for state of the art packing efficiency. Simplicity was a very important parameter and whatever solutions I came up with would have to be simple enough to ship it in a web browser without risks.

To recap, my goals were to:

allow packing more texture cache items into fewer textures,
reduce the amount of texture allocation/deallocation churn,
avoid increasing GPU memory usage, and if possible reduce it.

This was achieved by improving atlas packing to the point that we more rarely have to allocate multiple textures for each item type . The results look pretty good so far. Before the changes in Firefox, glyphs would often be spread over a number of textures after having visited a couple of websites, Currently the cache eviction is set so that we rarely need more than than one or two textures with the new allocator and I am planning to crank it up so we only use a single texture. For images, the shelf allocator is pretty big win as well. what used to fit into five textures now fits into two or three. Today this translates into fewer draw calls and less CPU-to-GPU transfers which has a noticeable impact on performance on low end Intel GPUs, in addition to reducing GPU memory usage.

The slab allocator improvements landed in bug 1674443 and shipped in Firefox 85, while the shelf allocator integration work went in bug 1679751 and will make it hit the release channel in Firefox 86. The interesting parts of this work are packaged in a couple of rust crates under permissive MIT OR Apache-2.0 license:

guillotiere (crate) (doc)
etagere (crate) (doc)

View Details

This is going to be a rather technical dive into a recent improvement that went into WebRender. Texture atlas allocation In order to submit work to the GPU efficiently, WebRender groups as many drawing primitives as it can into what we call batches. A batch is submitted to the GPU as a single drawing command … Continue reading Improving texture atlas allocation in WebRender →

View Details

Hey all, Jim Mathies here, the new Mozilla Graphics Team manager. We haven’t had a Graphics Newsletter since July, so there’s lots to catch up on. TL/DR – We’re shipping our Rust based WebRender backend to a very wide audience as of Firefox 84. Read on for more detail on our progress. WebRender Current Status … Continue reading moz://gfx newsletter #54 →

View Details

WebRender Current Status

The release audience for WebRender has expanded quite a bit over the last six months. As a result we expect to achieve nearly 80% desktop coverage by the end of the year and have a goal of shipping to 100% of our user base by next summer.

Operating System Support

MacOS – As of Firefox 84, we are shipping to all versions of MacOS including the latest Big Sur release.

Windows 10 – We are currently shipping to all versions of Windows 10.

Windows 8/8.1 – We are currently shipping to all versions of Windows 8.

Windows 7 – We are currently shipping to a subset of Windows 7 users. Users currently excluded are running versions of the operating system which have not received the first major platform update. Prior to this update Windows 7 lacked features WebRender relies on for painting to a window managed by a separate process. We are working on adding a fallback mechanism that moves composition into the parent browser process to work around this. We hope to ship support for this fallback mechanism in Firefox 85.

Android – We are currently shipping to devices that leverage the Mali-G chipset, Pixel devices, and a majority of Adreno 5 and 6 devices. Mali-T GPUs are our next big release target. Once we get Mali-T support out the door, we’ll have achieved 70% coverage for our mobile user base.

Linux – We have a little announcement to make here, in Firefox 84 will ship with an accelerated WebRender backend for the first time ever to a subset of Linux users. The target cohort leverages X11, Gnome, and recent Mesa library versions. We plan to expand this rollout to more desktop configurations over time, stay tuned!

Qualified Hardware

A note about qualified hardware – we have the ability to restrict who received the new pipeline based on a combination of hardware parameters – GPU manufacturer, generation, driver versions, battery power, video refresh rate, dual monitor configurations, and even screen size. We leveraged these filters heavily during our initial rollout to target specific cohorts. Thankfully most of these filters are no longer in use as our target audience has expanded greatly over the last six months.

As of Firefox 83, we are shipping to a majority of nVidia and AMD GPUs, and to all Intel GPUs newer than Generation 6. We actually shipped to Generation 6 GPUs in 83 but had to back off when some users reported rendering glitches on Reddit. The fix for this issue landed in 85 and has since been uplifted to 84 for rollout to Release, bringing WebRender support to the vast majority of modern Intel chipsets in Firefox 84.

The remaining GPUs we plan to target include older Intel Generation 4.5 and 5, a batch of mobile specific ‘LP’ Intel chipsets, and various older AMD/ATI/nVidia chipsets that represent the long tail of compatible chipsets from these manufacturers.

If you’re curious to see if your device is qualified and running with the new pipeline, visit about:support in a tab and view the Graphics section for this information.

The Long Tail

WebRender is an accelerated rendering backend. This means we leverage the power of your graphics hardware to speed getting pixels to the screen. Unfortunately there are some hardware configurations which will never be able to support this type of rendering pipeline. That’s a problem in that without 100% WebRender coverage for the Firefox user base, we’ll never be able to remove the old pipelining code these users leverage today. Our solution here involves a new fallback mechanism that performs rendering in software. Since WebRender currently supports an OpenGL based hardware backend, software fallback is essentially a software implementation of certain OpenGL ES3 features tailored for WebRender support. We’ve recently started testing software fallback in Nightly and are seeing better than expected performance. We’re not ready to ship this implementation yet but we’re getting closer. Once ready, software fallback will provide WebRender support to the ‘long tail’ of lower-end hardware, uncommon configurations, and users with specific issues like bad drivers.

Shipping software fallback will allow us to close the loop on 100% WebRender coverage, at which point the Graphics Team can move forward on new and interesting projects we’ve been itching to get too for a while now.

WebGPU

Independent of WebRender rollout we are continuing our work on Firefox’s WebGPU implementation currently available for testing in Nightly builds. The specification is on track to reach MVP status in the near future, after which it will go through a period of feedback and change on the road to the release of the final specification sometime in 2021.

Looking Beyond WebRender

WebRender development and shipping has taken a few years to accomplish. We’re finally at a point where the team is starting to think about what we’ll work on once we’ve shipped WebRender to our entire user population. There’s lots to do! An overall theme is currently emerging in our planning – Visual Quality and Performance! We’re investigating various opportunities to extend the WebRender pipeline deeper into Gecko’s layout engine, HDR features, improved color management, performance improvements for SVG and Canvas, and improvements in power consumption. We’ll post more about these projects in future posts, stay tuned!

Happy New Year from the Mozilla Graphics Team!

View Details

Bonjour à tous et à toutes, this is episode 53 of your favorite and only Firefox graphics newsletter. From now on instead of peeling through commit logs, I will be simply gathering notes sent to me by the rest of the team. This means the newsletter will be shorter, hopefully a bit less overwhelming with only the juicier bits. It will also give yours-truly more time to fix bugs instead of writing about it.

Lately we have been enabling WebRender for a lot more users. For the first time, WebRender is enabled by default in Nightly for Windows 7 and macOS users with modern GPUs. Today 78% of Nightly users have WebRender enabled, 40% on beta, and 22% on release. Not all of these configurations are ready to ride the trains yet, but the numbers are going to keep going up over the next few releases.

WebRender WebRender is a GPU based 2D rendering engine for the web written in Rust, currently powering Firefox‘s rendering engine as well as Mozilla’s research web browser Servo.

Ongoing work * Part of the team is now focusing on shipping WebRender on some flavors of Linux as well. * Worth highlighting also is the ongoing work by Martin Stránský and Robert Mader to switch Firefox on Linux from GLX to EGL. EGL is a more modern and better supported API, it will also let us share more code between Linux and Android. * Lee and Jim continue work on WebRender’s software backend. It has had a bunch of correctness improvements, works properly on Windows now and has more performance improvements in the pipeline. It works on all desktop platforms and can be enabled via the pref “gfx.webrender.software”.

Performance One of the projects that we worked on the last little while has been improving performance on lower-end/older Intel GPUs.

Glenn fixed a picture caching issue while scrolling gmail
Glenn fixed some over-invalidation on small screen resolutions.
Glenn reduced extra invalidation some more.
Dzmitry switched WebRender to a different CPU-to-GPU transfer strategy on Intel hardware on Windows. This avoid stalls during rendering.

Some other performance improvements that we made are:

Nical reduced CPU usage by re-building the scene a lot less often during scrolling.
Nical removed a lot of costly vector reallocation during scene building.
Nical reduced the amount of synchronous queries submitted to the X server on Linux, removing a lot of stalls when the GPU busy.
Nical landed a series of frame building optimizations.
Glenn improved texture cache eviction handling. This means lower memory usage and better performance.
Jeff enabled GPU switching for WebRender on Mac in Nightly. Previously WebRender only used the GPU that Firefox was started with. If the GPU was switched Firefox would have very bad performance because we would be drawing with the wrong GPU.
Markus finished and preffed on the OS compositor configuration of WR on macOS, which uses CoreAnimation for efficient scrolling.

Driver bugs * Dzmitry worked around a driver bug causing visual artifacts in Firefox’s toolbar on Intel Skylake and re-enabled direct composition on these configurations.

Desktop zooming * Botond announced on dev-platform that desktop zooming is ready for dogfooding by Nightly users who would like to try it out by flipping the pref. * Botond landed a series of patches that re-works how main-thread hit testing accounts for differences between the visual and layout viewports. This fixes a number of scenarios involving the experimental desktop zooming feature (enabled using apz.allow_zooming=true), including allowing scrollbars to be dragged with desktop zooming enabled. * Timothy landed support for DirectManipulation preffed off. It allows users to pinch-zoom on touchpads on Windows. It can be enabled by setting apz.windows.use_direct_manipulation=true

View Details

Hello everyone! I know you have been missing your favorite and only newsletter about software engineers staying at home, washing their hands often and fixing strange rendering glitches in Firefox’s graphics engine. In the last two months there has been a heap of fixes and improvements. Before the usual change list I’ll go through a few highlights:

A number of us focused on shipping WebRender to a wider range of Intel devices on Windows. This involved quite a bit of testing, profiling, puzzlement about driver behavior and of course heaps of improvements, a lot of which will benefit other configurations as well. See the “WebRender where” page for an overview of WebRender’s deployment.
DirectComposition integration with picture caching was a very important part of making this possible, yielding big performance and power usage improvements. It is now enabled by default on windows as of Firefox 75 in some configurations where WebRender is enabled and will expand to more configurations in 76 and 77.
In 76 we have improved DirectComposition usage for video playback. This reduces GPU usage during video play a lot. On an Intel HD 530 1080p60 video playback has 65% usage with WebRender on in 75, 40% with Webrender off, and 32% usage with WebRender on in 76.
The new vsync implementation on Windows, which improved our results on various benchmarks such as vsynctester.com and motionmark, reduced stuttering during video playback and scrolling.
Steady progress on WebRender’s software implementation (introduced at the top of the previous episode), codenamed SWGL (pronounced “swigle”), which will in the long run let us move even the most exotic hardware configurations to WebRender.
And hardware accelerated GL contexts on Wayland, and pre-optimized shaders, and WebGPU, and… So many other improvements everywhere it is actually quite hard to highlight only a few.

What’s new in gfx * Botond and Micah Tigley added initial support for double-click-to-zoom gestures in Responsive Design Mode * Botond and Agi Sferro tracked down and fixed a regression that was causing about:support to load zoomed in on Android. * Sotaro re-enabled video frame recycling with the RDD process. * Sotaro fixed an issue canvases to not be re-created after GPU context loss. * Lee worked around a memory leak. * Markus fixed a rendering glitch. * Jonathan Kew avoided defaulting to a monospace font as fallback on MacOS. * Nical attempted yet another fix at a crash that keeps coming back. * Jonathan Kew implemented distinguishing between OS-provided and user-installed fonts in the system font list. * Botond fixed non-unified build errors. * Kats improved the behavior of momentum scrolling. * Jonathan Kew fixed a crash. * Kats prevented the viewport clip from clipping position:sticky items. * Kris added desktop-zooming information to about:support. * Snorp made it possible to disambiguate top-level from other APZ events. * Roger Zanoni addressed some static analysis lints. * Chris Martin removed some usage of gdi surfaces on windows for sandboxing. * Andrew enabled color management for all images and not only tagged ones. * Jushua Gahan addressed some static analysis lints. * Timothy fixed an invalidation issue with display:none masks. * Sotaro fixed a canvas rendering issue when resuming on Android. * Jonathan Kew fixed a font metrics related issue. * Bob Owen fixed an intermittent issue with canvas remoting. * Kats fixed some issues with the dynamic toolbar sticky behavior on Android. * Miko fixed an issue with opacity optimization. * Bert improved the vsync implementation on Windows. * Arash Fotouhi addressed some static analysis lints. * Robert Mader implemented creating an OpenGL context with the Wayland backend on Linux. * Lee fixed a shutdown crash related to font loading. * Jonathan Kew fixed a printing issue with long SVG stroke-dasharray strings. * Sam Dalton improved the FPS counter implementation. * Kats removed the old android dynamic toolbar implementation. * Sotaro improved some OpenGL context debugging utilities. * Jonathan Kew fixed a font visibility issue with language-pack fonts on Windows.

What’s new in WebRender WebRender is a GPU based 2D rendering engine for the web written in Rust, currently powering Firefox‘s rendering engine as well as Mozilla’s research web browser Servo.

Jamie fixed a regression preventing picture caching from working well on Android.
Jamie implemented GLSL shader optimization at build time, resulting in quicker startup times.
Glenn added support for the tracy profiler in WebRender.
Miko added more items to display item caching.
Nical improved texture cache eviction and allocation heuristics to reduce the amount of texture reallocation.
Lee implemented a number of shader features in SWGL.
Nical prevented the texture cache from deallocating textures if it is about to allocate new ones.
Nical reduced the amount of alpha texture reallocations in the texture cache.
Connor Brewster fixed the max blur radii to take scale factor into account.
Jeff fixed a clipping issue with blob images.
Nical fixed clipping issues with shadows.
Nical fixed invalid glyph rasterization being requested continuously.
Glenn reduced picture cache invalidation with very large rectangles.
Miko added memory reporting to the display item cache.
Nical made WebRender’s builtin profiling tool more useful.
Timothy improved the stacking context scale factor selection to avoid blurriness under certain conditions.
Dzmitry fixed a crash caused by picture tasks exceeding the maximum supported texture size.
Glenn fixed picture cache tiles being evicted too eagerly.
Lee implemented perspective-correct rasterization in SWGL.
Glenn fixed a picture caching invalidation bug.
Kats and Kris implemented WebRender’s interaction between position:sticky and the dynamic toolbar.
Bert added a fast path for more gradient types.
Dzmitry fixed missing data in WebRender captures.
Kats removed the initial document splitting code.
Sotaro made it possible to enable WebRender without GPU process on Windows.
Timothy fixed an issue with incomplete SVG masks.
Glenn reduced the amount of texture reallocation in the render task target pool.
Nical reduced technical debt in WebRender’s message channels.
Jamie ensured upload strides are a multiple of 64 pixels on Adreno to avoid driver slow path.
Jeff disabled DirectComposition when the resolution is scaled and the compositor does not support stretching, to avoid rendering artifacts.
Nical reduced technical debt in the blob image rendering code.
Dzmitry fixed a clipping bug.
Lee implemented linear filtering for SWGL glBlitFrameBuffer.
Andrew fixed a memory leak with off-screen GISs.
Sotaro fixed some skipped frames when recving window expose events.
Jim added documentation to WebRender’s headless testing scripts.
Jim fixed a lot of compiler warnings in SWGL.
Sotaro simplified the DirectComposition integration code.
Andrew Added support for capturing sequences of frames when debugging WebRender.
Nical removed the deprecated recording tool in WebRender.
Kats fixed some github CI issues.
Glenn fixed a rendering issue involving a combination of blur, drop shadows and video.
Glenn simplified the image masking API.
Lee fixed a pixel rounding issuel with SWGL.
Kats fixed an APZ issue causing some frames to be skipped.
Jamie fixed a bug causing black screens on Android.
Nical refactored a lot of message passing code in WebRender in preparation for upcoming frame scheduling improvements.

To enable WebRender in Firefox, in the about:config page, enable the pref gfx.webrender.all and restart the browser.

WebRender is available under the MPLv2 license as a standalone crate on crates.io (documentation) for use in your own rust projects.

What’s new in WebGPU WebGPU is a new Web API to access graphics and compute capabilities of the hardware. Firefox and Servo have implementations in progress that are based on wgpu project written in Rust.

Dzmitry implemented render passes, samplers, textures, texture views, and other missing pieces of the API.
Dzmitry implemented support for presentation based on the CPU readback path.
Firefox Nightly is now able to run both Chrome’s WebGPU examples as well as the ones we cross-compiled from Rust to WASM in wgpu-rs.
Dzmitry published the article to Mozilla Hacks blog featuring the status of the API as well as our emerging implementation in Firefox (Nightly only).

To enable WebGPU, follow the steps in webgpu.io, which also shows the current implementation status in all browsers.

View Details

Bonjour, bonjour! Another long overdue episode of your favourite Mozilla gfx team newsletter is here. A few weeks ago, Jessie published a call to help us find steps to reproduce a mysterious glitch. Thanks a ton to everyone who helped out with this one! Glenn landed a fix to an issue that we suspect might be the cause of the issues . Don’t hesitate to let us know if you are still running into this particular glitch with Firefox Nightly and WebRender enabled.

Other than that there are a number of pretty exciting things going on in WebRender. One of them is Lee and Jeff’s work on a software backend for WebRender. In order to eventually move all Firefox users to WebRender, we need a backend that can accomodate for very old GPUs and very buggy drivers. Ideally this backend would use most of WebRender’s current code and infrastructure to avoid having too much new code (and bugs!) to maintain with our limited resources. One of the avenues that was investigated is using a software emulation layer for OpenGL such as Swift Shader or llvm-pipe as a black box to run our GPU code on the CPU. We unfortunately couldn’t get good enough performance this way so Lee is now experimenting with automatically translating our shaders into SIMD-optimized CPU code while compiling Firefox instead, with SIMD optimizations. These “software shaders” are then run into a simple custom rasterizer that only supports the few OpenGL features that we need and take advantage of the restricted featureset to run as fast as possible. This is still very much experimental but initial results are promising.

A lot of progress was also made towards DirectCompositor integration which I have mentioned a few times in this blog. Glenn, Sotaro and Jeff are ironing out the last few bugs (famous last words!) before the feature can ride the trains to release. A lot of work went into making video playback very efficient in this new compositing mode.

What’s new in gfx * Botond and Ting-Yu collaborated to fix an Android regression where pinch-zooming while text is selected could cause the view to jump to the top-left corner of the page. * Kats fixed a task exhaustion issue with the hit-testing dumping code. * Botond and Decoder fixed a race condition; * Botond fixed some race conditions in APZ caught by thread sanitizer. * Kats reenabled test_group_pointerevents.html for GeckoView. * Martin Stránský avoided passing invalid file descriptors through IPC with dmabuf textures on Wayland. * Bob Owen fixed a crash with canvas remoting. * Kris fixed a crash. * Martin Stránský made the GL compositor backend work with NV12 video textures produced by ffmpeg on Wayland. * Martin Stránský added support for wayland dmabuf textures with WebGL. * Martin Stránský implemented fence synchronization to dmabuf surfaces for WebGL on Wayland. * Sotaro prevented the creation of unnecessary compositor windows. * Sotaro improved the shutdown logic of the VR code. * Markus fixed an invalidation bug with CoreAnimation. * Botond fixed an issue with window.scrollY after maximizing the window. * Martin Stránský implemented dmabuf modifiers. * Tim Nguyen implemented conic gradients with the skia backend. * Nical fixed a crash in the D2D backend. * Jeff Gilbert fixed WebGLSL comment parsing. * Jeff Gilbert optimized GetDrawFetchLimits and vertexAttribPointer in WebGL. * Imanol Fernandez fixed a WebGL crash affecting Firefox reality. * Jeff Gilbert fixed failures in the CTS/conformance2/extensions test suite. * Boris Zbarsky made canvas error handling better match the specification when creating a pattern from an invalid image. * Jeff Gilbert fixed an issue with canvas drawImage and non-premultiplied source images. * Jeff Gilbert improved the performance of the VAO cache under in some cases.

What’s new in WebRender WebRender is a GPU based 2D rendering engine for the web written in Rust, currently powering Firefox‘s rendering engine as well as Mozilla’s research web browser Servo.

Jamie fixed several bugs in WebRender’s support for scrolling and pinch zooming on Android.
Miko implemented display-list interning.
Glenn made a ton of progress towards efficient video playback with direct composition.
Connor fixed a bug with large blur radii when a scale factor is applied.
Hiro Bug 1510030 running background color animations as WebRender animated properties.
Jamie enabled caching compiled shaders even after statup.
Andrew reduced quantization effects with text.
Andrew prevented quantization effects mismatch between CPU and text shader.
Glenn avoided generating segment instance vectors when there is only a single segment. This was intended as a workaround for running into a limit of segmented primitives which we were hitting in a benchmark, but also gave a nice 10% improvement on the displaylist-mutate benchmark.
Glenn fixed a race condition with memory pressure events causing invalid caches.
Miko made the serialized display list format easier to debug.
Nical made a lot of progress towards unifying the text and brush shading infrastructure.
Nical added a pref to tune batching behavior.
Sotaro fixed an invalidation bug witrh partial present.
Jeff improved render target creation with ANGLE.
Kris fixed a dashed/dotted border issue with Adreno GPUs on Android.
Glenn fixed a crash involving plane-splitting and invalid transform.
Glenn fixed a crash with picture cache dependency building.
Tim Nguyen implemented conic gradients in WebRender.
Glenn added a per-tile valid rect in the picture cache to reduce overdraw in some cases.
Bert improved support for picture cache invalidation debugging and testing.
Jdm removed now unused ipc-channel from WebRender.
Bert [usedd] primitive cluster flags to select picture cache tile size for scroll bars.
Lee fixed an issue causing opaque images to miss the fast image shader.
Nical added a pref to enable/disable rayon in WebRender.
Andrew fixed fixed a stuttering issue involving animated elements with filters.
Jim improved the line decoration shader.
Glenn fixed the invalidation of elements with an inflation factor.
Sotaro made profiler screenshots and frame recording work with DirectComposition integration.
Nical fixed incorrect border-image repetitions.
Nical implemented border-image-repeat: round in WebRender.
Nical reduced the amount of allocations in the render task graph.
Kvark fixed an issue with the solid border shader.
Connor moved the opacity filter to its own shader.
Nical removed accidental string manipulation in performance sensitive code.
Bert avoided using rayon for glyph rasterization unless the number of glyph is high enough to compensate for rayon’s overhead.
Simon Sapin avoided rounded corner paths in some cases where it isn’t needed.
Connor added support for alpha components on the SVG color matrix filter.
Jeff fixed a clipping issue when painting a xul tree.
Timothy fixed an issue with orphanned displaylist with fission and WebRender.
Jamie ensured the async scroll offset is in the correct coordinate space whilst async zooming.
Jamie fixed position:sticky during pinch-zoom.
Sotaro fixed a glitch with window recording on Windows.
Jeff allowed more blob images to be marked as opaque. This change allowed scrollbars to be marked as opaque on Windows which reduced the GPU utilization in the DWM with DirectComposite enabled from 21% to 17% at 1080p and 29% to 24% at 4k.

To enable WebRender in Firefox, in the about:config page, enable the pref gfx.webrender.all and restart the browser.

WebRender is available under the MPLv2 license as a standalone crate on crates.io (documentation) for use in your own rust projects.

What’s new in WebGPU Kvark implemented resource binding, compute pass recording, and reworked the asynchronous buffer mapping. All of that has landed now, and Firefox Nightly is able to run the standard compute example :tada: (when the pref is enabled)

Kvark spoke at Fosdem 2020 about our Rust-based WebGPU infrastructure. This presentation was derived from an internal talk about WebGPU that happened at Berlin All Hands, slides from which are also available. It had more focus on the Web API and Firefox architecture.

View Details

Bonjour, bonjour! Another long overdue episode of your favourite Mozilla gfx team newsletter is here. A few weeks ago, Jessie published a call to help us find steps to reproduce a mysterious glitch. Thanks a ton to everyone who helped out with this one! Glenn landed a fix to an issue that we suspect might … Continue reading moz://gfx newsletter #51 →

View Details

For the past little while, we have been tracking some interesting WebRender bugs that people are reporting in release. Despite best efforts, we have been unable to determine clear steps to reproduce these issues and have been unable to find a fix for them. Today we are announcing a special challenge to the community – help us track down steps to reproduce (a.k.a STR) for this bug and you will win some special, limited edition Firefox Graphics team swag! Read on for more details if you are interested in participating.

What we know so far about the bug: Late last year we started seeing reports of random UI glitching bugs that people were seeing in release. You can check out some of the reports on Bugzilla. Here is what we know so far about this bug:

At seemingly random intervals, either two things seem to happen:
- The text in the UI and content area of the browser seem to glitch out
  - Scrolling through the page or mousing over the affected area seems to fix it
- Or black boxes appear on the screen and also seem to disappear after scrolling around

Glitches! Black boxes! * The majority of the reports we have seen so far have come from people using NVIDIA graphics cards, although we have seen reports come in of this happening on Intel and AMD as well. That could be though because the majority of the people we have officially shipped WR to in release are on NVIDIA cards. * There doesn’t seem to be one clear driver version correlated to this bug, so we are not sure if it is a driver bug. * All reporters so far have been using Windows 10 * No one who has reported the bug thus far has been able to determine clear and consistent STR, and no one on the Graphics team has found a way to reproduce it either. We all use WebRender daily and none of us have encountered the bug.

How can you help? Without having a way to reliably reproduce this bug, we are at a loss on how to solve it. So we decided to hold a challenge to engage the community further to help us understand this bug better. If you are interested in helping us get to the root of this tricky bug, please do the following:

Download Firefox Nightly (if you don’t already use it)
Ideally you are using Windows 10 (but if you see this bug on other platforms, we are interested in hearing about it!)
Ensure WebRender is enabled
- Go to about:config and set gfx.webrender.all to true, then restart your browser
If you encounter the bug, help us by filing a bug in Bugzilla with the following details:
- What website are you on when the bug happens?
- Does it seem to happen when specific actions are taken?
- How frequently does the bug happen and can you ‘make’ it happen?
- Attach the contents of your about:support as a text file
The main thing we really need are consistent steps that result in the bug showing up. We will send some limited edition Graphics swag to the first 3 bug reporters who can give us consistent STR!

Even if you can’t easily find STR, we are still interested in hearing about whether you see this bug!

Challenge guidelines The winners of this challenge will be chosen based on the following criteria:

The bug report contains clear and repeatable steps to make the bug happen
- This can include things like having a specific hardware configuration, using certain add ons and browsing certain sites – literally anything as long as it can reliably and consistently cause the bug to appear
- BONUS: A member of the Graphics team can follow your steps and can also make the bug appear
We will choose the first 3 reporters who can meet this criteria (we say 3 because it is possible there is more than one bug and more than one way to reproduce it)
Winners will receive special limited edition Graphics Team swag! (t-shirt and stickers)

Update: we have created the channel #gfx-wr-glitch:mozilla.org on Matrix so you can ask questions/chat with us there. For more info about how to joing Matrix, check out: https://wiki.mozilla.org/Matrix

View Details

For the past little while, we have been tracking some interesting WebRender bugs that people are reporting in release. Despite best efforts, we have been unable to determine clear steps to reproduce these issues and have been unable to find a fix for them. Today we are announcing a special challenge to the community – … Continue reading Challenge: Snitch on the glitch! Help the Graphics team track down an interesting WebRender bug… →

View Details

Hi there! Another gfx newsletter incoming.

Glenn and Sotaro’s work on integrating WebRender with DirectComposition on Windows is close to being ready. We hope to let it ride the trains for Firefox 75. This will lead to lower GPU usage and energy consumption. Once this is done we plan to follow up with enabling WebRender by default for Windows users with (some subset of) Intel integrated GPUs, which is both challenging (these integrated GPUs are usually slower than discrete GPUs and we have run into a number of driver bugs with them on Windows) and rewarding as it represents a very large part of the user base.

Edit: Thanks to Robert in the comments section of this post for mentioning the Linux/Wayland progress! I copy-pasted it here:

Some additional highlights for the Linux folks: Martin Stránský is making good progress on the Wayland front, especially concerning DMABUF. It will allow better performance for WebGL and hardware decoding for video (eventually). Quoting from https://bugzilla.mozilla.org/show_bug.cgi?id=1586696#c2:

there’s a WIP dmabuf backend patch for WebGL, I see 100% performance boost with it for simple WebGL samples at GL compositor (it’s even faster than chrome/chromium on my box).

And there is active work on partial damage to reduce power consumption: https://bugzilla.mozilla.org/show_bug.cgi?id=1484812

What’s new in gfx * Handyman fixed fixed a crash in the async plugin infrastructure. * Botond fixed (2) various data races in the APZ code. * Sean Feng fixed another race condition in APZ code. * Andrew fixed a crash with OMTP and image decoding. * Sotaro fixed a crash with the GL compositor on Wayland. * Botond worked with Facebook developers to resolve a scrolling-related usability problem affecting Firefox users on messenger.com, primarily on MacOS. * Botond fixed (2) divisions by zero various parts of the APZ. * Sean Feng added some telemetry for touch input latency. * Timothy made sure all uses of APZCTreeManager::mGeckoFixedLayerMargins are protected by the proper mutex. * Boris Chiou moved animations of transforms with preserve-3d off the main thread * Jamie clamped some scale transforms at 32k to avoid excessively large rasterized areas. * Jonathan Kew reduced the emboldening strength used for synthetic-bold faces with FreeType. * Andrew implemented NEON accelerated methods for unpacking RGB to RGBA/BGRA. * Alex Henrie fixed a bug in Moz2D’s Skia backend.

What’s new in WebRender WebRender is a GPU based 2D rendering engine for the web written in Rust, currently powering Firefox‘s rendering engine as well as Mozilla’s research web browser servo.

Miko avoided calculating snapped bounds twice for some display items.
Kris fixed snapping and rounding errors causing picture caching invalidation when zoomed in.
Glenn fixed a picture caching invalidation bug.
Kvark ensured shader programs are bound after changing the blend mode. While not necessary for OpenGL, this makes it easier to efficiently implement backends for vulkan and other modern GPU APIs.
Glenn refactored the OS compositor abstraction.
Jamie implemented a texture upload path that plays better with Adreno OpenGL drivers.
Jonathan Kew reduced the emboldening strength used for synthetic-bold faces with FreeType.
Nical prevented invalid glyphs from generating expensive rasterization requests every frame.
Nical reduced the number of memory allocations associated with clip chain stacks.
Nical reduced the number of memory allocations in various parts of the picture caching code.
Glenn fixed a picture caching invalidation issue when scrollbars are disabled.
Glenn and Andrew adjusted tradeoffs between text rendering quality and performance.
Miko simplified some of the scene building code.
Jamie switched to local raster space when animating a double tap zoom to avoid uploading glyphs continuously on Android.
Glenn fixed an intermittent compositor surface creation bug.
Andrew fixed a shader compilation error on some Android devices.
Bert improved the way picture cache tile sizes are selected for scroll bars.
Gankra removed some unsafe code in wrench.
Glenn fixed an issue with picture cache tile merging heuristics.
Glenn fixed tile opacity getting out of sync with compositor surfaces.
Glenn added an API for tagging image descriptors as candidates for native compositor surfaces (typically video frames).
Sotaro followed up by tagging the approriate image descriptors on the content side.
Andrew removed removed pixel snapping from most shaders, now that it is handled earlier in the pipeline.
Glenn improved the invalidation logic for images with picture caching.
Glenn improved the logic to detect identical frames and skip composition.
Glenn fixed the shader implementation of rounded rectangles with very small radii.
Kris fixed misplaced text selection popup with GeckoView.
Markus fixed a ton of issues with WebRender/CoreAnimation integration.
Markus shared the depth buffer between OS compositor tiles on MacOS to save memory.
Sotaro fixed image bitmap canvases with WebRender.
Sotaro fixed a crash at the intersection between picture-in-picture and WebRender frame throttling.
Timothy implemented support for respecting fixed layer margins during hit-testing.
Timothy implemented GeckoView’s setVerticalClipping API for WebRender.
Jeff fixed an SVG rendering bug.
Jamie fixed an issue where the screen would remain black after resuming Firefox for Android.

To enable WebRender in Firefox, in the about:config page, enable the pref gfx.webrender.all and restart the browser.

WebRender is available under the MPLv2 license as a standalone crate on crates.io (documentation) for use in your own rust projects.

What’s new in Wgpu * Kvark implemented buffer creation and mapping, with an ability to both provide data and read it back from the GPU. * Kvark set up the synchronization from Mozilla Central to Github repository. * jdashg created a separate category for WebGPU mochitests. * Kvark heavily reworked lifetime and usage tracking of resources. * Many fixes and improvements were made by the contributors to wgpu (thank you!)

View Details

Hi there! Another gfx newsletter incoming. Glenn and Sotaro’s work on integrating WebRender with DirectComposition on Windows is close to being ready. We hope to let it ride the trains for Firefox 75. This will lead to lower GPU usage and energy consumption. Once this is done we plan to follow up with enabling WebRender … Continue reading moz://gfx newsletter #50 →

View Details

By way of introduction, I invite you to read Markus’ excellent post on this blog about CoreAnimation integration yielding substantial improvements in power usage if you haven’t already.

Next steps in this OS compositor integration saga include taking advantage CoreAnimation with WebRender’s picture caching infrastructure (rendering tiles directly into CoreAnimation surfaces), as well as rendering using a similar mechanism on Windows via DirectComposition surfaces. Markus, Glenn and Sotaro are making good progress on all of these fronts.

What’s new in gfx WebGPU Kvark landed WebGPU’s infrastructure and initial implementation on Nightly. It can be enabled via the pref “dom.webgpu.enable” in about:config. The functionality is limited to device creation at the moment, with support for Vulkan, D3D12, and Metal.

WebGPU Is a work-in-progress specification for the successor to WebGL 2, exposing a modern API to perform rendering and computation on the GPU.

Various bug fixes and improvements * Botond fixed a bug that caused WebExtension popups to render empty with desktop zooming. * Botond fixed a WebRender scrolling regression that affected some workflows in Tile Tabs and similar extensions. * Botond fixed some Android UX issues related to gesture handling (bug 1570559, bug 1586496). * Botond fixed some Android scrolling webcompat issues (bug 1592435, bug 1592902). * Botond fixed some more prerequisites for enabling a hiding URL bar in Firefox Preview (bug 1552608, bug 1590582). * Markus shared the GL contexts between open windows on Mac. * Bob Owen improved the recycling of shared textures. * Jonathan Kew fixed a sub-pixel anti-aliasing issue when the text color has some transparency and an emoji is present. * Sotaro fixed an issue with DWM, partial present and high-contrast mode on Windows. * Andrew allowed image decoders to choose RGBA and BGRA at compile time. * Nical Fixed a shutdown crash. * Sotaro fixed an issue with partial present and the snapshot mechanism used for reftests. * Jonathan Kew enabled a better default fallback font for Mongolian on macOS. * Jonathan fixed a whole lot of other font rendering issues. * Lee updated Skia to version m79. * Jamie avoided using glTexImage3D on android emulator.

What’s new in WebRender WebRender is a GPU based 2D rendering engine for web written in Rust, currently powering Firefox‘s rendering engine as well as the research web browser servo.

To enable WebRender in Firefox, in the about:config, enable the pref gfx.webrender.all and restart the browser.

WebRender is available as a standalone crate on crates.io (documentation) for use in your own rust projects.

Kvark made WebRender use the rounded rectangle fast path more aggressively, improving performance on some sites.
Glenn improved the performance of the Direct Composition integration of picture caching by only rebuilding the Direct Composition visual tree if different from last frame and using EGLImage instead of pbuffer for GL to Direct Composition interop.
Andrew fixed an issue with pixel snapping and backdrop filters.
Bert fixed the capture debugging tool on Windows.
Glenn added some debugging infrastructure for picture caching invalidation.
Lee fixed an issue with sub-pixel anti-aliasing and ClearType level settings.
Nical documented How the data that are passed to the shaders is organized out in GPU memory.
Lee improved the performance of text rendering with large sizes and during async zoom.
Sotaro made partial updates work with the OS compositor integration on Windows.
Jamie disabled picture caching during async zoom when the entire screen is invalidated every frame.
Glenn reduced the amount of surface allocations with picture caching.
Andrew fixed some pixel snapping issues.
Nical reorganized the data that is passed to the vertex shaders to be more consistent and enable better code generation with the multi-brush shader.
Nical refactored brush shaders and added the possibility to generate “multi-brush” shaders that support multiple types of primitives in preparation for a more flexible batching infrastructure.
Glenn added support for the built-in profiler debug overlays with the OS compositor mode.
Markus fixed vibrancy with the OS compositor mode on Mac.
Miko refactored display item building so that building function knows the item index beforehand.
Glenn fixed a picture caching bug with special pages such as about:support.
Glenn implemented CPU-side occlusion culling of picture cache slices to reduce memory usage and overdraw.
Jeff added a debugging tool to show when glyphs are re-rasterized.
Glenn reduced the work done when a picture cache tile is inside the display port but not currently visible on screen.
Nical improved the built-in profiler overlay to show more relevant information in some of the counters and highlight counters outside of their expected ranges.
Jamie ensured shader sources are null-terminated even when the source size is provided to work around a shader compiler bug.
Martin Stransky implemented dmabuf texture sharing for WebRender.
Kris made the screenshot infrastructure used in GeckoView work with WebRender.
Nical fixed a gradient rendering bug.
Nical improved WebRender’s API documentation.

View Details

By way of introduction, I invite you to read Markus’ excellent post on this blog about CoreAnimation integration yielding substantial improvements in power usage if you haven’t already. Next steps in this OS compositor integration saga include taking advantage CoreAnimation with WebRender’s picture caching infrastructure (rendering tiles directly into CoreAnimation surfaces), as well as rendering … Continue reading moz://gfx newsletter #49 →

Mozilla Gfx Team Blog: Recent Episodes

None