Kitten Tale

Scrolling Meets Compositor

A few weeks ago, I noticed that some engineers from Apple landed a small module behind THREADED_SCROLLING flag. Thread? The name sounds interesting. So I took a quick look. And here is a rough finding from that: This is a machinery aiming to make scrolling as smooth as CSS animation, it looks.

To find how it works, we need some basic understanding about implementation of CSS animation and scrolling. Let me start from the former, then proceed to the later, which includes the today’s highlight.

CSS Animation and ACCELERATED_COMPOSITING

WebKit allows CSS Animations to run smoothly. This is especially true for ones over CSS 3D Transforms. To make this happen, a WebKit feature called ACCELERATED_COMPOSITING was introduced at the beginning of 2009, which brought the power of GPU acceleration into WebKit.

Here is a basic idea behind the accelerated compositing: With accelerated compositing enabled, WebKit renders split the page into mutually exclusive parts called “layers”, then render each of them into an offscreen image separately, and “composite” it after all these images are rendered.

image

The point here is that each rendered image of the layer - it’s called a “backing store” in WebKit by the way - is located on the GPU side. Thus the composition of the set of backing stores into the final image can be done pretty quickly with little CPU load, thanks to the GPU speed.

Basically, a layer is corresponding to a subtree of of the page’s DOM. And the whole DOM tree is splitted into layer subtrees based on some CSS properties. For example, if there is a <div> which is styled to have 3d-transormation, the <div> will establish a new layer. And the <div>-rooted subtree will be painted into the backing store of that layer.

Since each layer has own backing store, we don’t need to repaint a layer even if others get modified and repainted. What we need is just re-composite these backing stores. This composition is orders of magnitude fater than repainting the whole page - GPU is a device desinged for these types of operations after all.

We also don’t need to repaint a layer if only its “transformation” is changed. The browser can handle such tranformations in the GPU side as a part of the composition process.

Backing stores, fast compositions and cost-free transformations; Thanks for these characteristics, WebKit can render 3D-ish CSS animations quite smoothly.

Core Animation for Better Slickness

Things go further if you are using Apple ports of WebKit. On Mac (or Apple Windows), the composition is implemented on top of Core Animation infrastructure. As the name implies, Core Animaton helps smooth animation like CSS 3D Transforms. And its implementation fits the backing store model of ACCELERATED_COMPOSITING. In reality, ACCELERATED_COMPOSITING was first desinged for Core Animations.

One great advantage of CA - WebKit calls it “CA” for short - is that it runs these animations in a different thread. So it can run smoothly even if the main thread is getting stuck by, for example, heavy JavaScript programs. Without threaded backends like this, animations for CSS transforms can stuck when big chunk of computations like layout or JS happen in the main thread.

image

This will especially work well for mobile devices. Because CPUs in mobile devices are relatively slow, and at the same time, they start to have another CPU core. CSS animation on mobile is one of the best place where CA-like system shines.

Chromium also has its own composition subsystem, which is called cc. But I don’t know much about that… Well, you can blame me ;-)

Scrolling - Slick or Quick?

Scrolling is another place where browsers can be slicker.

Don’t take me wrong. Browsers do scrolling well. But it’s more like quick than slick. Typical page scrollings happen by per-line, or per-page basis. If you hit “down” key, it scrolls the page about ten pixels. If you click scrollbar, it jumps over hundreds of pixels. (FYI, WebKit supports four types of scroll “granuality”: line, page, document and pixel. You can find them at ScrollTypes.h.) This large-step scrolling has been working well. But it may not be felt slick to skip dozen pixels per frame.

On Mac (again!), the situation is different. Safari (and Mac Chromium), does scrolling slickly intead of quickly; If you click the scroll bar, you’ll see an animated scroll. It scrolls down a page height using tens of frames. WebKit implements this slick scroll as ScrollAnimator class and the family. They live behind ENABLE(SMOOTH_SCROLLING) flag.

Note that the animated scrolling is a different gear from the accelerated compositing. These two don’t share same machinery for some reasons: First, scrolling makes previously-invisible regions visible - This is why people scroll the page. So browsers need to paint these regions. This prevents us from just compositing existing backing stores.

Second, the scrolling animation isn’t “declarative”. In other word, it cannot be described on top of CA’s animation model. It requires a custom-made code to compute the scrolled position.

First one is especially tough. What worse is, we might have to re-layout if the page has something like position: fixed. There are also pages which monitor scrolling to trigger JavaScript for something funny. All these stuff causes a pause and prevents smooth scrolling in a resonable frame rate. This is one of the reason why many non-Mac browsers choose quick, jumpy scrolling instead of slick one.

Scrolling on Touch Devices

This story has a blind side though. In fact, browsers for touch devices like Mobile Safari take an advantage of that.

When you use Mobile Sari, you’ll notice that their scrolling and zooming is extremely smooth. But at the same time, you’ll also notice that sometimes the page painting doesn’t catch up the scrolling (and the zooming). That is the trick: While scrolling, the browser skips some painting to gain the animation frame-rate. In theory, this behavior isn’t correct. An ideal browser shouldn’t show any blank region to the user. But in practice, this is a totally sensible trade-off.

WebKit itself doesn’t support this kind of scrolling-without-painting. Browsers do it by themselves. For example in iOS, I guess they do something like this: Grab the view, which ha CA backing store (called CALayer), pull it out, apply transformation to the backing store without letting WebKit know. Then once the CPU becomes less busy, they notify scrolling to it. WebKit won’t run during the animation thus doesn’t interrupt its slickness.

I don’t check how Android Browser works. But I can imagine it doing something similar.

Having read this, you may ask why WebKit doesn’t support this. Good question! That is exactly THREADED_SCROLLING is trying to do.

THREADED_SCROLLING and ScrollingCoordinator

Most of the THREADED_SCROLLING module is living under WebCore/page/scrolling. This piece of code is basically a scrolling complement for the CA animation model.

With THREADED_SCROLLING, WebKit pushes scrolling out to a background thread, which is named ScrollingThread.

image

This new machinery introduced an asynchronous scrolling model to make this work. In this model, an updated scroll-position, which is requested by some other WebCore object, isn’t applied immediately. Instead, asking a new position will trigger a scrolling animation, and the position applied after the animation is finished.

In FrameView.cpp, you can find FrameView::requestScrollPositionUpdate() where the asynchronous scrolling starts. It asks ScrollingCoordinator to run the scrolling animation. Then the coordinator schedules it on the dedicated thread. Once the animation is finished, the thread notifies back to the coordinator on the main thread, then it calls back FrameView::notifyScrollPositionChanged().

This new feature has one clear advantage over out-of-WebKit implementations: It can work with <iframe>. From a browser-side, it’s hard to access backing stores for each <iframe> because WebKit doesn’t expose such a detail. Actually its layer hierarchy isn’t so simple, It’s something WebKit should take care by itself.

What interesting is that no other objects except FrameView depends on stuff from the threaded scrolling shop. Also, no other newly-introduced objects other than ScrollingCoordinator touches FrameView. All of whom they communite with is CA’s backing stores - and a corresponting WebCore abstraction called GraphicsLayer. We could find some pattern for introducing new features from this minimized dependency.

Their rubber band scrolling animation is implemented in ScrollElasticityController under WebCore/platform/mac. It’s interesting to see that Mac port adopts Cocoa-like MVC pattern even for C++ world. If you take a look, you’ll find that ScrollAnimator I mentioned before is another “view” of the controller.

It looks I’ve stepped into too much now … Anyway, an enthusiasm to slickness like this is one of the most respectable characteristics of WebKit. It’s delightful to see such an improvement.