Brainstorming notes: Implementing alpha-masks in Haiku

This article attempts to sum up a discussion I had with stippi over IRC about how to implement this. Most of the ideas and design are his work, not mine. I’m just turning this into a more readable form and archiving it on the website.

Requirements: what are alpha masks, and why do we want them?

I’ve covered this in an existing blog post.

Short version: WebKit renders “blocks” of an HTML page (span, divs, and the like). It does this by entering one block, setting a clip region to it’s limits, drawing everything inside (possibly entering more blocks and constraining the clipping further), then moves on to the next block.

When you use some (not-so-)advanced CSS features, such as border-radius, you can end up with a block whose clipping isn’t a simple rectangle, but instead a more complex shape. We currently don’t implement this, leading to some drawing artifacts: things are either not being drawn, or drawn at places where they shouldn’t be.

API design

It turns out the BeAPI already plans for this feature. BView has a ClipToPicture (and ClipToInversePicture) call, allowing to clip the drawing of a view to any custom shapes. There are a few problems with this, however:

• There is no support for antialiasing
• Our implementation is extremely slow
• On BeOS, the clipping is done in view coordinates, ignoring the scale and origin of drawing

The first two points are implementation problems: we implement this method using ConstrainClippingRegion, which clips drawing using a region. A region can’t have half-pixels, and, a complex region that covers exactly the pixels covered by a picture is expansive to build. and, WebKit keeps changing the clipping all the time as it moves between rendering blocks.

The third item comes from a different need in BeOS: there, the method was used to have non-rectangle BViews. You would clip a child view to a picture of itself, so it doesn’t draw its background, then you would clip the parent to the inverse picture, so it draws the remaining pixels.

There doesn’t seem to be much apps using ClipToPicture this way (or at all: not a single call to it exists in the Haiku code base, except for test applications that are testing it). So, we decided to trade compatibility for sanity, and go with a clipping that works in object space instead of view space. This means the clipping can be scaled and translated with the rest of the drawing. The old behavior can still be achieved, by setting the clipping, then pushing the view state. While this way of working is not usual in the BeAPI, it will avoid a lot of nonsense when we get to implementing view transformations. If it proves to be a problem for compatibility (Gobe Productive is known to use this kind of clipping), we can add a compatibility test and have it work another way for old apps.

Implementation choices

To support the transformable mask we want, we have to keep the picture object around. While drawing, and as the state of the view changes, we need to generate a bitmap rendering of it, and use that as a clipping mask in app_server’s Painter. Whenever possible, the bitmap must be cached, so most of the usual cases, where the clipping isn’t changed very often, can be handled reasonably fast.

The masking code must be added and removed dynamically in the agg rendering pipeline. We don’t want views that don’t use ClipToPicture to be slowed down, and the alpha masking is an expensive operation that implies a lot of hit-testing and blending.

The masking must not be done per-pixel, this would be too slow. Instead, agg allows us to do the masking at the scanline renderer level. This is faster, because it actually clips out the drawing operations that are completely masked, and keeps the ones that are completely opaque reasonably fast. Other operations may be split in an opaque, a transparent, and a transition area between the two.

Architecture

We introduce an AlphaMask class that keeps the masking state. This includes at least:

• The masking picture, as a ServerPicture object, as we got it from the client,
• Part of the drawing state: pen size, drawing modes, and a few others. This should actually be part of the clipping picture, in order for it to be drawn in the same way each time.
• A lazily initialized cached rendering of the picture as a bitmap. This can be used as long as the drawing state stays the same. Changes to the drawing state don’t trigger an immediate redraw of this, however, they could mark it as ‘dirty’. When the alpha mask is about to actually be used for drawing, we make sure we have an up to date bitmap, and give it to the Painter so it can make use of it.

The actual masking happens in our Painter class. The two main methods that need rework are _StrokePath and _FillPath. Other methods need to be adjusted to call those (skipping eventual optimized versions) when a clipping picture is set. And these two methods must get an extra if-clause to handle the alpha masked case.

There also need to be dedicated code for bitmap and text drawing. These aren’t drawn using path code, but a different codepath ;). We have to find a way to clip them too.

ClipToInversePicture problems

The agg masks implementation lack some features we need. To avoid wasting memory, we want the alpha mask to be just big enough to hold the BPicture (this could be much smaller than the complete view side). but this means the mask may not be aligned with the view, and instead would start at an offset. Agg default clipping implementations don’t seem to allow this in an obvious way. It may be possible to trick them by using a render_buffer that itself offsets the picture.

We have a similar problem for the inverse picture case. Here, we may still have a very small picture, but we must also include anything that’s outside of the bitmap. AGG alpha masks consider anything outside the bitmap to be clipped out.

We have several ways to avoid that:

• Make the bitmap always the same size as the view. This means we don’t have to care about out of bounds access, but we don’t get to use the fast code path when drawing pixels outside of the picture area
• using a custom alpha mask subclass that can include everything outside the bitmap. This saves memory as the bitmap can be small again, but again, we may not get to use the fast codepath - have an hybrid region/pixel system, where we first clip using a BRegion, then fill the holes using the slower bitmap clipping. While this sounds like a great idea, the semantics are a bit unclear as the BRegion is only binary. We’d need a 3-state BRegion where each rect could be either included, excluded or “I don’t know, have a look at the bitmap”). This would get us the fast region-clipping in most cases, except in the area where pixel-clipping is in effect (or we could even reduce that to just the edges of that area, where antialiasing occurs). This is probably the best solution, but the most complicated to implement.

State stacking

An important feature we have to retain is the ability to push and pop view states, and have the clipping follow this. While this was relatively easy to implement for region clipping, it can get more tricky for alpha masking.

We have to intersect the BPictures from all stacked states to generate the actual clipping bitmap. Even when the current state changes, we may have to recompute bitmaps for the lower levels, for example because the view was resized or scrolled, and we don’t have a bitmap for the newly exposed areas.

In this case, we must do the following:

• Start with the lower layer in the stack
• Compute the mask for it, taking the current view size and scrolling into account
• Use the generated bitmap to mask the drawing of the above layer
• … and so on until we get to the top of the stack.

Note this will only happen in the cases where the view is scrolled or resized. This doesn’t happen during regular drawing operations, only between them, and we expect the state stack to be small in those cases. So, the overhead of rebuilding all the bitmaps isn’t that much of a problem. As changing the view origin and scale while drawing doesn’t affect pushed states in any way, they don’t trigger any such rebuild.