Polygon masks and better borders

[bobnik]

Layer mask and better borders (via boundary tracing and polygon machines)

Features

Mask effect: layer shape option

Lets you use one layer as the mask for another. Works with any shape, including self-intersecting patterns like Fractal Spirograph or text.

Control	Description
Mask shape: layer	Use another layer's shape as the mask
Source layer	Dropdown to select which layer defines the mask boundary
Boundary type	Auto (recommended), or manual: expand, concave, footprint, convex

Fine tuning effect: enhanced border

Provides several improved border options.

Control	Description
Border Type	Auto, expand, concave, footprint, or convex
Pad (%)	Expand/contract border relative to shape
Draw border only	Skip the pattern, output only the border path

Implementation

1. Polygon machines

New machine types that clip patterns to arbitrary polygon boundaries:

• PolygonMachine - Clips vertices inside a polygon boundary
• PolygonInvertedMachine - Clips vertices outside a polygon boundary

2. Boundary tracing

Methods for extracting outer boundaries from complex shapes. When drawing a boundary, it auto-detects the optimal algorithm based on shape characteristics.

Algorithm	Use Case
expand	Simple closed shapes (Star, Heart, Polygon)
footprint	Complex shapes (Text, Fractals, Mazes)
fillPattern	Disconnected cells (Voronoi, Tessellation)
concave	Uniform edge offset
convex	Convex hull

Use cases

Add a layer mask

1. Create Layer 1 with a Star shape
2. Create Layer 2 with a Spirograph pattern
3. Add Mask effect to Layer 2
4. Set mask shape to "layer"
5. Select Layer 1 (Star) as source
6. Spirograph is now clipped to the star shape

Add a border

1. Create a layer
2. Add Fine Tuning effect
3. Check "Add border"
4. Select a border type.

[jeffeb3]

I managed to make it a little weird by scaling the mask effect. I made a simple polygon (7 sides, in this case), then added a circle packer, then the mask effect, selected the polygon shape as the source layer.

Then I dragged the resize in the preview and the mask changed sizes, but the layer 1 polygon did not. That ended up with something a little weird:

I can think of a couple of ways to improve this:

1. When you use a shape as a mask, turn off the eye of that layer. That's what I expected in this situation anyway. Sort of like removing a tool in CAD when you use it in a binary operation. It could be tricky to not surprise people who aren't used to that behavior in CAD. The upside of this is that it would have drawn that border multiple times in sand (which is easy to miss in the preview).
2. Don't allow resizing of the Mask Effect if it is based on a layer. This is probably the simplest solution, but it leaves that layer being drawn (unless the user manually disables the eye for that layer). That may be preferred, from a UI standpoint. I'm on the fence. It's also possible this would be difficult, or nasty, from the implementation side.
3. Just let the user figure out (or describe it in a doc) to deselect that eye before or after making that mask. This would give the user the most control.

This is the .sdf file for that issue (ran on this branch):
edge_filter_scale_issue.txt

[bobnik]

@jeffeb3 I pushed a fix. I went with "hide mask source" layer (option 1). When you select a layer as the mask source, it hides it, adds a mask icon to the layer in the layer list, and adds a "masked source" description in the layer details when you select the layer. I also "dimmed" hidden layers in the layer list a bit to make them more obvious.

Test it out and let me know if this works, thanks.

[jeffeb3]

Looks good. I did find an issue, which I documented. This is ready to go.