Selection Gestures

title

Selection Gestures

dated

Q1 2025

related

Pens & Hands

Selection in drawing apps basically comes in two flavours, depending on how ink is modeled. In general, it seems most notetaking apps have settled on a whole stroke approach where you can only select entire strokes. On the opposite end, most drawing apps have a pixel model, and so when you select stuff, it’s really the pixels inside the lasso. Whole strokes are often a useful unit, but not allowing the user to select parts of strokes feels like premature formality. This begs the question: Can we create a selection system that can do both, while still being simple and intuitive?

In CAD tools, there’s a convention that distinguishes between a clockwise and counterclockwise lasso selection. Clockwise lasso selection typically selects only objects that are fully enclosed within the lassoed area, while counterclockwise selects all objects that the lasso crosses, even if they’re only partially inside the lasso.

So, I decided to take this idea and apply it to the two different styles of ink selection. A clockwise lasso (blue) selects only the ink that is fully enclosed within the lasso.

I built this ontop of simplified strokes, rather than Inklets. This makes the logic for things like intersections quite a bit more complicated and prone to numerical error.

While a counterclockwise lasso (green) selects all the ink that the lasso touches, as well as ink that is fully enclosed within the selection area.

While this might take some getting used to, it’s a great way of distinguishing between the two options that leans into habituation without needing to do toggling in a UI.

Self-healing ink

Note that to be able to move a selection, we may need to cut strokes into pieces. So, moving a selection might create a lot of small pieces of stroke. One possible way of thinking about units of ink is to treat all visually connected ink as a single unit. So, putting ink visually close together makes it self-healing (an idea we first explored in the Playbook design phase).

Here’s that idea in action. You can take ink apart by selecting partial strokes. You can put it back together again by placing the stroke close enough to another one.

Now, when you make a ccw selection, it’ll select both strokes as-if it were drawn in a single stroke. This approach prioritises what the user sees over the internal formal model.

Gradual specificity

The clockwise/counterclockwise distinction allows us to make two distinct selection types. But really, there’s a spectrum of different ways to select ink using a lasso, in order general to specific.

1. Select all visually connected ink that overlaps with the lasso
2. Select all strokes that overlap with the lasso
3. Select all ink exactly inside the lasso, including partial strokes
4. Select only the strokes that are fully inside the lasso

Rather than adding these as distinct toggles, I added an affordance along the side of the screen that allows the user to scrub up-and-down to change the specificity of the selection. Down is more specific; up is more general. Importantly, you can tweak the selection after you’ve drawn the lasso.

In my implementation, this is an addition to the cw/ccw distinction. But it could also a replace it entirely, it feels pretty good either way.

Possible next steps

• We could try more clever heuristics with visual/gestalt properties like the alignment of strokes or segmenting strokes at corners, which we could use to add more visually meaningful steps to the gradual specificity.
• Noodlification: we don’t need to cut up strokes when we move partial strokes, we could stretch them instead.