Foundations: pointer gestures

Posted on Friday, 17 March 2023 by Graeme Coleman in Design and development

A pointer gesture is an action a person performs using their finger, mouse, or an assistive technology like a head pointer or eye gaze system to operate a functional control. Some pointer gestures can be difficult or even impossible for certain people to perform, so an accessible alternative may be required.

This post introduces the different forms of pointer gestures and covers how and when to provide alternatives for people who can't perform them.

Pointer gestures and accessibility

Many of today's user interfaces require us to perform gestures beyond pointing and clicking. Changing the audio volume in a media player may involve moving a toggle up and down a slider, while arranging tasks in a to-do app may involve dragging items from one area of the page to another.

The popularity of touchscreen devices has also meant that many of us have had to learn new gestures using our fingers to perform actions that were otherwise impossible to perform using the keyboard or mouse alone.

Some people may find these gestures challenging to do accurately or cannot perform the gesture at all. For example:

People with limited dexterity or hand tremors may find it difficult to swipe, drag, or pinch, particularly if successful completion of the gesture depends on fine motor movements
People who use a head wand may only be able to perform comparatively simple gestures, and even then may encounter fatigue if a task requires multiple gestures
People with cognitive or learning disabilities may theoretically be able to perform the required gesture, but may not understand how to perform it

Understanding pointer gestures

The four main types of pointer gestures are as follows:

Single pointer gestures
Multi-point gestures
Path-based gestures
Dragging movements

Note that, from a conformance perspective, the Web Content Accessibility Guidelines explicitly distinguish between dragging movements and other forms of pointer gestures. Multi-point and path-based gestures are covered by 2.5.1 Pointer Gestures (Level A), introduced in WCAG 2.1, while dragging movements are covered separately by 2.5.7 Dragging Movements (Level AA), introduced in WCAG 2.2.

Single pointer gestures

Single pointer gestures only involve a mouse or a single finger/stylus on a touch screen, and are arguably the easiest to perform. Typical examples of single pointer gestures include:

Clicking: including double-clicking
Tapping: including double-tapping
Pressing: including press and hold
Swiping: if the angle of the swipe is irrelevant for operating the control (discussed in more detail in the note on path-based gestures below)

Buttons, links, and checkboxes normally only require single pointer gestures.

Multi-point gestures

Multi-point gestures are those that involve more than one pointer being used simultaneously. Such gestures are commonplace in touchscreen environments. Typical examples of multi-point gestures include:

Pinching/spreading two fingers: including pinch-to-zoom within a map application
Two-finger rotation: including two-finger rotating to operate a dial control
Tapping using more than one finger: including the selection of multiple items in a list

A person using pinch-to-zoom to zoom into/out of a map within the Google Maps app on iOS — Google Map's pinch-to-zoom feature that requires two fingers to operate

Path-based gestures

Path-based gestures involve an initial "down" press of the pointer, followed by a movement of the pointer along a specific path.

For example, tracing, drawing a shape, and steady-hand games where a person must move their pointer down a narrow twisting path from one end to the other all involve path-based gestures.

A game with a Start point on the left, and an End point on the right. There is a squiggly line between these points, and a toggle on the line that a person must move from the start point to the end point without deviating away from the line. — A steady-hand game that requires a path-based gesture to operate

Depending on other factors, gestures like sliding and swiping can also be path-based. This subject is covered in more detail in the note on path-based gestures below.

Dragging movements

Dragging involves grabbing, picking up, or otherwise selecting an item and moving it elsewhere.

The main difference between dragging movements and path-based gestures is that, when performing a dragging movement, only the start and end points matter, not the actual path that a person takes to move the item between those points.

For example, when dragging an item from the top of the screen to the bottom, the path that a person takes to move between these points is irrelevant.

Typical examples of dragging movements include:

Dragging cards between target areas of a Kanban board
Sorting a list of items
Dragging an item of furniture around a room
A volume slider

An example showing two boards that contain a number of tasks in each. The first board is labeled Today's tasks, and includes three tasks (Prepare presentation, Email John, and Finish report). The second board is labeled Tomorrow's tasks, and includes one task (Email Jane). There is a task labeled Book flights that is being moved from the first board to the second board. — A Kanban board that requires dragging movements to operate

Creating accessible alternatives

To support people with disabilities who may find it difficult or impossible to perform certain gestures, the related WCAG success criteria state that all functionality that uses a multi-point gesture, a path-based gesture, or a dragging movement for operation must also be operable using a single pointer gesture that is not path-based.

Exceptions include:

The gesture is essential: for example, steady-hand games rely on people being able to use gestures involving fine motor movements, otherwise the game is effectively redundant
The functionality is determined by the user agent and not modified by the author: for example, the functionality provided by a device's built-in screen reader

Two fingers performing a pinch-to-zoom gesture on the VoiceOver rotor. — VoiceOver's rotor feature requires a two-finger "dialling" gesture but, because it is functionality associated with the device's built-in screen reader, it's exempt from the related success criterion

In practice, this usually involves adding separate controls to your designs that perform the same functionality but only require a single pointer gesture like a single tap or click.

A note on path-based gestures

As mentioned above, a single pointer-based alternative that is not path-based must be provided for all functionality requiring a path-based gesture.

However, in some cases, interpreting whether a gesture is path-based, and therefore whether or not such an alternative is required, can be a confusing process. This is typical in cases where there is no explicit or apparent path for a person to follow when performing the gesture.

The answer comes down to how a person must perform the gesture for the related action to trigger; in other words, the accuracy of the gesture. In short:

If the angle or direction of the gesture has no impact on success, the gesture is not a path-based gesture, and an alternative is not required
If the angle or direction of the gesture does have an impact on success, the gesture is path-based, and therefore a single pointer-based, non-path-based, alternative is required

This is best explained through an example.

The following screenshot shows a typical carousel. A person moves slides into view by swiping left or right on the screen. The angle or direction of the swipe gesture has no impact on whether the carousel moves. Even if the gesture is performed at a slight angle, the carousel will still operate in the same way as if the person had swiped straight across the screen. Therefore, a single pointer-based, non-path-based alternative is not required to meet 2.5.1 Pointer Gestures (Level A).

Note, however, the lack of an alternative may cause the carousel to fail 2.5.7 Dragging Movements (Level AA) if the carousel also relies on dragging movements to operate.

A screenshot of an example carousel containing three slides, only one of which is fully visible. There is a finger icon below the carousel, and various arrows showing the direction of the swiping action. Ticks next to each arrow indicate that the carousel will move when the swiping gesture is performed in that direction. — A single pointer, non-path-based, alternative is not required if a person does not need to follow a specific path for a swipe to work successfully

The following screenshot shows the same carousel. This time, however, the angle of the swipe gesture must be perfectly straight from left-to-right or right-to-left. If there is deviation, for example a person swipes at a slight angle, the carousel does not move as expected. Therefore, in this case, a single pointer-based, non-path-based, alternative is required.

A screenshot the same carousel, finger icon, and arrows showing the direction of the swiping action. This time, there are ticks next to the arrows pointing directly left and directly left to show that the carousel will move when related swiping gestures are performed, but there are cross icons next to those arrows that demonstrate swipes performed at different angles. — A single pointer, non-path-based, alternative is required if a person needs to follow a specific path for a swipe to work successfully

That said, it is good practice to provide single-pointer, non-path-based alternatives in all cases, even if the gesture's angle, direction, or accuracy has no impact on the success of the gesture. Doing so benefits everyone, particularly people who may find gestures difficult to perform, even when the gestures are not path-based. Additionally, remember that alternatives are required if operating the component requires some form of dragging movement.

Examples of accessible alternatives

The following examples demonstrate how single pointer, non-path-based, non-dragging alternatives can be provided for controls that rely on multi-point gestures, path-based gestures, and dragging movements to operate.

Map application

The following screenshot shows the basic design for a touchscreen-based map application. A person can touch the screen and drag their finger around to pan the map, and they can use a two-fingered pinch-to-zoom gesture to zoom in and out of the map.

Equivalent and appropriately labeled buttons are provided for people who cannot perform these gestures.

An example map app that includes four buttons labeled with arrows representing north, south, east, and west, and two buttons labeled Zoom In and Zoom Out. — A map that includes separate buttons for navigation and zooming

Volume slider

A volume slider allows a person to touch or click on the toggle within the slider and then drag the toggle by moving their finger or mouse up or down.

For people unable to perform these actions, there are equivalent buttons above and below the slider that allow the toggle to be moved using a single pointer gesture without any path-based or dragging movements.

Additionally, tapping or clicking on any point in the slider should move the toggle to that point and change the volume accordingly.

Two instances of a vertical volume slider where the toggle is approximately halfway between the loudest and quietest points on the slider. There are plus and minus buttons at the top and bottom of the slider. In the first instance, there is a finger icon pointing to the minus button and an arrow showing where the toggle will move when this button is pressed. In the second instance, the finger points to a specific place on the slider, and an arrow shows where the toggle will move when the user activates this point on the slider. — A volume control that includes Volume Up and Volume Down buttons as alternatives for changing the volume, as well as functionality that allows people to click or tap on a specific point on the slider to set the required volume.

Carousel

A carousel includes Previous and Next buttons that allow people to navigate between slides without swiping, even if the swiping gesture isn't path-based, or dragging movements.

Drag-and-drop-based Kanban board

A Kanban board containing individual "Move" buttons that - when activated - open a dialog box of potential drop points. This allows people to complete the associated task using a control requiring only a single pointer gesture rather than a dragging movement.

A Today's Tasks board that includes two draggable items (Prepare Presentation and Book Flights). There are Move buttons next to each of these items. Activating one of the buttons brings up a Move This Task To dialog box that includes radio button options for tomorrow, next week, and next month, as well as Close and Confirm buttons. — A Kanban board containing "Move" buttons as an alternative to drag-and-drop

Path-based product selection

An application requires people to draw a circle around the item that they wish to select. Buttons labeled "Add" that only require a single pointer gesture to activate are positioned next to each option for people who are unable to perform this gesture.

A screen that begins with an instruction to draw a circle around the item that a person wishes to select, or press the related add button. There are three products on the page (water, milk, and bread) followed by an Add button. — The inclusion of equivalent Add buttons for a task that relies on a path-based gesture (drawing a circle)

Summary

When implementing interactive content, using interface controls that can be operated using a single pointer gesture and don't require any dragging - be it freeform or following a specific path - is the easiest and most accessible solution. Most components that people are familiar with, such as buttons, links, and form controls like checkboxes and radio buttons, all support single pointer gestures for operation.

If you must use controls that can only be operated using multi-point gestures, path-based gestures, or dragging movements, include additional controls within your designs that allow people to complete the related tasks using a single pointer, non-path-based, gesture that does not require any dragging.

More information

Next steps

Read more accessibility foundations posts, our WCAG primer or how assessments can help you identify issues in your websites, mobile applications, design systems, and other products and services.