Direct manipulation is broken: Why the IoT asks consumers to think like programmers and the UX challenges this creates

Claire Rowland examines how the IoT breaks direct manipulation, the UX convention underpinning most successful consumer interfaces. This creates significant UX challenges. Claire explores how using the IoT is conceptually similar to programming and why that proves a challenge to many consumers.

In direct manipulation—the basis of most examples of successful consumer user experience design in the last 30 years—users act on UI objects in front of them instead of learning a system of abstract representation. Direct manipulation user interfaces enabled personal computers to become mass market. But the Internet of Things (IoT) breaks direct manipulation. IoT systems allow remote control, automation, and the ability to coordinate multiple devices. When done right, these can be powerful features. But this causes significant UX challenges, particularly for consumers. User actions may produce consequences at future times, and in different places. Users must interact with abstract representations of devices, actions, and notation systems to coordinate devices (like IFTTT or smart rules). They may not get immediate or complete feedback on the consequences of their actions. Even if they can undo problematic actions, they may not know they need to. Conceptually, this is much more like software engineering, where users must anticipate their own future needs and plan a solution, configure the system to meet these needs using an abstract representation, and deal with unforeseen conditions that may make the system behave in unexpected ways, all while considering future conditions that might cause unexpected effects or conflicts. Empowering the widest possible audience to “program” their world is a noble aim, but for many users, this poses cognitive challenges and demands a heavy investment of time and attention. Claire Rowland explores how using the IoT is like programming and why that proves a challenge to many consumers. In the process, she addresses key considerations for designers, including to what extent it is feasible to offer consumers easier, quicker routes to do programming-like things, how we might make these easier to grasp, and when it is better for the system to be autonomous. . .without the user losing control.