superNatural User Interfaces

The interest and awareness of Natural User Interfaces (NUI) is growing quickly and I'm guessing that the term, like Ubiquitous Computing, will eventually become well-known. That's a good thing, but there is an aspect of NUI that has been bothering me since I first learned about it and started writing and talking about it: There is nothing natural about NUI.

Art: Magic Circle (1886) by John William Waterhouse

In a series of articles (part1, part2, part3) I wrote briefly about the metaphor of "magic" in the design of NUI interfaces. The idea is that you can design NUI interaction so that they feel like magic. I won't go into the specifics only to say that these articles have helped to shape my vision of the future Human Computer Interfaces more keenly than NUI has as described in mostly academic but increasingly commercial literature.

NUI is often described as leveraging interactions and affordances between humans and computers that are more natural. Gestures and touch are used to elicit information or behavior from computers rather than a mouse and keyboard. NUI is supposed to be more intuitive and quicker to learn. However NUI seems to have limited applications because the demands of being natural means that the fidelity of information and the duration of interaction is somewhat lower than you see with keyboard and mouse. That's why you see things like gestural interfaces and touch walls used mostly in museums and as gimmicks to attract attention at conferences than they are for everyday tasks.

While the use of NUI technologies to educate and attract users in public settings has enormous potential, I'm much more interested in how this new paradigm of interaction can be applied to productivity applications, personal software, and devices. In these cases the number of interaction made possible by "natural" gestures and affordences, which is rooted in real-world models, is far too limiting. In NUI the prevailing question seems to be: How do we make human computer interactions seem natural? I propose a very different question: How can we make the interaction between humans and computers seem magical?

Nature is limited by biology, chemistry, and physics. Why limit ourselves to that pallet? With technology today we can stretch reality and make interaction more magical by making them "super real". The concept of "super real" interactions is not my own, I first heard it from someone at Microsoft in a YouTube video, but I think it captures the essence of my interests.

The idea behind super real interactions in NUI is that the objects on a touch screen, for example, exhibit properties of real world physics - you can toss them around and rotate them - but they also exhibit properties that are not natural such as the ability to enlarge or reduce the size of a objects by stretching or pinching them with your fingers. These kinds of interaction are just as useful as the "natural" ones, but there is very little that is natural about them. They are super-real; more like magic than nature.

My objective is not to persuade you to start using the term superNatural User Interfaces, but simply to explain the reasons why I use it. I feel the term Natural User Interface is an oxymoron. A computer interface cannot, and in fact should not, be natural. It should be supernatural. The affordences and behavior of computing devices should leverage technology to go beyond what is possible with real world objects.