Friday, January 15, 2010

Proposed Definition of Natural User Interface (NUI)

"Natural User Interface" (NUI) is a term that is kind of like "pornography" in that its hard to define but you know it when you see it.

There have been a number of definitions offered for NUI, but none of them seem to capture the full spectrum of NUI technologies. I've decided to propose a definition that I believe captures the spirit of term Natural User Interface while helping to differentiate NUI from other kinds of human-computer interfaces.

There are four qualities of a NUI that should be in the definition of the term: Directness, Naturalness, and the fact that NUI is a type of Human-Computer Interface.

NUI is often differentiated from GUI by the directness of interaction. We say that NUI is more direct than GUI. The typical example of this is multi-touch where we can manipulate objects directly on the screen using your fingers rather than with a joystick or a mouse. The manipulation is said to be more direct, and therefor an example of NUI. The same could be said for audio interfaces where the user speaks commands directly to the computer rather than typing them in via a keyboard. In fact, the directness definition tends to apply pretty well to may different NUI technologies from touch screens, to direct voice input, to augmented reality, tangible computing, and even automatic identification. But directness, while an important dimension of NUI, its actually a bit too inclusive to stand alone. For example, the tuning in a radio station using a dial is pretty direct but its not NUI.

Another word often used to describe NUI s naturalness. Not natural as in Nature, but natural as in the learned and innate manor with which people interact with non-technological aspects of their environment. For example, its more "natural" for people to move things around with their hands than with a joystick. We don't use a joystick to eat food or wash ourselves. Similarly, its more natural for people to express their needs using speech than a keyboard. Although critical to the definition of NUI, naturalness, like directness cannot stand alone. For example, Augmented Reality - arguably a NUI technology - has little or no alignment with how we naturally perceive the world around us. Its not like objects naturally have meta data labels floating over them. In addition, many of the manipulations we use on multi-touch screens (e.g. pinch) have no natural counterpart in our interactions with everyday non-technical objects. You don't pinch a piece of paper to make it smaller while preserving its aspect ratio.

It would seem to me that directness and naturalness are both qualities that should be present in a NUI technology. A technology that is neither direct nor natural probably isn't NUI. For example, a joystick or mouse is neither direct nor natural. The same can be said for a keyboard.

Computer Input and Output
It's also important that we explicitly tie the definition of NUI to the physical devices used to input data and to output data to a computer. The way in which a computer processes information is not important to the definition of NUI. We don't care if a multi-touch computer uses artificial intelligence or mechanical gears to process information. What we care about when it comes to NUI is how do we give the computer data and how do we get information from the computer. These are the mechanisms that make up a computer's Interface. Defining NUI as a type of Human-Computer Interface may seem redundant but I think its important to emphasize the fact that we are talking about the inputs and outputs mechanisms of a computer and not the computational, memory, or storage aspects of a computer.

A Definition for NUI
Based on the proceeding paragraphs about directness, naturalness, and computer I/O, I propose the following definition for NUI.

A Natural User Interface is a human-computer interface that models aspects of direct interactions between people and their natural environment.

The term "models aspects" is the best I could think to express the idea that the NUI is not attempting to exactly duplicate a natural interaction, but is simply using some qualities natural interaction as a model. In fact, NUI interface are not indistinguishable from natural interactions. They simply resemble certain aspects of natural interactions.

UPDATE: January 22, 2010

I've come up with my final definition of NUI which you can find at this post.

UPDATE January 20, 2010

With the ink not even dry on this definition I've already written a new post which proposes an alternative definition of NUI based on commentary from Bill Buxton.


Stan said...

A good start!

Reg. the directness and our ability to "manipulate objects directly on the screen". It' has been described by Ben Shneiderman in 1983 and his definitions are used in the field of HCI.

The part saying that "The manipulation is said to be more direct" (by whom?) suggests that there is a spectrum of "directness" with mouse as less and touch as more direct examples. But is e.g. a command line less direct than a mouse that is pushed with a wooden stick held in somebody's hand? ;)

Maybe directness is about homogeneous medium of input and output - like using audio input (voice) to get audio feedback (sound from the speakers)? But what about touch then? Should that be a tactile feedback too (like vibration or change in surface's structure)?

I think the definition of "directness" should cover that distance between human-input method used and the place where the feedback is given. Quite significant one in case of mouse but not in case of touch/audio etc.

I'm not sure about all of that but hope to ignite some discussion.


sin said...

I am wondering what is the different between NUI and TUI ( tangible user interface), because TUI also emphasis on Directness and Naturalness of the way human interact with computer.

Multitouch Designer & Developer said...

Tangible User Interfaces (TUI), Originally called “Graspable User Interfaces”, is one kind of NUI technology. The fact that it's described using terms like direct and natural shows how well aligned it is with other NUI technologies like Direct Voice and multi-touch.

Tangible User Interfaces focus on the physical representation of information in objects that look and feel like the information they represent. An excellent example is the Wii Wheel which embeds a standard Wii remote into the center of a plastic cradle that looks like a steering wheel. You use the Wii Wheel while playing racing games and as you turn the wheel your car on the screen turns too. Another example is Siftables (

While TUI is a kind of NUI, its not the same as NUI. NUI is an umbrella term which also includes other UI technologies which are not TUI such as Audio UI (e.g. Direct Voice Input), multi-touch surfaces such as Microsoft Surface, and in my opinion Augmented Reality, Automatic Identification, and Calm Technology.

A while back I posted a blog entry that spoke about NUI and some of the technologies that are kinds of NUIs. You can find that post here :

Multitouch Designer & Developer said...

Directness is a perceived quality of NUI, not an electronic or mechanical property. I agree that in terms of electronics a touch surface or a microphone is no more direct than a keyboard or a mouse. But from the perspective of the user NUI technologies feel more direct. Talking to a computer or moving objects on a screen with their fingers, feels more direct than using a mouse and keyboard. That is the key to understanding what is meant by "directness".

sin said...

Thanks for your explanation.