Microsoft's Magic Book

A reader sent me a link to this Gizmodo article about a leaked Microsoft "booklet" computer called Courier which is in late prototype phase.

Art: The Crystal Ball (1902) by John William Waterhouse

I like Courier. A lot of designers - both industrial and interface - use this brand of note books called Moleskine Journals or Notebooks. What I've noticed is that these Moleskine notebooks are increasingly being used by executives - not just designers. Executives and designers like Moleskine notebooks because feel good in the hand when taking notes or sketching out ideas. You can jot down an idea and then reference them later. People accumulate these notebooks and use them as references for years.

The Courier is a digital example of a Moleskine notebook and I really like the way its designed. It's not just a slate with two opposable screens, its designed to feel and act like magical designer's notebook. You can take notes with a stylus, move stuff around with your fingers, snap pictures, and pull content off the web. Like the Surface, this device breaks normal GUI conventions and goes with a entirely new UI. I would definitely use one of these while doing my research - writing and sketching on lined pads is becoming a bit tedious.

The video (see blow) gives the Courier a magical feel. It's not trying to feel exactly like a book, more like a super book where you can peruse pages the way you do a normal notebook or search by keyword. You can move stuff from one page to another, copy images from web sites, take notes, make sketches, and import photos.

A couple things I would like to see in a product like this is some simple direct voice input (e.g. "Find notes on Engelbart") as an option and the ability to precisely clip material out of web documents. The later would require a really good web browser and the ability to choose content using some advance selection techniques. For example, I only want this paragraph and that photo but I want the content to link back to the original article if I need to. Add these capabilities and I think you would have a compelling interface for a note-taking product.

Another obvious use for this is as an eReader. Given the form factor and what appears to be a pretty powerful processor this booklet would be perfect for that market.

In terms of industrial engineering I would recommend a leather or leather-like exterior to give the impression that this is a private journal that users take everywhere they go. It will also help to make the connection with journals and those Moleskin notebooks I was talking about. It would also make it a more attractive eReader to curl up with at night. Note to all eReader makers: Make sure the rotation of the screen is optional for those of us who like to lay on our side at night while reading. I have some other ideas which I might post later.

Courier User Interface from Gizmodo on Vimeo.

superNatural User Interfaces

The interest and awareness of Natural User Interfaces (NUI) is growing quickly and I'm guessing that the term, like Ubiquitous Computing, will eventually become well-known. That's a good thing, but there is an aspect of NUI that has been bothering me since I first learned about it and started writing and talking about it: There is nothing natural about NUI.

Art: Magic Circle (1886) by John William Waterhouse

In a series of articles (part1, part2, part3) I wrote briefly about the metaphor of "magic" in the design of NUI interfaces. The idea is that you can design NUI interaction so that they feel like magic. I won't go into the specifics only to say that these articles have helped to shape my vision of the future Human Computer Interfaces more keenly than NUI has as described in mostly academic but increasingly commercial literature.

NUI is often described as leveraging interactions and affordances between humans and computers that are more natural. Gestures and touch are used to elicit information or behavior from computers rather than a mouse and keyboard. NUI is supposed to be more intuitive and quicker to learn. However NUI seems to have limited applications because the demands of being natural means that the fidelity of information and the duration of interaction is somewhat lower than you see with keyboard and mouse. That's why you see things like gestural interfaces and touch walls used mostly in museums and as gimmicks to attract attention at conferences than they are for everyday tasks.

While the use of NUI technologies to educate and attract users in public settings has enormous potential, I'm much more interested in how this new paradigm of interaction can be applied to productivity applications, personal software, and devices. In these cases the number of interaction made possible by "natural" gestures and affordences, which is rooted in real-world models, is far too limiting. In NUI the prevailing question seems to be: How do we make human computer interactions seem natural? I propose a very different question: How can we make the interaction between humans and computers seem magical?

Nature is limited by biology, chemistry, and physics. Why limit ourselves to that pallet? With technology today we can stretch reality and make interaction more magical by making them "super real". The concept of "super real" interactions is not my own, I first heard it from someone at Microsoft in a YouTube video, but I think it captures the essence of my interests.

The idea behind super real interactions in NUI is that the objects on a touch screen, for example, exhibit properties of real world physics - you can toss them around and rotate them - but they also exhibit properties that are not natural such as the ability to enlarge or reduce the size of a objects by stretching or pinching them with your fingers. These kinds of interaction are just as useful as the "natural" ones, but there is very little that is natural about them. They are super-real; more like magic than nature.

My objective is not to persuade you to start using the term superNatural User Interfaces, but simply to explain the reasons why I use it. I feel the term Natural User Interface is an oxymoron. A computer interface cannot, and in fact should not, be natural. It should be supernatural. The affordences and behavior of computing devices should leverage technology to go beyond what is possible with real world objects.

Sony Ericsson's NUI Head Phones

Sony has announced a set of pretty cool headphones, the Sony Ericsson MH907s, that can be controlled by simply removing them or placing them in your ears.

To start the music playing, place both ear buds in your hears. To pause, remove one ear bud. To answer a phone call, remove both ear buds and then place one back in your ear.

I'm not exactly sure about the gestures - I might have designed it a little differently - but the idea that you can control the earphones by just removing them or putting them in place is awesome.

This is an example of NUI if there ever was one. In fact, I would probably categorize this as Tangible User Interface since the object being manipulated is also the object providing the information. Anyway, check out their funny video demonstrating how they work.

(Btw - I've been really busy developing an iPhone application for a client and haven't had time to write but I'll get back to regular posting next week)

Hard Rock's Giant Touch Wall

Last week the Hard Rock Las Vegas re-opened - after remodeling - featuring not only Microsoft surface tables but also a custom made touch wall measuring 18 feet long and 4 feet high. The giant touch wall was created by Obscura Digital.

Judging by the video it looks like a great attraction application allowing visitors to see images of individual items in the Hard Rock's memorabilia collection and to pass those images around. I would think that the clicks and the audio of interviews and music would be distracting, but in a context such as this it probably encourages people to interact with each other as well as the wall.

Packaged Software for Multi-touch

It's really cool to see traditional packaged software vendors being shipping applications specifically designed to accommodate multi-touch. You can see the changes in the way these applications look with larger controls for thick fingers.

It seems appropriate to me that the first packaged applications to to support touch are productivity applications for multi-media and design. In July I posted a blog about the really cool 3D modeling software from SpaceClaim. Today, Corel announced that its new Digital Studio 2010 is fully touch enabled.

In the not to distant future (less than 6 months) there will be so many announcements about touch enabled PC software that I won't be able to keep up with it. Today supporting touch is a differentator but soon it will be expected that all PC be software designed to accommodate touch screens.

97 Things Every Programmer Should Know

Kevlin Henney and O'Reilly announced today that the web site "97 Things Every Software Programmer Should Know" is now public and open for contributions.

The "97 Things Every Software Programmer Should Know" book is part of a larger series of books which already includes "97 Things Every Software Architect Should Know" and "97 Things Every Project Manager should Know" both of which you can find at a book store or order from Amazon.com. The books are published by O'Reilly the folks that do the cool animal cover books. I edited the Software Architecture book and blogged about it back in February.

What is really cool about these "97 Things" books is that they are as close to open source as a book can get in my opinion. Each book contains contributions from people all over the world. Every contribution is licensed as Creative Commons, which is kind of like open source for literature. Each book contains 97 short essays from experts in a particular field.

The "97 Things Every Software Programmer Should Know" contains essays on every thing from how to improve performance of your software to advice about reading more from the humanities in order to improve your programming. Each bit of advice is written by a professional software programmer for other professional software programmers.

There are a lot of authors in this kind of book and you can be one of them! Anyone can make a contribution to the growing list. When the editors feel there are enough contributions, they will chose 97 of the best ones and put them in a paper book. This gives experts all over the world, who know programming inside and out and have really great advice, the chance to share their wisdom and get published for the first time.

Check out the growing list of submissions and learn a lot from people all over the world about the best practices in software programming. If you have an axiom or idea for a contribution, simply sign up to the wiki and submit it. If your contributions is poignant, coherent, and unique you may be published in a book from O'Reilly!

Humanoid Robot Interfaces with Multi-touch Surface

Engadget found a great video: A little humanoid robot uses his hands to interface with the photo application on Microsoft Surface. It's so weird I'm not even sure how to characterize it.

Snow Leopard Ready for Multi-touch Screen

When Apple hadn't announced support for touchscreen in Snow Leopard I was really disappointed and I wondered why Apple was giving the advantage for Windows 7.

Of course I continued to have high hopes for a tablet or slate form factor, but I figured Apple would use the iPhone operating system since the Mac's forthcoming operating system only supported gestures on its track pad (I don't really consider that to be true multi-touch).

But now, thanks to posting on 9to5 that Snow Leopard, it seems that Apple just might support true multi-touch on the Mac operating system. Check out this more detailed run down of clues to Snow Leopard's support for multi-touch by Leander Kahney on Cult of Mac. It's pretty exciting and it goes a long way to supporting the rumor that Apple will announce (but not necessary release) a tablet or slate next week.

I know its pretty silly to follow up on all these rumors about the Apple tablet, but I just can't help myself. The fact that Windows 7 has full support for multi-touch and the possiblity that Apple's Mac OS will too is just too exciting. It also helps to justify my switch last year from server-side software development wonk (15 years!) to multi-touch fanatic.

I told my wife last year, when I made the switch to multi-touch software development, that not only was I totally obsessed with the possibilities of multi-touch as a human-computer interface, but there is a sustainable market for those skilled in multi-touch design and development. This is just one more bit of evidence that my gambit will pay off! (I'm not saying, "I told you so"; I'm saying "Whooh! I'm glad that didn't backfire". )

I consider myself fortunate to be making a living doing both Microsoft Surface and iPhone development this year, and I'm looking forward to doing lots more multi-touch work on the Mac tablet and Windows 7. It's good to know that the software industry is going full speed ahead with multi-touch for the masses.