Widgets on a Slate

I've written extensively about the utility of desktop widgets in the past (here, here, and here) but I've been a little disappointed that the uptake by the masses has been less than dramatic. Than again, on reflection, I don't use the widgets I have on my own Mac - ever. I never open up the Dashboard and use the stock quote or wikipedia widgets. I just use my browser. Why? Why haven't these extremely focused, leight-weight applicaitons not been adopted by everyone?

One possibility is that its just as easy to google the information you want or launch an application as it is to launch a widget. But what about desktop widgets like Microsoft Gadgets or Google Desktop which are always up on your desktop? Why haven't these seen more adoption? Why aren't we excited about them? There are two possibilities as I see it: Either they are simply not as useful or the form factor is wrong.

An interesting post on TechCrunch (see video below) shows a concept of using Widgets on the iPad. The combination of widgets and the slate form factor doesn't seem to offer any advantages at first glance. But, if the widgets were on the unlock screen they would be more immediately accessible and useful. Now this won't work on an iPhone because the screen is too small but what about an iPad or other slate computers?

Imagine that when you pull your slate computer (iPad or otherwise) out that the first thing you see before going to the application launch screen are widgets you might want to glance at quickly. A date and time widget is an example. In fact, this widget functionality is already available on the iPhone's unlock screen. When you push the Home Button the unlock screen comes up with your background of choice and the date and time. I frequently glance at this without unlocking the iPhone just to know what time it is.

I suspect that the iPad will work the same way, but what if you could get a variety of information from the unlock screen. For example, if I can glance at my unlock screen and see that I have three emails from this, that and the other person; or if I could see my favorite stocks, or the weather, or wi-fi accessibility. that would all be very useful and I would not need to unlock the screen; I would just glance at the widgets that are already there.

The scope of widgets that would be useful on the unlock screen is fairly limited but like the date and time widget already on the unlock screen, other widgets may be equally as indispensable and frequently used. The widgets would be view only- they wouldn't be interactive because then they would be applications which are more appropriate when accessed from the launch screen. No, they should be configurable - the same way you can set the date and time - but in general they do not provide for interactivity but provide information at a glance. I think this could be really useful if placed appropriately.

Dashboard on iPad from Rich Hong on Vimeo.

Amazon Kindle for iPad (and other tablets)

Amazon.com announced that its developing Kindle eReader applications for tablet computers including the iPad. The design looks good and its pretty well aligned with other eReader applications I've seen such as the Kobo, Instapaper and, of course, Apple's on iBook. These applications appear to have been influenced by earlier eReader applications such as Classics and Delicious Library and classic Kindle.

It's an exciting area of exploration and I'm proud to announce that I have been contracted to develop a eReader that will work with an existing digital library you may already be familiar with. I can't say who it is yet, but I can say that design of this eReader has been underway for a couple weeks already. We (I've been assembling a small team of artists and developers) are already wrestling with the metaphors and implementation details and we are looking forward to releasing the new eReader on the Apple App Store when its ready.

Developing an eReader that works with an existing digital library is exciting and challenging. There are so many things to consider and so many ways of creating a user experience. The hard part is delivering what readers want and avoiding lots of features they won't find useful while keeping the interface somewhat aligned with existing digital channels. I'll keep you posted as to our progress.

Learn Objective-C for Java Developers

The Learning iPad SDK book I'm writing assumes that readers have little or no experience developing applications for Apple products. In particular it doesn't assume a lot of prior knowledge about Objective-C, Cocoa, or XCode. Rather than get bogged down writing definitive guides to Objective-C, Cocoa, and XCode I've adopted the strategy of explaining the basics while pointing readers to more thorough coverage of these areas.

One of the books I'll be leaning on a lot is Learning Objective-C for Java Developers by James Bucanek. The book is helping me to choose what to explain and what to defer to the book itself. I've read a lot of technical books and wrote a few of them myself so I'm sensitive to the quality of writing when reading. I have to say that Learn Objective-C for Java Developers is really excellent.

If you are a Java developer and you want to get up to speed on developing applications for the iPhone, iPad and Mac OS X than you absolutely must buy Bucanek's book. It's a must have on your shelf and it will become your primary resource for understanding the programming language used to develop applications for Apple devices.

Gestural Interfaces Must Be Part of a Multi-Modal Experience

In-the-air gestural interfaces, which are also called Spacial Interfaces, have some advantages and a lot of drawbacks. It's difficult for me see in-the-air gestures being used as the only mode of interaction. They will need to be augmented by other interfaces such as touch or speech in order to be effective.

Nicolas Nova posted an great pros & cons slide about gestural interfaces. While the pros are wonderful, the cons can be horrible. In particular fatigue and "immersion syndrome" seem to be really big problems.

I can't imagine using an interface that can only responded to in-the-air gestures. The potential for error seems enormous and I think users would find it frustrating. I can see, however, in-the-air gestures used in combination with a tangible like the Wii remote (that obviously works) or with voice commands to prompt interpretation of a gesture. I suspect that over time combining in-the-air gestures with tangibles (e.g. wands) and direct voice will become the most common multi-modal solution when using in-the-air gestures. You just can't make in-the-air gestural systems work without augmenting them with another mode of interaction.

Even Project Natal, which is probably one of the most advanced in-the-air gestural interfaces I've seen, uses multi-modal input. For example, Project Natal (or at least the demos they showed last year) used facial recognition and voice commands in its interfaces. While Project Natal is very exciting, I have some serious doubts about using in-the-air gesture interfaces as the primary mode of input to an interface.

Update on the iPad

Well the first iPad commercial aired during the Oscars, which I didn't see because I don't have a TV and I don't much care about who wins. The commercial is pretty cool but as demonstrated in the YouTube! video The SECRETS of the Apple iPad Ad its probably been digitally enhanced. The instant movie start up was the funnest in my opinion (see video below).

There has been some noise about Microsoft Courier being an iPad killer. While I like the Courier concept I don't see it as competing directly with the iPad. It's market seems smaller to me but I still can't wait see how well it works if it ever comes out. The fact that it relies so heavily on digital stylus worries me as my experience with that technology on the Windows platform as been very poor.

I'm already hard at work on my new book Learning iPad SDK and things are going well. I've been using the beta that was pulled and I'll keep using it until they post another.

I've been thinking of how iPhone apps will port to iPad in terms of layout and I have to agree with John Gruber at Daring Fireball that the experience will be, for the most part, less than awesome. If you want your application to work well on the iPad than you probably should rewrite it for that form factor otherwise the experience will be lacking.

Magazines to iPad: Just Scratching the Surface

Interview magazine, which was created by Andy Warhol 40 years ago has released a demo video of its upcoming iPad edition. If you are into photography its going to be great, but in my opinion Interview magazine's take on the iPad as a publishing platform only scratches the surface.

The iPad is so powerful that we can only begin to guess what publishers will do with the platform. A good article about realizing the full potential can be found here. The author of that article believes, as I do, that simply replicating the paper experience on the iPad is just the beginning.

At this point I'm actually talking seriously to a publisher about moving their content to the iPad and yesterday I was contacted by another different kind of publisher on this very same subject. To say that publishers are excited about the possibilities would be an understatement. The potential of the iPad has revitalized the creative thinking of many publishers. It's going to be a wonderful and wild ride as we all try to figure out the best approach for publishing different content on the iPad.

Blake's Definition of NUI

My friend Joshua Blake is, as noted in my last post, in the midst for writing a book on explaining how to develop multi-touch applications on Windows 7 and with Silverlight 3. Although that is the objective of the book (the cover is show on the left), his first paragraph is much more ambitious. It attempts to define Natural User Interfaces and to provide guidelines for the development of those interfaces.

As noted Josh his own blog post on the subject, I've taken a crack at the definition of NUI a few times with limited success. As is the case with so many new ways of doing things (e.g. Web 2.0, REST, Cloud Computing) defining it can prove difficult. NUI definitely falls into this category of technologies.

In Josh's new book he defines NUI as follows in his first chapter (you can read the chapter yourself here).

A natural user interface is a user interface designed to use natural human behaviors for interacting directly with content.

By itself the definition is not much clearer than others, but in his new book, Josh explains a lot about what has gone into this definition and his thoughts and writing are really very interesting. I really like the way he builds on Bill Buxton's off the cuff defintion of NUI as:

A user interface that "exploits skills that we have acquired through a lifetime of living in the world."

In particular Josh explains what Buxton may have meant by "skills" in a section titled "1.2.1 What are innate abilities and learned skills?" Here he explains that humans rely on both innate skills with which we are hard-wired as well as learned skills. For example, pushing things around with your fingers on a surface is an innate skill. Nearly all humans can do this and will be able to do it without in training. Learned skills, on the other hand, can be simple or complex. For example, using a mouse with a computer is a learned complex skill while learning the meaning of an arrow as an indicator of direction might be a simple skill. Simple skills are reusable while complex skills are not.

All that is very interesting as well as his writings about cognitive load, but what I like to focus on is innate vs learned skills and their importance to the definition of NUI. In particular I would like to see Josh change his definition of NUI slightly to read as follows:

A natural user interface is a user interface designed exploit both innate and learned skills for interacting directly with content.

In this definition I've combined Josh Blake's "behavior" definition with Buxton's "lifetime skills" definition to produce what I think is a pretty solid and clear definition. I left out "Humans" as this seems redundant - obviously we are talking about humans and not chimps or squirrels or whatever.

My next book: Learning iPad SDK

Update 4-12-2010

I'm sorry to say that I had to cancel this book project. My current workload developing iPad applications for clients prevents me from writing about iPad development. I suspect there are a lot of iPad development books in the works which will prove to be excellent resources for beginning iPad developers. Thanks to Brian Jespen and everyone else at O'Reilly Media for their kindness.

---

The ink is not even dry but I've already been shown the cover to my next book, Learning iPad SDK, which I'm writing for O'Reilly Media. Much thanks to Brian Jepson for helping to make it happen. Now I just have to write the book!

I'm pretty excited about writing this book. It's intended to help those who have never programmed for the iPhone get quickly up to speed and productive with the iPhone 3.2 SDK specifically for the iPad, which we are just referring to as the iPad SDK to avoid confusion.

My Animal for this book is a Cape Fox, which is pretty cool. I'm looking forward to working with Brian and getting a book out there early - there is going to be a huge level of interest in programming for the iPad. Hopefully, we'll be able to post some rough cuts for early feedback.

That reminds me, my digital friend Joshua Blake is writing his own book for Manning on touch programming for Windows 7. Check it out!