Windows 7 Will Take Multitouch Mainstream

Ars technica broke the story a few days ago that Microsoft would be publishing the RC (release candidate) for Windows 7 in May.

The RC is kind of like a beta+ and usually Microsoft will have a couple of them (i.e. RC1, RC2), but they've said that for Windows 7 there will only be one RC, so that means that the final release of Windows 7 isn't far away (probably this winter as suggested in an earlier blog post).

The thing that people may not realize is that Windows 7 is going to take multitouch to the masses in the same way that the iPhone did, only bigger. Not everyone owns an iPhone, but most of the world does their computing on Windows so if Windows 7 really is much better than Vista (personally I like Vista, but I seem to be in the minority) than its only a matter of a few years before multitouch is everywhere there's a computer.

If you've been thinking of learning multitouch programming and have been holding off, now is the best time to get started. Get a head of the herd so that when the wave of multitouch demand hits the IT industry you are ready and able to pick among the best projects. The availability of skilled and experienced multitouch designers and developers will lag demand which creates scarcity. That's a good thing if you are the scarce resource.

Update April 1, 2009

A short article by Fudzilla says that manufactures are gearing up for Windows 7.

"According to several Taiwanese LCD manufacturers, the second half of this year will bring a variety of capacitive multi-touch panels for use netbook and notebook markets. All in all, the firms are readying themselves for a highly anticipated Windows 7 launch later this year."

A Survey Of Digital Camera UIs

Thanks to Lynn Marentette for bring this great article from Gizmodo on the graphical interfaces - many of the touch - currently in use on digital cameras.

I don't usually talk much about digital cameras and touch interfaces because they are single-touch and also its not of great interest to me personally.

That said, I did enjoy reading this all-in-one review of what's out there. In my opinion, the digital camera industry needs a serious injection of UI design inspiration.

Phenomblue Sushi Surface Application

Walt Ritscher of WPF Wonderland posted about a cool Surface application developed by Phenomblue for Sushi restaurants (video).

There are a lot of things I like about this application. It's oriented for two-sided interaction, something I think is appropriate for a Sushi bar, and provides some nice animations and great video content. As someone who is a total neophyte when it comes to Sushi, I would love to have this when I eat out. Nice work guys!

Tesla Model S Car with Large Touch Screen

Tesla Motors, the Silcon Vally start up that makes high-performance all-electric cars, will be unwrapping the Model S concept car soon (it was leaked last night) which sports a large touch screen in the middle of the console. Some great pictures and a short story can be found at Gizmodo and Gawker respectively.

While I love the idea of multitouch on on all of our surfaces, I can't help but wonder if multitouch screens as the primary console for cars is a bit dangerious. I don't know about you, but I would prefer more tangable controlers which can be operated without looking. A little multitouch is a good thing, but ...

TacTable: Multitouch Tabel

It's always fun to discover new multitouch tables, which are my favorite multitouch interface. Thanks to this blog post by Operand, I've just discovered TacTable which comes in two sizes 30" x 40" and 60" x 40".

The device also supports multiple tagged objects, just like Surface, which is often a "must have" when it comes to interacting with multitouch tables.

According to Eric at Operand, the TacTable predates Surface and iPhone having first been commercially available four years ago. I couldn't confirm this using the way back machine, but I have an email out to the company for more details on that, device availability, and developer interface. I'll report back when I hear from them.

Gesture Dial

One of the coolest iPhone applications, in terms of multitouch, is Gesture Dial. The concept is simple and the engineering is excellent. Instead of looking up your "fab 5" assign your favorite contacts gestures on your phone. All you do is click on Gesture Dial, enter the right gesture and voila! Gesture Dial calls that person. As an alternative it can open up text messaging or email for a specific contact.

I had my doubts about the accuracy but its actually really good and the administration of gestures is really easy. I used gestures that looked like letters: "h" for home "Z" for my friend Zack and "M" from my friend Mike.

There is another gesture dialing software for iPhone that supports 16 pre-defined gestures, GDial, which might be better for people who don't want try to invent their own gestures. I prefer Gesture Dial because its easier to learn and use.

One of the biggest hurdles of multitouch, in my opinion, is learning new gestures. Applications that allow us to assign our own gestures to functionality - be it dialing a phone or something more complex - are an excellent option.

Eventually, de facto standards for gestures and their meaning will become common place (e.g. Pinch), but its going to be a while before we have enough de facto standards that people can easily move from one productivity application to another. Discoverable gestures are critical for one-off or sometimes-used applications, but productivity applications will require discoverable gestures as well as an arsenal of learned gestures.

Real Time Strategy Game on a Multitouch Table

Ubisoft has posted a promotional video for a new game R.U.S.E. which they will be releasing soon.

The video shows two guys playing the Real Time Strategy (RTS) War game on a large multitouch table. The visuals are stunning.

The product being released by Ubisoft won't be a multitouch table nor will it be playable on a Microsoft Surface - its being released for XBox 360 and Windows PCs - so the video is just a gimmick, but its a good one. It's a glimpse into the possible future of video games on multitouch tables. It's pretty cool to say the least.

Blackberry Storm's Haptic Feedback

The Mobile Spoon has an informative review of the Blackberry Storm which includes the pros and cons of the haptic (touch ) feedback mechanism it employs. There is also a great techno-porno shot of the inside of the Blackberry Storm showing a single bubble button on the back that provides the sensation of depressing individual areas of the screen. According to the review it works well but not so much at the perifery of the screen which makes sense. Still its an elegantly simple solution.

What I really like about this solution is not only the feeling of clicking, which doesn't really have value in anything other than tap and hold type gestures, but the fact that you can hover over an item, get feedback with highlilght, without activating it. That's something that the iPhone doesn't handle very well. Nice work guys!

Microsoft Sustainablity: Another Vision of the Future

Microsoft has published another extremely cool vision of the future called, Microsoft Sustainability. Although it starts out the same as the "2019" video and was part of the montage of images in "2019" it shows a lot of new stuff that's really cool. If you enjoyed "2019", "The Future of Healthcare", and other Microsoft Future visions, than you are going to love this latest video titled. It's awesome.

In this video you'll find all kinds of technologies imagined as mainstream that are currently in their infancy including multitouch (of course) and NanoTouch, augmented reality, e-paper, and more. It's inspiring to watch.

What I love about this video is that a lot of these technologies, multitouch, augmented reality, and so on are things we've been looking at my work at CSG for real customers today. As I've said before we are not that far off from seeing many of these visions realized - some can be done right now.

In an video interview with Ian Sands by Larry Larson, you can learn more about the creative process that goes into making Microsoft's future videos. There is also a good written interview which is really good but I can't seem to find again (help would be appreciated).

There another video by Bruce Branit which is very cool, but it should be titled "2060" because it will take about 50 years to create a virtual, tangible, city. Still its cool to watch.

IdentityMine Gesture Engine for Win 7 and Surface

IdentityMine announced a new product, IdentityMine Gesture Engine. Details are a bit light thus far.

Gesture Engine allows you to define more complex "multi-level cooperative" gestures. I'm not exactly sure what that means except perhaps that you can add gestures like drawing a question mark (?) or rubbing or other complex gestures. I have a call in to IdentityMine to get a clarification.

Assuming the above is correct this would be a nice tool to have on hand. I've built complex gestures but its a pain on Surface and each one requires custom code. I would much rather have a framework that supports complex gestures. Another cool thing about the Gesture Engine is that it can record gestures and use them as heuristics for identifying new gestures. That's really powerful, but I'm not sure how well it works - perhaps I'll get a chance to try it out when the software actually becomes available.

Struktable: the 70-inch Multitouch Table

Strukt a events and exhibitions design company in Austria demonstrated their Struktable multitouch table with a whopping 70-inch display. The device, of which there is only one at this time, is for rent or you can have one built for purchase.

Thanks to Luxuo for the heads up on this beautiful machine. There is a video of the device in action. Warning the video uses the worst sound track I've ever heard, but just hit mute and enjoy it anyway.

Update March 20, 2009

Thomas Hitthaler of Strukt was kind enough to return my email and answer some more questions about Struktable - here is what he said (with a little bit of editing and sans the intro and sign-off):

The Struktable runs on Windows XP and the Multitouch framework consists of two applications:

1. Tracker: a standalone app that handles calibration and blob tracking. That data is sent via OSC (or any other network protocol) to the development platform. We do also support APIs for C++, C# and vvvv(vvvv.org) that circumvent the few millisecond delay caused by sending network packages to the localhost.

2. In-house we are using vvvv to develop our applications. You can try it for free (http://www.vvvv.org/tiki-download_file.php?fileId=1861). Right now we are experimenting with Flash applications on the table. Works well... Any other development platform should work as well.

In principle we can build the Struktable out of any desired material. Right now we are using MDF with an steel cover. The surface consists of safety glass and a few other layers.

Microsoft Says New Version of Surface May Be Larger

In an interview with Matt Champagne, Microsoft Surface’s director of product management, it was revealed that larger size surface screens are under consideration.

"We are looking at a 46-inch model to complement the 30-inch version we already have," Champagne told Pocket-lint.

The current screen size is 30" (diagonal) so a 46" screen would be over 150% larger. And while Microsoft has had requests for even larger screens Champagne told Pocket-lint, "We are unlikely to go beyond 50-inches or 60-inches as in doing so changes the dynamic too much."

Personally, I would love to see have access to a larger Surface screen as I find the current size is sometimes pretty limiting.

Silverlight 3.0 Supports Multitouch

At MIX09 Microsoft announced that Silverlight 3.0, now in beta, will support multitouch!

In addition, there will be desktop version of Silverlight similar to Adobe AIR (I called that one) and they have made a Silverlight Eclipse plug-in for Mac users to develop Silverlight content (I never saw that coming).

Now that Silverlight 3.0 supports multitouch and Safari browser supports multitouch, when will Adobe get into the game?

Razorfish's Carville Surface Attract Screen

Personally, I've grown tired of the default attract screen offered by Microsoft with Surface. Yes its pretty cool, but I think its about time we pushed the envelope and did something other than an pond. Just adding animated Koi would be an improvement.

In a blog post from headlightblog.com, a video shows how the folks at Razorfish took the challenge and developed a new attract screen for a fictitious auto dealership. At first I sighed when I saw the attract screen: "Can't we please get serious with this device!" But after watching the video for a while it occurred to me that if you want people to walk over to the Surface at an auto dealership, than having little cars racing around a track is a great way to do it.

Another nice feature I liked about Carville was when the demonstration switched to a mock up showing how to implement a 180 degree orientation where users are on opposite sides of the machine. It was simple with combination of free floating and docked items, and it worked. Designing all Surface applications to have a 360 degree view is effective but its not always necessary and sometimes its counter productive. Nice work guys!

Samsung Omnia HD: How not to do touch screen on a mobile

Samsung makes some nice phones and I do like the haptics they built in to at least one of the trouch screen models, but I'm not a big fan of the Samsung Omnia HD i8910. The Omnia HD is Samsung new, very large, touch screen mobile phone based on the Symbian operating system.

If you watch this video you can see just how clumsy and unresponsive the touch screen on this phone is. Perhaps it will improve but the phone has bigger problems than that.

Notice in the video how the icons are dragged off the bar onto the main screen and then touched to activate. This is an great example of touch done wrong. Why do you need to drag the icon off the side bar before you can click it? There are other issues with its touch design but that's the one that really bothered me the most.

Drag and drop is a powerful feature in touch and multitouch applications but it really seems to be one of the most abused features as well.

iPhone OS 3.0 Announced

Well, Apple finally added copy-and-past to the Jesus phone and its about time. Today Apple announced the features available in the iPhone 3.0 operating system update and from a developer and user perspective they are pretty nice and much anticipated.

As usual Engadget has the best coverage the the operating system update. I think Apple put a lot of thought into copy-and-past - I like the double tap, menu, and shake to undo gestures they used. Better late than never! They also add phone-to-phone communication through bluetooth and access to Google maps from an SDK perspective.

According to AppleInsider a beta version of iPhone 3.0 OS will be available to registered developers here (it was not up when I posted this, however).

Microsoft Surface, version 2 Rumor

In what can only be described has pure rumor, techviews.org posted a story about the next version of Microsoft Surface being released later this year.

From what I can tell the mainstream press took a quote from Erik Kilmczak from that story and applied a liberal amount of creative license in some cases saying that SecondLight was the foundation of the new version of Surface due out this year, or in three years, depending on what article you are reading.

The actual original quote has Erik Kilmczack saying that Surface V2 would ship with higher resolution projection and camera input, which would be a very welcome upgrade. This seems more likely to me than the introduction of Second Light which I would expect to make an apperance next year or the year after.

The point is, if you want to get the source of the Surface v2 rumor just read the post made by techviews.org. Erik Kilmczack may be guessing or maybe he has inside information, but either way the news should be taken with a grain of salt.

Multitouch Resource for Flash Developers

I've mentioned the importance of Flash as a multitouch platform a couple of times in my blog.

For Flash developers interested in multitouch check out Christopher Yanc's web site, Idea Cyan[c] Design, he has been blogging about building his own multitouch surface and using tBeta and Touchlib along with Flash to develop his own multitouch applications.

He talks about his foray into multitouch on his blog and provides some nice insights into his experiences and how he got started. He's an excellent Flash developer and his blog is worth reading.

Imano Restaurant in London

Coolest Gadgets has an interesting story and video about the Imano Restaurant in London which features a multitouch table on which you also eat.

I'm not a big fan of over-head projection when it comes to multitouch and I would have thought that it would be a horrible experience in a restaurant with projections of bouncing balls and plasma on your food while you eat, but somehow the folks that implemented the over-head projection system set it up so it would not project on people's plates. That makes the over-head projection system more enjoyable. Over-projection also costs a lot less and you don't have to worry about wear and tear on the multitouch surface itself - its just a table after all.

Here are some great pictures of the restaurant complements of Wired.

iPhone applications banned by Google

One of my favorite iPhone development teams, innerfence, which developed the credit care processing iPhone application I blogged about back in December 2008 has done it again - well almost.

innerfence developed an iPhone application that gave users unlimited free SMS texting by accessing a free Google SMS service, which was originaly intended for its Google Talk instant messaging service.

The iPhone application, called Infinit SMS, was so popular that it quickly overwhelmed Google's beta SMS service driving Google to cut off access to its free SMS service to all thrid-party applications.

The guys a innerfence, Derek Del Conte and Ryan Johnson, explain the circumstances of Google's actions as "Their given reason for the block isn't abuse or wrongdoing; it's that we brought too many users (and thus too much cost) to an experimental service."

Derek and Ryan have developed other really useful applications such as a International Rates, which gives you the rates AT&T charges for international calls, and Excess Baggage which gives you the rates each Airline charges for excess baggage. innerfence is focus on applicaitons that are actually useful. I hope they continue to develop more of these awesome applications. (thanks to Slashdot for the heads up on this one).

iTunes App Store Needs Transparency

If you been involved in mobile application development then you probably know all about "closed gardens" which prevent or make it extremely difficult for developer to deploy commercial applications on many mobile phones and networks.

Apple iTunes, perhaps the most popular and successful mobile phone application market place ever, is also a closed garden. The nightmare stories about developers applications not getting approved are becoming legendary and the problem seems to only be getting worse. This article from PopSci.com does an excellent job of explaining the problem and providing solutions.

The bottom line is that Apple is killing the hands that feed them by making the process of approving applications completely opaque. What is needed is a clear process by which developers can track the progress of their submission through the App Store approval process.

Update March 17th, 2009

According to AppleInsider Greg Joswiak, Apple's VP of product marketing, said that " that Apple has managed to approve 96% of apps submitted this month, and that 98% of those apps were approved in less than 7 days.". If that's true than I think, transparent or not, Apple's iTunes App store is perhaps the easiest closed garden to get into in the history of mobile development.

The Palm Pre: A True iPhone Challenger

There are a lot of new touch-screen mobile phones coming out, but so far I haven't really seen anything that rivals the iPhone as a mobile phone multitouch device.

The more I look at the Palm Pre, however, the more excited I get about the possibilities. First, its really a nice looking phone and its UI is awesome - at least on par with the iPhone in my opinion.

Second, it's not locked down to a vendor's application store and there is an SDK that will be made available at the same time its released or shortly there after.

Third, it simply has some features that are better than the iPhone which is not a big feat since the iPhone is a pretty limited smart-phone. When you combine the additional features of the Pre with its beautiful multitouch UI you get a true challenger. I'm looking forward to getting one myself and learning how to develop applications for it - just as soon as my iPhone contract runs out and I've exhausted myself programming with Objective-C and Cocoa.

You can see some nice videos and find out a lot more at Engadget - if you are a Jimmy Fallan fan you might also enjoy this apperance by Engadget's own Joshua Topolsky showing off the Palm Pre.

Apple iTouch

Unless you've been ignoring Engadget and the most of the technology press, you probably already know about the rumors that Apple will be coming out with a touch screen device with a 10" screen in the second half of the year.

All of the speculation is based on an article posted by Reuters on Wednesday this week which got it's information from a Taiwan-based magazine. Apparently Wintek, the company that manufactures the iPhone screens for Apple, will be delivering 10" touch screens to Apple this summer for some, yet unnamed, product.

At first all the news outlets predicted it would be a Netbook but a lot of smart people consider that to be unlikely. The most likely product, if there is one, would be a tablet form factor with multitouch. To me this seems the most likely. I also think that the device will leverage the iPhone SDK and its UI will look like a cross between the UI for Mac OS X and the iPhone.

To really make a splash the tablet would need to support both multitouch and a stylus - so that it can be used for both precision drawing and less precise touching. It will be interesting to see what comes out of all this. With Safari now supporting multitouch gestures with JavaScript and Apple having established its multitouch SDK and its App Store application market place, I think a tablet - long desired but never successfully implemented for mainstream adoption - would be a great new product for the company.

If you take a look at the Air, which is not a tablet but a laptop, Apple has already gained some much need experience with the ultra-thin larger form factor. I think they'll leverage that experience to create a tablet that is thinner than Air but thicker than an iPhone. The device will include a touchscreen and a capacitance compatible stylus, and will pretty much be all screen. The device will probably sport the Snow Leopard operating system but its interface will be more like the iPhone than a Mac.

It's also likely that they will not call it a tablet or, God forbid, a "Netbook". Apple doesn't like mainstream category labels. Their iPhone has never been called a "smart-phone" by Apple and their laptops are called "PowerBooks", "Mac Books" but never "laptops" by Apple. I suspect they'll use "iTouch" as the product name - if some other company owns the "iTouch" name they'll just license it like they licensed the brand name "iPhone" from Cisco.

What's cool, as a blogger, is that if I'm right I can always point back to this blog as demonstrating how far sighted I am. If I'm wrong: no one will remember this post anyway. ;-)

Adobe on Multitouch

Adobe has produced a video that is at once illuminating and less than informative about Adobe's plans for integrating multitouch into their products and Flash player.

It's illuminating because the interviews with Senior Experience Designer Julie Meridian and Senior Computer Scientist Tim Kukulski shows that Adobe is actively working on not only enabling multitouch in its design products but also as a feature of Flash itself. They also say that this type of functionality will find its way into the Adobe family of open source projects, which was good to hear.

Although the video is really well produced and it does give some insights into Adobe commitment to multitouch, it provides very little in the way of details about how Adobe's multitouch capabilities would be manifested in its products and its Flash SDK.

In my opinion the faster Adobe gets into the multitouch game the better. With millions of Flash and Flex developers out there an Adobe ActionScript multitouch API would have instant developer uptake. In addition, the idea of designers using tools like Photoshop and Illustrator (CS6?) with a stylus and multitouch directly on a draft-table form factor is just plain exciting.

Mitsubishi's 3D touch panel

Engadget has an interesting, if brief, article on a new touch screen technology demonstrated by Mitsubishi that can detect fingers above the screen as well as their trajectory and rate of speed.

There are probably lots of ways to take advantage of this but the one that peaked my interest was the ability to have a "mouse over" like gesture in multitouch - something that you really can't do right now. The "touch above", if you will, would allow the application to provide clues (afforances) as to what an object does without actually touching the screen. This is useful because the afforance would only appear when the finger is hovering over the controler rather than being present all the time. Think of it as a Just-In-Time afforance.

Coldwell Banker's Real Estate Surface App

Coldwell Banker has announced the development of its own Surface application designed to help customers find homes and learn about candidate neighborhoods. You can see it in this video.

Before I started working for CSG I was actually in talks with a realtor to develop a very similar application. This kind of thing could be effective at bringing people into the Coldwell Banker office, but as I understand it brick-and-mortar locations for realtors are pretty much a thing of the past.

Most realtors work out of their home (or more likely their car) and interact with customers via mobile phones and email. This is why the realtor I spoke to decided not to pursue a Surface solution.

It will be interesting to see if Coldwell Banker's implementation has a real ROI or if its just a gimmick that ends up having no positive impact on the revenues . That's an important question because developing Surface applications is not cheap. It would not surprise me if it cost between 250k and 500k to develop the one demonstrated in the promotional video.

Flexible E-Ink and Touch

I'm a bit behind the ball on this one but I wanted to blog on this technology created by E Ink which includes support for touch. It's pretty cool and they expect the first product based on it, Readius, to available later this year.

This is the kind of technology that will lead to universal credit cards and ID cards, another concept that will be enabled by multitouch technology.

Stimulant on NUI, Mulitouch, and Surface

There are a series of video interviews with three designers at Stimulant, a Microsoft Surface partner, that are pretty illuminating. In the interviews, the three designers are questioned by Eric Havir of Microsoft in what reminds me of the Dating Game.

It's obvious to me that the Bachelor #2 is going to win Eric's heart as he does most of the talking. But Bachelor #2 is also pretty smart and his delivery and expression of concepts is excellent so I guess he deserves the pleasure of Eric's company on an ALL EXPENSES PAID DATE TO REDMOND!

Surface Now Available in Parts Europe and Middle East

Microsoft announced yesterday that the Microsoft Surface has become available in parts of Europe and in two countries in the Middle East.

"Microsoft Corp. announced expanded commercial availability of Microsoft Surface to 12 select markets in Europe, Middle East and Africa (EMEA) and has broadened its developer partner ecosystem to support those additional markets with local, relevant content. Previously available for purchase by companies in the U.S. and Canada, Microsoft Surface is now also available in Austria, Belgium, France, Germany, Ireland, Italy, Netherlands, Qatar, Spain, Sweden, the United Arab Emirates and the U.K."

The Future is Now!

I like to think of multitouch as an enabling technology for innovation on many levels of human-computer interaction (HCI). I think few people would disagree that products like the iPhone and Microsoft Surface are changing the market landscape, but there is so much more. This is what makes multitouch the most exciting HCI technology today.

The photo left is a design concept by Mac Funamizu an graphic artist from Japan (and likely a future industrial designer) based on the concept of augmented reality. Augmented reality is an idea that's been around for a long time but only recently with advances in imaging, display, and multitouch technologies is it starting to become a reality. Check out this story on real augmented reality solutions be developed right now.

Form factor of the devices we use has benefited the most from multitouch today. The iPhone and the Microsoft Surface are two examples of commercial products that are redefining end-users expectations, but there are others. Take for example, the Dual Screen that is supposed to the future of OLPC is becoming a reality with Asus, or the detachable touch screen laptop demoed by Always Innovating, touch screen cameras, music players, and even printers.

I could go on-and-on, but there is a lot other multitouch stories I want to tell not to mention real-world applications I need to develop! I welcome readers to post links that show how multitouch is changing the way we live today.

Update 20 minutes later

After making this short post I found out that the entire Blogesphere is talking about a video by Microsoft, called "2019", which is really cool. I don't know how accurate it will be but a couple of the ideas in the video are things we are already implementing right now at CSG for our customers - so you won't have to wait until 2019 to see a lot of this stuff become reality.

Updated March 20, 2009

For folks interested in the production of the video "2019" there is a great interview with its co-director complements of istartedsomething.com.