Monday, January 26, 2009

Apple Successfully Patents Multi-touch

According to the US Patent office records, last week on January 20th, 2009, Apple Inc. was awarded a patent to what is essentially multi-touch. If I understand the patent correctly this will have far reaching ramifications to the multi-touch industry.

Patent Number:


Patent Title:
Touch screen device, method, and graphical user interface for determining commands by applying heuristics

Patent Abstract:
A computer-implemented method for use in conjunction with a computing device with a touch screen display comprises: detecting one or more finger contacts with the touch screen display, applying one or more heuristics to the one or more finger contacts to determine a command for the device, and processing the command. The one or more heuristics comprise: a heuristic for determining that the one or more finger contacts correspond to a one-dimensional vertical screen scrolling command, a heuristic for determining that the one or more finger contacts correspond to a two-dimensional screen translation command, and a heuristic for determining that the one or more finger contacts correspond to a command to transition from displaying a respective item in a set of items to displaying a next item in the set of items.

I’m not an expert or even a novice at interpreting patents but as I read it they have patented not only multi-touch in general but also many of the gestures used in iPhone and elsewhere (e.g. Microsoft Surface), such as pinching, swipe, tap, rotation, and others. If this is the foundation of threats of legal action against Palm than its also the foundation of legal threats to all multi-touch devices. It’s scary. I would greatly appreciate comments from people who can clarify the meaning of the patent and its implications.

UPDATE January 29th, 2008

Engadget has a pretty good article on the Apple patent - probably the best analysis I've seen so far. It puts my mind to rest to some degree as it seems the patent is very focused on specific gestures rather than covering any and all multi-touch gestures which is what I was afraid of. Nice reporting guys!

UPDATE: January 27th, 2009 (morning)
Based on a comment on my posting on the forums and many comments on a post, it seems that the patent is for "heuristics" which is essentially the interpretation of gestures on the touch screen into actions on the interface. So, for example, the idea of placing your fingers on the screen and swiping down to scroll through a list is a heuristic of multi-touch; the interpretation of a finger contact gliding down the surface of a screen resulting in a scrolling motion. That's really bad for the industry.

Multi-touch is still new to the mass consumer market and the use of standardized gestures is key to the success of the technology. Take the scroll motion above. It's intuitive and natural way to scroll through a list based on decades of experience of using scroll bars on Windows, right? Now imagine that every multi-touch phone had to implement this gesture differently. What if on the Android phone you have to rotate your finger in a circle while on your Nokia phone you have to tap three times. That's a pretty ugly situation and if it had existed with WIMP (Windows, Icons, Mouse, and Pointers) it would have severely injured the development of the graphical computer interface. But that didn’t happen with WIMP. Instead, all WIMP-based interfaces treat scrolling the same way; you click on the scroll handle and drag it the direction you want to scroll. It's the same on a Mac, Windows, GNOME & KDE (Linux), X11 (Unix) and just about every other WIMP interface.

If Apple had patented just a few gestures that would have been bad but tenable; instead they seem to have patented the very idea of multi-touch gestures. That moving your fingers on a touch sensitive surface can be interpreted as some kind of interaction with the user interface. That's absolutely horrible and quite possibly the worst possible outcome. How will any other company be able to implement multi-touch (including Microsoft with its Surface device) if Apple holds patent on the very concept of gestures in multi-touch?

If my analysis is correct there is going to be one of two outcomes. Apple will license the technology to all comers which will be tolerable but also will hinder development of multi-touch systems by open source and small companies; OR Apple will simply use its patent to completely corner the market on multi-touch in general (not just phones but multi-touch on any device). The later scenario would result in serious injury to the advancement of multi-touch. I find it hard to believe that this type of patent will stand up in court given the abundance of prior art dating back at least to 1982. In my opinion, it’s up to Microsoft at this point to challenge the patent – if they don’t who will?


Anonymous said...

How is this possible given the prior work from bell labs and other academic work (eg Peter Hutterer and Jeff Han)? How do you see this impacting the marketplace, esp multi-touch capable software like android? In your opinion how does this reflect on the patent process?

Richard Monson-Haefel said...

I would like to think that this sort of patent will not hold up in court - that the US Patent Office will reverse its decision. However, given the long standing "One-Click" patent that may be wishful thinking.

Anonymous said...

The EFF has a program dealing with busting patents that it thinks are too broad, if you have ideas of prior art that seem to fit the patent then send it their way.

Anonymous said...

One hopes that this patent will be challenged, successfully, and the concept of multi-touch be an "open" User interface/interaction concept.

That said... can you blame Apple? Broken or not, this is the way the system works. Without an overhaul of the underlying concepts of business patent law, directing hate at Apple (or Amazon, or Microsoft, or Palm) is useless.

Anonymous said...

Are you seriously relying on Slashdot for valid information?

anirudh said...

you say that apple has patented Multitouch, does that hamper the chances of other console's Multitouch research? such as that of Jeff Han's Perspective Pixel

Anonymous said...

Isn't this akin to patenting intuition? Like other commentors, this should stand no chance of standing up in court.

Placing a finger on something and moving it in the direction you want a list or picture or item to move is basic human instinct. We learn it as a child. It should not be available to patented.

If it does, we're destined to abandon fingers, and use wheat crunchies to navigate our multi touch devices:

Daveed said...

The patent is quite careful to patent only aspects of multitouch that weren't prior art. E.g., dragging an object or rotating doesn't appear to be claimed by the patent. However, the iPhone "flick" effect to browse long lists is claimed, and certain specific rotation scenarios (e.g., rotating a document 90 degrees without actually performing a complete 90-degree gesture) appear claimed too.

IANAL, but my sense is that most of the claims can resist legal challenge.

Anonymous said...

Or maybe, just maybe, the patent will help make unify all the gestures... maybe apple did this to say these are the ways everyone should use it... We got rules, now use them OR ELSE! haha

Richard Monson-Haefel said...

This isn't about Apple bashing - I love Apple products - its about making people aware that a commercial entity has patented a technology that fundamental to the growth of an industry.

I don't blame Apple one bit. As some said, they are simply using the system to their advantage something I think most people would do in the same situation had they thought of it.

At the same time I don't think Apple's patent is good for the industry as a whole and I don't think they are doing it for the good of multi-touch in general. I think they are doing what companies are supposed to do, make profit.

The question is do we all roll over and allow Apple to own this intellectual property or not? I can't do anything on my own other then raise awareness but someone like Microsoft, who would be acting in their own interests, could put a stop to it. They have the resources to do that.

Jeff said...

I can understand why this is bad. I can also understand why this is bad for innovation or competition in the touch industry. However, Apple was first to market with this. Up until then, phones and computers were using joysticks and circular navigation keys. After the iPhone practically every competitor was falling over itself to implement such techniques (and I'm sure that any company could have been capable to lead here before Apple). So, to the first the rewards (as ugly as they may be to the industry).

Yes, the Mac GUI originated in XeroxPARC, but it was never implemented. Why was it fair that MS was allowed to successfully rip off that which Apple had the balls to develop? Windows 1 thru 3 were abominations until they tread on Apple's development (yes Apple lost; and they hadn't properly patented any of there IP).

Yes there is prior art for Touch, but none of it has previously come together, coalesced or found a usefulness until Apple has show the Industry how.

Why shouldn't Apple have a stranglehold? (As bad I think this is for the Industry, it's the industries own rot in the first place for not innovating and bringing this to fruition). Apple doesn't deserve to show the industry how it is done and then have that taken away by bumblers who couldn't put it together in the first place.

(This is not a dis to all the brilliant people who have put concepts down on paper or worked hard on the ideas in the labs. All the brilliant people at XeroxPARC did not end up making their management leaders today).

Just my thoughts.

Rip Ragged said...

You're joking, right?

Apple has:
1. The Patent
2. $30,000,000,000.00 in cash
3. No debt

The rest of the industry (including Microsoft) has:
1. Dreams of the patent
2. No money
3. Lots of debt

I'm not a legal or patent scholar, but, um, if this was a war Apple would have nuclear weapons. Everybody else is squabbling over who gets to hold the butter knife.

You go get that big bad old Apple there, little buddy.

Richard Monson-Haefel said...

So there are a couple of comments to which I need to respond.

Apple was not the first to market with multi-touch, not by a long shot, nor did they invent it. They were probably the first to really profit from it and they can be commended for that, but they shouldn't be awarded exclusive rights to the technology. They have their reward in terms hard earned cash - giving them a patent for multi-touch is really bad for our industry.

Apple is flush with cash this true but its not 30 billion, its 2 billion. Apple is still small compared to Microsoft, which has twice as much cash (4 billion compared to Apple's 2 billion), twice the Market Capitalization (160 billion compared to Apple's 83 billion) and twice the revenue (60 billion compared to Apple's 32 billion). So Apple is about half the size of Microsoft.

Rip Ragged said...

Richard, you look like a nice guy. So um, pull your head out. Apple has in excess of 35 Billion Dollars in cash, short term investments, Net receivables and inventory with no debt. It's right here:

If you continue looking you'll see that's roughly 2.5 Billion less than MSFT. If you keep looking, MSFT has about 8 Billion more in liabilities.

Richard Monson-Haefel said...

@Rip Ragged
You are right! I was wrong. It takes a big man to admit that OR just someone who has no idea what he's doing when reading an Income statement rather than the balance sheet. ;-)

That said, I still hope that Microsoft, HP, Nokia will band together to negate the patent in some way.

Jeff said...

I acknowledge that Apple "was not the first to market with multi-touch, not by a long shot, nor did they invent it." However, I feel that Apple deserves the spoils more than others who have been shown the way and are leaching.

That said, Engadget has a wonderfully sober analysis of a possible patent war which might put your mind at ease. Patents litigation, obviously, is a double-edged sword.

Maybe I'm optimistic, but maybe the Industry is safe (other than copycats like Miezu's M8).

Richard Monson-Haefel said...

I just finished reading the Engadget article and I agree its very good. It does put my mind at easy to some degree and I've posted and update to the blog to reflect that. In this case, I love being wrong. I had feared that the patent was far reaching covering all gestures but I'm relieved that it is probably much, much narrower than that. Kudos to Engadget!

Rip Ragged said...

If you've ever checked my blog, you'll know that I'm an Apple Fanboy of the first order. Given that I agree with you in one aspect. I hope somebody somewhere gives Apple some real technological competition soon.

If not, like any entity without challengers, Apple will eventually get complacent and lazy. I would hate to see that happen.