Paper Reading #13, "Mouse 2.0"

http://dlandinichi.blogspot.com/2011/03/paper-reading-14-madgets-actuating.html
http://chi2010-cskach.blogspot.com/2011/02/paper-reading-13-semfeel-user-interface.html

Mouse 2.0: Multi-touch Meets the Mouse

Nicolas Villar, Shahram Izadi, Dan Rosenfeld,

Hrvoje Benko, 

John Helmes, Jonathan Westhues,

Steve Hodges, 

Eyal Ofek, 

Alex Butler, 

Xiang Cao, and 

Billy Chen of Microsoft Research
Presentation venue not specified in paper.

Summary
This paper, as suggested in the title, presents research into expanding upon the current computer mouse. Five different implementations are presented, each incorporating "a different multi-touch sensing strategy". The authors emphasize that their goal is supplementing, not replacing the mouse; that touch screen requires more effort and creates fatigue, making it less than ideal; and that the goal of these designs is to produce a usable design allowing for a cursor with more than two degrees of freedom. The five implementations follow.

FITR Mouse (all images taken from the paper)

FITR Mouse
The acronym is for Frustrated Total Internal Reflection. The mouse consists of a curved piece of acrylic set in front of an infrared camera, which tracks the movement of fingers across the acrylic. Which movements on the surface map to what actions of the mouse is not explicitly stated. One of the diagrams makes it clear that there are physical buttons, but doesn't indicate which end of the acrylic they are located at. If the rear, the motion may be counterintuitive. The limitations noted are decreased ergonomics, and problems operating in areas with significant IR light already present. However, the mouse did accomplish basic design goals.

Orb Mouse (all images taken from the paper)

Orb Mouse

The Orb Mouse has an IR camera pointing in all directions under the dome, with the stated advantages of allowing all five fingers to be employed, avoiding broad environmental "noise" (as the IR light is generated internally here), and improved ergonomics relative to the FITR. However, the mouse is more susceptible to objects in proximity causing interference, and not just those that generate IR. It is specified that you can click the whole mouse, but not whether the mouse is limited to a single-button functionality. 

Cap Mouse (all images taken from the paper)

Cap Mouse

The Cap Mouse, or capacitive mouse, does not use the IR approach of the previous iterations but rather uses "a flexible matrix of capacitive sensing electrodes" to determine the position of the user's fingers. Cap Mouse is explicitly single button (in the front) and has the advantages of lower bandwidth required, complete immunity to ambient light, and increased compactness. However, it lacks the precision of the optical mice.

Side Mouse (all images taken from the paper)

Side Mouse

Side Mouse uses a wide angle camera lens to detect when the fingers of the user touch the table around it. It is IR, as in previous implementations described here. It is a single click device, with clicking performed by pressing down with the palm. The primary advantage of side mouse is that the scope of possible interactions is not limited by the physical size of the mouse itself.

Arty Mouse (all images taken from the paper)

Arty Mouse

The Arty Mouse, or articulated mouse, has the users place the mouse fingers on the projections, as you can see above, and then tracks the movement of those projections and the main mouse itself relative to each other by the means of an optical sensor under each element. It is the highest precision mouse described here. Arty Mouse was the most popular among test users.

The researchers offered a description of the multi-touch space which they were trying to manipulate, along with how they were attempting to manipulate it. This description, however, seems incomplete. I suspect a demonstration would be very helpful.

Finally, they outlined some conclusions of their research. Most surprising to them was that the more radical departures from conventional mice were better received. This may have been because users didn't have to relearn how to use the hardware, which was a particular problem with Cap Mouse.

Discussion

I think there are a lot of good ideas here, but overall I think that mice like this aren't going to be useful without a new paradigm of things for them to manipulate. Let me explain what I mean with an analogy.

In the current, 2DOF, mouse setup I can move around the two dimensional screen of the computer/building. In order to get to different levels, I have to go find an icon/staircase an utilize it. While a mouse with more degrees of freedom would allow me to fly around the floor, they ultimately wouldn't give me any advantage as anything I want to use is sitting on the floor and I can't to another by flying - I still have to use the stairs. Of course, if the floor was designed to accommodate flying types, being able to fly would be quite helpful. I don't which would have to be chicken and which would have to be the egg in this case, but its food for thought. (And, of course, if I know how to use a command line I can just teleport around the place.)

The details of this paper I can't really comment on too much, since I am very much a software guy and this is very much a hardware paper. The biggest weakness of this piece is a failure to have either a very clear description or a very clear demonstration of how manipulating each mouse mapped to manipulating the screen. I was unable to follow the discourse on that as written.

The next step, I think, would be to correct the flaws their research illuminated with their mice and begin to try to determine the implications of what multi-DOF mice would be. That is, what kind of software can we build to use this, and what function would that software server?