http://chi2010-cskach.blogspot.com/2011/03/paper-reading-18-speeding-pointing-in.html
Estimating User’s Engagement from Eye-gaze Behaviors in Human-Agent Conversation
I. Nakano Yukiko, Seikei University
Ryo Ishii, NTT Cyber Space Laboratories
Presented at IUI '10, February 7-10, 2010 in Hong Kong
Summary
<very technical, confusing?, long>
<discusses the basis for this paper, eye movements as feedback>
<past work summarized, very briefly>
<Experiment structure - cell phone sales WoOz>
<technical details>
<conclusion - probing + proper timing>
This is a long and very technical paper compared to those from the previous conferences. In it, some Japanese researchers attempt to quantify eye movements as a measure for engagement, presumably in the hope that in future algorithms using the data gathered and more like it will be able to better interact with humans.
The experiment conducted was on the Wizard of Oz model, where a human acting behind a computer interface played the part of the interacting algorithm to see what was effective and how it would be used. The theme was a cell phone salesman trying to keep a subject, also motivated by a 1,000 yen reward for being correctly able to identify which cell phones are most popular with which age groups, engaged in the conversation. Engagement was measured by both the subject and an observer, each of whom had a "boredom" button, referring to the state of the subject.
I see no point in including the many detailed graphs - they would take up far too much space - but I will note that the conclusion was that the best way to maintain engagement is with properly timed probing questions. Also worth noting is a very significant disconnect between when subjects and observers pushed their buttons.
From the paper.
Discussion
The paper is significant because improved interaction with humans is a clear goal of CHI. The worst flaw is that is very difficult for the un-initiated to read - although this may be the venue rather than a flaw per se. My next move in their position would be to repeat the Wizard of Oz experiment with a different frame than cell phone sales and see if a different topic gets different results.
This seems like a very in-depth study... Maybe too in-depth. Why did they include the observer? Who cares what an outside observer thinks about how bored somebody else is? How is that observer's opinion in that respect in any way more salient than the subject's? Weird.
ReplyDelete