http://aaronkirkes-chi.blogspot.com/2011/04/paper-reading-19-personalized-news.html
Lowering the Barriers to Website Testing with CoTester
Jalal Mahmud and Tessa Lau, IBM
IUI 2010, 7-10 Feb '10, Hong Kong
Summary
CoTester is, in the author's words, "a lightweight web testing tool which can help testers easily create and maintain test script". The intention, if I understand correctly, is to have a tool that can be used for easy, automated testing of website functions. The authors extended an existing, easy to learn scripting language (CoScripter) for the project, with the goal of creating a script testing tool that did not require knowledge of Java/Visual Basic to utilize.
I'm afraid that the implementation, which they went into a quite detailed explanation of, was a bit beyond my level and I do not feel I can relate it properly. Interested readers should refer to the paper.
The results were quite promising. The script did an excellent job of identifying problems, exceeding the comparison algorithm's success rate by 14% (91% to 77%), and using cosine similarity scores over straight up equality checking to determine which class instructions belong in was also quite successful.
Discussion
The item that jumped out at me here as questionable - and I may be a little off-base here - was that I'm not sure I want my debugging script to be written by someone without at least a minimal programming background. I mean, Java and Visual Basic aren't exactly the most difficult languages in the world. Still I'm sure there's some application for this and they seem to be getting good results.
A good automated testing system for anything development related could of course be of great benefit to any programmer, just ask the Extreme Programming guys.
I agree with the authors that a much-expanded user study would be a good next move, but I'm definitely not qualified to tell them what to do next in the technical sense.
The most accessible illustration from the paper. Pictures were not a strong suit.
I am going to have to agree with you on the usefulness of this product. If my script is going to be tested, I want it to be tested by someone that can actually read it and understand it. Looking at the graphic you provided though, this could be useful for other things besides testing though.
ReplyDeleteI agree that this could be questionable if people were making these without a full knowledge of scripting. However, I think the types of procedures these scripts are running are simple enough that almost anyone can create them.
ReplyDeleteI also read this paper, and I cannot agree more with what you said. Even though Java and Visual Basic may not be the toughest programming languages they are still programming languages and should therefore be used by programmers. I understand Luke's point about these scripts not being that difficult, but I still want someone with programming skills to develop website applications.
ReplyDeleteYea from what you have here it does seem like the authors need to look into how the code works some more, they seem to be making some generalizations that just simply don't hold.
ReplyDeleteWhile having code written by someone who is new is sort of scary, I think it's OK to make something like this more accessible to them. Once they get used to programming in this they can move on to doing for real in VB or Java.
ReplyDelete