Real-Time Bugs? uTest, the world’s largest QA marketplace, announced the results of its “Twitter Apps Bug Battle”, a week long contest for 600 of its 17,000 crowdsourced QA professionals. The mission: find the as many bugs as possible in the five most popular Twitter desktop apps – Tweet Deck, Seesmic Desktop, Twhirl, Tweetr and Twitteroo.
The results of the test are noteworthy: uTest community members found 317 bugs in a week long competition for $4000 cash prize. 20% of the bugs were classified as “show stoppers” or defects in need of immediate attention. The testers then ranked all five Twitter apps based on overall quality, usability, and feature set.
Some interesting stats from uTest’s official Bug Battle results page:
· Tweet Deck
o 77 bugs reported
o 13 percent classified as “showstoppers”
o 30 percent were Technical, 56 percent were Functional, and 14 percent were GUI
· Seesmic Desktop
o 80 bugs reported
o 20 percent classified as “showstoppers”
o 31 percent were Technical, 50 percent were Functional, and 19 percent were GUI
· Twhirl
o 61 bugs reported
o 16 percent classified as “showstoppers” and 33 percent as “high severity”
o 44 percent were Technical, 38 percent were Functional, and 18 percent were GUI
Matt Johnston, uTest’s VP of Marketing commented on the findings:
“Twitter users skyrocketed from 1.6 to 32.1 million users this year. Since filtering tweets is of top concern, sophisticated Twitter desktop apps are also booming. With this Bug Battle, uTest has once again demonstrated how quickly the tester community can mobilize and provide global testing coverage for the most cutting-edge apps.”
The second part of the content consisted of a usability survey, ranking the overall quality, usability, and feature set of the Twitter desktop apps. You can see the detailed results in the full pdf report. Tweet Deck came on top in terms of quality, followed by Seesmic Desktop and finally Twhirl, who got a high rank for usability. Interestingly, the more experienced testers (with over 3 years of testing experience), overwhelmingly favored Tweet Deck across all categories.
Most valuable features across apps:
· Easy access to Twitter actions such as Retweet (RT), Favorite, Reply, and Direct Message
· Filtering by groups, both multi-account and multi-column management
· Column or tab support for multiple searches, including trend searches
Most common complaints across apps:
· The location and grouping of the menu icons within the apps are not intuitive to the functionality to which they are related. (50 percent of respondents suggested adding a help menu or tool tips.)
· Applications consume too much memory (especially Tweet Deck) and compromise performance.
The winner of the uTest Bug Battle was Bernard Lelchuk of Israel, who reported the most valuable bugs and provided the best testing feedback, according to the uTest judges. uTest holds a Bug Battle each quarter, and is currently soliciting recommendations for future Bug Battle subjects on its blog: http://blog.utest.com.
Disclaimer: uTest is a VC Cafe Sponsor. You can find out more about uTest’s crowdsourcing QA solution by downloading this free white paper.
I'm a former general partner at google ventures, head of Google for Entrepreneurs in Europe and founding head of Campus London, Google's first physical hub for startups.
I'm also the founder of Techbikers, a non-profit bringing together the startup ecosystem on cycling challenges in support of Room to Read. Since inception in 2012 we've built 11 schools and 50 libraries in the developing world.
