When Netflix needed to improve its movie recommendation system in 2006, it posted a $1 million reward online for anyone who could improve its existing system by 10 percent rather than just hiring the best programmers.
That act of crowdsourcing, the increasingly popular practice of posting a task as a competition online, is the subject of McCormick Prof. Jason Hartline’s latest research. Hartline, who is in the electrical engineering and computer science department, delivered some of his findings to a crowd of 60 in Ford Motor Company Engineering Design Center Wednesday afternoon.
Hartline derived his findings using auction theory, a branch of game theory that studies auctions and bidding strategies. In his lecture, Hartline explained the basic mechanisms behind auctions and optimal bidding strategies, which shows that bidders on eBay should bid exactly how much they value a product no matter what others bid.
“It never really dawned on me that you could actually determine what an optimal bidding strategy on eBay … with mathematical proof,” said Weinberg senior Gabrielle Knight, who attended the lecture and is also currently taking one of Hartline’s classes.
Hartline gave his lecture as part of the Meet the Faculty lecture series organized by the EECS department. The goal is to publicize EECS faculty research to all students in McCormick, said EECS professor Thrasos Pappas, who organizes the weekly lecture series.
“A lot of times the University would invite outside speakers to campus, but it’s just as important to promote the work of our own staff,” Pappas said.
Hartline began the lecture with an example of his own crowdsourcing contest, a graphic design challenge for the same crowdsourcing lecture he was delivering. After posting his challenge on taskcn.com, the submissions he received ranged from incomprehensible flowcharts, complicated spider webs, clip art collages to the winning design: an elegant image of people working on computers around an apple tree.
What intrigued Hartline about the process was the wasted effort of the losers and the time they invested without reaping any benefit. Is crowdsourcing therefore a more wasteful practice than just hiring the best guy in the field to do it? Hartline said no.
“If you compare crowdsourcing as a means of procurement to more conventional things like grant application to the government, whatever, they’re actually not too far off,” he said. “You’re not wasting a lot of work.”
The reason, he said, is that the quality of the rest of the submissions deteriorates rapidly, just as his own example illustrated.
“There will be one really good submission, there might be another okay submission, but the other submissions are very, very bad,” Hartline said.
Hartline then modeled the mechanism of crowdsourcing contests mathematically and compared it with the model of a normal auction under ideal situations. He found the value of all the wasted effort in crowdsourcing is, at the most, equal to the value of the winning bid.
