05 August 2015
Crowdsourcing as Interesting Decisions: Update from BL Labs 2015 Competition Winner
Posted by Mahendra Mahey (BL Labs Manager) on behalf of Adam Crymble, a Lecturer in Digital History at the University of Hertfordshire, and one of the winners of the 2015 British Library Labs competition, describes the current progress of his project, ‘Mechanical Curator Arcade’.
When I was nine years old my friend Robbie and I spent an inordinate amount of time in the local video game arcade, and far more money than either of us would like to admit. We watched enviously as the teenagers hogged the Street Fighter II machine near the entrance. Robbie and I retreated deeper into the arcade, where we found a favourite in The Simpsons Arcade Game.
We even beat it once.
Like many children of the 1970s, 80s, and 90s, video games were a staple of our formative years. Many of us have developed a superhuman ability to stare at screens for long periods without blinking. We know instinctively that there is something behind this wall, and that some combination of buttons will help us discover it:
But how many of us know Why a game is fun? I only recently began to ask myself that question, and I came across a quote attributed to renowned video game maker Sid Meier, the creator of the Civilization franchise. Meier noted that 'a game is a series of interesting choices'.
Not everyone agrees with that definition, but it's a surprisingly simple and astute observation. Games lay down a series of rules - they generate the conditions of a virtual universe. We learn the rules, and our challenge is to win the game by making choices that lead us through that world, to victory.
But a game is about more than just choices. A game is about losing. Or at least, the threat of losing. If we make the wrong choice - jump on a prickly enemy, for example - we're punished. We die.
This revelation has been important for me, because for the past few months I've been trying to make crowdsourcing fun. Crowdsourcing is an increasingly common practice amongst historians, whereby a simple but repetitive task - such as transcription or tagging a huge set of images - is shared across a large number of volunteers. It adheres to the adage, 'many hands make light work'. Like games, crowdsourcing is inherently about choices. Depending on the task, the volunteer makes a choice. If they're transcribing handwritten documents, they have to decide what word they see on the screen. If they're asked to tag a historic image, they had to decide the appropriate tag.
In order to make crowdsourcing more fun, some projects have attempted to offer a series of incentives. High scores and leaderboards are popular now in 'gamified' crowdsourcing experiences. But I've yet to come across a crowdsourcing game in which you can REALLY lose. It's all carrot, and no stick, and that's why it's no fun.
Counterintuitive, perhaps, but once you hit the age of 5 and your competitive streak kicks in, it's the threat of losing that makes you want to win. And this is where crowdsourcing faces its biggest challenge if we want users to have a 'fun' experience. Because in order for you to lose, the maker of the game needs to know when you've done something wrong - when you've broken the rules of the virtual universe. That's easy enough for Super Mario, because the game is programmed to check when you've bumped into a bad guy, or fallen down a hole. But in crowdsourcing, we have no idea if you've given us the right answer - if you've tagged the image correctly, or transcribed the word right. If we knew that, we wouldn't have to ask you to do it in the first place. That means we can't punish you consistently. And it means you won't have fun the minute you realise that. Because at that point, your interesting decisions become meaningless and any correct information you provide comes down to your good will rather than your desire to win.
That's where we currently stand in our efforts to make crowdsourcing fun. It's a big challenge, but it's one I believe someone out there can tackle. So in the spirit of crowdsourcing, we're turning to the crowd, and we're hosting a virtual 'Game Jam' from 4-11 September 2015 to engage with amateur video game makers everywhere who think they've got the answer.
To help them get started with an appropriate crowdsourcing task, we've put together a sample set of these historic images - around 100 to 200 illustrations each of people, music, architecture, flora, fauna and even cycling - along with several hundred images that we know very little about. We thought this might help to validate the results of the crowdsourced content.
The sample link is: http://bl-labs.github.io/arcadeinterface/sample_images.html
An ideal game draws a random image from the set and through gameplay the player tells us something about the content of the image. Perhaps they choose from our limited set of tags (flora, fauna, mineral, human portrait, landscape, manmade - eg. machine, buildings, ship, abstract, artistic, music, map), or gamemakers can opt to be more creative.
If we like what we see, we've set aside up to £500 (courtesy of the Andrew Mellon Foundation) to work with someone to polish their game and release it as part of our 'Mechanical Curator Arcade Game', a 1980s-style arcade console that we're planning to install in the British Library this autumn. The Game Jam is open to anyone, but only those over the age of 18 are elligible to work for us.
All completed games (whether they fit the crowdsourcing theme or not) will also be eligible to enter the British Library Labs Awards, with a chance to win an additional £500 in prizes, as long as they use the British Library digital content such as the sounds and images from the open collections.
If you're up for the challenge, you can find out more on our Game Jam event page. We're looking forward to working with one of you, and get in touch at firstname.lastname@example.org if you'd like to discuss ideas. We're here to listen and learn.