Testing the wisdom of crowds

  

Only hours to go until the American presidential election kicks of for real in Iowa. Anyone else excited?

Of course, it should be noted that Iowa is a notoriously unpredictable place. This is because the event being run isn't a primary (New Hampshire has the honour of holding the first one of those) but a caucus. This is a very different form of democracy - very old-fashioned, very participatory. This also reduces the turnout. Indeed, this event - which is being covered by thousands of news people from all over the world, and has cost millions and millions of dollars - will likely only have (at the absolute most) a couple of hundred thousand people involved in it.

This also makes polling very, very difficult. In 2004, the Des Moines Register got a lot of kudos because it got things pretty much right in the run up to the contest - notably picking up the late rise of John Edwards through the field. Furthermore, a recent poll organised by opinion polling blogger Mark Blumenthal found that the Des Moines Register poll, undertaken by Selzer, was the most reliable in the field. A couple of days, then, the Register published its much anticipated last pre-Iowa poll. And the results were as follows:

Barack Obama - 32
Hillary Clinton - 25
John Edwards - 24

Mike Huckabee - 32
Mitt Romney - 26
John McCain - 13

The Democratic side of the poll has proved particularly controversial, with both the Clinton and Edwards campaign issuing immediate press releases rebutting some of the findings of the poll (for example, Clinton's pollster, Mark Penn). In particular, Obama's majority in the Selzer poll is largely constructed from a very high number of independents and even Republicans who said they were willing to come out and vote for him. And when I say high, I mean high - 45 per cent of the total predicted turnout. So, the argument goes, these numbers are flawed because, when push-comes-to-shove and its snowing in Iowa, these people won't turnout. We have to be careful here, there is a lot of spin doing the rounds, but the criticism does seem to make some sense. In 2004, for example, independents and GOP voters only constituted 19 per cent of caucus goers. At the very least, if Obama wins the caucus in the way that the Register poll suggest, it will be nothing short of a seismic event in American politics.

It is the question of turnout - and the unpredictability of it in a caucus environment - that perhaps does most to explain the extreme variance across the polls. At the same time as the Register poll came out, CNN were publishing another poll, which showed this:

Hillary Clinton - 33
Barack Obama - 31
John Edwards - 22

Mitt Romney - 31
Mike Huckabee - 28
Fred Thompson - 13
John McCain - 10

So what's the alternative? In a ridiculously ambitious (and, as it turned out, moderately time consuming) experiment I thought I would it would be fun to take the pollsters on with a different method - by harnessing the wisdom of crowds. I was flicking around the web, when I noticed that the Washington Post Fix blog had published a "predict the result" column. This was racking up dozens of predictions. So I copied and pasted them into an excel spreadsheet and then calculated the averages (the last comment I sampled was hal24, posted at 3:24 pm). Actually, the Fix column also has expert predictions on it, so this is in fact a test of three methods - pollsters, expert picks and the crowd. Here's the collective outcome of Fix commentators:

Barack Obama - 31.6
Hillary Clinton - 28.7
John Edwards - 27.9
[n = 75]

Mitt Romney - 29.9
Mike Huckabee - 29.5
John McCain - 18.5
[n = 69]

(Here is the spreadsheet with the workings, plus a proper comparison with the media polls).

The crowd results lead to a number of conclusions:

• Barack Obama will win for the Democrats, with a solid margin over Hillary Clinton.
  
• Mitt Romney will see off Huckabee in a very tight race.
• Edwards will do considerably better than the pollsters predict.
• John McCain will do considerably better than the pollsters predict.

It is these last two predictions that are perhaps most interesting, as that is where we see the greatest divergence between pollsters and the predictions of the Fix's readers, especially in the case of John McCain.

There are plenty of reasons to doubt the ability of the crowd - they might be multiple posting, many of them will have candidate preferences, their views might be warped by who can post at any given time - but it is still an interesting experiment. We'll have to wait until tomorrow morning to find out exactly who got it right though.

Update on 2008-01-04 17:04 by Nick Anstead

OK, so the results are in... and didn't the crowd do well! Here's the list, comparing them with the other predictions we had available. I did some quick (and probably horribly unscientific) calculations to assess who was closest. Basically, I took the prediction for all six candidates away from the actual percentage they got, then added them together and divided them by six to get an average predictive difference (you can see the workings here).

 

The crowd were only beaten by two of the experts and the Register poll. They managed to beat six experts and one multinational news organisations opinion poll. Not bad. Furthermore, the average of the crowd was also better than the average from the experts, which was 4.63, meaning the crowd was 0.75 more accurate.

The media hierarchy: who is borrowing from who?

Parody is, apparently, the sincerest form of flattery. But exactly who is parodying who says a lot about power and hierarchy. Once upon a time, if you logged into YouTube, you could have been sure of finding a host of parodies of plays, TV programmes, films and pieces of music, all put together by a group of amateurs. 

However, I was watching the Simpson's the other week (we are a bit behind anyone in the US - I suspect you would have got this a few months back), and I saw something interesting - the relationship has started to go the other way. In other words, old media forms (in this case, TV) have started to parody YouTube and particularly famous films stored there. This is interesting, as it shows the extent that YouTube has become embedded in our culture. It's not just that the Simpsons' writers use it, but that they know their audience will get the gag. What is most astonishing is the speed at which this has happened. Here's the original film and the Simpons clip.

PS. Oh, and a happy new year to everyone!  

BBC goes web 2.0

For an organisation that once insisted that radio news readers had to wear dinner jackets whilst delivering a bulletin, the BBC now seems to be making very brave advances into the world of web 2.0. Of course, the BBC website has been a global market leader for a number of years. However, I've always had the sense that it tended to rely on quantity of information, rather than any particularly flash web aps to achieve this (as well as piggy backing on the BBC's already well-established news and information gathering networks).

Now though, new innovations seem to really be harnessing the capabilities of the web to distribute this information in different ways. I first noticed this a couple of weeks back when I discovered that BBC Politics had set up a twitter feed, and have been regularly updating it with short banner headlines (which also link to the main story via tiny urls). It's obvious what this service has been set up to do - give people access to mobile news whilst they are on the go via RSS. And for that it is a brilliant service. 

Today, I logged onto the BBC website, and they were touting the beta version of their new website. It allows you to drag and drop different content boxes, so as select which aspects of the news your want to focus (my page will now, for example, always prioritise cricket, technology and politics headlines over others items), and receive updates from BBC journalists who blog.  For anyone who uses bloglines or even Google homepage, the features on it won't be very new. But it always seemed that lots of the web 2.0 stuff we speak about a lot (RSS maybe being the classic example) has never really penetrated into the consciousness of the mass of the population. However, with a major organisation like the BBC pushing these capabilities, it can really start to make a difference to the way media is consumed.

Media and terror: New book and journal

Media coverage of war and terrorism continues to raise ethical dilemmas for journalists and news producers. From the risk to journalists of reporting from Iraq and Afghanistan to decisions about rebroadcasting ‘citizen journalist’ footage, Al-Qaeda propaganda videos or hostage tapes, every week brings new difficulties about how to convey news in a credible way to increasingly distrustful and choosy audiences. These dilemmas were discussed on Tuesday 11^th December by media professionals and academics at a debate entitled ‘Media and Terror’, at the Frontline Club in London.

Led by Dr. Ben O’Loughlin of Royal Holloway, University of London and Dr. Andrew Hoskins, Associate Professor in Sociology at the University of Warwick, the debate marked the launch of a new journal entitled Media, War and Conflict published by Sage and a new research monograph by Hoskins and O’Loughlin entitled Television and Terror: Conflicting Times and the Crisis of News Discourse, published by Palgrave Macmillan.

Hoskins and O’Loughlin argue that television news since 9/11 has been marked by a series of uncertainties about the representation of terrorism and war. O’Loughlin spoke about ways for news media to convey events in a form that enable news consumers to engage with distant suffering and conflict. If events are presented as open to political intervention – that policy dilemmas remain unsolved, and a difference can still be made– this reduces the likelihood of compassion fatigue, fatalism and disengagement among news viewers. However, in Television and Terror Hoskins and O’Loughlin explain why it is that just as news brings the world’s wars and catastrophes onto the West’s horizon of responsibility, it simultaneously blocks them from clear view, failing to provide proportionate analysis of whether Western interventions are succeeding or what success might mean.

Other speakers included Prof. Philip Seib of the Annenberg School for Communication, University of Southern California , Prof. James Gow of King’s College, London , and Prof. Stuart Croft of the University of Warwick and Director of the Economic and Social Research Council’s (ESRC) New Security Challenges programme. The event was sponsored by the ESRC, Sage, Palgrave Macmillan, and Routledge.

Studies in Digital Politics: A New Book Series from Oxford University Press USA - Series Editor: Andrew Chadwick

Studies in Digital Politics

A New Book Series from Oxford University Press USA

Series Editor: Andrew Chadwick, Royal Holloway, University of London

Digital communication technologies are now central to our understanding of political, social, economic, and cultural life. Initiated in December 2007, this book series will bring together scholars with an interest in understanding the information and communication environments which shape - and are shaped by - politics and policy-making. The series will be concerned with theoretical and conceptual debates, political institutions and behavior, and policy issues. It will provide an important, high-profile publishing outlet for a range of talented authors, both established and up-and-coming.

Books in the series will analyze the politics of new communication technologies, broadly defined. Books will summarize and criticize existing literature as well as provide new departures. The field itself is currently undergoing a shift, as the impact of web 2.0, social networking, citizen journalism and related trends requires fresh perspectives.

 

For further details and information on how to submit a proposal, please download the guidance (pdf).

16 days of action for Darfur

A really interesting Facebook powered campaign organised by the people at Globe for Darfur. A while back they created a group called 16 Days of Action and started recruiting people to join it.  The aim of the group was to get members to engage in 16 forms of activism over 16 days, such as sending a video for Darfur, posting a blog entry about the crisis, and phoning your political representatives.

What is interesting about this campaign is that it seems to be wholly Facebook based (the only url on the press release launching it is the Facebook address). I reckon this can be interpreted in two ways (which aren't mutually exclusive).  It can certainly be seen as an inspired or powerful use of social networking to cheaply and quickly organise a campaign.  But additionally, it has to be asked whether it excludes people who don't use Facebook from participating - and in the process, overlooks potential supporters.

Postgraduate Research Studentships for 2008 entry

Postgraduate Research Studentships for 2008 entry. Five studentships in total for the Department of Politics and International Relations. Deadline for applications: February 29, 2008. For full details, please click here.

(Belatedly) happy Thanks Giving

There are certain rules I stick to on this very serious academic blog as opposed to my own weblog. Aside from the obvious "don't blog about cricket" rule, another really important one is "don't blog after drinking lots of margaritas"... but today I'm going to make an exception to that rule, as the party I have just been to was unexpectedly and very directly related to e-campaigning.

I was very lucky to be invited to a wonderful Thanks Giving party hosted by some friends this evening. The spread was nothing short of spectacular, as you can see from the picture - a soup starter, a main course with every conceivable trimming, and beautiful pies for dessert (and the aforementioned margaritas, which also made an appearance somewhere in the latter point of the meal - I'm a little hazy on the details).  It was a really wonderful evening. 

 

I can't claim any credit for the cooking.  Indeed, the one task I was given was to whip up a sauce containing milk, garlic, vinegar and extra virgin olive oil.  However, due to a cooking mis-communication, it got blended instead of whipped - the end result being what can only be described as a garlic smoothie going in the bin (you can see the rather hideous mess I made in the photo above), and someone being sent out to buy some humus as a replacement. I can only claim incompetence as a defence...

However, what was particularly interesting about this party was where many of the recipes had come from... or more specifically, who they came from - John Edwards. I had vaguely noted the story that Edwards was giving family recipes away to those who donated to his campaign, but I never thought that I would actually be able to go to a party where these recipes made up a large proportion of the meal served (I also should add that the hideous garlic smoothie was not an Edwards recipe - that belonged to the BBC Good Food magazine).

It struck me, as I was tucking into this fantastic spread, that this kind of campaigning would simply not have been possible without the Internet. The gathering of donations is faster than ever before, as is the ability of campaigns to communicate with those who have given them donations (or, most crucially of all, might give them more donations in the future). That's why John Edwards's Thanks Giving recipes makes sense and has been so effective (and tasty, I hasten to add).  It also globalises the campaign to a far greater degree. 

I know this blog has many reader all over the world, so can I just take this belated opportunity to wish anyone reading in the US a very happy Thanks Giving.   

What is Al-Jazeera English for?

We have recently witnessed a huge growth in the number of transnational English-language television channels. This workshop led by Prof. Marie Gillespie of the Open University and Dr. Ben O’Loughlin of Royal Holloway, University of London, focused on the purpose of these channels. Given that Qatar, France, Russia, Iran and China have all recently launched English-language TV stations, does this mean countries only feel they count as a ‘power’ if they have a voice alongside the BBC and CNN in the emerging ‘Anglosphere’? Are governments institutionalizing a new phase of public diplomacy in an attempt to influence other governments or publics? Or are these channels simply professional news providers, part of profit-making organizations?

Debate was anchored around two presentations. First, Dr. Mohammed El-Nawawy of the Queens University of Charlotte and Shawn Powers of the University of Southern California introduced their new study, Al-Jazeera English: Clash of Civilizations or Cross Cultural Dialogue? In the next year they will examine the impact of Al-Jazeera English in five countries, asking whether such media can have peace-making effects in world politics, acting as ‘conciliatory media’. Al-Jazeera English is an interesting case because journalists have an explicit mandate to give a ‘voice to the voiceless’ and produce news that does not offer casual demonisation, lack of context, or reduce debates to simplistic binary stand-offs. But why would the Emir of Qatar sanction this TV station alongside its Arabic version? And why recruit journalists from the West rather than from ‘voiceless’ regions? Debate focused not only on the purpose of this channel, however, but also on how researchers might ascertain its impact. For instance, few viewers have access to Al-Jazeera English at present, and those likely to reply to surveys or interviews about the channel are likely to be a self-selecting bunch: viewers who already approve of the journalistic ethos of Al-Jazeera English. How transnational media channels attempt to measure any ‘effects’ on audiences – conciliatory or otherwise – is a critical challenge in the coming years, and it remains to be seen how these channels will prove their value.

The second presentation drew on findings from the recent New Security Challenges Shifting Securities project by Marie Gillespie, Ben O’Loughlin, Prof. James Gow, King’s College, London and Dr. Andrew Hoskins, University of Warwick. The project explored how cultural and religious diversity affect news reception and the specific responses of British Muslims to media and security policy. It has also highlighted how changes in the technologies, ethics and practices of journalism shape the security stories and how they are interpreted. The study has been pioneering by connecting news producers, texts and audiences over time in ways that illuminate how audiences’ use of news contributes to their shifting perceptions of security and belonging. Insights from the Shifting Securities methodology will feed into several projects funded by the ESRC and AHRC in coming years, including in the New Political Communication Unit. An understanding of news consumption as a ritualised, social and situated process, not a matter of transmission of isolated messages to atomized viewers, can offer greater analytical purchase on questions of news credibility, the legitimation of security policy, and the ‘impact’ of media diplomacy around the world.

Darling, where did you leave my data?

It's certainly the biggest data protection scandal in British - if not world - history (details here, here and here). A junior tax clerk at HM Revenue and Customs copied a database containing 25 million records on to two CDs. The databases contained the personal details of every single UK citizen who claims child benefit - including their bank details and their mothers' maiden names, which has, for years, been the stock security question in any UK bank. The junior clerk then popped the discs (which were password protected but not encrypted) in the internal post, to be sent down to London. They never arrived. In all, the security leak is estimated to have put 7.25 million families at risk. Of course, there are no certainties that the discs have fallen into malevolent hands. But there is a horrible crushing uncertainty.

The political ramifications of this huge event are only really starting to be felt. It is quite easy to make the argument that this isn't really political in any meaningful sense, by which I mean related to the substance of policy (in fact, Jonathan Freedland cites just such an argument in today's Guardian). After all, some lowly HMRC official (who I imagine looks a bit like this) makes a balls up. What on earth does that have to do with the Chancellor of Exchequer? It is also easy to argue that (as Freedland goes onto, in fact) that the real impact of these events are perceptual.

The opening line of Simon Hoggart's Parliamentary sketch in today's Guardian rather neatly summarises the situation that Alistair Darling finds himself in:

Another day, another disaster - and this one was a stonker. The news that the private records and bank details of 25 million people were lying around on a computer disk, heaven knows where, like a Rockin' Good Christmas CD that's fallen out of a Sunday paper, was greeted by MPs with incredulity. They were less surprised by the fact that Alistair Darling was in charge. Poor Darling, or at least his department, is now seen as an ongoing accident blackspot.

I don't agree that this is wholly about perception. I think this this discussion has policy, as well as political content. Two arguments do suggest this is political in a meaningful sense. The first one was deployed by acting Liberal Democrat leader Vince Cable in the Commons yesterday, when he blamed the error on spending cuts that had occurred in the past ten years, which had ensured the civil servants were over worked and prone to making these kinds of errors. That seems a little like political semantics to me (Cable didn't actually quote any figures that I saw).

I think the second argument is much more compelling - and offers real reason to believe that, at the heart of this incident, lies a real policy problem. One of the defining technological developments in the past number of years has been the shrinkage, ubiquity and declining cost of massive data storage. To take one obvious example, your iPod nano can now hold considerably more data than a desktop could a decade ago.  By the same token, data has become more moveable, transferable and accessible. Access to the most complete and largest data sources has cascaded down the management structures of large organisations. There can be no doubt that this change has occurred on Labour's watch.

Therefore, it is no good simply arguing that this could have happened under a Tory government. It might have, of course. And there is no reason to suppose a Conservative ideology would pre-dispose to dealing with this any better. But the key point is this wasn't a one off event - a moment of madness where sensitive information was put in the post - but the near-inevitable end product of an inability to understand a new data environment created by a combination of technology and bureaucracy. Government should have spent a significant amount of time and effort in the past ten years trying to understand and develop systems for managing this new environment. Yesterday's events proved that they have not done so adequately and that is the great policy failure in this area.