I can’t argue with the ‘bad idea’ part, but the ‘everyone’ part is simply delusional. Yet many quite sensible people I speak to are genuinely bewildered that the filter is almost completely ignored by the mainstream media and barely registers on the radar of political debate.

(To be fair, the mainstream media’s reporting of internet censorship has been woeful and a prime example of what Jay Rosen calls “he said, she said journalism“, where a reporter simply records the opinions of opposing sides of an issue without subjecting their claims to any analysis. Most recently, on last night’s Q&A, Tony Jones only gave Small Business Minister Craig Emerson enough time to claim the government should filter all pornography that children shouldn’t see before shutting down the topic, preventing any debate.)

This is the kind of conversation I’m talking about:

renailemay: So let me get this straight. No #1 election issue on Twitter is the filter. And yet no questions from the floor during #ausvotes debate

vealmince: @renailemay Do you really not understand? Twitter is NOT the Australian public. It’s a tiny fraction of mostly like-minded people. #ausvotes

renailemay: @vealmince do you really not understand? Twitter is the Australian public. We live in Australia and we vote. Stop telling me I’m a minority

vealmince: @renailemay You and your 1000 mates. Either it’s not enough people, or you’re not organised enough to make a political difference.

Late last year, I argued that filter opponents were failing to cut through because they spent too much time agreeing with each other, debating nomenclature and deploying logic and sarcasm, rather than actual political lobbying, to sway the discussion in their favour.

But I think another factor at work is the inability of many in the twittersphere to see outside their small and mostly like-minded online social circle. This groupthink has led many online news outlets to publish polls finding that 95% or more of their readers were against an internet filter, unaware of or deliberately ignoring the massive selection bias inherent in asking that question to that audience.

The harsh reality is, even if everyone on Twitter thought and voted the same way, it would make no difference.

There are 13.9 million registered voters in Australia. There are 1.2 million Twitter accounts, of which no more than half could be active users who are eligible to vote. That makes 600,000 or about 4% of registered voters. It’s not a huge number, but 4% could gain a Senate seat, depending on how preferences fell, or swing the whole election.

Nice try. But of course, not all Twitter users would change their votes.

If Twitter is a representative sample of the Australian population (there are reasons to argue why it’s not), according to the latest polls, its users are split 50:50 on the two-party preferred vote. That means even if you could persuade every active Twitter user in Australia to vote for one party, it would only deliver a 2% swing.

But of course, you couldn’t get them all to vote the same way. Even though the filter is bad, some might argue that on the balance of all its policies, Labor is the less worse choice. Some of them might not care about the filter or, believe it or not, actually support it. (OMG, nowai!)

Still, a swing of less than 2% could be an election winner if Twitter users were disproportionately located in marginal seats such as western Sydney and the Brisbane suburbs. Whereas if a large number of Twitter users lived in safe seats, such as those in inner-city Melbourne and Sydney, even a 4% swing would make no difference.

Which do you think is more likely?

The impotent rage many Twits feel about the political-media establishment’s nonchalant treatment of the censorship issue is palpable. But it’s merely a symptom of the increasing influence of numbers men, marketing wonks and political strategists who use business intelligence technology to slice-and-dice, drill-down focus on winning a dozen or so marginal seats. If the issue that arouses your passion is not one that boils the blood of the residents of those seats, you’re irrelevant to the political process.

Viva democracy.

Which would you rather have in your home: Eddie McGuire or a steaming pile of poo?

The results were emphatic:

• Eddie McGuire: 2 votes (3%)
• Steaming pile of poo: 61 votes (97%)

While this was a terribly entertaining result, I did it to point out some of the problems with the many, many survey stories we see in the media.

If a journalist received a media release with these survey results in it, would he or she carefully scrutinise the details? Or write an amusing story to fill a gap in the news section? After all, nobody likes Eddie McGuire.

But a careful examination would reveal a large number of flaws in my technique.

Sample size

How many people did the survey interview? Is this a representative sample of the population the study claims to represent?

In this case, there were 63 respondents, which is definitely not enough to gain a valid idea of the opinions of all 22 million Australians. For a population of that size, you would need hundreds or thousands of votes to be statistically valid. Even then, there would be a large margin of error (ie, the true result could be 5% higher or lower than the one in the survey).

Avoiding duplication

How rigorous was the survey? Did it take any efforts to ensure people weren’t counted twice, such as deduplicating data? In the case of an online survey, did it use cookies or prevent people from voting from the same IP address?

My survey was not at all rigorous. I allowed people to vote multiple times if they wanted. I voted three times. If I were writing up a media release about my survey, I wouldn’t mention this, or I’d put it in tiny print at the bottom. No one would check.

Selection bias

Were the people who responded to the survey from a reasonable and broad cross-section of society or from a particular group? Would this affect their likelihood of voting one way or another?

My survey had 63 respondents, but who were they and how were they selected? In this case, I posted the survey on Twitter and Facebook. A couple of people retweeted it. Which means the respondents were:

• Regular internet users
• (Mostly) people I know

Would these factors make them more keen on poo? Or less likely to be fans of Eddie McGuire? Especially given the timing of the survey, after Eddie made homophobic comments about American figure skater.

A good example of selection bias is the recent, widely reported survey on ISP-whinge community site Whirlpool which found that 92% of its members were against mandatory ISP filtering. But 32.5% of respondents said they worked in IT. 70.9% were aged under 40. 72.1% said they were technical ‘gurus’ or ‘power users’.

So, the fact that 92% of the people most likely to oppose internet censorship were against internet censorship was hardly surprising. But it would be a giant leap to say this figure could translate to the broader Australian population.

Leading language

Did the language of the question favour one result over another? Why did the survey ask that particular question?

There is a definite art to choosing and framing survey questions. Asking the right question – or often not asking the wrong one – is a great way to get the answer you want.

I was quite deliberate in my choice of words, asking people which they would rather have in their homes. Not, for instance, which they would rather watch on TV. My aim was to make people contemplate the idea of having a steaming (smelly) pile of poo in their houses, which I thought would swing a few votes Eddie’s way.

Clearly I underestimated how much people disliked Eddie.

Manipulation

When people took the survey, how were they directed to the question? Did the surveyors couch it with a preamble or introduction that might sway the respondents one way or the other?

In directing people to the survey, I started out just tweeting the question but gradually became more manipulative.

• Which would you rather have in your home: Eddie McGuire or a steaming pile of poo?
• Disturbingly, a steaming pile of poo is outpolling Eddie McGuire 7 votes to 0. Can this be true? Have your say.
• I really thought at least one person would prefer Eddie McGuire to a steaming pile of poo. Do you?

Is it a coincidence that people only started voting for Eddie after I sent the third tweet?

Some thoughts for journos

Survey stories are great fun. They make for great headlines: a number, then a provocative or contentious topic. What’s more they give these matters of opinion or controversy a pseudo-scientific rigour.

But it’s fair to say of nearly all the survey press releases you receive that:

• The organisation paying for the survey has a very strong interest in the result being a particular way
• The organisation conducting the survey is aware of this interest (or is the same organisation that commissioned the survey)
• Surprise surprise, the survey results turn out the way the commissioning organisation wanted them to.

Journalists who reprint media releases verbatim, or rewrite them with no additional research, are often subjected to scathing criticism from their peers.

But journos who publish the results of obviously biased and flawed surveys without any critical analysis usually seem to get away with it. It’s time we got a lot smarter.