There’s chilling Internet news out of China. And as bad as it seems at first glance for human rights and privacy advocates, there could be something more disturbing in the wings.

The Chinese government has announced that, starting in July, it will require all computers sold in China to come with Internet blocking software. The goal, authorities say, is to protect children from pornography.

Given that the software is being created and sold by a company with ties to China’s security apparatus, and that China hasn’t hesitated in the past to block access to web sites critical of its record on democracy and human rights, the government’s critics are understandably skeptical. Software that blocks access to pornography can easily be configured to block access to, say, Amnesty International.

So speculation is rampant that the software, dubbed Green Dam Youth Escort, will be used for censorship or surveillance — if not immediately, then whenever China’s next human rights crisis arises. And while its makers say parents will be able to deactivate Green Dam at will, the government could well be tempted to make using the software mandatory… or at least hard to deactivate, and less than forthright about what content it’s blocking and what information it’s collecting.

None of that is good news. But consider this.

Any blocking software needs to update itself from time to time: at the very least to freshen its database of forbidden content, and more than likely to fix bugs, add features and improve performance. (Most anti-virus software does this.)

If all the software does is to refresh the list of banned sites, that limits the potential for abuse. But if the software is loading new executable code onto the computer, suddenly there’s the potential for something a lot bigger.

Say you’re a high-ranking official in the Chinese military. And let’s say you have some responsibility for the state’s capacity to wage so-called cyber warfare: digital assaults on an enemy’s technological infrastructure.

You’re idly surfing the web on your home computer late one night, when it starts to automatically download an update. And it occurs to you that, somewhere out there, a single central point is making the decision about what goes into that update.

It strikes you: there’s a single backdoor into more that 40 million Chinese computers, capable of installing… well, nearly anything you want.

What if you used that backdoor, not just to update blocking software, but to create something else?

Say, the biggest botnet in history?

A botnet is a network of dozens, hundreds or thousands of computers, all running a particular piece of software that allows them to operate in concert, autonomously. In its most benign form, a botnet is just distributed computing, done with the full knowledge and permission of the computers’ owners. If you’ve ever installed the SETI@home screensaver, you were part of such a botnet — in this case, helping to sift through radio telescope data to find any sign of intelligent life on other worlds.

But the term is more commonly used to describe a nastier kind of network, where the software is spread by stealth, especially through viruses (the recent Conficker outbreak created a massive botnet). And as you might imagine, that kind of network is used for more malicious ends: sending spam, for instance, or launching huge attacks on other networks.

The larger the botnet, the more devastating those attacks can be. And they can bring down more than just web sites. Conceivably, everything from hospitals to electrical power grids could be targets. That, at least, is the premise behind warnings of cyber warfare.

Now, those warnings are often overblown. And while China has been accused of conducting cyber-warfare — including incursions into Pentagon systems — proving the involvement of the government rather than nationalist zealots is difficult at best.

Still, a botnet 40 million strong (plus the installed base already in place in Chinese schools and other institutions) at the beck and call of the military is potentially a formidable weapon. Even if the Chinese government has no intention today of using Green Dam for anything other than blocking pornography, the temptation to repurpose it for military purposes may prove to be overwhelming.

In the past, Western governments have either stood by or even encouraged efforts by activists to help people in China circumvent domestic online surveillance and censorship. One project, Peekabooty, even used distributed computing – a benign botnet – to create a network of outside proxy servers that would allow web surfers living under repressive regimes to access forbidden content freely and privately. (Alex discusses Peekabooty in more detail in her dissertation. Psiphon, a project of the Citizen Lab, carries on Peekabooty’s legacy.)

But a botnet within China might be able to use such a network to disguise its own activity, making it harder for targets to defend themselves from attack. Governments that would normally look kindly on a Peekabooty-style initiative now might even look on it as a digital fifth column, and an unacceptable security vulnerability.

You’ll notice a lot of mights and coulds in what I’m saying; it would be speculative even if I’d looked at the code behind Green Dam, which I haven’t (I’m not holding my breath for the Chinese government to make the code available). And I don’t want to feed either the cyber-warfare hype machine or the anti-China sentiment being pushed by self-interested parties.

My point is this: we’re excited by the potential of networked conversation and collaboration. It has tremendous potential when control is in the hands of many. But there’s a real danger when centralized control intersects with networked power. And those of us who see the positive power and transformative potential of the web need to pay attention to that danger.

Both because we may not like the way governments respond to it (or exploit it), and because we might have solutions of our own to offer. Anyone for a cyber-peace movement?

Mastodon