Loosing the war while trying to win the battle (part1)

The BBC news today is full of articles about the extend and background to extensive spying the government does on private citizens personal communications such as Facebook, e-mails, phone calls, and so on. As a big data professional as well as a citizen of a democratic country there are a couple of angles worth taking. For one, what does commercially available software look like that gets sold to governments to achieve this sort of thing, and what can it do and how does a big data architecture like that do it. Secondly why is all this going on? Why does a government feel compelled to spy on its own citizens and non citizens.