Loosing the war while trying to win the battle (part1)

The BBC news today is full of articles about the extend and background to extensive spying the government does on private citizens personal communications such as Facebook, e-mails, phone calls, and so on. As a big data professional as well as a citizen of a democratic country there are a couple of angles worth taking. For one, what does commercially available software look like that gets sold to governments to achieve this sort of thing, and what can it do and how does a big data architecture like that do it. Secondly why is all this going on? Why does a government feel compelled to spy on its own citizens and non citizens.

A Most Peculiar Data moment

A manager of mine 10 years ago came out with “Perception is reality”. It has niggled at me ever since. Perceptions change, so reality changes? would it not be more insightful calling it perception is a reality? If I see a threat to my business, does it make that threat real? Odd – but maybe this story help me (and maybe you) to have a go at re-evaluating business in general.

Meta physical- not real?

from information-management.com

I like to focus on things that are surprising, I am obsessed with context and complexity, and I work day in and day out with people and organisations that face data challenges.  So here goes another odd story.

I had seen an advert for Heinz Large chunky soup (the advert showing large chunks of beef on a spoon) promising a more manly experience than tiny invisible beef remains I made up my mind to try this. The advert had been screened on US TV, but when I saw the soup in a UK supermarket I tried it.

Analyse this…

No, not the gangster movie, I do mean analytics as in maths and predictions. I am not sure if you noticed, but there is a it of a new buzz around regarding analytics. The last 3 projects I am was working on all ended up spending a lot of money on Analytics to gain more competitve advantage. Our team of analysts has grown from 3 to over 50. So I thought it might be worth noting a few things down on analytics that might be worth a few moments of your time. I’ll quote the CEO of my last multinational to set the questions (If you have seen the filme “Dogma” think of a character not unlike Jay, as in Jay and Silent Bob representing the CEO I have in mind.)

Who monitors the monitor?

We have all been there. The TV is 2 years old and just before (or after) new year the thing decides to try and desolve itself from all responsibilities. No news, no movies, no drama, no life sports. You might even have taken out the store’s extended warrenty, but the terms and conditions state that a chance of repair has to be exercised (which means that no-one comes for a week to look at the thing, then you wait another week for someone to take it away, and you wait a further 6 weeks for someone to say “you will need a new one”). It’s not what you were led to believe when you paid for the extended warrently. Either way you are left without the TV.

maths for data

I always felt an affinity with maths. That shouldn’t really be possible, but it is i can assure you. It is not a comfortable relationship, mainly due to the fact that I am poor at grasping math. So affinity might not be the most obvious word, but it is correct non the less. The affinity is due to the concept of math. For one thing math is not a constant. In it 1+1 can equal 2 or 10 simply depending on the number system you choose. Infinity can be massive in maths or very small (think of the largest number and you can always add 1, but you can also slice the differences between 1 and 0 by repeatedly and indefinitely multiply the higher number by 0.5. And the final straw for me was that there is proof in mathematics. It is one of the few disciplines where self reliance and consistency is more important than the subject itself. Wow! So when I come across abstracts like (a+b) to the power of 2 equals A^2 + 2AB + b^2 you have to stand in wonder not only be the facts, but also that it links you to the Greeks, who found these truths. Connecting to others in consistent language over time is the greatest gift of maths in my view.

networking at conferences

(c) Luca Longo

Berlin is cold in October. I never knew it was so close to the Russian low temperature zone. So only having a thin code wasn’t a great idea when I rolled up to SAP Teched 2010. The world of SAP has been my home for 17 years now, and there is very little I thought I didn’t know. Conferences are all about Networking.

Simply not true (or why IT doesn’t help)

Things can get really complex in life. When I was modifying my guitar last week I was trying to create a wiring diagram to keep track of things (as is and to be views). Now a computer would be a great help in this, using things like visio or the like, right? Well it wasn’t and in the end it was simpler to draw the thing by hand. My incompetence combined with prohibitive costs and the poor tool capabilities illustrated yet again that IT has tools for intelligent people, plasters for not so intelligent people and games for the rest. But real help with daily complexities seem to be thin on the ground. (Anyone who knows of a wire diagram creation software that is free and has all components in it that guitars may house (including ghost pickups) should leave me a message, please!)

Let me upscale this to the business world using my pet-subject Master Data Management. We know that Master Data Management is a worthwhile exercise. We know that having complete and accurate product information (for example) helps consumers pick the right product for them, establishing in the process a reference model. In practice the issue is more complex. Poorly manufactured products like to hide behind a wall of missinformation and hearsay to hide the lesser quality and the apparent lack of features. But the market is even more complex than that. Lifestyle and budget availability may drive people to a cheaper product under certain circumstances even or especially because of the lack of features (think older citizens or handicapped people).o where does that lead to?

Significantly misunderstood

The American Football season is restarting, and I must admit that I am a fan. I love the combination of tactics, strength and sports and although one of my best friends calls it “rugby for wimps with mattresses on” my son and I are hooked. There is just one gripe: The misuse of statistics. Great examples are:

” A team from New-England has never one one a Monday night while playing in Beige Socks”

“Quarterback Elway has only once been able to defeat a left-handed Quarterback”

“Chicago is more often defeated on uneven dates than Even numbered dates”

“R” you into statisitics?

“R” is a disruptive new open source analytics technology with a rapidly growing user community. It is a powerful and extensible statistical object oriented programming language. It was build to support rapid development of computational analytics and data visualisation. It can be and is easily extended (for example a new interpretative engine) and the benefits of these extensions are shared with the user community. There are now thousands of R analytics packages available to download made available by users in a growing academic and business community Several large software packages (such as Oracle, Greenplum and SAS), have integrations with the R language, or have released R language support. R has a vast array of standard graphical formats that can be reused, as well as good functionality for the production of bespoke plot and graphical format types

Page 1 of 212