Mining for Data. Interview: Spying on ourselves for the Numerati

[dataguru]

(Interview with Stephen Baker, on CBC News)

Did you check CBC.ca/news this morning, look up the weather for your
city, order a political biography online? Did you call your broker at
a stoplight from your BlackBerry, Google Prius reviews at the office,
use a loyalty card on the way home for heart-healthy groceries and the
Globe and Mail?

You are a data miner's dream, blithely shedding priceless data about
your life like breadcrumbs behind you.

The people following the crumbs, the people pulling the information
together and making sense or nonsense of it, the computer scientists,
the mathematicians, the algorithm-makers, are the subject of Stephen
Baker's new book The Numerati.

Whether they are using the data to model behaviour that can predict
illness or dementia, deliver the love match we've longed for or spell
terrorist, Baker says the Numerati are a powerful force shaping both
our world and us.

Stephen Baker spoke to CBC News by telephone from his home in
Montclair, New Jersey. The following are edited excerpts:
Stephen Baker: They have increasing power, because we have delivered
the information of our lives right to their doorstep in ones and
zeros. And so they can begin to understand and model us and predict
our behaviour as shoppers and voters and potential terrorists and
workers, and dramatically change our lives.

How can political parties use this kind of data to manipulate
opinions?
Traditionally, politicians have had to look at constituencies in large
groups. There are different ethnic groups and there are different
regional groups, and different generations. And they, in their clumsy
way, try to customize their message for each group … You go to the
Jewish neighbourhood and you eat a bagel, you go to the Italian
neighbourhood and you eat a pizza, you try to adapt and show that you
understand these people and their lives.

And you pick up a baby everywhere.
You pick up a baby everywhere. But the fact is... there is diversity
in every one of these groups. So if you are targeting a group, you are
mistargeting a certain percentage of that group. You are sending them
exactly the wrong message.

'We could talk about Al-Jazeera, we could buy hummus, we could talk
about bombs, and all of those words could go through the networks…'
—Stephen BakerBut if you go to a state that is very close and you can
find maybe only 4,000 voters that you didn't expect to be able to find
and because of their consumer data you can say these people are likely
to be closet Democrats, or closet Republicans, or swing voters who we
know are interested in this certain issue, then you can approach them
directly, either through the internet with e-mail or direct mail or a
phone call or a knock on the door. Then you might be able to turn
entire states.

We're in the middle of a global financial meltdown. What role have the
Numerati played?
All of the investments that are made by these banks are done on the
basis of risk algorithms so all of the decisions that they make — the
portfolio management, the asset management, and all the rest — has to
do with algorithms that are designed by the Numerati based on their
understanding of risks. So somewhere along the line, they
miscalculated risk. It's important to understand that you can have the
best math in the world, but if you don't understand human behaviour,
then you cannot calculate risk when it comes to market behaviour.

The chapter on terrorism was most chilling for me — the difficulty
security agencies have in predicting who's going to do something and
when and where, and the mistakes that they can make.

http://www.cbc.ca/technology/story/2008/10/20/numerati-baker.html