Hunches and the art of truth

[Sara Farmer]

Hi all,

We're starting to build the Hunchlab (https://github.com/global-
pulse). Which means that it's now 3am and I'm still chewing on a
philosophical problem about the nature of hunches. This goes
something like:

We've already got code that allows us to post hunches and add
'evidence' to each hunch. Let's call this version 0. We're about to
start building version 1, which inevitably has started a long
discussion in the office about what needs to go into version 3 (the GP
office is that kind of a place). Core to that is how we can include
output from the 'bots (tools using our ontologies etc to crawl the
interweb for signs of emerging crises) into hunches, and how to use
'support' scores from both them and people to rate how mature a hunch
is, and give a measure of how strong its support or refutation is.

There are lots of ways that we can 'score' evidence, and lots of ways
that we can combine those scores: we've been working through several
of these (thank you, uncertainty theory and robotics). But what's
keeping me awake at 3am is the nagging feeling that we need to ask a
more fundamental question. Which, in a sleep-addled sorta way is
this: are we automating a collaborative search for truth or an
argument, and is there a difference between them?

Or: do we assume that everyone (and everything) that adds evidence to
a hunch is giving us an uncertain belief about an underlying truth
that we can merge into a bigger picture about what's really going on,
or are we building an argumentation framework where the hunch mutates
over time. Oh - and what does that actually mean to both the humans
and the bots?

Or: I put up a hunch. Can that hunch change over time, given the
evidence that accumulates about it; and how can we represent that
change so it doesn't confuse the bots?

I have something niggling at the back of my mind, telling me that I've
asked this question before in a different context and that
argumentation frameworks were an important part of the answer. Are
there any other insomniacs on this list who'd care to join in the
discussion so I can finally get some sleep?

Thanks,


Sj.

[Jen Ziemke]

Glad to know I'm not the only one awake at 3 am : ). Would love to have a brief typing chat. Skype me, perhaps? jenziemke

--

Jen Ziemke, Ph.D.

Co-Founder & Co-Director
http://www.CrisisMappers.net

[Helena Puig Larrauri]

I'm awake, but also on a different time zone. Good morning :-)

Not going to answer any of your questions, but reading another question came to mind: how do you aggregate hunches? It's all fairly straightforward if people and robots are commenting on one hunch, but what happens when someone makes a competing hunch? Or a somewhat different but still relevant hunch? Perhaps this is a roundabout way to answer your first question - in a collaborative search for truth, we need to find a way to link hunches (that's what I think would be more useful); building an argument we'd just need to aggregate evidence around one hunch. I wonder whether you could link hunches by some measure of proximity so that you could get an idea (a visual even) of a network of hunches that gather around some truth (could you build a measure of cosine similarity to do this automatically somehow?).

Hope that helps somewhat.

Helena

[Sara Farmer]

Drat. I fell asleep just after I emailed.  Helena - you hit the master question there.  Here's how we think it goes at the moment. You can find similarities between hunches in many different ways, e.g. tag similarity (i.e. how many tags two hunches share) and clustering (i.e. where the tags 'bunch up' across all the hunches) text similarity in the hunches (yes, cosine metrics) shared links to evidence and other hunches (also again with the clusters) subject and geographical distance etc Linking evidence to hunches gives you something close to a naive Bayes (or other uncertainly-handling equlivalent) system - no more than one link at a time, no (or very few) messy interactions between links. But as soon as you start linking hunches, what you've got on your hands is a full-blown reasoning system, with concepts like generalisation/specialisation, reasoning chains, uncertainty handling and argumentation.  And that gets you quickly into places like inductive logic programming. Which is a long way of saying that in order to build something that *looks* like a very simple system (think Google search), we have to do some serious research into artificial reasoning methods.  What we probably need most right now is some good examples - does anyone want to post some ideas for this?  I'm still at the "I think there's a drought in Sudan"... "food prices are rising in Kenya" stage, and I'm sure you all have much better examples than that. Sj. --- On Fri, 13/5/11, Helena Puig Larrauri <hpuigl...@gmail.com> wrote:

[Eva Kaplan]

I have a question about this, with particular attention to the 'bots-- the danger I would worry about is less about truth vs. argument and more about whose truth and whose argument.  I think there's a danger of systematically excluding the "most vulnerable" from the conversation if the evidence for hunches is web-based even if it is in conjunction with "real people" who have direct reach into communities. 
So the questions for me are:
Somewhat fundamentally: is it still worth including automated hunches if we know ex ante they are only representing a particular subset?  How do we make sure it doesn't distort rather than reveal where we should be putting our attention/resources?
And a way around this might be: Is there a way we could interact the tools with text messages (assuming we had that data)? 

~Eva

[Patrick McNamara]

Hi Sara,

I'm wondering if there might be clues in qualitative data analysis software like Atlas.ti or TAMS Analyzer? These tools take text and create various analyses including mapping of how close the concepts are to each other - and logical links between them as well as some quantitate analysis. It wouldn't get into the depth of each hunch, but something like that might be useful in automating a collaborative search for truth.

I would think that hunches mutate and refine over time, and I wonder if there is any way the humans could interact real-time (via webchat or skype, for example) as hunches mature so at some point you go beyond aggregating or accumulating data toward creating a whole picture of a certain situation/hunch.

I''m looking at this from a systems thinking perspective...

Best,
Patrick

[Jen Ziemke]

Hi Sarah et al:

Sorry I am just now getting around to answering these older emails about Hunch.

Sara wrote: "I put up a hunch. Can that hunch change over time, given the

evidence that accumulates about it; and how can we represent that
change so it doesn't confuse the bots?"

In my view, all registered hunches should not change. The original "hunch"should be registered for historical/archival purposes in the system somewhere and should be recoverable. Of course, the collective understanding of the nature of the issue/question is something that can and will change in many cases, but we should always be able to trace back to the original idea/source. Others can register their countervailing hypotheses and evidence below and proximate to the original hunch. The original hunch should never be modified or erased permanently: this will allow us to see the evolution of ideas.

My 2 cents.

Jen