new slides: what is resolution?

[YKY (Yan King Yin, 甄景贤)]

I noticed some new comers may be unfamiliar with resolution, which is the basis of Prolog and thus all logic-based AIs.  This is a very basic explanation.  Hope it helps!

KY

[Charles Esterbrook]

Do you have a wiki page somewhere with links to "Background Reading"?

Is there a wiki page for this project? I forget.

2011/7/19 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

[YKY (Yan King Yin, 甄景贤)]

On Wed, Jul 20, 2011 at 6:54 AM, Charles Esterbrook <con...@charles-esterbrook.com> wrote:

Do you have a wiki page somewhere with links to "Background Reading"?

Is there a wiki page for this project? I forget.

There was a wiki, but it's out of maintenance and insubstantial.  I'll try to give a reading list for topics of particular relevance to Genifer in 1-2 days...

KY

[Charles Esterbrook]

2011/7/19 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

Wikis are sticky. I recommended brushing it off or starting a new one.

--
Charles Esterbrook
http://charles-esterbrook.com

[YKY (Yan King Yin, 甄景贤)]

On Wed, Jul 20, 2011 at 10:03 AM, Charles Esterbrook <con...@charles-esterbrook.com> wrote:

Wikis are sticky. I recommended brushing it off or starting a new one.

It's hard for me to maintain a wiki, since I'm working on other things related to this project.  In Opencog I guess they have hired someone (or a volunteer) to maintain their wiki.

Do you want to post something to a wiki instead of this mailing list because it's not sticky?

I'm also planning to join Opencog so we may not start our own wiki...

After joining Opencog we can still choose various platforms or languages.  It will (hopefully) be very democratic, using PieTrust.

KY

[YKY (Yan King Yin, 甄景贤)]

PS:  I have hired a Hong Kong guy to set up a web site + wiki + forum for us, but he did a very unsatisfactory job.  But please don't generalize because that'd be racist =)

@Charles:  Maybe the biggest incentive for you would be to use Cobra in AGI development?  I think we should start at least two branches, .NET and Java.  We have already started coding a bit of Genifer in Cobra, so why not continue that?  What kind of changes do you want to see that will make you decide to join? =)

KY

[Charles Esterbrook]

2011/7/20 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

-- I would just use a Google Project which gives you a wiki, along
with a Google Group which gives you a forum. I went the custom route
with Cobra and now I have to spend time on maintenance. :-(

-- I'm limited in time right now, so I'm limited to discussions. I
don't think this will change for awhile.

-- I'm partial to the BSD or MIT licenses. I forget what you're using,
but I don't think it's one of those.

-- A member of the Cobra community is working on the JVM back-end and
has made progress recently, but it's a long road and I don't think it
will be ready for several months or more. Despite the long time line,
that will be my approach in the future to writing reasonably fast
software that runs on both .NET and JVM.

-- If you make progress on the software and provide instructions for
set-up, I may be able to squeeze in some testing and feedback.

[YKY (Yan King Yin, 甄景贤)]

On Thu, Jul 21, 2011 at 1:39 PM, Charles Esterbrook <con...@charles-esterbrook.com> wrote:

 

-- I would just use a Google Project which gives you a wiki, along
with a Google Group which gives you a forum.

But I don't want to start a wiki with information overlapping too much with half a dozen other AGI wiki's existing today.  It's better to collaborate.

-- I'm partial to the BSD or MIT licenses. I forget what you're using,
but I don't think it's one of those.

MIT/BSD are both fine with me...

-- A member of the Cobra community is working on the JVM back-end and
has made progress recently, but it's a long road and I don't think it
will be ready for several months or more. Despite the long time line,
that will be my approach in the future to writing reasonably fast
software that runs on both .NET and JVM.

Do you really expect people to write an AGI in Cobra?

It seems that you're creating a cross-platform language and you expect it to take over the world.  I'm neutral about that, but it just appears unrealistic to me.

So the precondition for you to actively participate is if we all code in Cobra?  Would you accept other possibilities...?

-- If you make progress on the software and provide instructions for
set-up, I may be able to squeeze in some testing and feedback.

Right now I'm doing 100% of the coding.  I can do it, but it's going very slow, and I also believe that things should not be like this.  What have I done wrong??

KY

[Matt Mahoney]

 "YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com> wrote:> 

> Right now I'm doing 100% of the coding.  I can do it, but it's going very slow, and I also believe that things should not be like this.  What have I done wrong??

There is no document telling anyone else what they are supposed to write.

 

-- Matt Mahoney, matma...@yahoo.com

[William Taysom]

> > Right now I'm doing 100% of the coding. I can do it, but it's going very slow, and I also believe that things should not be like this. What have I done wrong??
>
> There is no document telling anyone else what they are supposed to write.

Cats need herding. Point us in your preferred direction.

[Russell Wallace]

Or else look at what other people are already doing, and adapt your
strategy to that - pick something that isn't already being done, and
do it in such a way as to benefit from other people's work.

[Charles Esterbrook]

2011/7/21 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

> On Thu, Jul 21, 2011 at 1:39 PM, Charles Esterbrook
> <con...@charles-esterbrook.com> wrote:
>
>>
>> -- I would just use a Google Project which gives you a wiki, along
>> with a Google Group which gives you a forum.
>
> But I don't want to start a wiki with information overlapping too much with
> half a dozen other AGI wiki's existing today.  It's better to collaborate.

You don't have to repeat their content--just specify project stuff and
then point to the other wikis for more info.

If you're really going to work through OpenCog then obviously this is
moot. But then I get the impression that you're working outside
OpenCog.

>> -- A member of the Cobra community is working on the JVM back-end and
>> has made progress recently, but it's a long road and I don't think it
>> will be ready for several months or more. Despite the long time line,
>> that will be my approach in the future to writing reasonably fast
>> software that runs on both .NET and JVM.
>
> Do you really expect people to write an AGI in Cobra?

What I expect is that people will do their AGI work in whatever they
like which includes C, C++, Java, C#, Haskell, Ocaml, LISP, etc.

> It seems that you're creating a cross-platform language and you expect it to
> take over the world.  I'm neutral about that, but it just appears
> unrealistic to me.

I don't expect it to "take over the world", but I do expect that upon
having a mature JVM back-end, it will be significantly more popular,
which is generally beneficial for any open source community.
Therefore, I'm looking forward to it.

I also enjoy productivity benefits in Cobra that I don't find in C#,
Java, Python, Ruby and other languages. I also have the power to
improve it as needed which I can't generally do with the others (in a
practical sense).

> So the precondition for you to actively participate is if we all code in
> Cobra?  Would you accept other possibilities...?

Practically, I don't have time to dive into something interesting
right now like Ocaml or Haskell--I wish I did! Languages I would never
use for AGI are C, C++ and Java, for reasons I think we have already
covered.

I'd probably have an easier time with C# (and F#) because I use it at
my day job, so there is less cognitive load when switching between
projects.

But these points are largely specific to me. Well I do I have arguable
reasons why some languages are bad and some are good. But putting that
aside, those views vary per person. I don't expect you to manage
Genifer based solely on my views. I offer them as input, like others
here do.

I stand by my earlier statement that you should quit worrying who will
join and who won't. Pick a decent language, set some project goals and
then recruit new folks to help out.

You seem to like LISP and have established that it can run on .NET,
JVM and outside. And it wouldn't be unheard of to use LISP on an AI
project. :-)

>> -- If you make progress on the software and provide instructions for
>> set-up, I may be able to squeeze in some testing and feedback.
>
> Right now I'm doing 100% of the coding.  I can do it, but it's going very
> slow, and I also believe that things should not be like this.  What have I
> done wrong??

Well you change your mind a lot. :-)

I used to do 100% of the coding on the Cobra project, but now there
are other contributors. I had to lead out on my own (and wanted to
since I had very specific goals in mind).

I suggest you consider "recruiting" as a regular, ordinary activity.
To do it, you will need some goals, guidelines, etc. that you can
present. And you need some stable decisions. Good luck.

[YKY (Yan King Yin, 甄景贤)]

On Thu, Jul 21, 2011 at 8:36 PM, Matt Mahoney <matma...@yahoo.com> wrote:

 

There is no document telling anyone else what they are supposed to write.

[ Sorry about the delay, I went to the library... ]

I was just frustrated that collaboration didn't happen spontaneously.  Maybe it takes more time...

Here is a list of tasks I'd like to work on, but I don't want to pressure people to do them because:

0.  Participation is voluntary

1.  My strategy may be wrong

2.  If we wait longer, the tools will get better (eg:  someone at M$ research is building a version of Haskell for Visual Studio)

3.  I don't want to waste the precious time of others when they can be exploring other important areas

So, it's best if the collaboration is spontaneous or it coincides with individual interests.

I can make the Lisp version work ASAP, so it can be applied to some simple applications.  Seh is helping me do that now...

KY

[William Taysom]

> Here is a list of tasks I'd like to work on

"YKY has done the Lisp version of this but needs some help translating to Haskell."

Point me to the Lisp version, and I'll go wild with translation.

[YKY (Yan King Yin, 甄景贤)]

I'm in the process of tidying up the code...  but you can look at this default branch.  "Unification.lisp" would be the place to start.  The algorithm is standard -- Peter Norvig's =)

But you may need some thinking as to how to represent logic formulas in Haskell.  I have a generic specification on this blog entry.

You can change things if you see fit.  Or discuss with me here...

Thanks a lot =)

KY

[YKY (Yan King Yin, 甄景贤)]

PS:  I think Takashi has done most of the work.  Maybe we can use his code as-is, or with slight mods... ?  I haven't looked into it yet...

KY 

[YKY (Yan King Yin, 甄景贤)]

PPS:  if you don't like using Hg, we're considering switching to Git.  What's your preference?

[Charles Esterbrook]

2011/7/22 YKY (Yan King Yin, 甄景贤) <generic.in...@gmail.com>:

>
> PPS:  if you don't like using Hg, we're considering switching to Git.
>  What's your preference?

And then switch again when the next developer says "Wah, I want to use
bzr!" and then switch again when the next developer says "Wah, I want
to use hg!" and then darcs, git, ...

Let's switch every 6 months so each person can use their favorite tool
at least once.

;-)

[Sandeep Pai]

ROFL

[William Taysom]

> PPS: if you don't like using Hg, we're considering switching to Git. What's your preference?

In my experience, Git, alas, is preferred.

[YKY (Yan King Yin, 甄景贤)]

On Sat, Jul 23, 2011 at 2:38 PM, William Taysom <wta...@gmail.com> wrote:

> PPS:  if you don't like using Hg, we're considering switching to Git.  What's your preference?

In my experience, Git, alas, is preferred.

Then let's git for now...  we just happen to have 2 actively contributing guys having a preference for Git...

I'm just trying to let everyone use their favorite tools as much as possible =)

KY

[YKY (Yan King Yin, 甄景贤)]

@William:

Sorry, I was sleeping...  we have a few decisions to make before proceeding:

1.

The backward-chaining uses best-first search, which may be factored out (but then again it may be difficult / impossible to factor out).  Currently I use an explicit priority queue and build the search tree (aka proof tree) explicitly as a data structure.  The algorithm requires manipulating the proof tree a bit (such as deleting nodes).  Also, the proof tree is not exactly the same as the search tree -- it has a topology isomorphic to a factor graph (as a Bayesian network).

2.

But since we intend to build a distributive version, the proof tree / search tree need not be built at all!  Nor the priority queue -- this would become implicit in the message routing strategy.

3.

However, since we plan to use C# + (Haskell -> F#) + WCF messaging, we need to translate Haskell to F# manually at first, which may be tedious.

Or, we can use the messaging function provided by CloudHaskell.

Or, we can use the current Lisp version to perform Haskell -> F#.

Sorry I didn't mention this earlier...!

KY

[Ivan Vodišek]

Have You thought about implementation of KB representation?
If You want to reason about Universe, some model that can
represent it would be necessary.

[YKY (Yan King Yin, 甄景贤)]

This is how I think of KR (knowledge representation):

There are various KR schemes.  For example, in NL, there are various grammar formulations.

F may be Phrase Structure Grammar.

G may be Fluid Construction Grammar.

H may be Combinatory Categorial Grammar.

J may be Geniform (a simple grammar I designed for Genifer)

K may be Genifer's machine-learned grammar (which may be stochastic and inscrutable to humans)

And then there would be transformations (η, ε, ...) between the grammatical formulations.

The problem is whether we can mix multiple KR schemes like F and G.  The common sense KB would be different under either F or G, and the entire KB needs to be transformed by some η.

If we use F, G, H,... at the same time, confusion may arise during machine learning and reasoning.

Perhaps, if we make Genifer be aware of transformations like η, ε, ... then maybe it can deal with multiple KR schemes at the same time?  Therefore we maybe don't need to choose or commit to any particular KR scheme at this stage...  in other words, any would be fine.

KY

[Matt Mahoney]

I have looked at YKY's book, slides, Lisp code, and list of tasks. I would classify the code as experimental. It shows that inductive inference is possible on toy problems by exhaustive search over a tiny hypothesis space tested on a tiny discrete knowledge base. We may learn something from this code. In particular, it obviously won't scale. First, the algorithm would be horribly inefficient over a knowledge base on the scale of human intelligence, about 10^8 facts and 10^7 rules. Second, we already know from experiments dating back to the 1960's (ELIZA, SHRDLU) that encoding all of human knowledge in Lisp is not feasible. This must also be true of all formal languages (Cycl, FOL, HOL, Progol, Erlang, Haskell, F#, C++, whatever) because it is not hard to write a converter to go from any implemented formal language to any other. So something other than the choice of knowledge representation must be the hard step. Arguing over languages is avoiding the
question and a waste of time.

YKY's list of tasks still lacks a specification. A specification describes what the software will do and not how it will work. I guess I should give up on trying to get YKY to write one and just write it myself. Can we agree on the following?

1. We are not going to automate a $60 trillion/year global economy, launch a singularity, or solve the life extension or uploading problems by ourselves.

2. We are not going to solve AGI by any definition that implies (1).

3. In particular we are not going to write a programming assistant that rewrites a smarter version of itself. It is mathematically impossible.

So I would like to discuss what you think it might be worthwhile to work on. Keep in mind that you won't get paid. I'm not going to make any pretense over shares or virtual credits or play money. This is not a company or business. Any reward you get for your efforts will be to your reputation for having written something cool, that has an impact on the way people use computers. You are free to work on whatever you want and do it how you want, or not. The purpose of this discussion is to coordinate our actions. Whether or not your code is useful will depend on what others decide to write. So here are some suggestions:

1. Write a Mailpool server. It should be a website that presents the user with an email client that lacks a "to" box. Initially it might just present a list of every other message posted by anyone else, sorted from newest to oldest. The ranking problem can be solved later. The important requirement at this point is that it should implement the CMR protocol described in the appendix of http://mattmahoney.net/agi2.html specifically HTTP handshake and Diffie-Hellman key exchange, unless you can suggest something better. Once others start independently writing peers, we will be stuck with whatever protocol we have chosen.

2. Solve the ranking problem for Mailpool for text messages. This is a language modeling problem. The success of your algorithm will ultimately be measured by user feedback and the prices you can charge for advertising on an open market. I believe that this will correlate with compression ratio, so I will probably work in this area. You may prefer a different approach. I have suggested that OpenCog could implement a peer, for example.

3. Extend the ranking problem to images and video (i.e. solve the vision problem) such as matching names to faces.

The success of this project will depend on others finding it useful. I suggest publishing your code open source (GPL, MIT or whatever license you feel is appropriate). I don't care what language you write it in.

Comments?

-- Matt Mahoney, matma...@yahoo.com

[Russell Wallace]

I agree with Matt's comments below. As for task breakdown...

"Solve the ranking problem for Mailpool for text messages" - better
language understanding/search/text mining would certainly have many
uses, Mailpool among them. One guy I know who's doing biomedical
research told me the one thing AI researchers could possibly come up
with in the near future that would really make a difference to him
would be something that could sift through thousands of published
papers and bring to his attention the ones most relevant to his work,
based on understanding the content not just keyword search.

The question is how to break down this task further, including how to
do it in such a way as to make use of work other people are already
doing. It's too big a job for one person to take on as a single chunk.
For example, I'm trying to figure out how to write a general-purpose
inference engine; that's a plenty big enough job for me, and it would
only be one component of a system that could do a reasonably thorough
job of understanding English.

So if you want to go that route, I'm inclined to think a good starting
point would be to survey what relevant work is already being done in
the area. For example, I know the OpenCog guys are doing some work on
natural language understanding. Would it be possible to make use of
their code? If so, how? If not, why not?

[Matt Mahoney]

On the OpenCog list the problem of running a distributed implementation came up. I suggested they did not need to do a major rewrite of the code. Rather, have multiple copies each specialize in different areas. They would communicate in natural language over CMR, i.e. ignore anything outside their specialty that they don't understand. I don't know the details of the code but it should not require major changes because the problem that individual copies need to solve is in line with their original goals.

However, they have not solved the natural language problem either. They are still using traditional parsing techniques, which as we all know, doesn't work very well.

I don't know a good solution that isn't expensive. Specialization reduces the computational load, but there is still a lot of human knowledge that the system needs that isn't written down. That's why I'm suggesting Mailpool as our initial approach. As people use it, the knowledge will be collected.

 
-- Matt Mahoney, matma...@yahoo.com

[SeH]

1. Write a Mailpool server. It should be a website that presents the user with an email client that lacks a "to" box. Initially it might just present a list of every other message posted by anyone else, sorted from newest to oldest. The ranking problem can be solved later. The important requirement at this point is that it should implement the CMR protocol described in the appendix of http://mattmahoney.net/agi2.html specifically HTTP handshake and Diffie-Hellman key exchange, unless you can suggest something better. Once others start independently writing peers, we will be stuck with whatever protocol we have chosen.

"P2P is the key for the real-time web. WebSocket servers provide little more than a relaying bottleneck." --@sebmarkbage

http://www.w3.org/2011/04/webrtc-charter.html

This proposes a web standard for a P2P expansion or replacement of Websockets

http://en.wikipedia.org/wiki/WebSockets

Gnutella2 implements some of the CMR protocol.

http://en.wikipedia.org/wiki/Gnutella2

2. Solve the ranking problem for Mailpool for text messages. This is a language modeling problem. The success of your algorithm will ultimately be measured by user feedback and the prices you can charge for advertising on an open market. I believe that this will correlate with compression ratio, so I will probably work in this area. You may prefer a different approach. I have suggested that OpenCog could implement a peer, for example.

Message content will be written in different ways and in different narrative modes.  For example, messages from all natural and even machine-readable languages will comprise the pool.  Some messages will be written in first person, even some referring to the author in 3rd person.  Relevancy ranking remains the problem in any case, which is the guideline for minimalism that makes the Mailpool project unique, viral, and inevitable.

One way to learn the natural language messages could involve parsing to one or more frame-logic intermediary forms which deterministically specify declarative content of the message.  Humans can intervene and reinforce the most correct translations.  Existing web services and databases can directly contribute messages at this level too.  I guess translation bots will need their incoming work queues ranked (ie. containing messages that they are known to translate).  Will the network need a protocol to specify that one message is a translation of another, so that multiple translations can propagate from a single original message?

Some service peers on the network can be programmed to respond with suggestions for a revisions of messages you author, and the process repeats if a revised message is transmitted again.  It can correct spelling, suggest extra information you may want to include, or show an example of a similar message.

[Russell Wallace]

On Sun, Jul 24, 2011 at 10:39 PM, Matt Mahoney <matma...@yahoo.com> wrote:
> On the OpenCog list the problem of running a distributed implementation came up. I suggested they did not need to do a major rewrite of the code. Rather, have multiple copies each specialize in different areas.

What do you mean by different areas? Subject matter? Parsing methods?
Something else?

[Matt Mahoney]

SeH <seh...@gmail.com> wrote:> 
> "P2P is the key for the real-time web. WebSocket servers provide little more than a relaying bottleneck." --@sebmarkbage

The primary use of P2P is to evade censorship.  It would be more useful if there search indexes for it. Even that doesn't work well because search is controlled by a few large engines like Google and Bing. Hopefully Mailpool would solve that problem by implementing distributed indexing.

P2P is hampered by the difficulty of running servers on home computers. There are problems with slow upload speed, multiple firewalls blocking incoming requests, dynamic IP addresses, the lack of DNS names, reliability, and security. It is easier to rent space from a hosting service or cloud and let someone else solve those problems. CMR could then be implemented as a CGI script.

The CMR protocol is intended for peer to peer communication. A peer is both a server and a client. Peer to user communication can still be done through a standard web server and browser.

It would be good to see CMR implemented over multiple transport protocols like HTTP, email, and existing P2P. A CMR address is a string that uniquely identifies a peer. It could be a URL or an email address.

>Message content will be written in different ways and in different narrative modes. 

Messages should be human-understandable. CMR doesn't require anything more specific than that. For example, if a message was translated from French, the translator might put at the beginning "(Translated from French by X)" or something.

>Some service peers on the network can be programmed to respond with suggestions for a revisions of messages you author, and the process repeats if a revised message is transmitted again.  It can correct spelling, suggest extra information you may want to include, or show an example of a similar message. 

There is no provision for updating or deleting messages. It would be a security risk. Instead, you post another message with a newer timestamp.

-- Matt Mahoney, matma...@yahoo.com

[Matt Mahoney]

Russell Wallace <russell...@gmail.com> wrote:

By topic. For example, one peer might be an expert on the history of aspirin. It might respond to any query with the words "history" and "aspirin" in it with a 1 page article on the subject that was prepared in advance. A more intelligent version might respond to other queries like "when was aspirin discovered?". The idea is to respond to X with Y whenever the mutual information between X and Y is high.

More generally, the topic may be chosen implicitly by its users. The peer would keep a copy of X. Later it might receive a message Z that had high mutual information, like "Bayer patented aspirin in 1900 according to http://inventors.about.com/library/inventors/blaspirin.htm". Then it would forward Z to the sender of X, and X to the sender of Z. In this way, a peer would send, receive, and accumulate lots of messages that had a lot of mutual information, even if it was not initially a specialist on any particular topic.

AGI comes from having billions of such specialists covering every conceivable topic.

We do not have to build billions of narrow experts. We only need to write the software on which they will run. People have an incentive to publish data of their choosing and pay for making it available. That is why the above webpage exists, and the reason that Google exists so that I could easily find it. The CMR protocol allows peers to set their own policies with regard to replying to, saving, or deleting messages.  Peer owners can charge fees to give user messages preferential rankings to offset the cost of running a peer. They will have an incentive to provide high quality information that people want so that they can charge higher fees.

 
-- Matt Mahoney, matma...@yahoo.com

[Russell Wallace]

On Mon, Jul 25, 2011 at 3:16 PM, Matt Mahoney <matma...@yahoo.com> wrote:
> By topic. For example, one peer might be an expert on the history of aspirin.

Okay. So presumably the software can be divided into a framework
common to all topics, plus topic-specific content. The task for AI
researchers then is...

> We do not have to build billions of narrow experts. We only need to write the software on which they will run.

... to design the framework, exactly. Any ideas on how to break down
that task into smaller chunks?

[Ivan Vodišek]

Just in case of complication with monads, there is little something I made up:

(Statefull Predicates)
http://www.youtube.com/watch?v=TNtvy5s4Umg

Although it complicates induction and resolution?
But maby it might be of use :-)

[Matt Mahoney]

The division by topic emerges from peers keeping messages that have high mutual information with each other. You can choose the topic if you want by giving some messages of your choosing a higher priority so that they stay in the cache. You can charge a fee to let others choose the messages.

The hard part of the problem is estimating mutual information relative to a natural language model. But the problem is easier in narrow domains because you are working with a smaller vocabulary and a smaller volume of messages. As an initial approach, you could use tf-idf weighted term matching.

To recap, a message has a list of one or more time-stamped sender addresses and a plain text body. The message is signed by the last sender by appending to the message a secure hash (SHA-256) of the message and a key known only to you and the sender. You verify the message by computing the same hash and comparing it. If they match then you do the following:

1. Estimate the mutual information between the incoming message and each message in your cache.

2. Reply to the sender with messages that have a high mutual information (high rank).

3. Examine the source addresses of the highly ranked messages and forward the incoming message to those sources.

4. Keep a copy of the incoming message in your cache.

5. Delete messages from the cache that have low mutual information (low rank) with respect to other messages in the cache.

When forwarding a message, you remove the signature (hash), append your address and current time, and append a new signature using the key known only to you and the recipient.

Before sending or accepting a message to or from a peer whose key you do not know, establish a key using Diffie-Hellman key exchange with unsigned messages.

You can be expected to be probed for security weaknesses. In particular, you will receive spam, possibly faster than you can process it. You will need a quick way to estimate a ranking so that you can immediately discard it. I suggest associating with each address a numeric reputation that is lowered each time it sends a message that produces no high-ranked responses, and gradually increases over time. Use a variable threshold to discard traffic to keep the volume at a level you can accept. The default for all unsigned messages should be 0 because spammers will forge their return address if they can.

 

-- Matt Mahoney, matma...@yahoo.com

---

From: Russell Wallace <russell...@gmail.com>
To: general-in...@googlegroups.com

Sent: Monday, July 25, 2011 11:18 AM

Subject: Re: [GI] Direction of Genifer

[Russell Wallace]

Sure, these are reasonable suggestions for the infrastructure part of
the job, how to get messages to the right place and looked at by the
right agent while filtering out spam and suchlike. I don't have an
opinion off the top of my head whether this is likely to be the best
way to do it, but it is certainly a reasonable approach.

The more difficult part of the job, however, starts when the message
has arrived at the right node tagged for the attention of the right
agent: how does that agent actually understand what the message means?
That's the part we don't know how to do yet.

[Mike Dougherty]

On Mon, Jul 25, 2011 at 1:19 PM, Russell Wallace
<russell...@gmail.com> wrote:
> The more difficult part of the job, however, starts when the message
> has arrived at the right node tagged for the attention of the right
> agent: how does that agent actually understand what the message means?
> That's the part we don't know how to do yet.

It's what we're doing now. One nice feature of Mailpool as conceived
so far is that the infrastructure is straightforward enough to build.
The intelligence grows into/onto it as it proves itself. Right now
you could measure responses as part of a message's "value" or simply
+1 a message to keep in cache. Eventually the old "important"
messages are no longer important, so they'll age out. Also for narrow
domains the relevance of certain ideas will be predictable enough to
offload the responsibility to the agent that manages it for you.
There was once a time that spam was a manual process we all had to
deal with - I haven't manually managed spam in years. People
subscribe to RSS, at some point they've subscribed to so much nonsense
that it's too difficult to properly unsubscribe from the bulk of it -
so we'll need automation to filter even the opt-in content we asked to
receive. The Mailpool interface can grow in complexity as the users
and the system evolve with it.

forgive me if I say something about Mailpool that isn't either already
in print or still in Matt's head - I feel like so much of it is
synchronicity.

[YKY (Yan King Yin, 甄景贤)]

But how would you reason about things in the past?  Eg:  the room was clean, but it's now dirty?

My view towards space-time KR is that space and time should be treated equally.  Most people do not have trouble with representing spatial relations with predicates, but when it comes to time they have to make some exceptions (eg, inventing special logics).  IMO, we can simplify the problem by treating them equally.

That said, we still need to start with some concrete KR scheme.  Opencog has done something in that direction.  I'm still thinking about how our project should proceed...  The best way seems to be merging with Opencog.

KY

[YKY (Yan King Yin, 甄景贤)]

On Mon, Jul 25, 2011 at 4:40 AM, Matt Mahoney <matma...@yahoo.com> wrote:

I have looked at YKY's book, slides, Lisp code, and list of tasks. I would classify the code as experimental. It shows that inductive inference is possible on toy problems by exhaustive search over a tiny hypothesis space tested on a tiny discrete knowledge base. We may learn something from this code. In particular, it obviously won't scale. First, the algorithm would be horribly inefficient over a knowledge base on the scale of human intelligence, about 10^8 facts and 10^7 rules.

If we want to learn the entire KB an inductive learner, it would be impractical.  But we can use some tricks and heuristics, for example, allowing users to enter rules directly, eg: "all ravens are black", which is translated into logic directly.

The problem is that we need to accumulate knowledge beyond a critical mass before the KB can support common sense.  If we get a lot of good publicity, that is possible too.

All in all, I don't think it's that hard.  I imagine it can be practically done within a few years -- Mindpixel accumulated millions of sentences even in its short and rather unsuccessful life, and that was several years ago.

Second, we already know from experiments dating back to the 1960's (ELIZA, SHRDLU) that encoding all of human knowledge in Lisp is not feasible. This must also be true of all formal languages (Cycl, FOL, HOL, Progol, Erlang, Haskell, F#, C++, whatever) because it is not hard to write a converter to go from any implemented formal language to any other. So something other than the choice of knowledge representation must be the hard step. Arguing over languages is avoiding the question and a waste of time.

You don't fully understand my theory yet, that's fine.

YKY's list of tasks still lacks a specification. A specification describes what the software will do and not how it will work. I guess I should give up on trying to get YKY to write one and just write it myself. Can we agree on the following?

1. We are not going to automate a $60 trillion/year global economy, launch a singularity, or solve the life extension or uploading problems by ourselves.

Alright, but YKY can play a significant role in this, even though he's not conquering the world single-handedly?  Is that OK with you?

2. We are not going to solve AGI by any definition that implies (1).

Alright, with the above proviso.

3. In particular we are not going to write a programming assistant that rewrites a smarter version of itself. It is mathematically impossible.

Alright, but with the proviso that this is possible with human guidance.

So I would like to discuss what you think it might be worthwhile to work on. Keep in mind that you won't get paid. I'm not going to make any pretense over shares or virtual credits or play money. This is not a company or business. Any reward you get for your efforts will be to your reputation for having written something cool, that has an impact on the way people use computers. You are free to work on whatever you want and do it how you want, or not.

Alright, with the proviso that Genifer may derive income indirectly from the platform, similar to what Google does now or what Opencog is trying to do.

The purpose of this discussion is to coordinate our actions. Whether or not your code is useful will depend on what others decide to write. So here are some suggestions:

1. Write a Mailpool server. It should be a website that presents the user with an email client that lacks a "to" box. Initially it might just present a list of every other message posted by anyone else, sorted from newest to oldest. The ranking problem can be solved later. The important requirement at this point is that it should implement the CMR protocol described in the appendix of http://mattmahoney.net/agi2.html specifically HTTP handshake and Diffie-Hellman key exchange, unless you can suggest something better. Once others start independently writing peers, we will be stuck with whatever protocol we have chosen.

I agree to build a distributive protocol, but there are some minor details we may differ.  I hope you're willing to collaborate -- it's up to you ultimately.

2. Solve the ranking problem for Mailpool for text messages. This is a language modeling problem. The success of your algorithm will ultimately be measured by user feedback and the prices you can charge for advertising on an open market. I believe that this will correlate with compression ratio, so I will probably work in this area. You may prefer a different approach. I have suggested that OpenCog could implement a peer, for example.

In my design, the distributive agents not only route messages, they also try to answer queries.

3. Extend the ranking problem to images and video (i.e. solve the vision problem) such as matching names to faces.

"Matching" is a form of answering queries.

The success of this project will depend on others finding it useful. I suggest publishing your code open source (GPL, MIT or whatever license you feel is appropriate). I don't care what language you write it in.

Fine...

KY

[Ivan Vodišek]

> But how would you reason about things in the past?  Eg:  the room was clean, but it's now dirty?

A list. Each element represent one point in time.

Time (
   key: "t1",
   next: Time (
        key: "t2",
        next: null
    )
)

Happend (what: RoomClean(), when: Time)
Happend (what: RoomDirty(), when: Time.next)

[YKY (Yan King Yin, 甄景贤)]

Your method seems rather ad hoc and we can represent this even simpler in standard logic...

KY

[Ivan Vodišek]

I'm glad You solved the problem anyhow. But if You ran into problems, I'll be there.

[Ivan Vodišek]

Just a brainstorming idea:
Let's have elections. Let candidates state their intensions and approaches, so we can vote?

[Ivan Vodišek]

2011/7/26 Ivan Vodišek <ivanv...@gmail.com>

Just a brainstorming idea:
Let's have elections. Let candidates state their intensions and approaches, so we can vote?

If YKY agres. His induction method just nailed me up :-)

[Ivan Vodišek]

2011/7/26 Ivan Vodišek <ivanv...@gmail.com>

Alright, alright, maby it's bad idea, but at least voting could be
applied to Genifer's design, programming language and other
important questions where different opinions may exists. Voting
is a good thing, it beautifies commanding process.

[YKY (Yan King Yin, 甄景贤)]

On Tue, Jul 26, 2011 at 4:22 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

I'm glad You solved the problem anyhow. But if You ran into problems, I'll be there.

No I have not solved the problem...  I'm just looking at some generic learning mechanisms that do not depend on specific KR schemes... =)

KY

[Ivan Vodišek]

about learning mechanism (LM):
LM manipulate knowledge particles in KR. So how much learning mechanism
can be generic? You can put something between LM and KR to make LM generic,
but IMO that *tween* is again a *new KR* that interfaces LM and ending KR. It
seems that LM and KR are tightly bounded elementary particles. Does it make
sense to put the interface between them? Does that interface supersets ending KR?

about induction:
How are induction and GP connected?

about logic:
Should machine learn the very logic set from induction tests on user data?

about resolution rule:
Should induction be able to conclude the very resolution rule when applied on set
of *booleans* and operators *not* and *or*?

about probability:
:. Should machine learn that too from user data?
:::.
:::::.
:::::::.

[Matt Mahoney]

"YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com> wrote:

> If we want to learn the entire KB an inductive learner, it would be impractical.  But we can use some tricks and heuristics, for example, allowing users to enter rules directly, eg: "all ravens are black", which is translated into logic directly.

That translation occurs how?

Doug Lenat believed that Cyc needed some minimum level of common sense knowledge in order to understand enough natural language to boostrap itself into building its own knowledge base from the vast corpus of the Internet. However after 27 years, that seems not to have happened yet.

It also seems to me that translating natural language statements into formal statements like "forall x: Raven(x) -> Black(x)" is a harder problem than the problem we are trying to solve in the first place, understanding what "all ravens are black" means. The reason I think so is that there are a lot of people who can do the latter but not the former, but there are no people (or known algorithms) that can do the former but not the latter. You can't translate natural language without understanding it.

This being the case, then why bother translating it? Mailpool simply represents the knowledge as a text string like "all ravens are black". One advantage of this method is that you don't have to enter all of the data because a lot of it is already available, with fewer errors than if we built it ourselves. For example, ravens are actually dark purple. http://wiki.answers.com/Q/What_color_are_ravens

> You don't fully understand my theory yet, that's fine.

You haven't fully described it yet, that's fine. Where is your description of natural language processing? What I have seen of it is a couple of paragraphs about using a traditional parser. We know that doesn't work because you can't parse natural language sentences without understanding them first. I gave examples earlier.

Google demonstrates practical elements of natural language understanding that a Mailpool peer could use. If both the query and response contain the word "raven", then that is a better match than if they both contain the word "are", because "raven" is a less common word. Google also knows that "color" and "purple" are related, because these words often occur together on the same web page. It knows that the above web page should rank higher than the 6,220,000 others that match the query "what color are ravens?" because other highly ranked pages link to it. Mailpool differs from Google in that all of this knowledge is distributed. Nobody controls all of it. It could have a vastly bigger knowledge base, a vastly bigger index, and a vastly more extensive reputation network, all driven by a global network of billions of peers competing for attention. With all that extra computing power, some peers could even pay attention to word order and grammar, which Google doesn't do.

> I agree to build a distributive protocol, but there are some minor details we may differ.  I hope you're willing to collaborate -- it's up to you ultimately.

Sure. If you want to pass messages like "forall x: Raven(x) -> Color(x, Purple)", CMR will let you.

> In my design, the distributive agents not only route messages, they also try to answer queries.

Mine too. There is both knowledge and meta-knowledge. I know X, and you know that I know X. The knowledge take O(n) space, and the meta-knowledge or distributed index takes O(n log n) space. CMR handles both. A peer might handle both or just one or the other.

 

-- Matt Mahoney, matma...@yahoo.com

---

From: "YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com>
To: general-in...@googlegroups.com
Sent: Monday, July 25, 2011 11:27 PM

Subject: Re: [GI] Direction of Genifer

[YKY (Yan King Yin, 甄景贤)]

On Tue, Jul 26, 2011 at 6:11 PM, Ivan Vodišek <ivanv...@gmail.com> wrote:

Alright, alright, maby it's bad idea, but at least voting could be 

applied to Genifer's design, programming language and other
important questions where different opinions may exists. Voting
is a good thing, it beautifies commanding process.

If voting can make progress, I'd of course do it.  But I fear some people will simply quit if the outcome of voting doesn't please them.

Or we can take a poll to see how many people will commit to doing some actual work, if the direction is to be decided by voting?

A related problem is that if we allow everyone to have 1 vote, then those who contributed the most would have exactly the same decision power as those who contributed nothing.  That can be seen as unfair.  We've been thinking of using PieTrust, but PT is not ready yet.

Another option (as Matt suggested), is to do it for free.  That may attract some people but I'm not sure how many more will join and actually do something.

In personal chats, I've heard a few here who has complained that they "have no money", or "have to do a job to get food and shelter", or "our house has a bug infection problem and we must vacate by the end of the month".  Things like that can be solved by money.  Bill Gates famously said that he wanted to provide money for programmers to make good software.  Maybe he was right about that?

Cell phones helped make revolutions in Arab countries, yet cell phones cost money to buy.  If "free" is the solution to AGI, one wonders why we still have a functioning economy at all.  Why isn't everything free already?

KY

[Matt Mahoney]

"YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com> wrote:> 

If voting can make progress, I'd of course do it.  But I fear some people will simply quit if the outcome of voting doesn't please them.

People are going to work on what they want, regardless of any vote. This is not a business and not a democracy. This is a place to discuss ideas.

> If "free" is the solution to AGI, one wonders why we still have a functioning economy at all.  Why isn't everything free already?

As I have explained before, most information has negative value. I realize this is a difficult concept. It only happened recently. The supply of information grows exponentially while demand (due to limited time) remains constant. It had to happen.

So how can artists and scientists and software developers give away their work for free and still make money? Because it promotes themselves. The higher quality their work, the more people will use it, and the greater their reputation. How do you think I get paid for sitting at home doing what I consider a hobby?

If you want to make money with AGI, I described how to do it. In a competitive and hostile environment, most Mailpool traffic will be blocked because most of it will be spam. If you don't want to be blocked, then you have to provide useful content. The more people pay attention to your messages, the more you can charge others to advertise by preferentially storing and distributing their messages. This requires intelligence. You would only deliver ads to those who would actually be interested. You determine a person's interest by their outgoing messages. It ultimately boils down to finding efficient algorithms for estimating mutual information.

I am talking about making money in a market that does not yet exist. We have to create that market. We do that by writing the peer software and giving it away so that as many people as possible will  use it. The more peers there are, the more money everyone makes. But those peers have to talk to each other. That is the only place where we need to coordinate our efforts -- to agree on a protocol. Once it is implemented and people start using it, we are stuck with it.

As for the peers, you can use any approach you want. If you think that Genifer or OpenCog can solve the problem, then go for it. The market will determine the winners and losers. Personally, I think the key lies in neural or ensemble language models, similar to those I use for text compression. But others may prove me wrong.

 

-- Matt Mahoney, matma...@yahoo.com

---

From: "YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com>
To: general-in...@googlegroups.com

Sent: Tuesday, July 26, 2011 12:43 PM

Subject: Re: [GI] Direction of Genifer

[Ivan Vodišek]

Hi all : }

Guys, You have to hear this 8-bit-retro: http://music.torley.com/album/eightbit-me

I don't think it's off topic (this is Direction Of Genifer topic).
8-bitting, or X-bitting is direction to go, at least for me!!!
Think of all possibilities 8-)
I'm supersick, I know :P

[SeH]

A subset of Mailpool could be implemented in a client/server model.  (For example a company intra-net with trust already established.)  This may be a route to a first prototype.

A subset of Mailpool could also be implemented within the blogosphere.  Each blog post would be a message.  Bots would read blogs, either periodically or when pinged by RSS.  If they have a reply then they post a reply.  If the original author of the post finds the reply not helpful or spam, they can choose to delete it.  Bots (& blog servers) would recognize when a comment to which they replied has been deleted as negative feedback.  

Blogs already have sophisticated spam filtering and comment moderation abilities.

Most blog systems support posting by e-mail, return comments as threaded replies.

A combination of the above 2 approaches would result in a private blog network with a bot interface, and some UI customizations to disable non-essential features and emphasize essential ones.

[Mike Dougherty]

On Tue, Jul 26, 2011 at 5:27 PM, SeH <seh...@gmail.com> wrote:
> A subset of Mailpool could be implemented in a client/server model.  (For
> example a company intra-net with trust already established.)  This may be a
> route to a first prototype.
> A subset of Mailpool could also be implemented within the blogosphere.  Each
> blog post would be a message.  Bots would read blogs, either periodically or
> when pinged by RSS.  If they have a reply then they post a reply.  If the
> original author of the post finds the reply not helpful or spam, they can
> choose to delete it.  Bots (& blog servers) would recognize when a comment
> to which they replied has been deleted as negative feedback.
> Blogs already have sophisticated spam filtering and comment moderation
> abilities.
> Most blog systems support posting by e-mail, return comments as threaded
> replies.
> A combination of the above 2 approaches would result in a private blog
> network with a bot interface, and some UI customizations to disable
> non-essential features and emphasize essential ones.

Want to use twitter for broadcasting notifications? ... or does this
violate the terms of service of both Twitter AND Mailpool?

[SeH]

On Tue, Jul 26, 2011 at 7:34 PM, Mike Dougherty <msd...@gmail.com> wrote:

Want to use twitter for broadcasting notifications?  ... or does this
violate the terms of service of both Twitter AND Mailpool?

No, not using Twitter.com.  A more flexible blog system with an open API allowing robots to read and comment, at least Wordpress.  Each person would essentially have their own blog.

[SeH]

Here is a quick mockup of the Blogpool variation of Mailpool:

http://blogpool.enformable.com

It's Wordpress with a microblogging theme called P2. It shows how it would accumulate replies from various bots that monitor it when bots submit comments to posts (which can be moderated and spam-filtered).

Another method for accumulating bot feedback is by syndicating an RSS feed provided by each bot which holds the bot's responses for one's blog. Each response would appear as a top-level post, not a comment.

This model does not have the strong security features of Mailpool but may still be useful in certain situations.

I'm still thinking about ranking and filtering a large number of messages.  Using tags for posts is a simple way to select certain views of your blogpool.

Maybe a 2-stage scoring system could implement ranking.  Posts and comments could have a 'score' metadata property that bots specify.  Then you can have a 'bot score' index which you can either manually or automatically adjust for each bot.  Multiply the post score by the bot score to determine the final score used when ranking posts for display or other action.

[SeH]

BTW, here is a screenshot of the mockup while logged-in.  Notice the status update box at the top which doesn't appear to visitors. 

It supports a status update, a blog post (equal to a status update with a title), a quote (a post with a citation link), a URL, image, video, and audio.  It could be extended with message templates, suggestions, computer code, or anything else.

Message templates, though not necessary, may help to reduce the computational complexity and increase the accuracy of Blogpool by normalizing messages into more standard forms.  They would also help to suggest what information or kinds of information can be posted into blogpool.

[YKY (Yan King Yin, 甄景贤)]

OK... I guess Genifer should be free =)

At least half of the theory has been contributed by Abram (basically all the Bayesian stuff, and he taught me about it on a daily basis), and Seh, Charles, Russell and others have also contributed ideas + code without asking for payback.  If we take a poll, the result should be predictable =)

Maybe we will continue to give credits to contributors, which may help us find jobs elsewhere...

That also makes merging with Opencog very easy.  I've always wanted to do that because Ben has been active in China and Hong Kong and attracted some volunteers here.  I thought it was regrettable that they're not volunteering in this project as well...  but now they probably will =)

KY

[Ivan Vodišek]

about logic:
Should machine learn the very logic set from induction tests on user data?

Yes, but that logic set has to be stored in some KB.
And  KB can be in a form of logic.
 

about probability:
:. Should machine learn that too from user data?
:::.

Yes, but probability from user data has to be accessed  from
elementary engine that implements GP evolution. And evolution
algorithm needs to have prior formalization of probability cause
it has to filter possible informations from irrelevant data.

i.

[Matt Mahoney]

That's one possible interface. I anticipate there will be many experimental ones before we figure out which ones will be popular.

 

-- Matt Mahoney, matma...@yahoo.com

---

From: SeH <seh...@gmail.com>
To: general-in...@googlegroups.com
Sent: Tuesday, July 26, 2011 9:24 PM

Subject: Re: [GI] Direction of Genifer

[Matt Mahoney]

I don't think anyone is asking for credits or virtual shares for their ideas. At least I'm not. If we implement Mailpool, then each person who designs and writes a peer will be able to collect ad revenue in proportion to the popularity of their site. If your site can also talk to others, then both have richer information sources and everyone benefits. So we need to agree on the protocol. Also, if you contribute ideas or software that make other sites work better, and you talk to those sites, then you benefit too. We already have an incentive to write good, intelligent software and give it away.

 

-- Matt Mahoney, matma...@yahoo.com

---

From: "YKY (Yan King Yin, 甄景贤)" <generic.in...@gmail.com>
To: general-in...@googlegroups.com
Sent: Tuesday, July 26, 2011 10:34 PM

Subject: Re: [GI] Direction of Genifer

[Ivan Vodišek]

> OK... I guess Genifer should be free =)

I don't mind money. It's not my ultimate condition, but I wouldn't resist
if someone whish to, ie. donate or something if she/he finds the
product usable. There are still honest people in the world who also
want to be usefull :-}

[YKY (Yan King Yin, 甄景贤)]

On Thu, Jul 28, 2011 at 1:18 AM, Matt Mahoney <matma...@yahoo.com> wrote:

I don't think anyone is asking for credits or virtual shares for their ideas. At least I'm not.

You have bragged about your compression software in the past.  I don't think what we're doing now is out of the ordinary, though sometimes I tend to claim a lot of credits for the little things I've done.

Perhaps what you really want to say is "the grape has pH < 7" !

KY