How to run chatbot with cogita

[Masayuki Hatta]

Hi,

I added Debian packages of RelEx (and JWNL) to my Debian APT repository[1], so now a whole set of the OpenCog NLP pipline is available as .deb packages.  They can be used (almost) right out of the box if you are using Debian, Ubuntu or some other Debian-derivatives.

Now I'm trying to test with opencog/nlp/chatbot, and basically it works as expected.  But cogita (IRC bridge) doesn't work.  I appreciate if you could help.

-----

The following is the verbose log of what I do, step by step (basically following https://github.com/opencog/opencog/tree/master/opencog/nlp/chatbot).  I hope some other Debian/Ubuntu users out there would give it a try and re-test:

1) Running RelEx server

Install "opencog-relex" package, then

$ opencog-server

(It's supposed to be opencog-server.sh, but Debian Policy doesn't allow executable extensions like .sh or .pl, so I changed the file name)

Seems it works nicely:

$ telnet localhost 4444
Trying ::1...                                                      
Connected to localhost.                                            
Escape character is '^]'.                                                      
This is a pen.                                                                 
; (S (NP this.p) (VP is.v (NP a pen.n)) .)          

(snip)

; END OF SENTENCE                                                    
Connection closed by foreign host. 

2) Running cogserver for chatbot

I ship opencog/nlp/chatbot/run-chatbot.scm and lib/opencog-chatbot.conf as examples in my "opencog" package, so copy them to my home dir:

$ cp /usr/share/doc/opencog/examples/run-chatbot.scm /usr/share/doc/opencog/examples/opencog-chatbot.conf ~

And edit run-chatbot.scm a bit to change the path accordingly:

--- ./run-chatbot.scm   2017-09-01 17:53:28.080287457 +0900
+++ ./run-chatbot.scm.new       2017-09-01 17:51:23.480283129 +0900
@@ -1,5 +1,5 @@
-#! /usr/local/bin/guile \
-  -L /usr/local/share/opencog/scm --
+#! /usr/bin/guile \
+  -L /usr/share/opencog/scm --
 !#
 ; For users who are not aware of readline ...
 (use-modules (ice-9 readline))
@@ -17,4 +17,4 @@
 ; See https://github.com/opencog/opencog/issues/2021
 (load-r2l-rulebase)
 
-(start-cogserver "../lib/opencog-chatbot.conf")
+(start-cogserver "./opencog-chatbot.conf")

Then setting up a simple ~/.guile (at least adding /usr/share/opencog/scm to load path is needed):

(add-to-load-path "/usr/share/opencog/scm")                                    
(add-to-load-path ".")                 
(use-modules (ice-9 readline))         
(activate-readline)

Now run guile:

$ guile -l run-chatbot.scm

(snip)

Listening on port 17004                
GNU Guile 2.0.13   
Copyright (C) 1995-2016 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.                
scheme@(guile-user)>                   

And also it works as expected:

$ telnet localhost 17004

Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
(process-query "mhatta" "Are you a bot?")
Entering scheme shell; use ^D or a single . on a line by itself to exit.
guile> Hello mhatta, you said: "Are you a bot?"
You asked a Truth Query
I can't process truth query for now

guile>

3) Now the problem.  I ship a pre-copmpiled binary of cogita in my "opencog" package as /usr/bin/cogita. And it seems to work:

$ cogita -c mhatta-test

And it can enter #mhatta-test@freenode.

[18:03] == cogita-bot [~mha...@12-345-678-99.some.example.org] has joined #mhatta-test

[18:03] -cogita-bot:#mhatta-test- ola

[18:03] <cogita-bot> here we are

But it doesn't respond my queries. Seems it received the message anyway:

:mhatta!3ba8fe13@gateway/web/freenode/ip.12.345.678.99 PRIVMSG #mhatta-test :cogita-bot: Hello!
#mhatta-test: <mhatta> cogita-bot: Hello!
input=:cogita-bot: Hello!
nick=mhatta ident=3ba8fe13 host=gateway/web/freenode/ip.12.345.678.99 target=#mhatta-test
opencog reply:

And seems cogita is sending to opencog cogserver something like:

scm hush
(process-query "mhatta" " Are you a bot?")

But got no reply at all.  I think this query is correct abd cogserver is listening -- what's wrong?

[1]: How to use my Debian APT repository for OpenCog

I explained this in the previous mail, but anyway reprise here.

1) Add the repository:

Do

$ sudo touch /etc/apt/sources.list.d/opencog.list

and put the following 2 lines in it:

deb https://people.debian.org/~mhatta/debian mhatta-unstable/
deb-src https://people.debian.org/~mhatta/debian mhatta-unstable/

2) Add my GPG key:

$ wget -q -O - https://people.debian.org/~mhatta/mhatta.asc | sudo apt-key add

3) Then

$ sudo apt-get update; sudo apt-get install opencog opencog-atomspace opencog-moses opencog-relex

Some other libraries and updated link-grammar packages will be installed, too.

Also, files are directly available at https://people.debian.org/~mhatta/debian/mhatta-unstable/

Please let me know if you have any difficulties.

[Linas Vepstas]

Hi,  Wow!

On Fri, Sep 1, 2017 at 4:20 AM, Masayuki Hatta <mha...@gmail.com> wrote:

Hi,

I added Debian packages of RelEx (and JWNL) to my Debian APT repository[1], so now a whole set of the OpenCog NLP pipline is available as .deb packages.  They can be used (almost) right out of the box if you are using Debian, Ubuntu or some other Debian-derivatives.

Very good!

Now I'm trying to test with opencog/nlp/chatbot, and basically it works as expected.  But cogita (IRC bridge) doesn't work.  I appreciate if you could help.

-----

The following is the verbose log of what I do, step by step (basically following https://github.com/opencog/opencog/tree/master/opencog/nlp/chatbot).  I hope some other Debian/Ubuntu users out there would give it a try and re-test:

1) Running RelEx server

Install "opencog-relex" package, then

$ opencog-server

(It's supposed to be opencog-server.sh, but Debian Policy doesn't allow executable extensions like .sh or .pl, so I changed the file name)

This should be renamed to something else, because it is not really the "opencog server", it is the "relex server that generates opencog-style output".

Maybe call it "relex-opencog" or "relex-oc-bridge" or something like that?

Yes, that all looks correct.
 

3) Now the problem.  I ship a pre-copmpiled binary of cogita in my "opencog" package as /usr/bin/cogita. And it seems to work:

$ cogita -c mhatta-test

And it can enter #mhatta-test@freenode.

[18:03] == cogita-bot [~mha...@12-345-678-99.some.example.org] has joined #mhatta-test

[18:03] -cogita-bot:#mhatta-test- ola

[18:03] <cogita-bot> here we are

But it doesn't respond my queries. Seems it received the message anyway:

:mhatta!3ba8fe13@gateway/web/freenode/ip.12.345.678.99 PRIVMSG #mhatta-test :cogita-bot: Hello!
#mhatta-test: <mhatta> cogita-bot: Hello!
input=:cogita-bot: Hello!
nick=mhatta ident=3ba8fe13 host=gateway/web/freenode/ip.12.345.678.99 target=#mhatta-test
opencog reply:

And seems cogita is sending to opencog cogserver something like:

scm hush
(process-query "mhatta" " Are you a bot?")

 

Yes, that would be correct.  Try removing the "scm hush" and see what happens.

Are you sure that cogita connected to port 17004? Because if it did, it should work ... The socket interface between cogita and the cogserver is very simple...

Try altering the cogita source code, and see if you get a reply to (+ 2 2) or (display "this is a test\n")  If you get a reply, then the socket connection works, and (process-query..) is broken.

I cannot test the apt repo today, I've got too much to worry about right now.

--linas

But got no reply at all.  I think this query is correct abd cogserver is listening -- what's wrong?

[1]: How to use my Debian APT repository for OpenCog

I explained this in the previous mail, but anyway reprise here.

1) Add the repository:

Do

$ sudo touch /etc/apt/sources.list.d/opencog.list

and put the following 2 lines in it:

deb https://people.debian.org/~mhatta/debian mhatta-unstable/
deb-src https://people.debian.org/~mhatta/debian mhatta-unstable/

2) Add my GPG key:

$ wget -q -O - https://people.debian.org/~mhatta/mhatta.asc | sudo apt-key add

3) Then

$ sudo apt-get update; sudo apt-get install opencog opencog-atomspace opencog-moses opencog-relex

Some other libraries and updated link-grammar packages will be installed, too.

Also, files are directly available at https://people.debian.org/~mhatta/debian/mhatta-unstable/

Please let me know if you have any difficulties.

--
You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+unsubscribe@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.
To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/e1284c3f-efb2-4ed6-929d-cd1b762ec1a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

"The problem is not that artificial intelligence will get too smart and take over the world," computer scientist Pedro Domingos writes, "the problem is that it's too stupid and already has."

[Masayuki Hatta]

Hi,

$ opencog-server

(It's supposed to be opencog-server.sh, but Debian Policy doesn't allow executable extensions like .sh or .pl, so I changed the file name)

This should be renamed to something else, because it is not really the "opencog server", it is the "relex server that generates opencog-style output".

Maybe call it "relex-opencog" or "relex-oc-bridge" or something like that?

Agreed, I'll file a issue or PR.

 

Yes, that would be correct.  Try removing the "scm hush" and see what happens.

I removed "scm hush" (or "hush", the same result) from cmdline in nlp/irc/go-irc.cc, then I got

04:13 (cogita-bot) ola
04:13 (cogita-bot) here we are
04:13 (mhatta) cogita-bot: hello
04:13 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.
04:13 (cogita-bot) [0;34mguile [1;34m> [0m

(the last line is a decorated guile prompt, seems "ANSI_PROMPT=" " in opencog-chatbot.conf doesn't work.  if you want a plain prompt, need to set "ANSI_ENABLED = false", but anyway basically the same result)

So cogita is successfully communicating with a local cogserver:17004,  but this "welcoming message" (?) might confuse it.  What is it supposed to be?

[Linas Vepstas]

On Fri, Sep 1, 2017 at 2:24 PM, Masayuki Hatta <mha...@gmail.com> wrote:

 

Yes, that would be correct.  Try removing the "scm hush" and see what happens.

I removed "scm hush" (or "hush", the same result) from cmdline in nlp/irc/go-irc.cc, then I got

04:13 (cogita-bot) ola
04:13 (cogita-bot) here we are
04:13 (mhatta) cogita-bot: hello
04:13 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.
04:13 (cogita-bot) [0;34mguile [1;34m> [0m

(the last line is a decorated guile prompt, seems "ANSI_PROMPT=" " in opencog-chatbot.conf doesn't work.  if you want a plain prompt, need to set "ANSI_ENABLED = false", but anyway basically the same result)

Yes, that's OK, that's just for debugging. We need to put back the `scm hush` after

the bug is solved...

So cogita is successfully communicating with a local cogserver:17004, 

Yes.
 

but this "welcoming message" (?) might confuse it. 

 

No, its fine.
 

What is it supposed to be?

All that cogita does is to take whatever is printed to stdout by the cogserver, and send it to IRC. It does not process this text in any way (except maybe to remove newline chars or something trivial like that)   

The fact that you are seeing the `Entering scheme shell` is a very good thing.  It means that there is a connection to the cogserver, and that stdout really is being returned to IRC.  So that's good.

So now, if you hack cogita to send `(+ 2 2)` to the cogserver, you should seem something like:

04:13 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.

04:13 (cogita-bot) [0;34mguile [1;34m> [0m (+ 2 2)
04:13 (cogita-bot) 4

04:13 (cogita-bot) [0;34mguile [1;34m> [0m

or something close to that.  Sending `(display "this is a test\n")` should result in

04:13 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.

04:13 (cogita-bot) [0;34mguile [1;34m> [0m (display "this is a test\n")
04:13 (cogita-bot) this is a test

04:13 (cogita-bot) [0;34mguile [1;34m> [0m

and sending  ` scm hush\n (display "this is a test\n")`  should result in

04:13 (cogita-bot) this is a test

without the guile prompts.

If the above all works, then it seems that `process-query` must somehow be broken ... even though it worked when you typed it by hand!? Maybe it breaks when you call it a second time?

Another possibility is that the socket to the cogserver got closed, before the reply arrived.  Its hard to guess why this would happen.  You can test this by sending `(sleep 3) (display "after big sleep\n")`

Another possibility is that the reply was generated, but somehow, the output buffer was not flushed, and so the guile reply was not sent back to cogita. But if the above sleep/display works, then this does not make sense ....

Another possibility is that the reply from the cogserver was posted, but cogita did not pick it up.  But if the above sleep/display works, then this also does not make sense ....

Its supposed to be a super-simple send-recv interface, there's not much that can go wrong.  I tried to keep it super-simple, for exactly this reason -- to minimize any room for bugs.

-- linas

--
You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+unsubscribe@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/cb602d69-d8ec-483d-af1e-54e63001da5c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Masayuki Hatta]

Hi, thanks for suggestion.

So now, if you hack cogita to send `(+ 2 2)` to the cogserver, you should seem something like:

04:13 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.
04:13 (cogita-bot) [0;34mguile [1;34m> [0m (+ 2 2)
04:13 (cogita-bot) 4
04:13 (cogita-bot) [0;34mguile [1;34m> [0m

I changed go-irc.cc like:

//strcpy (cmdline, "scm hush\n(process-query \"");
strcpy (cmdline, "scm \n(+ 2 2)\n");

Then it gives me the same response on IRC side:

17:59 (mhatta) cogita-bot: Hello!

17:59 (cogita-bot) Entering scheme shell; use ^D or a single . on a line by itself to exit.
17:59 (cogita-bot) [0;34mguile [1;34m> [0m

cogita's output is:

(snip)

#mhatta-test: <mhatta> cogita-bot: Hello!
input=:cogita-bot: Hello!

nick=mhatta ident=1549 host=www4.inetd.co.jp target=#mhatta-test
Sending to opencog: scm
(+ 2 2)

opencog reply: Entering scheme shell; use ^D or a single . on a line by itself to exit.
guile>

Changing to  (display "foobar\n") basically gives the same result.

From telnet, (+ 2 2) works:

$ telnet localhost 17004
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

(+ 2 2)

Entering scheme shell; use ^D or a single . on a line by itself to exit.

guile> 4
guile>

So, somehow scheme expressions ares not passed to cogserver at all.  I got pretty much stuck.  Any hints?

[Linas Vepstas]

Hmm. well, beats me. That needs debugging. let me see if I can reproduce it.

Either the string is not being sent (maybe a missing flush on the socket?) or the answer is not being sent.

--linas

--

You received this message because you are subscribed to the Google Groups "opencog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opencog+unsubscribe@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
Visit this group at https://groups.google.com/group/opencog.

To view this discussion on the web visit https://groups.google.com/d/msgid/opencog/01f1288a-666f-455f-ba25-3bd42f3bac47%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Linas Vepstas]

On Sun, Sep 3, 2017 at 5:23 AM, Masayuki Hatta <mha...@gmail.com> wrote:

Hi,

I may be completely wrong, but is this the expected behavior?

1) touch test.txt and edit something like:

scm
(process-query "mhatta" "Hello now!)
(+ 2 2)

2)

$ cat test.txt | netcat localhost 17004

Entering scheme shell; use ^D or a single . on a line by itself to exit.

guile> ... ... 

Yes, that should work.  And it should reply like the chatbot does.

If I change "scm" to "scm hush", then the prompt is not shown, so seems the first line is evaluated somehow.

Yes. The only thing that "scm hush" does is to hide the prompt.  That way, the text output from scheme/guile  can be sent directly to IRC.  That is, if you call some scheme function, and that scheme funcction prints some text, that text goes to IRC without any changes.   So "scm hush"  just prevents the prompt from being printed, so that it does not garbage up IRC.  Otherwise, there should be no difference at all between what IRC does, and what telnet to port 17004 does.

what version of guile are you using?  Version 2.2 works best.  Be careful not to compile with 2.0 and then use the 2.2 guile libraries, strange things will happen. Don't forget to experiment with  "make clean" ...

--linas

[Masayuki Hatta]

Hi,

I realized I can increase the log level of cogserver, and now I got:

(cogita is running, get a message via IRC)

[2017-09-07 08:46:46:377] [DEBUG] [GenericShell] dtor, wait for eval thread 0xdd7fa700.
[2017-09-07 08:46:46:404] [DEBUG] [GenericShell] exit eval loop
[2017-09-07 08:46:46:411] [DEBUG] [GenericShell] dtor, joined eval thread
[2017-09-07 08:46:46:411] [DEBUG] [GenericShell] dtor finished.
[2017-09-07 08:46:46:412] [DEBUG] [ConsoleSocket] destructor finished
[2017-09-07 08:46:46:412] [DEBUG] ServerSocket::~ServerSocket()

(and nothing follows)

So I guess scheme expression given by cogita is evaluated, but somehow ServerSocket::Send is not called?

[Linas Vepstas]

It is very unlikely that the shells (generic shell, scheme shell) are the cause of your issues. These are very heavily used all over the place; e.g. for language learning, I'm blasting maybe a dozen of these per second, for weeks on end, without any issues.

--linas

​

[Linas Vepstas]

I think I fixed this in some recent commits. ... It was a bug in the shell, after all. 

--linas

On Thu, Sep 7, 2017 at 3:26 PM, Linas Vepstas <linasv...@gmail.com> wrote:

It is very unlikely that the shells (generic shell, scheme shell) are the cause of your issues. These are very heavily used all over the place; e.g. for language learning, I'm blasting maybe a dozen of these per second, for weeks on end, without any issues.

--linas

​

[Masayuki Hatta]

Hi,

I updated my Debian packages of OpenCog (including link-grammar 5.5.0).  Seems cogita is broken again(the same symptom).  Did you change something in cogita or cogserver? I was away from OpenCog for a while, so it might not be in the most recent changes.  I'll try to investigate this by myself later.

Best regards,
MH

2017年9月9日土曜日 1時09分15秒 UTC+9 linas: