Channels/Targets for Component.fire

[Mark Young]

I'm trying to implement a TCP server and client using this package,
but in the process I need to send a message to the server exclusively
(avoding sending it back to the client's event queue), which I can't
figure out how to do. I've tried using the default "server" channel
when firing but that just sends it to the client's server channel
(which it doesn't have) and the server doesn't notice it at all. I've
tried specifying the server's component in every way I can think of,
but I can't get it to work properly.

I realize if I could access the actual component object, I could
probably specify the target that way, but my server and client classes
are in distinct files (and processes, for that matter).

When I don't specify the target or the channel at all, the client
sends to the server and itself as expected.

If needed, I can try to throw a small example together showing my
problem.

[James Mills]

Hey Mark,

It would be helpful if we could see some code?

http://circuits.codepad.org/

Also, what version of circuits are you using?

cheers

James

James Mills / prologic

E: prol...@shortcircuit.net.au

W: prologic.shortcircuit.net.au

--
You received this message because you are subscribed to the Google Groups "circuits" group.
To post to this group, send email to circuit...@googlegroups.com.
To unsubscribe from this group, send email to circuits-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/circuits-users?hl=en.

[James Mills]

Hi Mark,

Thanks for the samples!

Couple of questions:

1) Why do you create your own startup mechanisms?

Example:

    def go(self):

        self.start()

        while True:

            pass

And later on:

client.go()

2) I'm not following the flow of events all too well by reading your sample code. But that's ok!

3) Can you paste (from your real code) the specific snippets you're having trouble with?

Also could you show/paste a Debugger() log of the events

that demonstrate your problem (in your real system)?

Thanks heaps!

As for your comment on examples, I'll be sure to write one up before next release (Just after Christmas).

cheers

James

James Mills / prologic

E: prol...@shortcircuit.net.au

W: prologic.shortcircuit.net.au

On Sun, Dec 16, 2012 at 2:46 PM, Mark Young <mark...@gmail.com> wrote:

Ah, I didn't realize easy_install didn't grab the most recent version. I'm now using 2.0.2 but I'm still seeing the same problem.  (Apparently the target parameter is removed in circuits 2?)

Here's an example version of my server code: http://circuits.codepad.org/US4pqj3E

Here's an example version of my client code: http://circuits.codepad.org/XI31U0wx

The real versions rely on the rest of my code and would be a waste of your time to read.

As an aside, a client example and a slightly more complicated server example (one using channels, for example) in the documentation would be nice.

Thanks for trying to help so far.

--
You received this message because you are subscribed to the Google Groups "circuits" group.

To view this discussion on the web visit https://groups.google.com/d/msg/circuits-users/-/zmJ8_al_03EJ.

[Mark Young]

Here's a zip of the real program, if you want to see absolutely
everything in context: http://marky1991.dyndns.org/archivos/CRABcon.zip
(The client is defined in client.py, server in server.py)

Here's my debugger output (This may not be 100% what the debugger
outputs. It's intermingled with output from some of my print
statements. I tried to remove all of mine.)
(It crashes at the end because the client tries to send a "Make map
event" to the server, which ends up getting put in its own event
queue. The client doesn't have a make_map method, causing it to
crash.): http://marky1991.dyndns.org/archivos/debug.txt

Here's the snippet from the client that I'm trying to write so as to
only send to the server:

def send(self, event):
if event:
#This just turns objects into strings.
sent = util.bytify(event) + constants.message_delimiter
sent= sent.encode("utf-8")
logger.print("Client Sent: " + sent.decode("utf-8"),
subject="CLIENT")
logger.print("Client is sending a message of size: ",
sys.getsizeof(sent), subject="CLIENT")
#I thought TCPServer objects by default listen to a server
channel, so
#I would think this would only be heard by the server.
self.fire(circuit.Write(sent), "server")

And here's the server's write handler:
@handler("write")
def read(self, source, data):
logger.print("The server recieved a response: ", data,
subject="SERVER")
for line in
data.decode("utf-8").split(constants.message_delimiter):
if line:
event = util.debytify(line)
print(line, event, "HERE")
event_tuple = event.handle(self)
if event_tuple:
print(event_tuple, "tup")
self.send(None, event_tuple, socket=source)

logger.print("Server Received: {}".format(responses),
subject="SERVER")

A simple example showing how to only send events to the server would
be great. Thinking about it now, I have the client listening to every
channel, so I can see how it would hear it. But I didn't think it
would notice it because it doesn't have a server channel. I would have
thought I could have just done it by specifying target="server" or
something similar, but the target parameter was removed from fire in
circuits 2 as far as I can tell from reading the source.

RE the purpose of the go method: I have to do some other startup stuff
that most conveniently goes there. The example was too simple, making
it look pointless.

Sorry if I'm being an unhelpful help_asker_forer.

[James Mills]

I will look at this today and try to get an answer to you by the end
of today. ok the meantime I committed a simple chat server example to
our dev branch. feel free to draw ideas from it!

Cheers
James

Sent from my iPad

> --
> You received this message because you are subscribed to the Google Groups "circuits" group.

[James Mills]

Hi Mark,

Okay to be honest, I'm not going to be able to get my head around your codebase.

It's quite large! Over 20kLOC already! Wow!

But a few things...

I did write a chatserver.py example as request

to demonstrate some aspects of a multi-client

server: http://hg.softcircuit.com.au/circuits-dev/src/d165d46a53dd6cf9a3265932226390d97bb35f0b/examples/chatserver.py?at=default

Simply run with optional -b/--bind argument and

use the standard UNIX telnet command to connect.

A couple of questions:

1) Are you running your client and server code together in the same process?

2) Are you using a request/response tcp model?

A few notes on channels to help clear things up:

A Component has event handlers:

Example:

class Foo(Component):

   def hello(self):

      ...

Defines a simple component called Foo with a single

event handler "hello" which will listen for events of the same name.

By default, the channel of a Component is "*" meaning.

Component channels are merely used as a separation of potentially conflicting events.

The following defines a similar component but one in which specifically listening on a separate channel (or group):

class Foo(Component):

   channel = "foo"

   def hello(self):

      ...

The event handler "hello" will only receive "hello" events

if and only if:

1) a channel of "foo" was specified in .fire(Event.create("hello"), "hello")

or

2) a channel of "*" was specified in .fire(Event.create("hello"), "*")

or

3) The event handler was defined as:

@handler("hello", channel="*")

def _on_foo(self):

   ...

Does this help at all?

cheers

James

James Mills / prologic

E: prol...@shortcircuit.net.au

W: prologic.shortcircuit.net.au

--
You received this message because you are subscribed to the Google Groups "circuits" group.

[Mark Young]

"Okay to be honest, I'm not going to be able to get my head around
your
codebase."

I really didn't expect you to. I was providing it just in case. (It's
totally not 20k though. There's a gui package included by mistake in
my directory. (Not sure why it's there.))

"1) Are you running your client and server code together in the same
process?"

No. The client and server are run in distinct processes.

"2) Are you using a request/response tcp model?"

I believe so. Both sides send events and wait for some sort of
response. (At the very least an ACK event)

I've managed to make a more clear example of what's going wrong.

Here's the client code: http://circuits.codepad.org/B7umeHoR
Here's the server code: http://circuits.codepad.org/GEnxwAF6

If you run the two programs, you'll see that the server doesn't get
the "hi" message. (No one does.) If you remove the "server" channel
parameter from the fire method call in the say_hi method call in the
client code, the server will get the message.

I.e. this doesn't work:

def say_hi(self):
print("Client is sending hi")
self.fire(circuit.Write("Hi!".encode("utf-8")), "server")

but this does:
def say_hi(self):
print("Client is sending hi")
self.fire(circuit.Write("Hi!".encode("utf-8")))

What am I doing wrong?

(Another solution to this will be figuring out why the client is
getting its own message back. In the simple example above, the
client's message doesn't get placed in its own event queue. Even once
I figure this out, I'd still like to understand how to properly use
channels.)

Thanks for being so patient with me.

[James Mills]

I'm going to top post here! sorry!

Judging from your example, you're sending events to and from the
client and server right?

If so, you really ought to use and help us improve circuits.node which
is specifically designed around remote message passing to and from
other nodes (which is what you seem to be trying to do!)

Sorry it took me this long to figure this out!

Cheers
James

Sent from my iPad

[Mark Young]

Oooooh, I didn't even look at circuits.node. Will look at this
tomorrow hopefully. Thanks!

[James Mills]

No problems! let me know how it goes?

Cheers
James


Sent from my iPad

On 20/12/2012, at 17:47, Mark Young <mark...@gmail.com> wrote:

> Oooooh, I didn't even look at circuits.node. Will look at this
> tomorrow hopefully. Thanks!
>

[Mark Young]

1. When using the Write events, should the data be bytes or a string?
Looking at the sockets version, it appears that the data is supposed
to be in byte format (it directly uses the results from recv. E.g.
"data = self._sock.recv(self._bufsize)"), but the client and server in
the node package use a string as their buffer. If data is sent as a
stream of bytes, each node tries to directly append the recieved data
to the buffer, which causes a typeError in python 3, as you're not
allowed to directly append a bytes instance to a string instance. If
it's supposed to be a string, I get exceptions because TCPServer
expects data sent to support the buffer interface, which strings don't
do.

I would assume it's supposed to be bytes, but then I see things like
line 52 of circuits.node.server: "self.fire(Write(v.node_sock, "%s%s"
% (data, DELIMITER)))", where a string is sent. Is this just something
missed in the python2 -> python3 conversion?

2. Is the remote event supposed to be used directly? If so, what is
the meaning of the node parameter of the Remote init function "def
__init__(self, event, node...") How should it be specified?

3. Why does node.client take a host and a port instead of the bind
tuple that everything else uses? The inconsistancy makes using super
(when using multiple inheritance) more inconvenient. (Plus, the
inconsistancy is just a special case to have to learn.)

3. I've run into the same problem when trying to specify channels. If
no channel or the asterisk is specified, the messages arrive
successfully. Otherwise, nothing gets through. In my particular case,
it matters less, as I don't really need the channels, but I'd like to
understand what I'm doing wrong in the event that I eventually need
it.

Here're my updated example files using circuits.node:
test_server.py: http://circuits.codepad.org/5PrwqhTn
test_client.py: http://circuits.codepad.org/zm9xqKoa

Here's the output when the asterisk is used (Showing the bytes/string
exception):
http://circuits.codepad.org/1q0etP2k

Here's the output when "server" is specified in the fire call (Nothing
happens):
http://circuits.codepad.org/0BPUrUSm

If you happen to read this before Christmas, Merry Christmas by the
way! (I understand completely if this is ignored until after Christmas)

[James Mills]

On Mon, Dec 24, 2012 at 3:10 PM, Mark Young <mark...@gmail.com> wrote:

1. When using the Write events, should the data be bytes or a string?
Looking at the sockets version, it appears that the data is supposed
to be in byte format (it directly uses the results from recv. E.g.
"data = self._sock.recv(self._bufsize)"), but the client and server in
the node package use a string as their buffer. If data is sent as a
stream of bytes, each node tries to directly append the recieved data
to the buffer, which causes a typeError in python 3, as you're not
allowed to directly append a bytes instance to a string instance. If
it's supposed to be a string, I get exceptions because TCPServer
expects data sent to support the buffer interface, which strings don't
do.

I would assume it's supposed to be bytes, but then I see things like
line 52 of circuits.node.server: "self.fire(Write(v.node_sock, "%s%s"
% (data, DELIMITER)))", where a string is sent. Is this just something
missed in the python2 -> python3 conversion?

We apologize for this. We do not officially support Python 3

very well (yet). If you would be willing to supply simple patches to fix this

that would be greatly appreciated. Although please ensure unit tests pass

for anything you change :)

Yes they are supposed to be bytes.

2. Is the remote event supposed to be used directly? If so, what is
the meaning of the node parameter of the Remote init function "def
__init__(self, event, node...") How should it be specified?

Every node you add using the .add(name, host, port) API

gets a name. You reference the Node you wish to send the event

to via Remote(event, name)

Make sense?

 

Please by all means suggest improvements if there are improvements to be made :)

3. Why does node.client take a host and a port instead of the bind
tuple that everything else uses? The inconsistancy makes using super
(when using multiple inheritance) more inconvenient. (Plus, the
inconsistancy is just a special case to have to learn.)

You are right. This should probably be fixed and be more consistent with circuits.net.sockets components.

Can you fix this?

 

3. I've run into the same problem when trying to specify channels. If
no channel or the asterisk is specified, the messages arrive
successfully. Otherwise, nothing gets through. In my particular case,
it matters less, as I don't really need the channels, but I'd like to
understand what I'm doing wrong in the event that I eventually need
it.

I'll have a look at your code you've pasted below to see if I can figiure out whatyou're saying.

It would be useful if you could take tests/node/test_node.py and modify it to show the failure. 

Here're my updated example files using circuits.node:
test_server.py: http://circuits.codepad.org/5PrwqhTn
test_client.py: http://circuits.codepad.org/zm9xqKoa

Here's the output when the asterisk is used (Showing the bytes/string
exception):
http://circuits.codepad.org/1q0etP2k

Here's the output when "server" is specified in the fire call (Nothing
happens):
http://circuits.codepad.org/0BPUrUSm

If you happen to read this before Christmas, Merry Christmas by the
way! (I understand completely if this is ignored until after Christmas)

A Merry Christmas to you too!

cheers

James

[Mark Young]

(Sorry if my quoting etiquette is bad. I'm used to a more forum-based
style.)

On Dec 24, 12:20 am, James Mills <prolo...@shortcircuit.net.au> wrote:

> On Mon, Dec 24, 2012 at 3:10 PM, Mark Young <marky1...@gmail.com> wrote:
> > 1. When using the Write events, should the data be bytes or a string?
> > Looking at the sockets version, it appears that the data is supposed
> > to be in byte format (it directly uses the results from recv. E.g.
> > "data = self._sock.recv(self._bufsize)"), but the client and server in
> > the node package use a string as their buffer. If data is sent as a
> > stream of bytes, each node tries to directly append the recieved data
> > to the buffer, which causes a typeError in python 3, as you're not
> > allowed to directly append a bytes instance to a string instance. If
> > it's supposed to be a string, I get exceptions because TCPServer
> > expects data sent to support the buffer interface, which strings don't
> > do.
>
> > I would assume it's supposed to be bytes, but then I see things like
> > line 52 of circuits.node.server: "self.fire(Write(v.node_sock, "%s%s"
> > % (data, DELIMITER)))", where a string is sent. Is this just something
> > missed in the python2 -> python3 conversion?
>
> We apologize for this. We do not officially support Python 3
> very well (yet). If you would be willing to supply simple patches to fix
> this
> that would be greatly appreciated. Although please ensure unit tests pass
> for anything you change :)

Sure, I'll try. I've never contributed to a open-source project
before. How exactly do I do this? Does this look like a correct guide:
https://ariejan.net/2009/10/26/how-to-create-and-apply-a-patch-with-git
? (I mostly know how to use git, I've just never done patches)

(I'll have to fix the tests to run on windows and make sure they work
with python 3.)

> 2. Is the remote event supposed to be used directly? If so, what is
>
> > the meaning of the node parameter of the Remote init function "def
> > __init__(self, event, node...") How should it be specified?
>
> Every node you add using the .add(name, host, port) API
> gets a name. You reference the Node you wish to send the event
> to via Remote(event, name)
>
> Make sense?

Ooooh, I assumed add was called automatically during the connection.
This makes perfect sense now.

>
> Please by all means suggest improvements if there are improvements to be
> made :)
>
> 3. Why does node.client take a host and a port instead of the bind
>
> > tuple that everything else uses? The inconsistancy makes using super
> > (when using multiple inheritance) more inconvenient. (Plus, the
> > inconsistancy is just a special case to have to learn.)
>
> You are right. This should probably be fixed and be more consistent with
> circuits.net.sockets components.
> Can you fix this?

Once I figure out how to contribute, definitely.

Once I figure out the code a little better, I'd really like to add
more documentation throughout.

Thanks for being so helpful.

[Mark Young]

Oh, I see this uses hg, not git. I figured out how to create a patch
file now.

[James Mills]

Yes I was going to say (earlier):

Signup to http://bitbucket.org/

Clone circuits-dev

Make your changes, run tests, ensure everything is "good" :)

And send a pull request :)

cheers

James

James Mills / prologic

E: prol...@shortcircuit.net.au

W: prologic.shortcircuit.net.au

On Mon, Dec 24, 2012 at 4:23 PM, Mark Young <mark...@gmail.com> wrote:

Oh, I see this uses hg, not git. I figured out how to create a patch
file now.