monitoring api

[Julien Ammous]

Hi,

I wanted to give a try building a visualization tool for some time now and I finally got it started, but as I feared I did not get far on the celluloid side of things...

My problem is to have a way into celluloid to be notified about important events, some are easy to find others not so easy, here is what I have so far:

- Actor creation (working)

- Actor death (working)

- Actor named (working)

- Actors linked (failure)

The real challenge is to find ways to be notified without disturbing the core and with constant identifiers, currently I have reliable notifications for creation, death and naming but

I am going nowhere with linking, I tried hooking into two different spot (look for the commented "link_created" line) but both have their problems.

The constant problem is to get a reference to the Celluloid::Actor object, so far I sometime get the ruby object itself or the actor depending which one is linking which the other.

There is also other interesting events to monitor but I think those are the main ones to get me started.

Here is my current work: https://github.com/schmurfy/celluloid/tree/monitoring_api (This is a testing branch)

I anyone can help I would love to work on the interesting part instead of being stuck at ground level :/

[Tim Carey-Smith]

This sounds pretty awesome!
Have a look at my logging branch: https://github.com/halorgium/celluloid/compare/logging

We could have a google hangout and scheme about how to achieve this!

Jump on IRC sometime, I'd really enjoy this tooling to be built!

Cheers,
Tim

> --
> You received this message because you are subscribed to the Google Groups "Celluloid" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to celluloid-rub...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Julien Ammous]

I wanted to have some sort of proof of concept before digging further but when I get something working sure I will getback to you :)

for now I fear I will not be able to get all the events I really need without killing the Celluloid core (I get some pretty weird errors with my current link notification when trying to do anything more that just a print).

It looks like you hooked into the same places as me for the linking events, if am not sure why the target at this particular point in the codebase is not an actor :(

[Tim Carey-Smith]

On May 19, 2013, at 11:44 AM, Julien Ammous <schm...@gmail.com> wrote:

> I wanted to have some sort of proof of concept before digging further but
> when I get something working sure I will getback to you :)

Ah, I was thinking you might garner some insights from talking which would help a PoC.
I know I would get some understanding of what Celluloid might need.

> for now I fear I will not be able to get all the events I really need
> without killing the Celluloid core (I get some pretty weird errors with my
> current link notification when trying to do anything more that just a
> print).

I am not sure what you mean by killing?
Performance? Monkey-patching?

> It looks like you hooked into the same places as me for the linking events,
> if am not sure why the target at this particular point in the codebase is
> not an actor :(

This is what we need to make happen, an event-bus similar to the Notifications actor.
At the moment, we haven't really tackled this, but it would be great to have someone to take a stab at it.

Glad to talk, working code is optional :)

Ciao,
Tim

>
> On Sunday, 19 May 2013 19:31:07 UTC+2, Tim Carey-Smith wrote:
>>
>> This sounds pretty awesome!
>> Have a look at my logging branch:
>> https://github.com/halorgium/celluloid/compare/logging
>>
>> We could have a google hangout and scheme about how to achieve this!
>>
>> Jump on IRC sometime, I'd really enjoy this tooling to be built!
>>
>> Cheers,
>> Tim
>>

>> On May 19, 2013, at 3:20 AM, Julien Ammous <schm...@gmail.com<javascript:>>

>> an email to celluloid-rub...@googlegroups.com <javascript:>.

[Julien Ammous]

by killing I meant really killing, I managed to crash the system actors ^^

When I said I wanted to have some working code I did not meant the final product, I prefer to gather informations on what is possible and what is not and

eventually what might need to change before engaging into serious talk on what is actually needed, once I figure out some things that still bother me

I will try to catch you on IRC.

[Tony Arcieri]

On Sun, May 19, 2013 at 10:31 AM, Tim Carey-Smith <g...@spork.in> wrote:

This sounds pretty awesome!
Have a look at my logging branch: https://github.com/halorgium/celluloid/compare/logging

We could have a google hangout and scheme about how to achieve this!

Jump on IRC sometime, I'd really enjoy this tooling to be built!

Note that I'd recommend guarding this sort of debugging with a global variable, e.g.

    $CELLULOID_LOGGING = true 

On JRuby, thanks to InDy, when this is flipped off, it literally has zero performance overhead, as JRuby will use switchpoints to allow HotSpot to compile this completely out of the code.

--
Tony Arcieri

[Julien Ammous]

Tony, do you know how can I achieve what I wanted ? Since this is a question about celluloid internal I think you are well placed to answer it ^^

To clear up my question what I wanted to do is being able to write this (I don't want to write it per say but that's close enough):

Actor 1 created

Actor 2 Created

Actor 1 linked to Actor 2

Actor 1 died

Actor 2 died

Where 1 and 2 are any string/number identifying the actors to help me track their state change, so far I have it working for creation/destruction of actors but I cannot

find a way when actors are linked to get the same ids I got for creation/destruction :(

I tried using the object_id of the Actor object and the bare object, none of them are matched with the link (one of them is depending on where you place your "print")

What  currently manage to get with my branch above is:

Actor 1 created

Actor 2 Created

Actor 1 linked to Actor 45

Actor 1 died

Actor 2 died

(note the really unhelpful "45")

Any pointer/idea would be greatly appreciated :)

[Tony Arcieri]

Try mailbox.address... it's a UUID

--

You received this message because you are subscribed to the Google Groups "Celluloid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to celluloid-rub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Tony Arcieri

[Tim Carey-Smith]

The other thing to look at is the StackDumper.

>>>> Have a look at my logging branch: https://github.com/halorgium/**
>>>> celluloid/compare/logging<https://github.com/halorgium/celluloid/compare/logging>

[Julien Ammous]

Thanks but my problem is even lower than that, to catch the "link" event I tried two places in actor.rb, the first is when the LinkingResponse is received inside "linking_request" method:

Celluloid.link_created(self, receiver)

the second place is also in actor.rb in the method "handle_system_event" when LinkingRequest is received:

Celluloid.link_created(self, event.actor)

in both cases event.actor and receiver are not actors... it looks like they are bare object :s

I tested their class with __send__(:class) so I think they really are bare object but it makes no sense to me

any ideas ?

PS: mailbox.address will definitely prove useful nonetheless !

PS2: what the link_created method does nothing more than printing the class of the arguments

[Julien Ammous]

no ideas ?:(

[Tim Carey-Smith]

I think we'd want to publish these kinds of things instead of using global methods.
The current logging/notifications framework could be made to support these things.

It would be good to wait for some of the overhaul pieces which give better reasoning about Celluloid.
https://github.com/celluloid/celluloid/wiki/Overhaul-Plan
Number 4 particularly.

The other thing to consider is waiting for us to split the actor from the object.
https://github.com/celluloid/celluloid/pull/204

This might give a cleaner separation for understanding.

Ciao,
TIm

>>>>>> Have a look at my logging branch: https://github.com/halorgium/**
>>>>>> celluloid/compare/logging<https://github.com/halorgium/celluloid/compare/logging>

[Julien Ammous]

For now how the system is "plugged" into celluloid is really not a concern, I am confident there will eventually be a clean way to do it and my

current patch is just temporary. What I really wanted to do is work on the visualization itself but the problem is that I can't even get the basic informations I need out

of the celluloid core :/
For me there are two completely different problems and how to properly interface with the celluloid core is not my current one but it would surely be the next one

if only I could get started.

I wish to visualize a celluloid system I am currently working on now, so if anyone has any ideas on how to get a useable and unique identifier for an actor

allowing to know which one was linked to which other one no matter where in the code or how dirty gathering this information might look I get it !

I really can't imagine nobody knows, this seems like a pretty basic question for anyone who knows the celluloid core which I don't know enough about.

I don't even ask for code, just point me to the right place in the code I will take care of the rest.

I am sure I am not the only one wanting to see a celluloid system so please help me help you ;)

While this is not the way I want to take here is an example of what might be possible: http://www.youtube.com/watch?v=lHoWfeNuAN8

What I want to see are actors created/destroyed, message queues states (mean size, max, ...), actor links maybe even more.

Any help to find the right spot to insert my code into is welcome and really don't bother with whether this is a clean/maintainable way to do it,

I will really take anything in the current state of things.

[Josh Adams]

So there was a GSoC proposal to build something like the Typesafe Console for Celluloid, which would provide a lot of what you're talking about here.  I'm presently working on a decent-sized celluloid system, using websockets heavily for the user interface.  Since I already have a really good bedrock that gets me a lot of the way to something like the typesafe console in my existing ui, I am hoping to convince my customer to let me spend a little bit of time building something similar.

Obviously the big hold up with any tool like this would be getting detailed information on the currently-running system.  I know that Tim's celluloid debugging bits go a decent way towards this.  At present, actors aren't stored hierarchically afaik, but I believe that's on the list to be changed.  Building the system I have in mind wouldn't require that, for what it's worth - I'd want to show number of actors, let you dig into which ones are running, see messages per second stats, etc.  The basics from the typesafe console demo.  Don't intend to get the heap, GC, etc. stats in the first iteration.

What are your thoughts guys?  How close are we to having the metadata needed to get at this sort of information?  Would the best way to get the data out of the system into a visualizer be to inject a ConsoleDataManager actor into the running system, and have him ship the metadata over zmq or something to the actual visualizer, running in a different process?

Josh Adams
CTO | isotope|eleven
[cell] (205) 215-3957
[work] (877) 476-8671 x201

[Tony Arcieri]

On Tue, May 28, 2013 at 2:17 AM, Julien Ammous <schm...@gmail.com> wrote:

I wish to visualize a celluloid system I am currently working on now, so if anyone has any ideas on how to get a useable and unique identifier for an actor

I already told you that...

    2.0.0p0 :001 > class Foo; include Celluloid; end

     => Foo

    2.0.0p0 :002 > Foo.new.mailbox.address

     => "60246c16-f2d0-4e4c-8c98-000000000000" 

This is *the* address that uniquely identifies any actor within Celluloid. Please use it.

--
Tony Arcieri

[Julien Ammous]

Tony I got this and it provided me with an uid for the actors which is a start but what I can't get a hold of is a "link" event with the same mailbox id, what I am looking for is where to

plug myself to be able to call (that's pseudo code but you should get the idea):

actors_linked(actor1_uid, actor2_uid)

knowing that I already managed to get these two as a start:

actor_created(actor_uid)

actor_destroyed(actor_uid)

PS: actorX_uid would be the mailbox address for the associated actor

[Tony Arcieri]

I was talking to Tim about this a bit yesterday. I think it would be interesting for every actor to publish events about itself that other actors can optionally subscribe to. That idea isn't exactly trivial though, especially if we want to be performance conscious.

--

You received this message because you are subscribed to the Google Groups "Celluloid" group.
To unsubscribe from this group and stop receiving emails from it, send an email to celluloid-rub...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Tony Arcieri

[Julien Ammous]

Something like this would definitely be nice and I am sure we can find a way to only load it when needed (you decide whether or not to activate it).

[Julien Ammous]

I managed to get my missing data, here is what I was speaking about: https://dl.dropboxusercontent.com/u/1313066/temp_forum/celluloid_monitoring.png, that's a webapp using d3.js library to show a live view of your celluloid system. The application itself is is a set of actors and the client updates as things happen through a websocket (the prototype supports only one client).

Now that I finally managed to have the prototype working I can finally start working on the ui itself. There is still the issue of the celluloid changes themselves, for now I have a very minimal layer (the one I showed here) which is the bare minimum to gather the data, everything else is handled by the web node.

This is clearly not in a shareable state but I am curious to know if someone has any interest in helping out on this.

For now my goal is to have a simple view of your system to help understand it and how it behaves, it may evolve later but something like the TypeSafe console is clearly not in my current goals.

What I plan to show:

- actors (of course)

- links between them

- messages flow (not sure how to represent that yet)

[Tim Carey-Smith]

Very nice! I would love to see this available as a part of the Celluloid.dump.
SIGUSR2 -> stack dump + pic written out to a file.

In the future, the default links will be in a tree-like hierarchy.
Perhaps having an annotation on the default link to allow differentiation between manual and automatic linking.

Could you point us to the changes to Celluloid?
I'd like to incorporate anything which makes this easier.

Thanks,
Tim

>>>>> => "60246c16-f2d0-4e4c-8c98-**000000000000"

[Schmurfy]

I will try making a cleaner patch in the coming days but I don't think it can be more than the start of a discussion on how to allow this, I would like the core the provide some kind of hooks but unconnected by default to provide negative impact on performance and allow an external gem to plug itself in these hooks.

You received this message because you are subscribed to a topic in the Google Groups "Celluloid" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/celluloid-ruby/pitudmgoIQI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to celluloid-rub...@googlegroups.com.

[Tony Arcieri]

On Mon, Jun 17, 2013 at 12:31 PM, Schmurfy <schm...@gmail.com> wrote:

I will try making a cleaner patch in the coming days but I don't think it can be more than the start of a discussion on how to allow this, I would like the core the provide some kind of hooks but unconnected by default to provide negative impact on performance and allow an external gem to plug itself in these hooks.

If we can toggle the hooks on and off with a global variable, it will provide zero performance impact on JRuby when disabled

 

--
Tony Arcieri

[Julien Ammous]

I will open a pull request to continue the discussion, having a global variable enabling this is ugly though :s

[Julien Ammous]

Here it is: https://github.com/celluloid/celluloid/pull/284

On Monday, 17 June 2013 23:02:08 UTC+2, Tony Arcieri wrote: