Everything in groovy opstring must be serializable?

[Dawid Loubser]

Hmm, an interesting limitation I came across today: There are certain things (like ranges) that you cannot use in your Rio opstrings, becuase the are not Serializable. For example, using something like this in an opstring:

[ 1 .. 5 ].each { n -> /* Do stuff... */ }

Results in this deployment error:

java.rmi.MarshalException: error marshalling arguments; nested exception is: 

java.io.NotSerializableException: groovy.lang.IntRange

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:785)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659)

at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528)

at $Proxy9.deploy(Unknown Source)

at org.rioproject.monitor.ProvisionMonitorAdminProxy.deploy(ProvisionMonitorAdminProxy.java:214)

at org.rioproject.tools.cli.MonitorControl$DeployHandler.deploy(MonitorControl.java:241)

at org.rioproject.tools.cli.MonitorControl$DeployHandler.process(MonitorControl.java:145)

at org.rioproject.tools.cli.CLI.doInit(CLI.java:1531)

at org.rioproject.tools.cli.CLI.initCLI(CLI.java:1431)

at org.rioproject.tools.cli.CLI.main(CLI.java:1581)

However, this works fine (since it's just a list):

[ 1, 2, 3, 4, 5 ].each { n -> /* Do stuff... */ }

This might be something to think about - I think Rio should explicitly document these limitations in the Groovy opstring, or perhaps a different approach could be taken during deployment, i.e. the Groovy text should be sent over the remote interface (and parsed in the monitor itself) instead of the parsed OperationalString object?

Just thought I'd share. I hit this while building some scalability experiments to try and determine the cause of my random-non-deployment problems. Might this open a more general discussion as to what is, and is not, allowable in an Opstring? (I'd assumed anything you could do in Groovy would be fair game before today).

Dawid

[Dennis Reedy]

On Jan 2, 2013, at 1024AM, Dawid Loubser wrote:

Hmm, an interesting limitation I came across today: There are certain things (like ranges) that you cannot use in your Rio opstrings, becuase the are not Serializable. For example, using something like this in an opstring:

[ 1 .. 5 ].each { n -> /* Do stuff... */ }

Can you give a more complete example of what you're trying to do?

Results in this deployment error:

java.rmi.MarshalException: error marshalling arguments; nested exception is: 

java.io.NotSerializableException: groovy.lang.IntRange

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethodOnce(BasicInvocationHandler.java:785)

at net.jini.jeri.BasicInvocationHandler.invokeRemoteMethod(BasicInvocationHandler.java:659)

at net.jini.jeri.BasicInvocationHandler.invoke(BasicInvocationHandler.java:528)

at $Proxy9.deploy(Unknown Source)

at org.rioproject.monitor.ProvisionMonitorAdminProxy.deploy(ProvisionMonitorAdminProxy.java:214)

at org.rioproject.tools.cli.MonitorControl$DeployHandler.deploy(MonitorControl.java:241)

at org.rioproject.tools.cli.MonitorControl$DeployHandler.process(MonitorControl.java:145)

at org.rioproject.tools.cli.CLI.doInit(CLI.java:1531)

at org.rioproject.tools.cli.CLI.initCLI(CLI.java:1431)

at org.rioproject.tools.cli.CLI.main(CLI.java:1581)

However, this works fine (since it's just a list):

[ 1, 2, 3, 4, 5 ].each { n -> /* Do stuff... */ }

This might be something to think about - I think Rio should explicitly document these limitations in the Groovy opstring,

Sure thats reasonable.

[Dawid Loubser]

On Wednesday, January 2, 2013 5:30:35 PM UTC+2, Dennis Reedy wrote:

Can you give a more complete example of what you're trying to do?

Oh, this is just in some scalability testing I'm doing - firing up many distinct copies of the same service (with different names) to see at what point things break down.

So, inside my loop, I am declaring a service:

[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]

.each { n ->

        service (name: 'Worker' + n )

        {

            ...

            maintain 1

        }

    }

Obviously this is only useful in production, if one perhaps wants to control the services one deploys via some other factor or configuration. In our case, for example, I deploy differently-configured workers based on a map of client information (more complex version of what you do in your workflow example). This works fine though, because it does not use ranges. I wonder if the Groovy guys simply forgot to make IntRange serializable?

I just thought it was interesting that it works with an explicit list, but not a range. Not a problem for me.

This is all part of experiments I'm running to try and determine Rio's thread usage and practical scalability limits (versus perhaps limits in the JavaSpace (e.g. Outrigger) itself.

You'd be happy to know that, as a test, I can deploy 400 services in Rio, each with an association to a JavaSpace and a TransactionManager, and each creating three internal threads at startup.

For too many threads (over 2700!) but it runs, and CPU usage settles to about 10% (all that context switching, I imagine).

I'd love to talk about the possibility of a Rio Cybernode offering perhaps a shared cached thread pool - an ExecutorService perhaps - for use in (injected into?) deployed services, and possibly also for use internally in the association management? With many many services, the threads overhead becomes rough, even though all other system parameters are very happy. Each Jini Service acting totally on it's own, creating entirely it's own Threads etc, becomes very inefficient real fast. But that's for another day...

I have to say, my testing so far points to weirdness either in my own framework, or in Outrigger (and not in Rio, inherently). Rio seems to behave well - with the one caveat that, for eagerly-injected "uses" associations, it very often Advertises a service before an association proxy has been injected. I was under the impression that it would always first inject a proxy (that blocks until the service is available) before advertising. Works fine for "requires" though, so no problem.

I've not been wanting to abuse your extreme helpfullness until I have a very clear idea about what is going on, and I'm getting closer to that clarity :-) 

Have a great day.

Dawid

[Dennis Reedy]

On Jan 3, 2013, at 852AM, Dawid Loubser wrote:

On Wednesday, January 2, 2013 5:30:35 PM UTC+2, Dennis Reedy wrote:

Can you give a more complete example of what you're trying to do?

Oh, this is just in some scalability testing I'm doing - firing up many distinct copies of the same service (with different names) to see at what point things break down.

So, inside my loop, I am declaring a service:

[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]

.each { n ->

        service (name: 'Worker' + n )

        {

            ...

            maintain 1

        }

    }

Obviously this is only useful in production, if one perhaps wants to control the services one deploys via some other factor or configuration. In our case, for example, I deploy differently-configured workers based on a map of client information (more complex version of what you do in your workflow example). This works fine though, because it does not use ranges. I wonder if the Groovy guys simply forgot to make IntRange serializable?

I just thought it was interesting that it works with an explicit list, but not a range. Not a problem for me.

There should be no reason why the range should not work. The Groovy DSL gets parsed, and then the OperationalString is created as a result of parsing the DSL. Let me try and recreate this. If its a bug I'll add an issue and it will be slated for M2.

This is all part of experiments I'm running to try and determine Rio's thread usage and practical scalability limits (versus perhaps limits in the JavaSpace (e.g. Outrigger) itself.

I've been trying to recreate your issues with the Workflow example, have created > 100 services in the Cybernode and have yet to see the lifecycle issues you see.

You'd be happy to know that, as a test, I can deploy 400 services in Rio, each with an association to a JavaSpace and a TransactionManager, and each creating three internal threads at startup.

For too many threads (over 2700!) but it runs, and CPU usage settles to about 10% (all that context switching, I imagine).

Thats good to know, although the CPU usage should be better. One thing that can also be done is to fork services that may require their own JVM (for memory purposes). Just add fork:'yes' and you're all set.

I'd love to talk about the possibility of a Rio Cybernode offering perhaps a shared cached thread pool - an ExecutorService perhaps - for use in (injected into?) deployed services, and possibly also for use internally in the association management? With many many services, the threads overhead becomes rough, even though all other system parameters are very happy. Each Jini Service acting totally on it's own, creating entirely it's own Threads etc, becomes very inefficient real fast. But that's for another day...

You can add something like this to the platform. It would be added to the CommonClassLoader, and would be available to all services. Look at how to do that in config/platform/readme.txt

I have to say, my testing so far points to weirdness either in my own framework, or in Outrigger (and not in Rio, inherently). Rio seems to behave well - with the one caveat that, for eagerly-injected "uses" associations, it very often Advertises a service before an association proxy has been injected. I was under the impression that it would always first inject a proxy (that blocks until the service is available) before advertising. Works fine for "requires" though, so no problem.

No, eager means that you get the injection right away (hence the eager-ness).

I've not been wanting to abuse your extreme helpfullness until I have a very clear idea about what is going on, and I'm getting closer to that clarity :-) 

NP :)

Dennis

[Dennis Reedy]

On Jan 3, 2013, at 905AM, Dennis Reedy wrote:

On Jan 3, 2013, at 852AM, Dawid Loubser wrote:

On Wednesday, January 2, 2013 5:30:35 PM UTC+2, Dennis Reedy wrote:

Can you give a more complete example of what you're trying to do?

Oh, this is just in some scalability testing I'm doing - firing up many distinct copies of the same service (with different names) to see at what point things break down.

So, inside my loop, I am declaring a service:

[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50]

.each { n ->

        service (name: 'Worker' + n )

        {

            ...

            maintain 1

        }

    }

Obviously this is only useful in production, if one perhaps wants to control the services one deploys via some other factor or configuration. In our case, for example, I deploy differently-configured workers based on a map of client information (more complex version of what you do in your workflow example). This works fine though, because it does not use ranges. I wonder if the Groovy guys simply forgot to make IntRange serializable?

I just thought it was interesting that it works with an explicit list, but not a range. Not a problem for me.

There should be no reason why the range should not work. The Groovy DSL gets parsed, and then the OperationalString is created as a result of parsing the DSL. Let me try and recreate this. If its a bug I'll add an issue and it will be slated for M2.

I just verified that:

(1..50).each { n ->

    service(name: 'Hello-'+n) {

        ...

        maintain 1

    }

}

Works just fine. 50 services get created.

Is your syntax correct?

Dennis

[Dawid Loubser]

Thanks for all the helpful info, lots for me to mull over. 

And thanks for the thought that went into Rio (I didn't know about the platform capabilities, it's great...)

[Dawid Loubser]

On Thursday, January 3, 2013 4:17:35 PM UTC+2, dennisr wrote:

I just verified that:

(1..50).each { n ->

    service(name: 'Hello-'+n) {

        ...

        maintain 1

    }

}

Works just fine. 50 services get created.

Is your syntax correct?

I'm so sorry, you're right. I'm used to Haskell, where the syntax for declaring a list containing 0 to 50 is [0..50] - I somehow assumed that it's the same in Groovy, but now I see it's simply 0..50 . The funny thing is, [0..50] is clearly not syntactically invalid, but probably means something different - perhaps even declaring a list containing a list of 0 to 50 (nesting) ?

I changed my code, and it works fine now :-)