SimpleResolver

[Ian]

has anyone tried writing a service with a large dependency graph?

dependency resolution (classpath generation) seems to be taking a long
time with SimpleResolver... i.e. over 10 minutes and counting in my
ITFooServiceTest:


- locked <7f393fd80> (a java.io.BufferedInputStream)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:687)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:632)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:
1195)
- locked <7f392ace8> (a sun.net.www.protocol.http.HttpURLConnection)
at java.net.URL.openStream(URL.java:1010)
[snip]...
at org.rioproject.resolver.maven2.ArtifactUtils
$getArtifactPomURLFromRepository.call(Unknown Source)
[snip]...
at org.rioproject.resolver.maven2.PomParser.parse(PomParser.groovy:
174)
at org.rioproject.resolver.maven2.PomParser$parse.callCurrent(Unknown
Source)
at org.rioproject.resolver.maven2.PomParser.parse(PomParser.groovy:
48)
at org.rioproject.resolver.maven2.PomParser$parse$0.call(Unknown
Source)
at
org.rioproject.resolver.maven2.SimpleResolver.parseDependency(SimpleResolver.groovy:
157)
[snip ( a lot of duplicates here. lot of recursion) ]...
at org.rioproject.resolver.maven2.SimpleResolver.this
$2$parseDependency(SimpleResolver.groovy)
at org.rioproject.resolver.maven2.SimpleResolver$this
$2$parseDependency.callCurrent(Unknown Source)
at
org.rioproject.resolver.maven2.SimpleResolver.parseDependency(SimpleResolver.groovy:
168)
at org.rioproject.resolver.maven2.SimpleResolver.this
$2$parseDependency(SimpleResolver.groovy)
at org.rioproject.resolver.maven2.SimpleResolver$this
$2$parseDependency.callCurrent(Unknown Source)
at
org.rioproject.resolver.maven2.SimpleResolver.doResolve(SimpleResolver.groovy:
125)
[snip]...
at
org.rioproject.resolver.maven2.SimpleResolver.getClassPathFor(SimpleResolver.groovy:
89)
[snip]...
at
org.rioproject.test.ProjectModuleResolver.getClassPathFor(ProjectModuleResolver.groovy:
82)
[snip]...
at
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:
159)
at
org.rioproject.test.ProjectModuleResolver.getClassPathFor(ProjectModuleResolver.groovy:
61)
[snip]...
at
org.rioproject.resolver.ResolverHelper.resolve(ResolverHelper.groovy:
44)
at
org.rioproject.cybernode.ServiceBeanLoader.provisionService(ServiceBeanLoader.java:
915)
- locked <7fc457bc8> (a java.lang.Class for
org.rioproject.cybernode.ServiceBeanLoader)
at
org.rioproject.cybernode.ServiceBeanLoader.load(ServiceBeanLoader.java:
263)
at org.rioproject.cybernode.JSBDelegate$1.run(JSBDelegate.java:540)

I wrote an implementation of Resolver using Maven 3's Aether API and
it seems to run pretty fast.

before I go down the route of trying to use that ,i wanted to check to
see if maybe something is not right with my setup or maybe if
SimpleResolver is not actually slow and it's just hung doing the
initial population of the local maven repo.

but in reviewing the code, I saw that it seems to be trying to use
sensible maven defaults (therefore there should be no initial
population of the local repo if I've already downloaded most of the
artifacts...)

one thing that may be unique about my setup is that I have a mirror of
central in my settings.xml (rather than adding my corporate repo as an
additional maven repo, I proxy central).

any thoughts or guidance?

[Dennis Reedy]

On Apr 25, 2011, at 514PM, Ian wrote:

> has anyone tried writing a service with a large dependency graph?

Yep

>
> I wrote an implementation of Resolver using Maven 3's Aether API and
> it seems to run pretty fast.

Thats planned, I just havent gotten to it (http://jira.rio-project.org/browse/RIO-290). Care to share? :) Perhaps you could contribute what you've got to the referenced issue?

>
> before I go down the route of trying to use that ,i wanted to check to
> see if maybe something is not right with my setup or maybe if
> SimpleResolver is not actually slow and it's just hung doing the
> initial population of the local maven repo.

If it's a complicated dependency graph SR can be very slow

[iang...@gmail.com]

ok - I'm still trying to get the hang of git here, trying to learn too many things at once. 

This patch (now attached to JIRA)  somehow has an earlier revision of my changes and then another change set, but I'm guessing if you apply it it should come out with a working version. Let me know and I can make another one. 

Probably lots of things worth stealing here, not sure if my implementation of the interface is even correct (i noticed RemoteRepositories are passed into Resolver, that I don't handle, i'm guessing it's because the opstrings support the specification of repos somehow).

Any hints on how to set it up so the ServiceLocator finds my instance rather than SimpleResolver? I wasn't able to grok where that binding happens.

thanks

ian

[Dennis Reedy]

On Apr 25, 2011, at 550PM, iang...@gmail.com wrote:

> ok - I'm still trying to get the hang of git here, trying to learn too many things at once.
>
> This patch (now attached to JIRA) somehow has an earlier revision of my changes and then another change set, but I'm guessing if you apply it it should come out with a working version. Let me know and I can make another one.

Looks great, thanks!

>
> Probably lots of things worth stealing here, not sure if my implementation of the interface is even correct (i noticed RemoteRepositories are passed into Resolver, that I don't handle, i'm guessing it's because the opstrings support the specification of repos somehow).

The repos are obtained from reading the pom at deployment and sent along with the ServiceElement to a Cybernode when the service is being instantiated.

>
> Any hints on how to set it up so the ServiceLocator finds my instance rather than SimpleResolver? I wasn't able to grok where that binding happens.

Under resolver/src/main/resources we'll need to add a file named:

META-INF.services/org.rioprojecy.resolver.Resolver, and within that file add a single line:

org.rioproject.resolver.aether.AetherResolver

This will allow the ServiceLocator to load the AR when ResolverHelper.getInstance() is invoked

Dennis

[iang...@gmail.com]

On Mon, Apr 25, 2011 at 5:18 PM, Dennis Reedy <dennis...@gmail.com> wrote:

The repos are obtained from reading the pom at deployment and sent along with the ServiceElement to a Cybernode when the service is being instantiated.

ahh, yes, i see, we can't depend on the settings.xml of the deploying machine to be the same on the remote machines.

we will probably need to support mirrors, too, in that case (as in my setup - which you will see if you read my first cut of code)

thanks

ian

[Dennis Reedy]

On Apr 25, 2011, at 550PM, iang...@gmail.com wrote:

> ok - I'm still trying to get the hang of git here, trying to learn too many things at once.
>
> This patch (now attached to JIRA) somehow has an earlier revision of my changes and then another change set, but I'm guessing if you apply it it should come out with a working version. Let me know and I can make another one.

Actually, it would be helpful if you could send your AetherService and AetherResolver.

Thanks

Dennis