design integration tests with JenkinsRule in remote env

[Kanstantsin Shautsou]

I'm searching the way to programatically describe integration tests that will be possible to copy and run jenkins instance with test code to some remote environment.

ATH JenkinsController do logically similar thing, but it uses UI/rest for interaction with remote jenkins instance while i need have ability call/interact with jenkins object (instance). 

From test code i want run some pre-defined environment in docker container, place jenkins instance (with test code) and execute it. In the end of test kill containers and continue composing different environment around the test. Ideally it should look like JenkinsRule that will be run in remote env. 

Is there any ways to do such communication? AFAIK slave.jar uses channel connection, but will require wrapping calls in Callables that is not very convenient.

[Jesse Glick]

On Thu, Sep 10, 2015 at 9:55 AM, Kanstantsin Shautsou
<kanstan...@gmail.com> wrote:
> Is there any ways to do such communication?

`/script` (or the CLI equivalent) is your best bet. From an ATH test
you can simply use `jenkins.runScript("…")`.

[Kanstantsin Shautsou]

Then i need install some junit jars to have ability execute asserts and matchers and produce somehow junit xml output that should be fetched back…
I also thought putting files under JENKINS_HOME/groovy.d and run remote instance.

> --
> You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/R2_tHJDF0KI/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3P249QALRnWRLuajyR7Us03%2BAbiCd35aVspA48H9WjpQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Kanstantsin Shautsou]

Seems that existed ATH code can be extended for creating channel, that can support automatic classloading and tests execution on remote side. 

The idea is to programmatically prepare complex environment (my test code), run remote JenkinsController (existed ATH), get channel connection to it (add such feature in ATH), call Junit test execution on channel and continue with other tests. How jenkins->jenkins communication organised in JoC(?)? Could somebody point on some examples?

Also for testing domain cookies i want get webclient from JenkinsRule that will connect using domain address (even if it localhost), that is hardcoded to http://localhost.  (Trying add ATH to project and mix with Docker container run). Is it possible to have changeable hostname?

On Thursday, September 10, 2015 at 8:20:43 PM UTC+3, Kanstantsin Shautsou wrote:

Then i need install some junit jars to have ability execute asserts and matchers and produce somehow junit xml output that should be fetched back…
I also thought putting files under JENKINS_HOME/groovy.d and run remote instance.
> On Sep 10, 2015, at 20:16, Jesse Glick <jgl...@cloudbees.com> wrote:
>
> On Thu, Sep 10, 2015 at 9:55 AM, Kanstantsin Shautsou
> <kanstan...@gmail.com> wrote:
>> Is there any ways to do such communication?
>
> `/script` (or the CLI equivalent) is your best bet. From an ATH test
> you can simply use `jenkins.runScript("…")`.
>
> --
> You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/R2_tHJDF0KI/unsubscribe.

> To unsubscribe from this group and all its topics, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

[Jesse Glick]

On Sun, Sep 13, 2015 at 11:50 AM, Kanstantsin Shautsou
<kanstan...@gmail.com> wrote:
> Seems that existed ATH code can be extended for creating channel, that can
> support automatic classloading and tests execution on remote side.
> The idea is to programmatically prepare complex environment (my test code)

I guess you mean you want Java test code (`src/test/java/`) which
links against core/plugin types, but which ought to be loaded not by
the Java application class loader, but by a loader inside the remote
Jenkins JVM.

http://hg.netbeans.org/main-silver/file/a976d656a742/nbjunit/src/org/netbeans/junit/NbModuleSuite.java#l819

is an example (in the NetBeans functional test system), though it
would be more applicable to adding this feature to `JenkinsRule` (if
we wanted it to use a more realistic class loading scheme).

> run remote JenkinsController (existed ATH), get channel connection to it
> (add such feature in ATH), call Junit test execution on channel

I suspect you would need to add a custom support plugin to the Jenkins
server (see, e.g., `LocalController` and `RemoteJenkinsProvider` which
add `form-element-path`) which would add a `AgentProtocol` that would
use `ChannelBuilder` to set up a remoting channel (see
`JnlpSlaveAgentProtocol` for an example) that trusts the client to
send class files (your test code) to the master, loading them in a new
`ClassLoader` based on `uberClassLoader` (cf. `GroovyshCommand`).

All should be possible, but difficult code to write—you will need to
know Remoting pretty intimately.

[asoto...@cloudbees.com]

Maybe we can use Arquillian to deploy Jenkins to server side, and since Arquillian can be executed in-container (this means that test is executed within your application), it can be a possible solution to fix this. Of course I don't know deeply the details of what you want to test and maybe it would need some hack on Arquillian part, but it could be another solution.

Alex.

[Kanstantsin Shautsou]

I tried to use standard CLI connection from test and it works fine with Describables that comes from core. Custom (plugin) Describable initialised in Callable through channel missing Descriptor because this object doesn't match by java link (more specifically `type` from clazz) for Describable loaded on remote side (master jvm). 

Looks like RemoteClassloader vs WebApp classloader works differently for different Describables. Can't standard CLI.channel() works with plugin Describable or i must implemented custom hole with Agent+uberclassloader?

[Kanstantsin Shautsou]

Finally i got it working with standard CLI, but:

1)  commit in master was related to my issue with absent Descriptor for plugin objects created from CLI callables. I defined slaveAgentPort in config.xml but it wasn't returned in headers for CLI after jenkins become available. CLI fallbacks to http connection, that

2) httpCli connection with -Dhudson.diyChunking=false https://issues.jenkins-ci.org/browse/JENKINS-23232 has different classloading?