Stuck creating local jenkins update center

[jje]

ello Folks!
 
As my jenkins server is running in a subnet without internet access i was 
wondering if it's possible to configure jenkins to check for plugin updates 
on variable URL different from jenkins plugin centre one. 
As prop'ed on 
https://github.com/ikedam/backend-update-center2/wiki/How-to-create-your-own-Jenkins-Update-Center  
I've tried to create my own json file, but now I'm stuck.

I've did exactly the proposed steps and set up proper toolchain. As it is 
compiled as an maven prj., compiling was no problem. But maven actually 
fails creating the json Files.

Here is toolchain I've used:
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 14:51:28+0100)
Maven home: /usr/local/apache-maven                                                                                                                                                                                                                                           
Java version: 1.7.0_79, vendor: Oracle Corporation                                                                                                                                                                                                                            
Java home: /opt/Oracle_Java/jdk1.7.0_79/jre                                                                                                                                                                                                                                   
Default locale: en_US, platform encoding: UTF-8                                                                                                                                                                                                                               
OS name: "linux", version: "3.16.0-38-generic", arch: "amd64", family: "unix"           



As maven is supposed to, it gets the plugins needed automatically - I 
didn't modified pom-File anyway.
But trying to run generated jars, build process fails with null-ptr. 
execption:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.186s
[INFO] Finished at: Thu Jul 09 14:36:08 CEST 2015
[INFO] Final Memory: 12M/301M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:java (default-cli) on project update-center2: An exception occured while executing the Java class. null: InvocationTargetException: NullPointerException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:java (default-cli) on project update-center2: An exception occured while executing the Java class. null
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. null
        at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:346)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
        ... 19 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:291)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
        at java.lang.String$CaseInsensitiveComparator.compare(String.java:1176)
        at java.lang.String$CaseInsensitiveComparator.compare(String.java:1170)
        at java.util.TreeMap.getEntryUsingComparator(TreeMap.java:369)
        at java.util.TreeMap.getEntry(TreeMap.java:340)
        at java.util.TreeMap.get(TreeMap.java:273)
        at org.jvnet.hudson.update_center.LocalDirectoryRepository.listHudsonPlugins(LocalDirectoryRepository.java:132)
        at org.jvnet.hudson.update_center.Main.buildPlugins(Main.java:274)
        at org.jvnet.hudson.update_center.Main.buildUpdateCenterJson(Main.java:203)
        at org.jvnet.hudson.update_center.Main.run(Main.java:167)
        at org.jvnet.hudson.update_center.Main.run(Main.java:144)
        at org.jvnet.hudson.update_center.Main.main(Main.java:131)




What is going wrong? Check cmd-line:

mvn exec:java -Dexec.args="-id com.XXX.UC.jenkins -h /dev/null -o 
update-center.json -r release-history.json -repository 
http://192.168.10.XXX/apt-mirror/jenkins-plugins/ -hpiDirectory 
/var/lib/jenkins/plugins -nowiki -key jenkins-update-center.key 
-certificate jenkins-update-center.crt -pretty" -X 

Key and Certificate has been created as well. Exported variables look like 
follows:

export LANGUAGES="en"
export LC_MESSAGES="en_GB.UTF-8"
export LC_CTYPE="en_GB.UTF-8"
export LC_COLLATE="en_GB.UTF-8"
export LANG="en_GB.UTF-8"
export JAVA_HOME="/opt/Oracle_Java/jdk1.7.0_79/jre"
export PATH=$PATH:$JAVA_HOME/bin
export M2_HOME=/usr/local/apache-maven
export M2=$M2_HOME/bin
export PATH=$M2:$PATH




Afterwards I'm going to define additional update centers on the Advanced 
tab of the plugin manager. Anything I've missed?


Thanks for support in advance!

Joe

[jje]

As I've stated, i get null pointer exception in protected JSONObject buildPlugins() -->Main.java;

There is no NULL Pointer check for this statement:

    /**
     * Build JSON for the plugin list.
     * @param repository
     * @param redirect
     */
    protected JSONObject buildPlugins(MavenRepository repository, PrintWriter redirect) throws Exception {

        ConfluencePluginList cpl = nowiki?new NoConfluencePluginList():new ConfluencePluginList();
        cpl.initialize();

        int total = 0;

        JSONObject plugins = new JSONObject();
       
        for( PluginHistory hpi : repository.listHudsonPlugins() ) {

This actually causes build to crash....

[Andy Pemberton]

Totally biased, but I'd be remiss not to point out this functionality exists in CloudBees Jenkins Enterprise: https://www.cloudbees.com/products/cloudbees-jenkins-platform/enterprise-edition/features/custom-update-center-plugin

It comes with a slick UI for managing either your own manually uploaded plugins or proxying the upstream OSS update centers, etc.

[jje]

Hi Andy,

thx....but I'm not willing to pay for it. BTW - this is not a sales event, is it?

[Daniel Beck]

Maybe this fits your needs better?

https://github.com/yandex-qatools/juseppe

Doesn't work with a Maven index I think, but as you presumably want to offer one of every plugin in a flat(ish) structure for download, you can use that as data source for the JSON.

On 10.07.2015, at 10:36, jje <j...@huber-online.com> wrote:

> ....given one means - plugins available on your jenkins plugin folder.
>
> Am Freitag, 10. Juli 2015 10:35:42 UTC+2 schrieb jje:
> Now I've found out, that it does not matter, which installation is used(probably installed plugins are supposed to be available to final jenkins installation).
> But only the given one will apear on your Update-Center Site on Jenkins.
>
> That means, probably using a maven repository may be a better soltuion
>
> Am Freitag, 10. Juli 2015 08:56:44 UTC+2 schrieb jje:
> Further I've found out, that my local jenkins installation may be corrupted. Did I get this right:
> Either maven-created repo with plugins or jenkins installation with plugins used is needed, isn't it?
>
> Does creation of JSON with an other jenkins installation then the final one used make a difference to the created UC?
>
> I tried to put only one plguin into my JENK_HOME/plugins folder and JSON has been created as expected. But unfortunally our main jenkins server is not connected to www, so building UC with maven is not an option as mvn. plugin dependencien cannot be resolved.
>
> Joe
>
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/27f452b1-e847-47b9-8a66-607617d7c1c9%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[jje]

One more thing:
Null-Pointers are rather ugly......Finally reason for NULL-Ptr. exception has been probably file-type which couldn't be handled(have seen some backup-files e.g.). Havn't reproduced it yet.


Am Freitag, 10. Juli 2015 13:21:28 UTC+2 schrieb jje:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED
!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED!!!!!!!!!!!!!!!!!!!!!!!!!!!!SOLVED

FINAL SOLUTION:

As hpi-folder means actually only route to folder containing hpi files, I've failed because I gave path to JENK_HOME/plugins folder.
But simply path to folder containing plugins, which you expect to be listed in UC is expected instead.

1. Create your folder containing the HPI files(installed/not installed doesn't mattter)
2. Step through given manual with -hpiDirectory pointing to those

Then your json finally provides list of all the parsed headers of plugins provided in step 1, which actually should match your content on plug-in server.

Prost!!

[jje]

...As I've stated: Since some of manifest files are corrupted on plug-in repository we actually cloned to access localy, i found that exception handling is to be reworked.
Simply adding some exception handlers to proper place will fix this issue.

Jenkins UC seems to be able to handle created JSON though.

Cheers

[jje]

Last but not least:

Gettin' metadata via browser(JSON Index File) is now working as expected. But it is nearly impossible to move jenkins to download plugins. Even though either machine and jenkins user has access to local repository granted, which actually has been tested by getting plugin files(*.hpi) via wget http:/sljdflsjdfj/plugin.hpi out of console.

Having said that bare mind it is guaranteed that trusty chain is working as expected, as metadata  file wasn't downloaded otherwise if not.

My workaround is to put tinyproxy on my jenkins machine and configure jenkins properly. That way I'll get the plugins downloaded as expected. Hope that helps anyone.
BTW:Anyone a better solution?

Cheers

[jje]

Pls. note that our local repo containing plugin-files and jenkins master reside in same network either.

[jje]

I think that the following I'd giving a try as well:

-->Setup your local DNS server to redirect requests to official jenkins update center to your local webserver hosting the plugin repository.(updates.jenkins-ci.org to 192.168.168.10 e.g.)
Actually this is very simple by running a webserver indexing a folder containing your plugins.

In case there is no need to host your own plugins, one don't even need to generate JSON index file either. Rather consider to keep a local mirror having the same folder structure as official UC.

That's it!

Am Dienstag, 21. Juli 2015 15:50:11 UTC+2 schrieb jje: