SCM Export/Import limits?

[Andrzej Penkowski]

Hello,

has anybody tried checking the above? From my understanding, if you want to version jobs in a single repo, you can set both import and export to use the same remote. While doing that for export, rundeck generates local repository files before proceeding any further. I have a couple of projects in for now, one of which has almost 750 jobs. The process takes ages, almost 10 minutes for these 750 jobs, during which opening Jobs tab results in a "Cannot serialize job" error, which goes away once all of the job files are generated. Nginx, doing as my front proxy, times out of course but the process continues in the background. Fortunately, all scheduled executions seem to be working fine during that time. Moreover, configuring Import afterwards, results either in null tree error or in jobs not being tracked. Configuring the same elements for a much smaller project (31 jobs) seems to be working okay, although for the jobs to start being tracked (import) I have to restart rundeck entirely - after that it both detects changes made from rundeck ui and in remote repo, which then can be pushed/pulled.

Project config is file based but I am wondering if datastorage for jobs and executions being mariadb is the culprit here ?? 

ps. Once i reproduce the errors again I may paste stacktraces.

[Andrzej Penkowski]

Updated to 2.6.9 today. Using separate dirs for export and import just in case. Displaying Job list after export setup after a few tries seems to be causing no error. Configuring Import however afterwards, results in 

Cannot get property 'tree' on null object

java.lang.NullPointerException: Cannot get property 'tree' on null object
	at org.rundeck.plugin.scm.git.GitImportPlugin.importSynchStateForStatus(GitImportPlugin.groovy:302)
	at org.rundeck.plugin.scm.git.GitImportPlugin.refreshJobStatus(GitImportPlugin.groovy:242)
	at org.rundeck.plugin.scm.git.GitImportPlugin.getJobStatus(GitImportPlugin.groovy:349)
	at org.rundeck.plugin.scm.git.GitImportPlugin.getJobStatus(GitImportPlugin.groovy:337)
	at rundeck.services.ScmService$_importStatusForJobs_closure15.doCall(ScmService.groovy:961)
	at rundeck.services.ScmService.importStatusForJobs(ScmService.groovy:956)
	at rundeck.controllers.MenuController$_closure10.doCall(MenuController.groovy:323)
	at rundeck.controllers.MenuController$_closure9.doCall(MenuController.groovy:231)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1360)
	at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:198)
	at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:539)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:288)
	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:115)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:97)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at com.dtolabs.rundeck.server.filters.AuthFilter.doFilter(AuthFilter.java:58)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1331)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:477)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:486)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
	at org.eclipse.jetty.server.Server.handle(Server.java:349)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:910)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) 

 at java.lang.Thread.run(Thread.java:745)