Exception executing and following a job via the rundeck cli (SAXReader.java invalid XML character - Unicode: 0x8)

[Andrew Steady]

Hello fellow Rundeck users,

I get the following exception when running and following a long running job from the CLI using this command: run -f -p <my-project-name> -j my-group/my-job -- <my parameters>

The same job works when I run it from the rundeck UI. In fact even when kicked off via the CLI and it crashes, the job carries on running fine. It is just the CLI follow that is crashing.

ERROR WebserviceHttpClientChannel: Unable to parse result document: Error on line 1 of document  : An invalid XML character (Unicode: 0x8) was found in the element content of the document. Nested exception: An invalid XML character (Unicode: 0x8) was found in the element content of the document.
org.dom4j.DocumentException: Error on line 1 of document  : An invalid XML character (Unicode: 0x8) was found in the element content of the document. Nested exception: An invalid XML character (Unicode: 0x8) was found in the element content of the document.
	at org.dom4j.io.SAXReader.read(SAXReader.java:482)
	at org.dom4j.io.SAXReader.read(SAXReader.java:343)
	at com.dtolabs.client.utils.WebserviceHttpClientChannel.postMakeRequest(WebserviceHttpClientChannel.java:283)
	at com.dtolabs.client.utils.HttpClientChannel.makeRequest(HttpClientChannel.java:396)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:182)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:134)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:97)
	at com.dtolabs.client.services.RundeckAPICentralDispatcher.followDispatcherExecution(RundeckAPICentralDispatcher.java:819)
	at com.dtolabs.rundeck.core.cli.queue.QueueTool.followAction(QueueTool.java:484)
	at com.dtolabs.rundeck.core.cli.run.RunTool.followOutput(RunTool.java:504)
	at com.dtolabs.rundeck.core.cli.run.RunTool.jobrunAction(RunTool.java:489)
	at com.dtolabs.rundeck.core.cli.run.RunTool.go(RunTool.java:392)
	at com.dtolabs.rundeck.core.cli.BaseTool.run(BaseTool.java:107)
	at com.dtolabs.rundeck.core.cli.run.RunTool.main(RunTool.java:124)
Nested exception: 
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 5841; An invalid XML character (Unicode: 0x8) was found in the element content of the document.
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.dom4j.io.SAXReader.read(SAXReader.java:465)
	at org.dom4j.io.SAXReader.read(SAXReader.java:343)
	at com.dtolabs.client.utils.WebserviceHttpClientChannel.postMakeRequest(WebserviceHttpClientChannel.java:283)
	at com.dtolabs.client.utils.HttpClientChannel.makeRequest(HttpClientChannel.java:396)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:182)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:134)
	at com.dtolabs.client.services.ServerService.makeRundeckRequest(ServerService.java:97)
	at com.dtolabs.client.services.RundeckAPICentralDispatcher.followDispatcherExecution(RundeckAPICentralDispatcher.java:819)
	at com.dtolabs.rundeck.core.cli.queue.QueueTool.followAction(QueueTool.java:484)
	at com.dtolabs.rundeck.core.cli.run.RunTool.followOutput(RunTool.java:504)
	at com.dtolabs.rundeck.core.cli.run.RunTool.jobrunAction(RunTool.java:489)
	at com.dtolabs.rundeck.core.cli.run.RunTool.go(RunTool.java:392)
	at com.dtolabs.rundeck.core.cli.BaseTool.run(BaseTool.java:107)
	at com.dtolabs.rundeck.core.cli.run.RunTool.main(RunTool.java:124)
Error: Failed following output for execution: 2548

Looking in this file: ./logs/rundeck/ProjName/

bootstrap/bootstrap-hap/2548.txt

I can see the last line which comes out before the crash, and the first line which doesn't sitting side by side:

^^^08:54:26|INFO|root||3-rundeck-jobref:3-rundeck-jobref:1-exec|FTSJG5J-pxe||INFO: Creating new partitions on disks^^^
^^^08:54:28|INFO|root||3-rundeck-jobref:3-rundeck-jobref:1-exec|FTSJG5J-pxe||INFO: Setting LVM flags^^^

I understand the special char 0x8 to be the backspace key - not sure how that could be coming.

I am making this call from Jenkins as I have refactored a series of Jenkins jobs into a single, multi-step rundeck job. Much neater, except for this issue. I am specifically not using the Rundeck plug-in for Jenkins because it does not follow the jobs output.

Any help or suggestions would be much appreciated.

Many thanks,

Andy

[Moses Lei]

Hi Andy, the command that's generating that output, try piping it through tee and see if that fixes it. It might get rid of any control codes that are coming through to the output.

The issue that we don't escape it before we put it in the XML API output is a bug though.

Moses

--
Moses Lei
[ Professional Services | DTO Solutions, Inc. ]
[ mobile: +1 703.901.5969 | e-mail: ml...@dtosolutions.com | aim/gtalk: ml...@controltier.com | yahoo: moseslei | windows live (msn): ml...@dtosolutions.com ]

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Andrew Steady]

Hi Moses,

Thanks for the tip it got me thinking on the right lines. I wasn't sure how tee could solve it (feel free to share) but in the end I found that this worked:

mycommand  | tr -d "[:cntrl:]"

Thanks again,

Andy

[Greg Schueler]

I logged this bug here: https://github.com/dtolabs/rundeck/issues/313

[Andrew Steady]

Hi Greg,

I see there was some progress on this issue, thanks.

I wasn't quite sure how to work out when or how this might be made available, either GA or LA (private fix etc.).

I still see the issue in 1.5.1.

Thanks,

Andy

[Greg Schueler]

Hi Andy,

are you building rundeck yourself? if not perhaps you can try the snapshot builds on http://build.rundeck.org/ this is the latest: http://build.rundeck.org/job/rundeck-development/209/

let me know if that issue still remains, thanks

[Andrew Steady]

Hi,

Will do.

Is there a quick way to use a snapshot (copy a jar/war somewhere?) or do I need to re-install? I'm using debian.

Cheers,

Andy

[Greg Schueler]

Hi Andrew,

You would have to replace the expanded webapp dir (/var/lib/rundeck/exp/webapp) with the contents of the war file.

Using the snapshot is somewhat "bleeding edge" so the cleanest method would be to just use the standalone jar somewhere to verify the feature, but that might be harder to setup the appropriate node and command to run for you.

Otherwise, I would suggest making sure you have a backup of the vital parts of your install before replacing the webapp contents, or installing the snapshot .deb file.

[Andrew Steady]

Hi,

I just tried it using the expanded WAR technique on my local environment (not a big deal to rebuild if it goes belly up).

It worked :) thanks!

Now the question is how to get this into my production environment? What do you think / recommend?

Andy

--
You received this message because you are subscribed to a topic in the Google Groups "rundeck-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rundeck-discuss/JJ5RFAa3Of0/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to rundeck-discu...@googlegroups.com.

[Andrew Steady]

 I decided to use the snapshot in production, if any instability is found I can always revert quite easily as all the jobs etc. are source controlled. This bug fix is really useful so worth to try it.