StringIndexOutOfBoundsException: String index out of range: -4

[Jeff]

Using an exiting Ubuntu 16.04 VM running go-agent v16.9.0 I created an ESX VM template of the VM and am deploying more Go Agents from it.  

The go-agent won't start on some of the deployed VM's.  There doesn't seem to be any specific reason or error in the logs as it appears there are no errors in the logs, but the go-server doesn't see it.

So I resort to doing 'apt-get purge go-agent' and then forcefully delete the /var/lib/go-agent folder before doing 'apt-get install go-agent'.

Once I reinstall, restore the /etc/default/go-agent configuration and restart the agent it shows up on the go-server, but I then can't do anything with it such as assign an environment.  The following error is in the logs:

2016-09-16 16:19:36,186 [loopThread] ERROR go.agent.service.SslInfrastructureService:104 - [Agent Registration] There was a problem registering with the go server.

java.lang.StringIndexOutOfBoundsException: String index out of range: -4

        at java.lang.String.charAt(String.java:658)

        at org.bouncycastle.util.encoders.Base64Encoder.decode(Unknown Source)

        at org.bouncycastle.util.encoders.Base64.decode(Unknown Source)

        at org.bouncycastle.util.io.pem.PemReader.loadObject(Unknown Source)

        at org.bouncycastle.util.io.pem.PemReader.readPemObject(Unknown Source)

        at com.thoughtworks.go.security.Registration.fromJson(Registration.java:43)

        at com.thoughtworks.go.agent.service.SslInfrastructureService$RemoteRegistrationRequester.readResponse(SslInfrastructureService.java:186)

        at com.thoughtworks.go.agent.service.SslInfrastructureService$RemoteRegistrationRequester.requestRegistration(SslInfrastructureService.java:178)

        at com.thoughtworks.go.agent.service.SslInfrastructureService.register(SslInfrastructureService.java:102)

        at com.thoughtworks.go.agent.service.SslInfrastructureService.registerIfNecessary(SslInfrastructureService.java:86)

        at com.thoughtworks.go.agent.AgentController.rpcLoop(AgentController.java:181)

        at com.thoughtworks.go.agent.AgentController.loop(AgentController.java:173)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)

        at org.springframework.scheduling.support.MethodInvokingRunnable.run(MethodInvokingRunnable.java:65)

        at org.springframework.scheduling.timer.DelegatingTimerTask.run(DelegatingTimerTask.java:70)

        at java.util.TimerThread.mainLoop(Timer.java:555)

        at java.util.TimerThread.run(Timer.java:505)

2016-09-16 16:19:36,186 [loopThread] ERROR thoughtworks.go.agent.AgentController:192 - [Agent Loop] Error occurred during loop:

java.lang.StringIndexOutOfBoundsException: String index out of range: -4

        at java.lang.String.charAt(String.java:658)

        at org.bouncycastle.util.encoders.Base64Encoder.decode(Unknown Source)

        at org.bouncycastle.util.encoders.Base64.decode(Unknown Source)

        at org.bouncycastle.util.io.pem.PemReader.loadObject(Unknown Source)

        at org.bouncycastle.util.io.pem.PemReader.readPemObject(Unknown Source)

        at com.thoughtworks.go.security.Registration.fromJson(Registration.java:43)

        at com.thoughtworks.go.agent.service.SslInfrastructureService$RemoteRegistrationRequester.readResponse(SslInfrastructureService.java:186)

        at com.thoughtworks.go.agent.service.SslInfrastructureService$RemoteRegistrationRequester.requestRegistration(SslInfrastructureService.java:178)

        at com.thoughtworks.go.agent.service.SslInfrastructureService.register(SslInfrastructureService.java:102)

        at com.thoughtworks.go.agent.service.SslInfrastructureService.registerIfNecessary(SslInfrastructureService.java:86)

        at com.thoughtworks.go.agent.AgentController.rpcLoop(AgentController.java:181)

        at com.thoughtworks.go.agent.AgentController.loop(AgentController.java:173)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)

        at org.springframework.scheduling.support.MethodInvokingRunnable.run(MethodInvokingRunnable.java:65)

        at org.springframework.scheduling.timer.DelegatingTimerTask.run(DelegatingTimerTask.java:70)

        at java.util.TimerThread.mainLoop(Timer.java:555)

        at java.util.TimerThread.run(Timer.java:505)

If I go to the server, disable the agent, I can then assign the environment and re-enable the agent and it seems to work, but it is annoying that I have to do this.

Is this normal behavior?  If not, anyone know why it does this?  Is there something maybe left over that relates to the old agent state that the 'apt-get purge' doesn't remove that I hadn't deleted yet?

Thanks!

--

Jeff Vincent
See my LinkedIn profile at:
http://www.linkedin.com/in/rjeffreyvincent

[Ketan Padegaonkar]

Jeff,

This issue was reported here and has been fixed for 16.10, due sometime this week.

For now, it's perfectly safe to ignore this exception in the agent logs — the agent is attempting to parse an empty JSON object for its certificates and failing on it, instead of gracefully handling an empty JSON response.

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ketan Padegaonkar]

I think I did not read the problem correctly, I feel that perhaps you're facing multiple issues because you're using VM templates.

Could you confirm if your VM template contains the directory `/var/lib/go-agents/config` and what files that directory contains? Ideally, that directory should not exist in the template, because the contents of that directory are supposed to be unique per agent instance.

It usually contains 2 things in it —

• the guid.txt file, which contains the unique identifier for each agent
  
• agent.jks file, which contains the unique client certificate for each agent

[Jeff]

Okay, those files are/were there and that did help me on the first issue where a new cloned VM couldn't connect to the go-server.

Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "go-cd" group.

To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.