Elastic Agent Autoregistration failing v21.3.0

[Reza Etezal]

We're working on an upgrade from v19.7 to v21.3. For the upgrade, I've restored postgres and config backups into a whole new environment.

* Ran the db-migrator and upgraded from old to new db schema

* Purged some nodes from the cruise-config.xml (to get rid of all elastic configs, including all elastic and non-elastic agents), and changed other values to prevent conflict with "prod" instance.

* Installed latest "release" version (7.0.0-4) of the ECS Elastic Plugin and configured a new cluster

I then unpaused a single pipeline and assigned it the Elastic profile. The task gets created in ECS and I can see it running on the EC2 instance. This is where the trouble starts. At this point, the agent tries to "auto-register" with the server and is rejected

2021-11-09 02:23:30,499 ERROR [qtp1711838422-42] AgentRegistrationController:242 - Rejecting request for registration. Error: HttpCode=[422] Message=[Elastic agent registration requires an auto-register agent key to be setup on the server. The agentAutoRegisterKey: [89a52addc1b241c0be4c81e9fd2068cb] is either not provided or expired. Agent-id: [GoCD64cbbd28b581471386946cc75bc03ad2], Plugin-id: [com.thoughtworks.gocd.elastic-agent.ecs]] UUID=[78552c7b-e179-4713-8a96-4df9ba7845a0] Hostname=[727b6411a118]ElasticAgentID=[GoCD64cbbd28b581471386946cc75bc03ad2] PluginID=[com.thoughtworks.gocd.elastic-agent.ecs]

I've checked on the agentAutoRegisterKey and made sure that the value in the cruise-config.xml and the one being sent by the agent match. I've switched it over to new GUIDs several times (with and without dashes) and every time I get the same message logged on the server and a similar message on the elastic agent:

2021-11-09 02:24:25,694 ERROR [scheduler-1] SslInfrastructureService:173 - Error occurred during agent registration process: Elastic agent registration requires an auto-register agent key to be setup on the server. The agentAutoRegisterKey: [89a52addc1b241c0be4c81e9fd2068cb] is either not provided or expired. Agent-id: [GoCD64cbbd28b581471386946cc75bc03ad2], Plugin-id: [com.thoughtworks.gocd.elastic-agent.ecs]

There is an NGINX reverse proxy that sits in front of the server and is configured according to https://docs.gocd.org/21.3.0/installation/configure-reverse-proxy.html#agents-and-reverse-proxies.

The way it seems to me is that the agentAutoRegisterKey="89a52addc1b241c0be4c81e9fd2068cb key inside the server node in the XML is not being parsed properly as the server is unaware of this key having been set.

On a side note, I came across from discussions and (merged) PRs about using ephemeral EA autoregister keys, but wasn't able to find much in the way of documentation. Is this "ephemeral" autoregistration process implemented? Is the process documented anywhere?

[Reza Etezal]

I found out what we were doing wrong.

We build our own agent docker images. In the init script, we examine env vars to set the autoregister file values. We keep the agentAutoRegisterKey in a secret store and grab it during the init script. 

This allowed us to use the same image for Elastic and Static agents.

The Elastic Agent plugin sets two of the EA env vars that we required, one of which we were not making any accommodation for (GO_EA_AUTO_REGISTER_KEY). By modifying the init script slightly to look for the presence of this env var and making it set precedence over other ways of setting the autoregister key, I was able to get the elastic agent registered with the cluster.

[Chad Wilson]

Thanks for sharing that you managed to get this fixed. I must admit I was scratching my head a bit thinking what the issue could have been :-)

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/8bbd87dd-dcb6-438d-8a56-78baeab2cd30n%40googlegroups.com.