Warnings about authentication not being enabled from golang clientv3 library

[Jille Timmermans]

Hi,

I'm trying to use the Golang clientv3 library to create a watcher that writes out new config files and reloads e.g. nginx. However, the watcher doesn't actually seem to trigger. I'm getting a warning "etcdserver: authentication is not enabled" in the logs of my application which I hope is the root cause.

This is an empty etcd cluster, where I just ran 1) user add root and 2) auth enable.

It does work if I just use `etcdctl --endpoints https://etcd_etcd_1:2379 --ca-file /etcd-keys/etcd-ca.pem -u root:password watch /nginx/test`, so I suppose I'm somehow failing to use the client library properly.

$ ./etcd-file-sync --key /nginx/test
{"level":"warn","ts":"2022-01-15T22:26:23.545Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0001e5500/etcd_etcd_1:2379","method":"/etcdserverpb.Auth/Authenticate","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: authentication is not enabled"}
2022/01/15 22:26:23 Connected.
{"level":"warn","ts":"2022-01-15T22:26:23.546Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0001e5500/etcd_etcd_1:2379","method":"/etcdserverpb.Auth/Authenticate","attempt":0,"error":"rpc error: code = FailedPrecondition desc = etcdserver: authentication is not enabled"}
(clientv3.WatchResponse) {
 Header: (etcdserverpb.ResponseHeader) cluster_id:1213032040247930871 member_id:15572083284620800828 revision:1 raft_term:2 ,
 Events: ([]*clientv3.Event) {
 },
 CompactRevision: (int64) 0,
 Canceled: (bool) false,
 Created: (bool) true,
 closeErr: (error) <nil>,
 cancelReason: (string) ""
}

And nothing happens when I use etcdctl to change that key (which does get picked up by etcdctl-watch).

The code is fairly straightforward:

  c, err := clientv3.New(clientv3.Config{
    Endpoints:        []string{"https://etcd_etcd_1:2379"},
    DialTimeout:      15 * time.Second,
    AutoSyncInterval: time.Minute,
    TLS: &tls.Config{
      InsecureSkipVerify: true, // TODO
    },
    Username: os.Getenv("SYNCER_USER"),
    Password: os.Getenv("SYNCER_PASSWORD"),
  })
// error handling

  go c.Sync(ctx)
  wc := c.Watch(ctx, *key, clientv3.WithCreatedNotify())
  for e := range wc {
    spew.Dump(e)
  }

and the config is too:

      - ETCD_CERT_FILE=/etcd-keys/etcd_etcd_1.pem
      - ETCD_KEY_FILE=/etcd-keys/etcd_etcd_1-key.pem
      - ETCD_TRUSTED_CA_FILE=/etcd-keys/ca.pem
      - ETCD_PEER_CERT_FILE=/etcd-keys/etcd_etcd_1.pem
      - ETCD_PEER_KEY_FILE=/etcd-keys/etcd_etcd_1-key.pem
      - ETCD_PEER_TRUSTED_CA_FILE=/etcd-keys/ca.pem

[Jille Timmermans]

This turned out to be version mismatches.

gcr.io/etcd-development/etcd:latest points to v3.3.12. (I'll file an issue to get that updated)

etcdctl (from ubuntu focal) was 3.2.26.

I was using golang's clientv3 at v3.5.1.