Secrets Engine plugin warnings: error closing client during Kill + plugin failed to exit gracefully

[Ryan Treat]

We (Venafi) have developed 2 secrets engine plugins for Vault (https://github.com/Venafi/vault-pki-backend-venafi and https://github.com/Venafi/vault-pki-monitor-venafi).  Both are working properly in terms of their functionality but we're seeing warnings like the following being logged repeatedly which is concern both to us and our mutual customers:

2020-01-02T21:27:06.607Z [WARN]  secrets.pki-backend-venafi.pki-backend-venafi_4459c816.pki-backend-venafi: error closing client during Kill: metadata=true err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2020-01-02T21:27:06.607Z [WARN]  secrets.pki-backend-venafi.pki-backend-venafi_4459c816.pki-backend-venafi: plugin failed to exit gracefully: metadata=true

This doesn't appear to be anything we've done wrong in our implementation because I see the same warnings when I load other secrets engine plugins I found, one of which is in a HashiCorp repo (https://github.com/hashicorp/vault-plugin-secrets-gcp and https://github.com/sethvargo/vault-secrets-gen/releases).  I didn't find an open issue for this in the Vault repo so I'm checking here before opening one.

Thanks!

Ryan

[Nick Cabatoff]

Hi Ryan,

Yes, this is unfortunately something you'll always see when enabling an external plugin. We launch such plugins first in metadata mode, which is a special mode that disables TLS, prevents the plugin from talking to storage or handling commands, and allows us to query the plugin to get some information needed for a proper initialization.  Then we kill it and allow the plugin to be lazily loaded later.  This was added in https://github.com/hashicorp/vault/pull/3255 in case you're curious for more context.

Feel free to open a GH issue as you say, I agree it would be desirable not to see these warnings.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vault-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vault-tool/611e4751-46c8-444b-b262-6d819e55b1f4%40googlegroups.com.

[Ryan Treat]

Thank you for the details, Nick.  I will follow through and open a GitHub issue for the enhancement.

On Thursday, January 2, 2020 at 2:34:36 PM UTC-8, Nick Cabatoff wrote:

Hi Ryan,

Yes, this is unfortunately something you'll always see when enabling an external plugin. We launch such plugins first in metadata mode, which is a special mode that disables TLS, prevents the plugin from talking to storage or handling commands, and allows us to query the plugin to get some information needed for a proper initialization.  Then we kill it and allow the plugin to be lazily loaded later.  This was added in https://github.com/hashicorp/vault/pull/3255 in case you're curious for more context.

Feel free to open a GH issue as you say, I agree it would be desirable not to see these warnings.

On Thu, Jan 2, 2020 at 4:59 PM Ryan Treat <ryan...@gmail.com> wrote:

We (Venafi) have developed 2 secrets engine plugins for Vault (https://github.com/Venafi/vault-pki-backend-venafi and https://github.com/Venafi/vault-pki-monitor-venafi).  Both are working properly in terms of their functionality but we're seeing warnings like the following being logged repeatedly which is concern both to us and our mutual customers:

2020-01-02T21:27:06.607Z [WARN]  secrets.pki-backend-venafi.pki-backend-venafi_4459c816.pki-backend-venafi: error closing client during Kill: metadata=true err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2020-01-02T21:27:06.607Z [WARN]  secrets.pki-backend-venafi.pki-backend-venafi_4459c816.pki-backend-venafi: plugin failed to exit gracefully: metadata=true

This doesn't appear to be anything we've done wrong in our implementation because I see the same warnings when I load other secrets engine plugins I found, one of which is in a HashiCorp repo (https://github.com/hashicorp/vault-plugin-secrets-gcp and https://github.com/sethvargo/vault-secrets-gen/releases).  I didn't find an open issue for this in the Vault repo so I'm checking here before opening one.

Thanks!

Ryan

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/vault/issues
IRC: #vault-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Vault" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vault...@googlegroups.com.