watch keyprefix if one key modified, why return all keys?

[Ivan Jobs]

Hi guys,

I'm new to consul, and study it. 

`The "keyprefix" watch type is used to watch a prefix of keys in the KV store. It requires that the "prefix" parameter be specified. 

This watch returns all keys matching the prefix whenever any key matching the prefix changes.` 

why not just return the modified key instead of return a whole tree of keys?

really confusing now.

[Michael Fischer]

Hi Ivan,

This is normal behavior for Consul, due to the way the underlying query API is designed.  (The underlying API request is a blocking query to retrieve all keys with the matching prefix.)

If you want to retrieve only a modified key, you'll need to set up separate watches for each of the keys you're interested in, or write your own client to maintain the previous state of each key and then calculate the differences after the result set is returned.

Best regards,

Michael

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/0e247c47-7412-4bfb-b614-9e2bffcf3074%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gary OTC]

The way watch API (in keyPrefixWatch) is implemented is to 'List' all keys values and their metadata, after kv change is detected. If data size is large for a set of keys, when there is only one key/value is modified, the overhead is quite huge.

There are only two APIs operate on keyprefix, List() and Keys(). The first one returns all values, and the second one returns only the keys. If there exists an API that returns all keys and their metadata, I think it will help improve the performance in some cases.

Thanks

Gary

On Saturday, December 24, 2016 at 10:08:23 AM UTC-8, Michael Fischer wrote:

Hi Ivan,

This is normal behavior for Consul, due to the way the underlying query API is designed.  (The underlying API request is a blocking query to retrieve all keys with the matching prefix.)

If you want to retrieve only a modified key, you'll need to set up separate watches for each of the keys you're interested in, or write your own client to maintain the previous state of each key and then calculate the differences after the result set is returned.

Best regards,

Michael

On Sat, Dec 24, 2016 at 2:05 AM, Ivan Jobs <lone...@gmail.com> wrote:

Hi guys,

I'm new to consul, and study it. 

`The "keyprefix" watch type is used to watch a prefix of keys in the KV store. It requires that the "prefix" parameter be specified. 

This watch returns all keys matching the prefix whenever any key matching the prefix changes.` 

why not just return the modified key instead of return a whole tree of keys?

really confusing now.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.

To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.

[Kresten Krab Thorup]

.... but it seems that it should be fairly easy to filter returned entries/keys based on the ?index=I passed in, just like entries are filtered based on the acl.  Doing this filtering server side would be much more efficient for someone who wants to monitor changes/additions in a given tree.

[Michael Fischer]

That's a great point. Why don't you file a an issue describing your idea?  Some code would be even better. 

On Wed, Mar 8, 2017 at 5:12 AM Kresten Krab Thorup <kreste...@gmail.com> wrote:

.... but it seems that it should be fairly easy to filter returned entries/keys based on the ?index=I passed in, just like entries are filtered based on the acl.  Doing this filtering server side would be much more efficient for someone who wants to monitor changes/additions in a given tree.

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/60f356a2-4d68-4f91-8293-5fd97ff9073b%40googlegroups.com.