from nameko.rpc import rpc from nameko_cachetools import CachedRpcProxy class Service(object): name = "demo" other_service = CachedRpcProxy('other_service', failover_timeout=3)
@rpc def do_something(self, request): # this rpc response will be cached, further queries will be # timed and cached values will be returned if no response is # received in 3 seconds or an exception is raised at the # destination service other_service.get_value('hi')
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/57e23c2b-9e2f-46e4-bea8-728114b0def9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Looks like a good tool to use with config managers, if used as services.
2018-06-11 14:18 GMT-03:00 <santi@blameless.com>:
Hey everyone, I figured I'd announce here that I recently worked on a caching RPC implementation for nameko services and decided to make it open source for everyone to look at and improve:The idea is to increase resiliency in our infrastructure by caching specific services that tend to be read-only and could become single-point-of-failure for the rest of the infra.Here's the gist of it:from nameko.rpc import rpc from nameko_cachetools import CachedRpcProxy class Service(object): name = "demo" other_service = CachedRpcProxy('other_service', failover_timeout=3)@rpc def do_something(self, request): # this rpc response will be cached, further queries will be # timed and cached values will be returned if no response is # received in 3 seconds or an exception is raised at the # destination service other_service.get_value('hi')
I did build two different strategies: cache on issues (use the cache if something goes wrong at the destination), and cache first (use the cache before even talking to the destination). Please share your thoughts and feedback!Santi @ Blameless
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
Yeah, that was our actual use case. All our system's settings are centralized in a service and others pull those on demand when needed. Caching was essential to improve reliability of the whole thing if settings went down.
On Mon, Jun 11, 2018 at 11:03 AM, Guilherme Caminha<gp...@cin.ufpe.br>wrote:
Looks like a good tool to use with config managers, if used as services.
2018-06-11 14:18 GMT-03:00 <sa...@blameless.com>:
Hey everyone, I figured I'd announce here that I recently worked on a caching RPC implementation for nameko services and decided to make it open source for everyone to look at and improve:The idea is to increase resiliency in our infrastructure by caching specific services that tend to be read-only and could become single-point-of-failure for the rest of the infra.Here's the gist of it:from nameko.rpc import rpc from nameko_cachetools import CachedRpcProxy class Service(object): name = "demo" other_service = CachedRpcProxy('other_service', failover_timeout=3)@rpc def do_something(self, request): # this rpc response will be cached, further queries will be # timed and cached values will be returned if no response is # received in 3 seconds or an exception is raised at the # destination service other_service.get_value('hi')
I did build two different strategies: cache on issues (use the cache if something goes wrong at the destination), and cache first (use the cache before even talking to the destination). Please share your thoughts and feedback!Santi @ Blameless
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+...@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
If you have two dependencies that you want to use together (e.g. RabbitMQ and a database) you have two good options:1. Glue them together using a service method. This is the "obvious" way but will feel wrong if you want to have the interaction abstracted away and not cluttering up your service.2. Make a combined DependencyProvider that talks to both dependencies. This will nicely abstract them away behind a single interface.A third option is to use a 'contract" of sorts between DPs. All bound Extensions have access to the others via the service container. so you can access another DP by name if it exists. This couples the extensions together though, so is not my preference.The downside of the "combined" DP is that you may end up with duplicate resources. For example, your combined DP with an RPC proxy may be used in a service that also declares an RPC proxy. This may feel slightly wasteful but I generally prefer it for the cleaner interface that it gives you. Decent chunks of the "duplicate" resources are often shared at the process level anway, so it's not as wasteful as it may feel. For example Kombu uses a process level connection pool; even if you have duplicate message publishers, ultimately they won't be using many more underlying connections, which are the expensive bit.I guess the use-case here is a DependencyProvider that configures itself using settings fetched over RPC? In this case I would definitely recommend an embedded RPC proxy -- the reason is that you really want this setup to happen "out of band". The initial setup would need to happen during container setup, at which time the other extensions may not be ready to use.
On Thursday, June 14, 2018 at 3:11:35 AM UTC+1, Guilherme Caminha wrote:
I was wondering something now - is it possible for a dependency provider to access others? For example, suppose I want to have a database dependency provider. It would be interesting to use this to manage the access to the database. Is using a ClusterRpcProxy or something like that from within the dependency provider implementation a good way?Ideally, though, even the connection to RabbitMQ should be handled using a centralized settings manager.
2018-06-13 10:54 GMT-03:00 Santiago Suarez Ordoñez <santi@blameless.com>:
Sounds good! I'll tweak things to make sure the right version of cachetools can track each major version!
Thanks for the heads up
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/a5b78318-2097-41c8-b641-55617fd30694%40googlegroups.com.
This answer is super relevant and something we struggled with quite a bit for the same reason Guilherme pointed out. Thanks for the tip! Could we drop this into the docs somewhere? Maybe an FAQ or Troubleshooting section. I'd be happy to get that started and send a PR if you guys this it's a good idea.
On Thu, Jun 14, 2018 at 12:37 AM, Matt Yule-Bennett<bennett.matthew@gmail.com>wrote:
If you have two dependencies that you want to use together (e.g. RabbitMQ and a database) you have two good options:1. Glue them together using a service method. This is the "obvious" way but will feel wrong if you want to have the interaction abstracted away and not cluttering up your service.2. Make a combined DependencyProvider that talks to both dependencies. This will nicely abstract them away behind a single interface.A third option is to use a 'contract" of sorts between DPs. All bound Extensions have access to the others via the service container. so you can access another DP by name if it exists. This couples the extensions together though, so is not my preference.The downside of the "combined" DP is that you may end up with duplicate resources. For example, your combined DP with an RPC proxy may be used in a service that also declares an RPC proxy. This may feel slightly wasteful but I generally prefer it for the cleaner interface that it gives you. Decent chunks of the "duplicate" resources are often shared at the process level anway, so it's not as wasteful as it may feel. For example Kombu uses a process level connection pool; even if you have duplicate message publishers, ultimately they won't be using many more underlying connections, which are the expensive bit.I guess the use-case here is a DependencyProvider that configures itself using settings fetched over RPC? In this case I would definitely recommend an embedded RPC proxy -- the reason is that you really want this setup to happen "out of band". The initial setup would need to happen during container setup, at which time the other extensions may not be ready to use.
On Thursday, June 14, 2018 at 3:11:35 AM UTC+1, Guilherme Caminha wrote:
I was wondering something now - is it possible for a dependency provider to access others? For example, suppose I want to have a database dependency provider. It would be interesting to use this to manage the access to the database. Is using a ClusterRpcProxy or something like that from within the dependency provider implementation a good way?Ideally, though, even the connection to RabbitMQ should be handled using a centralized settings manager.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/a5b78318-2097-41c8-b641-55617fd30694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/jietgx5y.2709b9be-95eb-44b9-b36e-c44f9931765a%40we.are.superhuman.com.
Santiago, this might be a little off-topic (sorry for that) but could you share what tool you're using for your centralized settings?
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/a5b78318-2097-41c8-b641-55617fd30694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/jietgx5y.2709b9be-95eb-44b9-b36e-c44f9931765a%40we.are.superhuman.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAJ23YE95J0WL6uscF8xMoJyvSqNpxOcFOCr5cWZBicVsc%2B-mYw%40mail.gmail.com.
This answer is super relevant and something we struggled with quite a bit for the same reason Guilherme pointed out. Thanks for the tip! Could we drop this into the docs somewhere? Maybe an FAQ or Troubleshooting section. I'd be happy to get that started and send a PR if you guys this it's a good idea.
On Thu, Jun 14, 2018 at 12:37 AM, Matt Yule-Bennett<bennett.matthew@gmail.com>wrote:
If you have two dependencies that you want to use together (e.g. RabbitMQ and a database) you have two good options:1. Glue them together using a service method. This is the "obvious" way but will feel wrong if you want to have the interaction abstracted away and not cluttering up your service.2. Make a combined DependencyProvider that talks to both dependencies. This will nicely abstract them away behind a single interface.A third option is to use a 'contract" of sorts between DPs. All bound Extensions have access to the others via the service container. so you can access another DP by name if it exists. This couples the extensions together though, so is not my preference.The downside of the "combined" DP is that you may end up with duplicate resources. For example, your combined DP with an RPC proxy may be used in a service that also declares an RPC proxy. This may feel slightly wasteful but I generally prefer it for the cleaner interface that it gives you. Decent chunks of the "duplicate" resources are often shared at the process level anway, so it's not as wasteful as it may feel. For example Kombu uses a process level connection pool; even if you have duplicate message publishers, ultimately they won't be using many more underlying connections, which are the expensive bit.I guess the use-case here is a DependencyProvider that configures itself using settings fetched over RPC? In this case I would definitely recommend an embedded RPC proxy -- the reason is that you really want this setup to happen "out of band". The initial setup would need to happen during container setup, at which time the other extensions may not be ready to use.
On Thursday, June 14, 2018 at 3:11:35 AM UTC+1, Guilherme Caminha wrote:
I was wondering something now - is it possible for a dependency provider to access others? For example, suppose I want to have a database dependency provider. It would be interesting to use this to manage the access to the database. Is using a ClusterRpcProxy or something like that from within the dependency provider implementation a good way?Ideally, though, even the connection to RabbitMQ should be handled using a centralized settings manager.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/a5b78318-2097-41c8-b641-55617fd30694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/16a24ad9-5830-4c09-b010-3183b8ba58ad%40googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/bc3ecae0-a571-40ae-b967-857f41bd6755%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "nameko-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/CAEqTAZajuwDwsUgRqBgL%2BFRuaAP%2B9LCjgprMpHgLo1f3hjBaZg%40mail.gmail.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/a5b78318-2097-41c8-b641-55617fd30694%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to namek...@googlegroups.com.
--
You received this message because you are subscribed to a topic in the Google Groups "nameko-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nameko-dev/gvgr-tB5MW4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nameko-dev+unsubscribe@googlegroups.com.
To post to this group, send email to nameko-dev@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/nameko-dev/7f0f5788-187f-4b0e-b6e5-793a5863871f%40googlegroups.com.