Skip to first unread message
in...@ecsystems.nl
Jan 6, 2015, 6:32:29 AM1/6/15
to openre...@googlegroups.com
In order to add a lot more control to nginx upstream(s) I am trying to get some things done, if anyone can assist, comment or suggest fixes/ideas see:
https://github.com/openresty/lua-upstream-nginx-module/issues/10
At the bottom there is a link (zip file) to what I've done so far, but it needs more work in .c
https://github.com/openresty/lua-upstream-nginx-module/issues/10
At the bottom there is a link (zip file) to what I've done so far, but it needs more work in .c
in...@ecsystems.nl
Feb 27, 2015, 5:11:27 PM2/27/15
to openre...@googlegroups.com
We've gotten a few steps further, it's working! with a huge thanks to vozlt (github) for a huge code boost!
What we have so far can be found here http://nginx-win.ecsds.eu/devtest/EBLB_upstream_dev2.zip
With the source, example nginx.conf, a Lua web GUI/control and curl examples.
(and yes this work on any OS)
As with the other functions it only works with one worker.
To get this to work for all workers I am thinking about some sort of Lua trigger, ideas / example code anyone ?
1. using shared memory (ngx.shared.DICT) to store the change request (http request) and issue some sort of trigger
2. a Lua function which is loaded in a worker (init_worker_by_lua), go to sleep and is triggered by ?? (ngx.sleep or ngx.timer.at ?)
3. avoid running trigger twice and avoid missing a trigger message when some workers are too busy to handle it strait away
Anyone any ideas ?
What we have so far can be found here http://nginx-win.ecsds.eu/devtest/EBLB_upstream_dev2.zip
With the source, example nginx.conf, a Lua web GUI/control and curl examples.
(and yes this work on any OS)
As with the other functions it only works with one worker.
To get this to work for all workers I am thinking about some sort of Lua trigger, ideas / example code anyone ?
1. using shared memory (ngx.shared.DICT) to store the change request (http request) and issue some sort of trigger
2. a Lua function which is loaded in a worker (init_worker_by_lua), go to sleep and is triggered by ?? (ngx.sleep or ngx.timer.at ?)
3. avoid running trigger twice and avoid missing a trigger message when some workers are too busy to handle it strait away
Anyone any ideas ?
Yichun Zhang (agentzh)
Feb 27, 2015, 9:22:20 PM2/27/15
to openresty-en
Hello!
On Fri, Feb 27, 2015 at 2:11 PM, info wrote:
> We've gotten a few steps further, it's working! with a huge thanks to vozlt
> (github) for a huge code boost!
>
Cool. Let's see if we can incorporate your new features into ngx_lua_upstream.
> To get this to work for all workers I am thinking about some sort of Lua
> trigger, ideas / example code anyone ?
> 1. using shared memory (ngx.shared.DICT) to store the change request (http
> request) and issue some sort of trigger
> 2. a Lua function which is loaded in a worker (init_worker_by_lua), go to
> sleep and is triggered by ?? (ngx.sleep or ngx.timer.at ?)
We need both. Take a look at how the lua-resty-upstream-healthcheck
library implements this based on existing features in
ngx_lua_upstream:
https://github.com/openresty/lua-resty-upstream-healthcheck
It works pretty well in production as far as I've heard :)
Regards,
-agentzh
On Fri, Feb 27, 2015 at 2:11 PM, info wrote:
> We've gotten a few steps further, it's working! with a huge thanks to vozlt
> (github) for a huge code boost!
>
> To get this to work for all workers I am thinking about some sort of Lua
> trigger, ideas / example code anyone ?
> 1. using shared memory (ngx.shared.DICT) to store the change request (http
> request) and issue some sort of trigger
> 2. a Lua function which is loaded in a worker (init_worker_by_lua), go to
> sleep and is triggered by ?? (ngx.sleep or ngx.timer.at ?)
library implements this based on existing features in
ngx_lua_upstream:
https://github.com/openresty/lua-resty-upstream-healthcheck
It works pretty well in production as far as I've heard :)
Regards,
-agentzh
in...@ecsystems.nl
Feb 28, 2015, 3:39:37 PM2/28/15
to openre...@googlegroups.com
Cool. Let's see if we can incorporate your new features into ngx_lua_upstream.
I've updated the dev2 archive with todays code and just finished testing it, ok so far, vozlt will do a fork merge request to your master so we can keep things in one place (after you are ok with the new / changed code) when this happens make some space for the Gui-Lua files if you can.
https://github.com/openresty/lua-resty-upstream-healthcheck
It works pretty well in production as far as I've heard :)
Almost what we need :) definitely worth using, I was thinking about some inter-worker communication protocol which could then also be used across multiple instances / farms for better (auto)scaling.
in...@ecsystems.nl
Mar 1, 2015, 5:43:32 PM3/1/15
to openre...@googlegroups.com
Almost what we need :) definitely worth using, I was thinking about some inter-worker communication protocol which could then also be used across multiple instances / farms for better (auto)scaling.
Well I've made an inter worker communication system and it works quite well.
small example logging with 4 workers taking a peer up:
2015/03/01 23:20:26 [error] 2748#3304: [lua] iworkcomproto.lua:15: timer activation for worker: 2748, context: ngx.timer
2015/03/01 23:20:31 [error] 2912#2888: [lua] iworkcomproto.lua:15: timer activation for worker: 2912, context: ngx.timer
2015/03/01 23:20:31 [error] 3568#3776: [lua] iworkcomproto.lua:15: timer activation for worker: 3568, context: ngx.timer
2015/03/01 23:20:31 [error] 3568#3776: [lua] iworkcomproto.lua:25: message for worker: 3568: #2912#luaupstfoo,2,1, context: ngx.timer
2015/03/01 23:20:31 [error] 3768#1028: [lua] iworkcomproto.lua:15: timer activation for worker: 3768, context: ngx.timer
2015/03/01 23:20:31 [error] 3768#1028: [lua] iworkcomproto.lua:25: message for worker: 3768: #3568#2912#luaupstfoo,2,1, context: ngx.timer
2015/03/01 23:20:31 [error] 2748#3304: [lua] iworkcomproto.lua:15: timer activation for worker: 2748, context: ngx.timer
2015/03/01 23:20:31 [error] 2748#3304: [lua] iworkcomproto.lua:25: message for worker: 2748: #3768#3568#2912#luaupstfoo,2,1, context: ngx.timer
2015/03/01 23:20:36 [error] 2912#2888: [lua] iworkcomproto.lua:15: timer activation for worker: 2912, context: ngx.timer
2015/03/01 23:20:36 [error] 3568#3776: [lua] iworkcomproto.lua:15: timer activation for worker: 3568, context: ngx.timer
2015/03/01 23:20:36 [error] 3768#1028: [lua] iworkcomproto.lua:15: timer activation for worker: 3768, context: ngx.timer
2015/03/01 23:20:37 [error] 2748#3304: [lua] iworkcomproto.lua:15: timer activation for worker: 2748, context: ngx.timer
2015/03/01 23:20:41 [error] 2912#2888: [lua] iworkcomproto.lua:15: timer activation for worker: 2912, context: ngx.timer
2015/03/01 23:20:41 [error] 3568#3776: [lua] iworkcomproto.lua:15: timer activation for worker: 3568, context: ngx.timer
Each worker deals with a message and adds itself so it doesn't process the message again.
but I am missing a feature, I'd like to do
QResult = ngx.shared.iworkcomproto:get("IWCP_MSG_PD%")
but obviously it doesn't work, other then going through all the keys 'ngx.shared.DICT:get_keys' is there a way to search with a wildcard?
The reason is that I need to add a random value to the key value (IWCP_MSG_PD_qwerty12345) to make the message unique to sort of create a Que of messages, going through all the keys all the time is not really efficient, unless someone else has a better idea how to Que such messages and pull them back out :)
Message has been deleted
in...@ecsystems.nl
Mar 2, 2015, 7:20:02 AM3/2/15
to openre...@googlegroups.com
but I am missing a feature, I'd like to do
QResult = ngx.shared.iworkcomproto:get("IWCP_MSG_PD%")
but obviously it doesn't work, other then going through all the keys 'ngx.shared.DICT:get_keys' is there a way to search with a wildcard?
The reason is that I need to add a random value to the key value (IWCP_MSG_PD_qwerty12345) to make the message unique to sort of create a Que of messages, going through all the keys all the time is not really efficient, unless someone else has a better idea how to Que such messages and pull them back out :)
Reply all
Reply to author
Forward
0 new messages