Populate in-memory cache lua_shared_dict

23 views
Skip to first unread message

Kshitij Joshi

unread,
Apr 30, 2021, 5:23:33 PMApr 30
to openresty-en
I need to call an external API & populate an in-memory cache in lua_shared_dict at start of OpenResty (open to considering other possibilities).

I also want a kind of cronjob to run every X mins to update the in-memory cache.

Since I need to call an external API, I cannot use init_worker_by_lua as I cannot use http lua or ngx.location.capture.

Also, I want it to be a pure cronjob which means I don't want to invoke this through an external call.

Any help would be appreciated.

Thanks!

Piotr Przybylski

unread,
May 4, 2021, 12:55:34 PMMay 4
to openresty-en
You can use http clients in init_by_lua  - check out https://github.com/thibaultcha/lua-resty-socket. To refresh your cache you can run a timer within a privileged agent.

Kshitij Joshi

unread,
May 4, 2021, 2:50:40 PMMay 4
to openresty-en

So I have the following piece of code in the init_by_lua_file in the http context


local http = require "resty.http"

local httpc = http.newCoSocket()

where I have added


function _M.newCoSocket()

    local sock = socket.tcp() -- similar to ngx.socket.tcp(). 

    return setmetatable({. 

        sock = sock. 

    }, {__index = _M}). 

end. 

function _M:connect(host, port)

    local ok, err = self.sock:connect(host, port)

    if not ok then

        return nil, err

    end

    local times, err = self.sock:getreusedtimes() -- cosocket API

    if not times then

        return nil, err

    elseif times == 0 then

        -- handle connection

    end

end

to the http.lua file.


I get the following error on the startup


socket.lua:167: stack overflow


The line 167 is

local phase = get_phase()

Piotr Przybylski

unread,
May 4, 2021, 7:19:24 PMMay 4
to openresty-en
I'm not sure what's broken in your case, but what I am personally using is a global patch for ngx.socket, something like:

ngx.socket.tcp = require('resty.socket').tcp

at the top of init_by_file, before any other files/modules are loaded. This way external libraries, like resty.http, can be used without any changes.
Reply all
Reply to author
Forward
0 new messages