Populate in-memory cache lua_shared_dict

Skip to first unread message

Kshitij Joshi

Apr 30, 2021, 5:23:33 PMApr 30
to openresty-en
I need to call an external API & populate an in-memory cache in lua_shared_dict at start of OpenResty (open to considering other possibilities).

I also want a kind of cronjob to run every X mins to update the in-memory cache.

Since I need to call an external API, I cannot use init_worker_by_lua as I cannot use http lua or ngx.location.capture.

Also, I want it to be a pure cronjob which means I don't want to invoke this through an external call.

Any help would be appreciated.


Piotr Przybylski

May 4, 2021, 12:55:34 PMMay 4
to openresty-en
You can use http clients in init_by_lua  - check out https://github.com/thibaultcha/lua-resty-socket. To refresh your cache you can run a timer within a privileged agent.

Kshitij Joshi

May 4, 2021, 2:50:40 PMMay 4
to openresty-en

So I have the following piece of code in the init_by_lua_file in the http context

local http = require "resty.http"

local httpc = http.newCoSocket()

where I have added

function _M.newCoSocket()

    local sock = socket.tcp() -- similar to ngx.socket.tcp(). 

    return setmetatable({. 

        sock = sock. 

    }, {__index = _M}). 


function _M:connect(host, port)

    local ok, err = self.sock:connect(host, port)

    if not ok then

        return nil, err


    local times, err = self.sock:getreusedtimes() -- cosocket API

    if not times then

        return nil, err

    elseif times == 0 then

        -- handle connection



to the http.lua file.

I get the following error on the startup

socket.lua:167: stack overflow

The line 167 is

local phase = get_phase()

Piotr Przybylski

May 4, 2021, 7:19:24 PMMay 4
to openresty-en
I'm not sure what's broken in your case, but what I am personally using is a global patch for ngx.socket, something like:

ngx.socket.tcp = require('resty.socket').tcp

at the top of init_by_file, before any other files/modules are loaded. This way external libraries, like resty.http, can be used without any changes.
Reply all
Reply to author
0 new messages