Attempt to yield across C-call boundary

[Maanas Royy]

I have code a Single Sign On module on openresty using drizzle and lua. 

https://github.com/maanas/sso

Now i m in process of porting the same to use mysql.resty and more functionality. First and fore most is to provide multi domain authentication and redirection and full fledges web application firewall.

I m trying to evolve a minimal framework but i m hitting wall by getting this error. 

Attempt to yield across C-call boundary

I assume i m mixing C (nginx) and lua and that is causing this error. I tried to remove all the nginx call and still hitting this error. 

The files can be see here

https://github.com/maanas/lua-resty/tree/master/sso/lua.d

In db.lua i m trying to abstract the data base connection but i hit the unable to import error.

In user.lua i m trying to make a module of user which does some interaction with database though some function but i hit the 

Attempt to yield across C-call boundary

 

Any suggestion to push me to right direction. Any documentation or example module which uses a mysql database in modular way. I tried reading code for Lapis but no help.

Thanks

[Maanas Royy]

Does Statically linking pure Lua Module example is key.

If yes, then do i have to compile every time i make some changes?

[Yichun Zhang (agentzh)]

Hello!

On Thu, Sep 3, 2015 at 12:03 AM, Maanas Royy wrote:
> I have code a Single Sign On module on openresty using drizzle and lua.
> https://github.com/maanas/sso
>
> Now i m in process of porting the same to use mysql.resty and more
> functionality. First and fore most is to provide multi domain authentication
> and redirection and full fledges web application firewall.
> I m trying to evolve a minimal framework but i m hitting wall by getting
> this error.
>>
>> Attempt to yield across C-call boundary
>

Please check out the related section in ngx_lua's official documentation:

https://github.com/openresty/lua-nginx-module#lua-coroutine-yieldingresuming

One common mistake is to do operations that may require yielding (like
cosockets and ngx.sleep) on the top-level scope of your own Lua module
files (and thus the require CFunction in the LuaJIT VM will throw this
error).

If you still don't have any clue, please try to provide a minimal and
standalone example that we can comment on and help you further.

> Does Statically linking pure Lua Module example is key.
> If yes, then do i have to compile every time i make some changes?

Seems like a require() thing mentioned above :) No, static linking
should not be a must. Maybe you're just abusing some side effect of
it.

Regards,
-agentzh

[Maanas Royy]

Here is my code

nginx.conf

worker_processes 1;

error_log logs/error.log;

events {

    worker_connections 1024;

}

http {

    include /vagrant/sso/conf.d/*.conf;

}

sso.conf

lua_package_path "/vagrant/sso/lua.d/?.lua;;";

server {

listen 80;

server_name sso.maanas.co;

default_type 'text/plain';

# Set Database variable

set $db_host "127.0.0.1";

set $db_name "lua";

set $db_user "lua_dba";

set $db_pass "myluadatabasepassword";

location / {

content_by_lua "ngx.say('Hello,world!')";

}

location /test {

content_by_lua_file  lua.d/test.lua;

}

}

test.lua

-- Test lua file

local user = require "user"

ngx.say(user:get_hash())

— user.lua

local mysql = require "resty.mysql"

local db_host = ngx.var.db_host

local db_name = ngx.var.db_name

local db_user = ngx.var.db_user

local db_pass = ngx.var.db_pass

local db, err = mysql:new()

-- Instantiate a db connection

if not db then

    return nil

end

db:set_timeout(1000) -- 1 sec

local ok, err, errno, sqlstate = db:connect{

    host = db_host,

    port = 3306,

    database = db_name,

    user = db_user,

    password = db_pass,

    max_packet_size = 1024 * 1024 

}

if not ok then

    return nil

end

local _M = {

_VERSION = '0.01'

}

function _M.get_hash()

return db:query("SELECT SHA2(UUID(), 256) AS hash")

end

return _M

I m trying to push the all the function dealing with database into a sub class db.lua in some kid of ORM but i m stuck in first stage it self. I was trying to put the connection routine in db.lua and returning db variable but that also lead to C call boundary error. I them move then in user.lua and again tried to access though test.lua it gave same error.

db.lua 

local mysql = require "resty.mysql"

local db, err = mysql:new()

if not db then

    -- ngx.say("failed to instantiate mysql: ", err)

    return nil

end

db:set_timeout(1000) -- 1 sec

local ok, err, errno, sqlstate = db:connect{

    host = ngx.var.db_host,

    port = 3306,

    database = ngx.var.db_name,

    user = ngx.var.db_user,

    password = ngx.var.db_pass,

    max_packet_size = 1024 * 1024 

}

if not ok then

    -- ngx.say("failed to connect: ", err, ": ", errno, " ", sqlstate)

    return nil

end

return db

Thanks

--
You received this message because you are subscribed to a topic in the Google Groups "openresty-en" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openresty-en/bZR6D4Le-Xw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[DeJiang Zhu]

Hi,

You can try like this :)

function _M.get_hash()

        local db, err = mysql:new()

        local ok, err, errno, sqlstate = db:connect ...

return db:query("SELECT SHA2(UUID(), 256) AS hash")

end

--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.

[Maanas Royy]

Thank you,

How can we encapsulated the db:connect and all related function in to separate function. Is that possible?

Does this mean for each function call we need to load new connection.

I was thinking on using connection pooling so that in one request we open one connection only. 

We would be hitting db connection limit if we get large number of request per second.

Thanks once again.

[DeJiang Zhu]

Hi,

Usually, this is my way :)

-- db module

local _M = {}

local mt = { __index = _M }

function _M.new()

    local db = mysql.new()

    db:connect()

    return setmetatable({db = db}, mt)

end

function _M.get_hash(self)

    local db = self.db

    local sql = "..."

    db:query(sql)

end

function _M.keepalive(self)

    self.db:set_keppalive()

end

2015-09-06 10:45 GMT+08:00 Maanas Royy <m4m...@gmail.com>:

Thank you,

How can we encapsulated the db:connect and all related function in to separate function. Is that possible?

Does this mean for each function call we need to load new connection.

no, we don't have to. we can use one db connection in  different functions in one request.

but, we should not share the db connection cross request :)

I was thinking on using connection pooling so that in one request we open one connection only. 

We would be hitting db connection limit if we get large number of request per second.

There is an connnection pool inside:)

It will auto pick an idle connection from the pool in every `connect `

So, we should call `db:set_keepalive` to put an connection to pool when the connection not more need in the request

https://github.com/openresty/lua-resty-mysql#set_keepalive

http://wiki.nginx.org/HttpLuaModule#tcpsock:setkeepalive

[Yichun Zhang (agentzh)]

Hello!

On Sun, Sep 6, 2015 at 10:23 AM, Maanas Royy wrote:
> — user.lua
>
> local mysql = require "resty.mysql"
>
> local db_host = ngx.var.db_host
> local db_name = ngx.var.db_name
> local db_user = ngx.var.db_user
> local db_pass = ngx.var.db_pass
> local db, err = mysql:new()
>
> -- Instantiate a db connection
> if not db then
> return nil
> end
>
> db:set_timeout(1000) -- 1 sec
>
> local ok, err, errno, sqlstate = db:connect{

This is exactly what I was talking about in my previous email in this thread:

"One common mistake is to do operations that may require yielding (like
cosockets and ngx.sleep) on the top-level scope of your own Lua module
files (and thus the require CFunction in the LuaJIT VM will throw this
error)."

Here, you call db:connect() (which may yield) in the toplevel scope of
your own Lua module file user.lua, which may yield across the boundary
the C function require() you're calling in test.lua.

It does not make sense to initialize a cosocket object (here, the
mysql object) on a worker level since cosockets are per-request
objects. See

https://github.com/openresty/lua-resty-mysql#limitations

Try only calling such things in your own Lua functions instead, which
has a per-request context (i.e., a separate Lua stack), just as what
DeJiang Zhu has suggested.

Regards,
-agentzh

[Pierre-Yves Gérardy]

On Sun, Sep 6, 2015 at 3:54 PM, Yichun Zhang (agentzh)
<age...@gmail.com> wrote:
> This is exactly what I was talking about in my previous email in this thread:
>
> "One common mistake is to do operations that may require yielding (like
> cosockets and ngx.sleep) on the top-level scope of your own Lua module
> files (and thus the require CFunction in the LuaJIT VM will throw this
> error)."

I'm a bit late to the party, but I've rewritten require in pure Lua to
work around this very issue.

https://github.com/pygy/require.lua

It is as close as possible to the original C code, and in practice identical.

Add "require" as a Luarocks dependency, then, at the start of your code, do

require = require"require".require

Maybe it could be mentioned in the docs, if not incorporated in either
LuaJIT or OpenResty...

—Pierre-Yves