Problem with ngx.timer.at and resty-ression

[Ming Fang]

After calling proxy_pass, I want to store the result into a session using resty-session.
To to that I'm using the ngx.timer.at workaround.
Here is what my code look like...

proxy_pass http://someserver;

header_filter_by_lua_block {
    local function saveProfile(premature, profileJSON)
        if premature then
            return
        end

        local profile = require("cjson").decode(profileJSON)
        local session = require "resty.session".start{ secret = secret }
        for key, val in pairs(profile) do
            session.data[key] = val
         end

         session:save()
      end

       local profileJSON = ngx.resp.get_headers()['profile']
       if profileJSON then
          ngx.timer.at(0, saveProfile, profileJSON)
       end
}

There error I'm getting is this...

2016/04/19 01:24:34 [error] 55#55: lua entry thread aborted: runtime error: /usr/local/openresty/luajit/share/lua/5.1/resty/session.lua:223: API disabled in the current context
stack traceback:
coroutine 0:
    [C]: in function '__index'
    /usr/local/openresty/luajit/share/lua/5.1/resty/session.lua:223: in function 'open'
    /usr/local/openresty/luajit/share/lua/5.1/resty/session.lua:285: in function 'start'
    header_filter_by_lua:11: in function <header_filter_by_lua:3>, context: ngx.timer, client: 192.168.1.175, server: 0.0.0.0:443

Question is, am I not using the ngx.timer.at workaround correctly?
Or is this a problem with resty-session?

Thank you for your help.
--ming

[Lord Nynex]

Hello,

Please have a look at the header_filter_by_lua directive documentation at https://github.com/openresty/lua-nginx-module#header_filter_by_lua. This tells you which API's are disabled in this phase. There are a few possible reasons you're getting this error and it depends on your configuration.

1) Resty-session is configured with an external store for session data. In this case, if session data is stored in redis or memecached, you will not have access to the cosocket API to read or write session data.

2) In it's default state, resty-session uses cookie based sessioning. Since you are calling save() on the session, you are attempting to generate a set-cookie header during an nginx phase that does not support any sort of response output (including headers).

Further, (in my humble opinion) it is ill advised to create a timer (light thread) per request in this way. It is unclear to me what you are trying to accomplish with this. It seems that you are persisting some sort of session state, which makes this operation best suited for a content_by_lua block. 

If you provide more details on your goal, I or someone else on the list may be able to provide some alternatives that will be more performant and work around the issue you've described. 

If your intention is to persist some data asynchronously, you'd want to use ngx.shared.DICT to create an in memory queue to push 'profileJSON' onto to be processed by a timer thread that reschedules itself in an init_by_lua block. Better still would be using an async queue. I highly recommend https://github.com/pintsized/lua-resty-qless for a redis backed priority queue for async work. It is quite scalable as all nginx instances (across servers) would chip away at the work queue. I've used this library in high volume situations and it's yet to fail me. 

Lastly, I'm not sure what 'profileJSON' is but the name leads me to believe it's a JSON serialized string. Your example seems to be reading this JSON out of headers. I sincerely advise against storing this type of data in HTTP headers. Simply put, it's not what headers are meant for. It is accident prone and may lead to truncated data. Further, a major problem with HTTP as a whole is the highly variable nature of headers. This is why HTTP2 (SPDY) has focused so much on an efficient header compression standard. 

-Brandon

--
You received this message because you are subscribed to the Google Groups "openresty-en" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openresty-en...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ming Fang]

Brandon,

Thank you very much for your (very detailed)response.

I am trying to implement SAML authentication using Lua.

Because I'm unable to find the pieces to do all of it natively in Lua, I had to delegate pieces to an external NodeJs service, hence the proxy_pass.

The flow goes like this.

1-SAML assertions comes into Nginx and hits an access_by_lua_block. 

2-That block sets up a proxy_pass to the NodeJs service.

3-NodeJs service authenticates and return the profileJSON as response header.

   I agree that's a hack, but it is necessary due to the way SAML works.

   Basically a positive response to SAML is a 302 redirect so the only thing I can do is attach the profileJSON as part of the redirect.

4-the header_filter_by_lua_block reads the profileJSON and stores it into resty-session

5-resty-session will save the profile into redis

Based on your information, it appears I will have to find a different way to do this.

I'm very grateful for your help.

Thanks,

--ming

[Lord Nynex]

Hello,

I think I follow your thought process with regard to upstream SAML. I think this is very possible to do but you may have overcomplicated it slightly. 

Here are just some notes/thoughts/possible alternatives. I leave it to you to independently validate which works best for you:

- First, do not underestimate the power/speed of lua/openresty. It is very possible for you to perform the upstream check inside your access_by_lua block. Unfortunately you would not have the power of the proxy_pass module but there are plenty of alternatives. Here are a few:

  * https://github.com/openresty/lua-nginx-module#ngxlocationcapture You can use ngx.location.capture (which is already a part of openresyt) to call an 'internal' (emphasis on internal. see named locations) to execute this upstream for you. Like ''location @saml { proxy_pass http://nodejs/; } location / { access_by_lua_block { res = ngx.location_capture("@saml"); check res.... persist res to redis... } }'

  * Using an HTTP client like https://github.com/pintsized/lua-resty-http and perform a raw HTTP request to your saml upstream directly. 

If this is performed in an access block, your content handlers are free to operate as usual. Provided your upstream is healthy and responsive, I think you will be surprised at the speed/performance you will achieve here. 

-Brandon

[Ming Fang]

Brandon,

I would love to use a pure Lua solution.

In fact I have one here

https://github.com/mingfang/docker-knginx/blob/master/saml/saml.lua

The problem with my current implementation is that I can't find a Lua library to do XML signature verification.

Therefore I resorted adding this https://github.com/bergie/passport-saml to the mix.

Using the information you've provided, I'm going to try to use lua-resty-http to hit passport-saml.

Basically replace lines 41-52 here https://github.com/mingfang/docker-knginx/blob/master/saml/saml.lua#L41

Thanks,

--ming

[Lord Nynex]

Hello, 

I think you are on the right track. 

I am unfamiliar with SAML message formats (detached vs enveloped signatures), however, as far as I understand it adheres to the XML Sig specification. Thankfully, libxmlsec is pretty standard for most linux operating systems. With a bit of care you can create FFI bindings into libxmlsec to perform signature validation for you. Please be aware that xmlsec makes use of pointers to context structures to maintain state, you must use care when allocating/deallocating this memory. 

Here is an example demonstrating the workflow for validating xml signatures https://www.aleksey.com/xmlsec/api/xmlsec-verify-with-key.html. This should be enough to quickly create module. 

This is, of course, not necessary though. Openresty/nginx comes with enough cryptographic primitives that you can implement a validation function in pure lua without the need for external libraries. There are many PHP implementations available to use as a point of reference. Here is one example https://github.com/MrMarchello/php-XmlDigitalSignature/blob/master/src/XmlDigitalSignature.php

- Brandon