custom data linking and lifecycle

[Hans]

Hi,

Moving from fastcgi to nxweb here, as fastcgi with C++ has proven to be too slow and fragile for me. Looks good so far (just have remember to include #include <stdbool.h>). 

Can you clarify 5 questions that I have?

1) on the linking of custom data to a request.

As I understand, nxweb_set_request_data will allow attaching some custom data to a request. The mechanism is based upon a custom key. The problem that I have is the following:

• If that key is static, there is a high risk that multiple simultaneous requests use the same custom data, so static keys (like you use in all of your examples) are forbidden.
• If that key is dynamic, the only way that I see to share that custom data between callbacks like on_post_data and on_request is to set up a custom lookup mechanism based on the request ID (req->uid). There is no way for me to derive the custom data or the key directly from a request.

In other words, I see no need for using nxweb_set_request_data for the linking of custom data to the request, as it is up to me to do the linking between the custom data and the request anyway. As I see it, it would have much more programmer friendly if the request struct held a free-to-use pointer that one could set and get. 

Do I understand this correctly or am I missing something here?

2) If I want to allocate memory (re: C++ object instances in libraries, etc) myself for the duration of the request, without going through nxb_alloc_obj, I know it is up to me to free the memory once the request is over. What is the best callback for that? I will want to use that allocated memory for objects that get referenced in callbacks like on_request, on_post_data_complete and in the finalize from nxweb_set_request_data. I obviously do not want to free the memory too early and do not want to create leaks in error situations. I understand that the finalize call is the right place for releasing objects?

3) If I want to use the finalizer (for example for closing the fd of a nxd_fwbuffer, or to release any objects), I am obliged to use nxweb_set_request_data. But I can use 0/NULL as the key, and still provide a custom data structure that I maintain myself, as that data structure will get passed to the finalize call. Am I correct here?

4) where can I find a good lifecycle description of the callbacks? What callbacks get fired when? I am missing that. If you can confirm that on_post_data is called before on_request (on any request with a body, otherwise on_request is the first), and that on_post_data will have all the headers loaded, then I personally do not need a full description anymore, but a full description might still be useful for other people.

5) do the callbacks risk being run from different threads over the lifetime of a request? If so, one would need cooperation mechanisms on any custom data linked to a request, which will complicate things a bit.

6) (bonus): any plans on moving to cmake?

Kind regards,

Hans

[Yaroslav]

Hi Hans,

On Mon, Aug 25, 2014 at 5:24 PM, Hans <hansb...@gmail.com> wrote:

Hi,

Moving from fastcgi to nxweb here, as fastcgi with C++ has proven to be too slow and fragile for me. Looks good so far (just have remember to include #include <stdbool.h>). 

Can you clarify 5 questions that I have?

1) on the linking of custom data to a request.

As I understand, nxweb_set_request_data will allow attaching some custom data to a request. The mechanism is based upon a custom key. The problem that I have is the following:

• If that key is static, there is a high risk that multiple simultaneous requests use the same custom data, so static keys (like you use in all of your examples) are forbidden.
• If that key is dynamic, the only way that I see to share that custom data between callbacks like on_post_data and on_request is to set up a custom lookup mechanism based on the request ID (req->uid). There is no way for me to derive the custom data or the key directly from a request.

In other words, I see no need for using nxweb_set_request_data for the linking of custom data to the request, as it is up to me to do the linking between the custom data and the request anyway. As I see it, it would have much more programmer friendly if the request struct held a free-to-use pointer that one could set and get. 

Do I understand this correctly or am I missing something here?

There should be no problem with static key, as there is a separate map of key->data for each individual request.

There was a pointer for custom data in earlier versions of nxweb. I removed it as there could be several modules/filters that need to attach their data. nxweb_set_request_data() has another advantage of allowing you to attach finalizer function that will be called after request is complete.

2) If I want to allocate memory (re: C++ object instances in libraries, etc) myself for the duration of the request, without going through nxb_alloc_obj, I know it is up to me to free the memory once the request is over. What is the best callback for that? I will want to use that allocated memory for objects that get referenced in callbacks like on_request, on_post_data_complete and in the finalize from nxweb_set_request_data. I obviously do not want to free the memory too early and do not want to create leaks in error situations. I understand that the finalize call is the right place for releasing objects?

The best way is to use finalizer function passed to nxweb_set_request_data(). You can attach finalizers without data by specifying key=0.

3) If I want to use the finalizer (for example for closing the fd of a nxd_fwbuffer, or to release any objects), I am obliged to use nxweb_set_request_data. But I can use 0/NULL as the key, and still provide a custom data structure that I maintain myself, as that data structure will get passed to the finalize call. Am I correct here?

Correct. 

4) where can I find a good lifecycle description of the callbacks? What callbacks get fired when? I am missing that. If you can confirm that on_post_data is called before on_request (on any request with a body, otherwise on_request is the first), and that on_post_data will have all the headers loaded, then I personally do not need a full description anymore, but a full description might still be useful for other people.

Unfortunately there are no docs apart from this mailing list.

I can confirm the order you described for on_post_data.

5) do the callbacks risk being run from different threads over the lifetime of a request? If so, one would need cooperation mechanisms on any custom data linked to a request, which will complicate things a bit.

All request callbacks are invoked from the same network thread servicing the request. Therefore no locks.

 

6) (bonus): any plans on moving to cmake?

Probably not. I've invested so much time in this bloody automake so I'd prefer sticking to it unless it breaks some day.

 

Kind regards,

Hans

--
You received this message because you are subscribed to the Google Groups "nxweb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nxweb+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hans]

Cool, thanks. Even better than I thought. The map per request thing was not something I knew it supported.  Nice.

Kind regards,

Hans