Documentation

[Huhlig]

Is there any documentation on how to use nxweb other than the example
source code. Especially on things like the various options, INWORKER,
etc?

[Yaroslav]

Unfortunately there is no full documentation. Please post questions here. I will try to answer them.

As for INWORKER-handlers these are specified by NXWEB_INWORKER flag. See benchmark.c module for example:

nxweb_handler benchmark_handler_inworker={.on_request=benchmark_inworker_on_request, .flags=NXWEB_HANDLE_GET|NXWEB_INWORKER};

[Yaroslav]

I will post here excerpts from my mailings that might be useful to nxweb users.

1. Memory allocation

When a module allocates memory for data it wants to send
to the client, the last access to this memory (ie. the actual send to client)
happens after the module on_request handler returned. So the
module can't free memory buffers it allocated before returning.

Is there an allocating/buffering API which would automatically handle free()
at the right time (a la libevent's buffer_events)?

Yes, there is. Look at nx_buffer.h. Both request and response structs have a pointer to the same nxb_buffer; it is called nxb. It gets initialized when request arrives and is automatically freed when request is complete. It is very similar to obstack (from glibc) and can be used in two ways:

1) Object allocation:

void* ptr1=nxb_alloc_obj(req->nxb, 1024);

void* ptr2=nxb_calloc_obj(req->nxb, 256);

void* ptr1copy=nxb_copy_obj(req->nxb, ptr1, 1024);

2) Character streaming (variable length buffer):

nxb_printf(req->nxb, "Test %d\n", 12);

// printf-like functions are very slow, avoid them if you care for speed:

nxb_append_str(req->nxb, "Test ");

nxb_append_uint(req->nxb, 12);

nxb_append_char(req->nxb, '\n');

// or even faster (no checking for room every call):

nxb_make_room(req->nxb, 20);

nxb_append_str_fast(req->nxb, "Test ");

nxb_append_uint(req->nxb, 12);

nxb_append_char_fast(req->nxb, '\n');

// finish the stream:

int result_size;

char* result=nxb_finish_stream(req->nxb, &result_size);

Beware that you can not have more than one unfinished stream per nxb, and all nxweb_response_append_* kind of functions do use the same request's nxb, so if you need to stream some characters in your handler do it and finish your stream before using any of these response construction functions. Whatever is left unfinished in nxb on return from on_request considered to be response body.

As opposed to standard obstack implementation nx_buffer has no restriction on interleaving stream and object allocation calls.

nx_buffer is also the most efficient way to allocate small memory blocks in nxweb. Technically there is no limit on size of objects you allocate through nx_buffer (apart from size parameter being of type int), but I think that huge memory chunks are better allocated in some other ways, which are not defined in nxweb so far.

[Marcel Wijnen]

Hi Yaroslav, Hi Huhlig,

@Yaroslav:
Thanks for writing this code! I'm very interested in it and will try to set it up this weekend. I'm also interested in documentation and willing to write something up once I understand your code. I'm somewhat of a beginner though. I'm quite experienced in C# but a lot of hard core C stuff is new to me.

@Huhlig: I guess you have been playing around with the code. Do you have some tips and tricks for me?

Thanks!

Marcel 

[Yaroslav]

Hi Marcel,

As I said earlier there is almost no documentation for nxweb. But I will try to answer specific questions in this group.

Yaroslav

[Hans]

Hi, 

can you elaborate a bit on the practical memory allocation limits you see with nx_buffer, as you seem to imply in the below message? nx_buffer uses (the apparently obsolete function) memalign() underneath. You do not use a list of 16KB buffers for the output, like you seem to do for the input, so I don't really see big problems here. 

OK, I shouldn't allocate gigabytes here, but would say 50MB be acceptable? Any problems you foresee with nxweb's internals (ignore overall server RAM starvation issues please)? 

Related question: Apart from the buffer that the worker thread would allocate, do you do any other significant memory allocations for output? How much memory would be needed for 1 reply using for example a 50MB output buffer? 

Kind regards,

HB

[Yaroslav]

The only limitation I see is the size of int. As I use int type to store buffer size, any size not higher than maximum signed int value should be safe enough. If you allocate more than one chunk (multiple nxb_finish, or nxb_alloc_obj) then you can store even more in one nx_buffer, as every chunk can store up to max_int bytes.

50Mb should not be an issue at all. If you use nx_buffer streams and know in advance that you will need that amount of memory I would advise using nxb_make_room() in order to avoid unnecessary memory re-allocations and data copying.

After worker completes its job nxweb might need to allocate more memory from the same nx_buffer in order to build response headers (typically less than 1K). If you use filters these might also need to allocate something (eg. templates filter might allocate as much as whole output buffer if you marked this response with Templates-ON flag). Otherwise nxweb does not duplicate your generated output, so no substantial additional memory is required.

--
You received this message because you are subscribed to the Google Groups "nxweb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nxweb+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Tolga Hoşgör]

How does make_room work exactly? Does it assure the response has a total of X bytes or allocate more on each call?

Thanks.

[Yaroslav]

It makes sure there are X more bytes pre-allocated in nx_buffer so you can use *_fast functions. Using make_room in advance with proper amount might reduce number of memory re-allocations.

Calling make_room with same number of bytes several times does not allocate more memory.

On Fri, Jun 28, 2013 at 3:25 PM, Tolga Hoşgör <fasdf...@gmail.com> wrote:

How does make_room work exactly? Does it assure the response has a total of X bytes or allocate more on each call?

Thanks.