libuv and timerfd

[Kees k]

Hello all,

I have a difficulty with integrating timerfd worker threads with another worker thread. Basically my app has two types of external threads:

1. worker thread that sleeps 10 ms ('does some blocking task') and returns

2. worker thread that is polls a timerfd and returns.

The difficulty starts only when I create more than three timers (TIMERFDS > 3):

1. The worker thread that sleeps only returns after 1 second instead of 10 ms.

2. timers work fine.

Test code below:

#include <v8.h>

#include <uv.h>

#include <node.h>

#include <iostream>

#include <time.h>

#include <errno.h>

#include <unistd.h>

#include <sys/timerfd.h>

#include <sys/select.h>

using namespace v8;

using namespace node;

using namespace std;

struct timerfd_t {

unsigned int fd;

int ret;

};

void wait_async(int);

void timerfd_async(uv_work_t* req) {

timerfd_t* request = (timerfd_t*)req->data;

char buf[32];

fd_set rfds;

FD_ZERO(&rfds);

FD_SET(request->fd, &rfds);

request->ret = select((request->fd+1), &rfds, NULL, NULL, NULL);

if (request->ret >= 0)

{

//poll ok, clear data

if (FD_ISSET(request->fd, &rfds))

read(request->fd, buf, 32);

//do not handle read values (POLLPRI and POLLIN

}

}

void timerfd_async_after(uv_work_t* req, int arg) {

timerfd_t* request = (timerfd_t*)req->data;

cout<<"timer fires"<<endl;

wait_async(request->fd);

}

void wait_async(int fd)

{

timerfd_t *request = new timerfd_t();

request->fd = fd;

uv_work_t* req = new uv_work_t();

req->data = request;

uv_queue_work(uv_default_loop(), req, timerfd_async, (uv_after_work_cb) timerfd_async_after);

}

Handle<Value> js_start(const Arguments& args)

{

HandleScope scope;

struct itimerspec interval;

interval.it_interval.tv_sec = 1;

interval.it_interval.tv_nsec = 0;

interval.it_value.tv_sec = 0;

interval.it_value.tv_nsec = 1;

int fd = timerfd_create(CLOCK_MONOTONIC, 0);

int ret = timerfd_settime(fd, 0, &interval, NULL);

wait_async(fd);

return Undefined();

}

timeval t1, t2, t3,t4;

struct sleep_t {

Persistent<Function> cb;

};

void sleep_async(uv_work_t* req)

{

gettimeofday(&t2, NULL);

usleep(10000);

gettimeofday(&t3, NULL);

}

void sleep_async_after(uv_work_t* req, int arg) {

HandleScope scope;

sleep_t* request = (sleep_t*)req->data;

gettimeofday(&t4, NULL);

printf("T1 %ld|%ld\n", t1.tv_sec, t1.tv_usec);

printf("T2 %ld|%ld\n", t2.tv_sec, t2.tv_usec);

printf("T3 %ld|%ld\n", t3.tv_sec,t3.tv_usec);

printf("T4 %ld|%ld\n", t4.tv_sec,t4.tv_usec);

Handle<Value> argv[1];

argv[0] = Undefined();

request->cb->Call(Context::GetCurrent()->Global(), 1, argv);

request->cb.Dispose();

delete request;

delete req;

}

Handle<Value> js_sleep(const Arguments& args)

{

HandleScope scope;

Local<Function> cb = Local<Function>::Cast(args[0]);

sleep_t* request = new sleep_t;

request->cb = Persistent<Function>::New(cb);

uv_work_t* req = new uv_work_t();

req->data = request;

gettimeofday(&t1, NULL);

uv_queue_work(uv_default_loop(), req, sleep_async, (uv_after_work_cb) sleep_async_after);

return Undefined();

}

extern "C" void init (Handle<Object> target) {

HandleScope scope;

target->Set(String::New("start"), FunctionTemplate::New(js_start)->GetFunction());

target->Set(String::New("sleep"), FunctionTemplate::New(js_sleep)->GetFunction());

}

void Initialize (Handle<Object> exports);

NODE_MODULE(test_module, init)

And javascript code

var util = require('util');

var events = require('events');

var test = require("./build/Release/test_module");

var TIMERFDS = 4;

for (var i = 0; i < TIMERFDS; i++) {

test.start();

}

var doSleep = function() 

{

test.sleep(doSleep);

}

doSleep();

[Ben Noordhuis]

On Wed, May 7, 2014 at 2:01 PM, Kees k <keeskwe...@gmail.com> wrote:
> Hello all,
>
> I have a difficulty with integrating timerfd worker threads with another
> worker thread. Basically my app has two types of external threads:
> 1. worker thread that sleeps 10 ms ('does some blocking task') and returns
> 2. worker thread that is polls a timerfd and returns.
>
> The difficulty starts only when I create more than three timers (TIMERFDS >
> 3):
> 1. The worker thread that sleeps only returns after 1 second instead of 10
> ms.
> 2. timers work fine.

That's because your tasks are clogging the thread pool. It's
currently of a fixed size and it probably won't come as a surprise
that said size is 4. (You're probably wondering why. It's
complicated.)

There are two things you can do:

1. `env UV_THREADPOOL_SIZE=32 node app.js`

2. Create your own threads with uv_thread_create().

[Kees k]

I was already guessing something like that (uv_loop_t definition), because setting all intervals to 0.5 sec... 

Why the pool size is 4; is it a pragmatic choice?

My workaround is to create only one timerfd with a fixed 1 sec interval (much more accurate than setInterval, no drift, and does not go crazy on system time changes).

The callback executes an array with callback functions and intervals that are a multitude of 1 sec.

[Ben Noordhuis]

On Thu, May 8, 2014 at 10:26 AM, Kees k <keeskwe...@gmail.com> wrote:
> Why the pool size is 4; is it a pragmatic choice?

More or less. The actual number is fairly arbitrary but the reason
it's fixed is that I couldn't make an auto-scaling thread pool always
perform better than a fixed one. I've tried several approaches but
they all regress some of the benchmarks.

[Kees k]

Auto-scaling will 'never' perform better than fixed one. At best perform equal I guess (e.g. compare std::vector to normal arrays)