why threads instead of processes

[Alex Koch]

Hi,

When designing nxweb, what motivated you to use a thread based architecture instead of process workers? Is the nxweb event loop not able to work with child processes?

Thanks,

Alex

[Yaroslav]

Hi Alex,

One of the main goals of nxweb was programmer friendly C API. Writing modules for multi-process servers such as Apache or Nginx seemed nightmare to me. Inter-process communication is nowhere close in convenience to inter-thread communication. So I knew from the start that I am going to use threads, not processes.

I had concerns though about performance, cause there are opinions that processes scale better. After studying the reasoning behind those opinions I found one potential problem, which is called 'false sharing' of memory between threads running on different CPU cores. To avoid false sharing I implemented guard memory areas between allocated blocks.

Funny thing is that I did not see any performance change after adding those guard areas. So I do not think that false sharing was ever a performance factor for nxweb.

Another performance problem could be with multi-threaded malloc, which internally uses some kind of blocking. I use my own free-list memory allocation (separate for each thread) for most frequently allocated blocks, but there are still calls to malloc in code. The problem can be mitigated by linking against tcmalloc (or similar) library, I think.

The event loop of nxweb is single-threaded. Threads in nxweb work independently and share minimum information to avoid locks, etc.

By the way what benefits do you see of process workers over threads?

Yaroslav

--
You received this message because you are subscribed to the Google Groups "nxweb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nxweb+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Koch]

Thanks for your follow-up Yaroslav.

>> The event loop of nxweb is single-threaded. Threads in nxweb work independently and share minimum information to avoid locks, etc.

I see... I looked at the code and it looks to me that if you would instead of creating threads, just fork child processes and pass in the listener_fd, you basically achieve the same thing - behavior looks similar where threads run, is that correct?

>>By the way what benefits do you see of process workers over threads?

One thing that come to mind is memory corruption and race conditions - cache update would need locks which in a sense lead to also a blocking behavior. I think for the file cache of nxweb, you had to workaround race condition of the threading model?

Alex

---

Date: Wed, 25 Jun 2014 00:44:57 +0400
Subject: Re: {nxweb} why threads instead of processes
From: yar...@gmail.com
To: nx...@googlegroups.com

[Yaroslav]

On Thu, Jun 26, 2014 at 1:50 AM, Alex Koch <alex.k...@outlook.com> wrote:

Thanks for your follow-up Yaroslav.

>> The event loop of nxweb is single-threaded. Threads in nxweb work independently and share minimum information to avoid locks, etc.

I see... I looked at the code and it looks to me that if you would instead of creating threads, just fork child processes and pass in the listener_fd, you basically achieve the same thing - behavior looks similar where threads run, is that correct?

Basically yes. Instead of forking child processes I simply launch threads.

 

>>By the way what benefits do you see of process workers over threads?

One thing that come to mind is memory corruption and race conditions - cache update would need locks which in a sense lead to also a blocking behavior. I think for the file cache of nxweb, you had to workaround race condition of the threading model?

What is the difference of race condition between processes from the race condition between threads? If you have shared data in any scenario you would need locks. I am not sure if inter-process locks are any faster than inter-thread ones.

As for file cache I do not use locks. Whichever thread creates file first it writes to it. Second thread simply cannot create file with the same name, so it fails and skips writing to cache. Effectively the file acts as a lock, but non-blocking one.

[yiifburj]

Hi Yaroslav,  I am puzzled about how does multi-threads uses multi-cores. In my computer, they are all running on one core. Is it the asynchronous way that make the cores busy in the kernel threads?  

在 2014年6月25日星期三 UTC+8上午4:44:58，Yaroslav写道：

[Yaroslav]

How do you know they are running on single core? Did you run nxweb on a regular computer or virtual machine? Need more details to find out what's going on.

[yiifburj]

Sorry, I have some misunderstanding about how multi-thread programs behave, because I have conducted a test on a multi-thread busy looping, and the cpu usage told me that threy were on the same core , I don't know why, today I reconfirmed it, multi-thread programs really can use multi-cores, maybe they are affected by the kernel version and the meachine environment, thank you.

在 2015年7月3日星期五 UTC+8下午4:22:56，Yaroslav写道：

[kaga...@gmail.com]

> By the way what benefits do you see of process workers over threads?

I believe, the robustness would be a significant benefit.

Fault Detection, Isolation and Recovery (FDIR) is an important aspect in the aerospace industry.

If I write a buggy module in my server, which leads to a crash in some rare conditions, only the one process, which is executing that problematic case would die, but the others would continue. That means the next request would be still processed normally. That would be (fault isolation). The main process would have a chance to detect if any of the worker processes has died (fault detection) and would be able to create a replacement (fault recovery).

[Yaroslav]

I do not think the difference here is really significant. Each process/thread handles many concurrent connections anyway. All of them fail in case of process failure. For example if you have 4 CPU cores you will have either 4 processes (as with NGINX) or 4 threads (as with NXWEB), i.e. in case of single request failure you drop either 25% or 100% of open connections.

It also worth noting that whole nxweb restart takes only a fraction of a second. So in case of failure you have very little downtime.

In case you really need fault isolation you'd better go with a server that forks (launches new process) on every incoming request with keep-alive disabled. Performance will drop significantly in this scenario but it is gonna be more reliable.