[akaros] Pipes not working (sometimes) (#17)

[Kevin Klues]

I have a simple pipe test sitting on origin/failing-pipetest that periodically dies with a page fault. It should always run to completion.

—
Reply to this email directly or view it on GitHub.

[Kevin Klues]

It turns out, it wasn't the pipes, but the fact that I was using fork() without an exec(), which is slightly broken on Akaros. The gist of the problem is that child processes are not able to properly handle syscalls that do not complete immediately (i.e. when a kthread is put to sleep on a rendez and returns to userspace with SC_DONE == 0). I noticed the problem because I was page faulting in the child as an early SCP, but the parent was clearly already a normal SCP (so the child should have been as well). This info is stored in procdata, and is not currently being copied over to a forked process properly.
This is a result of not properly keeping our fork() code up to date with additions to procinfo/procdata that affect forked processes.

I have a fix I am putting together that copies relevant portions of procinfo and procdata into a forked process, and then resets them properly if we happen to do an exec later. I will push it out for review soon.

[Davide Libenzi]

Related (almost) question. Is there any plan to support COW on fork?

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ron minnich]

On Mon, Oct 12, 2015 at 9:33 PM 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Related (almost) question. Is there any plan to support COW on fork?

That's a great question. I've seen opinions on COW go back and forth the the point that I have no clue. I remember a great paper in Usenix once on bovaphobic behavior..

Davide, is COW considered a win nowadays?

ron 

[Davide Libenzi]

I have actually never measured the cost of a "dumb" (as in, copy/split the whole VM before syscall exit) fork WRT a COW-enabled one.

My guess is yes, it matters quite a bit in most cases.

If both parent and child are going to have a long life, and ultimately write every single page in their VM space, then actually the overall cost of COW is higher WRT the dumb fork. There is extra cost per page with COW (taking fault, figuring out things, setup PTEs, and memcpy()).

But, such cost would be diluted over the processes lifetime, and not paid upfront. Which might result in a better behavior for the caller in terms of latency.

Then there is the most common case, fork+exec, for which the dumb fork is a total waste, as what you copied in the child, will be thrown away 100ns later.

There are also secondary issues. Think a process with huge VM footprint like search.

Now, search is unlikely to be doing any fork, but if it does, and fork tries to double up all its physical pages at syscall time, you may get into OOM pretty quickly.

Having a Win32 CreateProcess() equivalent will be helpful to optimize the fork+exec case, though, apps will have to be rewritten for that.

Now, if g3 has an equivalent for that, and most g3 apps we plan to port, use it, the task can become easier.

--

[barret rhoden]

On 2015-10-12 at 21:33 'Davide Libenzi' via Akaros wrote:
> Related (almost) question. Is there any plan to support COW on fork?

My take on this for Akaros is that fork/exec is a nasty model for
process creation, especially for an OS that uses parallel processes,
that we support only for legacy reasons. The native method for making
processes is sys_proc_create(). That is the only way for an MCP to
spawn a child process too - fork() fails for MCPs.

Our common use for fork/exec is the shell. In which case, I'm not
overly concerned about process creation time - the shell is not a huge
process (or shouldn't be!).


For some background reading on this topic:

"Evolving the Unix System Interface to Support Multithreaded Programs"
http://www.hpl.hp.com/techreports/Compaq-DEC/SRC-RR-21.pdf

My dissertation (Sections 4.3 and 2.2)
http://www.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-223.pdf

Barret

[Dan Cross]

On Tue, Oct 13, 2015 at 9:18 AM, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

I have actually never measured the cost of a "dumb" (as in, copy/split the whole VM before syscall exit) fork WRT a COW-enabled one.

My guess is yes, it matters quite a bit in most cases.

If both parent and child are going to have a long life, and ultimately write every single page in their VM space, then actually the overall cost of COW is higher WRT the dumb fork. There is extra cost per page with COW (taking fault, figuring out things, setup PTEs, and memcpy()).

But, such cost would be diluted over the processes lifetime, and not paid upfront. Which might result in a better behavior for the caller in terms of latency.

Then there is the most common case, fork+exec, for which the dumb fork is a total waste, as what you copied in the child, will be thrown away 100ns later.

There are also secondary issues. Think a process with huge VM footprint like search.

Now, search is unlikely to be doing any fork, but if it does, and fork tries to double up all its physical pages at syscall time, you may get into OOM pretty quickly.

Having a Win32 CreateProcess() equivalent will be helpful to optimize the fork+exec case, though, apps will have to be rewritten for that.

Now, if g3 has an equivalent for that, and most g3 apps we plan to port, use it, the task can become easier.

Please note: this is an *external* mailing list. These specifics are not appropriate for public discussion.

[ron minnich]

For long lived processes, in NIX, we always split the VM up front but the VM design there was far simpler and lower

overhead than in Linux for such things. But the Linux VM could certainly scale to much larger images I bet.

I'm kind of on the "fork is a bad idea" side of this discussion I guess :-)

Which counts as me being a bovaphobe unless I'm drinking milk :-)

ron

[barret rhoden]

On 2015-10-13 at 14:14 ron minnich wrote:
> Which counts as me being a bovaphobe unless I'm drinking milk :-)

Took me a second. =) I'm also in the 'eat more chicken' camp, unless
it turns out we need to eat the CoW. =)

Barret

[Davide Libenzi]

We agree that threads, or create process, can cope with almost all the cases, without resorting to the need of fork.

It all depends on the level of support we want to have for standard Unix processes in Akaros.

IMHO, in 2015, designing a Unix like OS without support for standard POSIX stuff, is not a good idea if we expect developers to move to it.

Also, I think it is OK for Akaros fork to be slow [1], but it is less OK for it to fail under certain conditions.

[1] Cygwin folks went a LONG way in implementing it, in a world which has no idea of fork, from userspace. It sucks, but you can run a standard POSIX software on it.

[Kevin Klues]

Resolved by:

https://groups.google.com/forum/#!topic/akaros/DFfhXDH8tCs

[Kevin Klues]

Closed #17.