Using proot to replace attempts to create hard links with either a file copy or a symlink

[Corbin Champion]

Some programs, like apt-get, make hard-links.  This is another action that is often not permitted for non-root users.  Is this something that:
a) proot does have a way to handle?
b) proot could handle with extensions?
I think this could be done fairly easily with LD_PRELOAD, but then you have a dependency of compiling the libfakelink.so (like libfakeroot.so) with the same libc as the user space programs.  This isn't awful, but limits their ability to ever upgrade their libc. 
Thanks,
Corbin

[Cédric VINCENT]

I've just pushed an extension named "link2symlink" that converts call
to link/linkat into calls to symlink/symlinkat in about 80 lines of
code:

https://github.com/cedric-vincent/PRoot/blob/link2symlink/src/extension/link2symlink/link2symlink.c

Without this extension:

$ touch foo
$ ln foo bar
$ ls -il foo bar
6078531 -rw-r--r-- 2 cedric users 0 Aug 19 20:56 bar
6078531 -rw-r--r-- 2 cedric users 0 Aug 19 20:56 foo

With this extension:

$ rm foo bar
$ touch foo
$ ./src/proot ln foo bar
$ ls -il foo bar
6078532 lrwxrwxrwx 1 cedric users 31 Aug 19 20:57 bar ->
/usr/local/cedric/Git/proot/foo
6078531 -rw-r--r-- 1 cedric users 0 Aug 19 20:57 foo

Do you think this could help?

Cédric.

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "PRoot" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to proot_me+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Corbin Champion]

Very cool.  This illustrates to me better how proot works and what an simple extension looks like.  This could very much help.  Now, about building.  I have never built proot.    Where is the best link to get started on?  Also, do extensions get compiled with the other source code or do you compile them the separately and include them in some other manner...how do you add an extension?).

Separate question, I saw static linked proot builds for various architectures, but not for MIPS.  Has it been built for MIPS previously?  Should it work?

Thanks!!
Corbin

[Cédric VINCENT]

On Mon, Aug 19, 2013 at 10:13 PM, Corbin Champion <corb...@gmail.com> wrote:
> Very cool. This illustrates to me better how proot works and what an simple
> extension looks like. This could very much help. Now, about building. I
> have never built proot. Where is the best link to get started on?

You can build PRoot with the following command:

make -C src

Important note: PRoot depends on libtalloc.

> Also,
> do extensions get compiled with the other source code or do you compile them
> the separately and include them in some other manner...how do you add an
> extension?).

For the moment extensions are embedded in PRoot:

https://github.com/cedric-vincent/PRoot/commit/0192b7cdbe62ea1fb701059491bb331e2b3bd240

> Separate question, I saw static linked proot builds for various
> architectures, but not for MIPS. Has it been built for MIPS previously?
> Should it work?

It has to be ported to this architecture first:

https://github.com/cedric-vincent/PRoot/blob/master/src/arch.h

https://github.com/cedric-vincent/PRoot/blob/master/src/syscall/sysnums-arm64.h
(for instance)

This is a matter of hours, if you wish to contribute ;)

Cédric.

[Corbin Champion]

Thanks for everything.  I will try building it myself soon. 

About porting to MIPS.  I will happily give it a go if I get there before others.  Main focus is ARM, but there are MIPS android users and I don't want to leave them out in the cold :)

Thank again,
Corbin

[hhm]

Neat! Just a bit of tinkering with, and imitating, link2symlink, and
then hopefully I (or someone else) can make a symlink2fakelink to have
symlink support on filesystems like FAT32!

@Corbin it seems like I have goals which are somewhat similar to
yours, to get more "desktop linux" code working on Android without
root. I am working on tal-dew (which is mentioned in some posts
around). So I guess I am not the only one interested in such an
approach!

Now, just need to simulate mmap etc., enabling running executables
from noexec mounts, and we can run a rootfs off the SD card! A big win
for devices with a tiny internal storage.

On 8/19/13, Corbin Champion <corb...@gmail.com> wrote:
> Thanks for everything. I will try building it myself soon.
>
> About porting to MIPS. I will happily give it a go if I get there before
> others. Main focus is ARM, but there are MIPS android users and I don't
> want to leave them out in the cold :)
>
> Thank again,
> Corbin
>
> On Monday, August 19, 2013 1:27:51 PM UTC-7, Cédric VINCENT wrote:
>>
>> On Mon, Aug 19, 2013 at 10:13 PM, Corbin Champion

>> <corb...@gmail.com<javascript:>>

[Corbin Champion]

I am not familiar with tal-dew.  Can you provide a link?  

Corbin 

[hhm]

https://sites.google.com/site/taldewandroid/

It is very much a work-in-progress; some ideas i want to do:
- test/fixes for using other rootfs's (besides OpenWrt rootfs)
- allow any/all parts of rootfs to be on an SD card, even if
formatted with a filesystem that is mounted noexec or doesn't have
symlink support
- (maybe) userspace tun/tap device emulation (for networking support
via tunnels and the like
- (maybe) implement userspace /dev/fuse (or libfuse) so can mount
things (loopmounts particularly)


In short: the goal is to simulate a complete linux system, without
root access on the host.

(There are already some tools which do this, Linux Namespaces
(especially userns in 3.8), which is still new, and UMView
<http://wiki.virtualsquare.org/wiki/index.php/UMview>, which is
tailored more towards full-desktop distros, often with root; a
multifunctionality unified toolset is easier here. Some of the ideas
from these systems, and some of their code if licensing permits, maybe
can be used.)

>> > email to proot_me+u...@googlegroups.com <javascript:>.

[Corbin Champion]

Alright, now I am dangerous.  Getting libtalloc and compiling is easy enough.  That being said, while I see that hard links are being replaced with symbolic links when use that branch, I still get a ln: cannot create hard link from 'foo' to 'bar': Operation not permitted when trying to do the same on Android.  No investigation beyond that yet.

Corbin

[Cédric VINCENT]

On Thu, Aug 22, 2013 at 9:09 AM, Corbin Champion <corb...@gmail.com> wrote:
> That being said, while I see that hard links are being replaced with
> symbolic links when use that branch, I still get a ln: cannot create
> hard link from 'foo' to 'bar': Operation not permitted when trying
> to do the same on Android. No investigation beyond that yet.

Maybe the link(2) restriction is applied *before* the ptrace
notification. Please, could you send the output of "proot -v 5 ..."
from your test on Android?

Cédric.

[Corbin Champion]

Error message looks slightly different if using a debian root instead of a busybox based rootfs.  /bin/ln: failed to create hard link 'foo' => 'bar': Operation not permitted 

[Corbin Champion]

What is the best way to capture the proot output into a file?  I think adding a > at the end of the line is getting treated as part of the initial command.

[Corbin Champion]

Looks like I didn't reply to all last time.  I as wrong (up too late).  The error has changed.  It was "operation not permitted" before and it is now "bad address".  You can see proot logs of the new (with extension) and old (without extension) doing a hard link and doing a soft link here:

https://www.dropbox.com/sh/m8qd9sg6padumj5/9-siVY7dNp

Thanks,
Corbin

On Thursday, August 22, 2013 12:30:22 AM UTC-7, Cédric VINCENT wrote:

[Cédric VINCENT]

Hello Corbin,

I wonder if you encountered a generic bug in the "symlinkat()"
handling in PRoot/ARM, I'll check that tomorrow... In the mean time,
could you please test the new version of the "link2symlink" extension,
now based on "symlink()" only:

https://github.com/cedric-vincent/PRoot/commits/link2symlink

Regards,
Cédric.

[Corbin Champion]

New log posted in the same place "hardnewest.txt".  Now get a different error failed to create hard link `testln2' => `test.txt' : No such file or directory 
The file test.txt really does exist.  If I swap the arguments (which would be wrong as test.txt is a text file that exists), I get a similar but different accessing `testln2': No such file or directory.  So, it is not the same as pointing it at a target that doesn't exist.

Thanks for working on this and the crazy fast response!  I haven't had any time other than to build, run and capture the log.

Thanks,
Corbin

[Cédric VINCENT]

As of my understanding, substitution of the syscall number (eg. linkat
-> symlink) doesn't work at all on ARM. It's under investigation.

> Thanks for working on this and the crazy fast response! I haven't
> had any time other than to build, run and capture the log.

You're welcome. Your tests and reports are valuable: they contribute
to the quality of PRoot.

Cédric.

[Cédric VINCENT]

On Fri, Aug 23, 2013 at 9:39 AM, Cédric VINCENT
<cedric....@gmail.com> wrote:
> As of my understanding, substitution of the syscall number (eg. linkat
> -> symlink) doesn't work at all on ARM. It's under investigation.

I pushed a fix into the "link2symlink" branch:

https://github.com/cedric-vincent/PRoot/commit/cea97c4ba0cfdf383ab617467d0880d141acc7cc

Cédric.

[Corbin Champion]

> As of my understanding, substitution of the syscall number (eg. linkat -> symlink) doesn't work at all on ARM.  It's under investigation. 

Well that is inconvenient!  I wonder why that would be different.  

For now I have used a preloaded shared library to do the trick.  That works, but I like using proot for this too, if it turns out to be possible.   

Pasted here for reference.

#include <unistd.h>

int link (const char *oldname, const char *newname) {

   return symlink(oldname, newname);

}

int linkat(int olddirfd, const char *oldpath, int newdirfd, const char *newpath, int flags) {

   return symlinkat(oldpath, newdirfd, newpath);

}

Thanks again!  I will provide feedback on the other threads we have going soon.

Corbin

[Corbin Champion]

Is the problem related to the differences in how arm does a system call?  That which is discussed some here?  

http://stackoverflow.com/questions/12946958/system-call-in-arm

The links in the selected answer are interesting.

[Cédric VINCENT]

On Sat, Aug 24, 2013 at 7:06 AM, Corbin Champion <corb...@gmail.com> wrote:
> Is the problem related to the differences in how arm does a system call?

Somewhat. on ARM, the syscall number used to be encoded in the
instruction stream (ie. as an immediate), so a specific trick was
required to change the syscall number without pre-patching the
instruction stream:

http://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/?id=3f471126ee53feb5e9b210ea2f525ed3bb9b7a7f

However, this was obsoleted by the EABI, and now the syscall number is
stored in R7 [0], so it doesn't really make sens in our case.

[0] https://wiki.debian.org/ArmEabiPort#System_call_interface

> On Fri, Aug 23, 2013 at 9:39 AM, Cédric VINCENT

> I pushed a fix into the "link2symlink" branch:
>
> https://github.com/cedric-vincent/PRoot/commit/cea97c4ba0cfdf383ab617467d0880d141acc7cc

This patch was successfully tested on ARM/EABI (PRoot doesn't support
the OABI yet).

Cédric.

[Corbin Champion]

Interesting.  I will try this soon.  Do I understand correctly that the hard link->symlink should work on arm now with this?

Thanks!

Corbin

[Cédric VINCENT]

On Sun, Aug 25, 2013 at 1:34 AM, Corbin Champion <corb...@gmail.com> wrote:
> Do I understand correctly that the hard link->symlink should work on arm now
> with this?

Yes :)

[Corbin Champion]

I have tried it now, this works great.  Thank you!  This is better, for my use, than the LD_PRELOAD trick.  Wonderful!
Corbin

[Corbin Champion]

A couple of things:

1) It seems that many times copy is a better replacement for a hard link than symbolic link.  For example, in managing the group.lock file, they create a link of group.lock (turned into a symlink) to another temporary file group.#### .  When they delete the temporary file group.###, group.lock is left as a hanging symlink and this causes things to fail later on that test for the existence of and contents of the group.lock file.  A) Can you think of a more robust way to emulate a hard link, because neither a copy nor a symlink is a great answer?  A binding is actually a good answer, but it would be a binding from within the rootfs to another file within the rootfs.  Maybe these could be created when a hard link is created and get deleted when the file (that is now a binding) is removed.  The list of these could be written to a file, for launching in the future.  This feels like it could work, but maybe is a little clunky.  Any better ideas?  Currently my initial rootfs have no hard links (I replace them with copies initially) B) There is no system call to do a copy by itself, so if I wanted a hard link to be a copy, how do I replace the one system call with multiple, or just drop it and then do some code to perform the copy?

2) This is something that proot could do well, would help Android users a lot, but might be outside the scope of what you want to do.  If you take a normal linux rootfs, say debian, 90% of the files are non-executable and 10% are executable.  Android often has a small partition that supports symlinks and executables and a larger partition (sometimes external) that is mounted noexec and may be formatted such that it does not support symlinks.  So, what if the rootfs is split into 3 pieces...  the rootfs itself has every file replaced with links to one of two bindings exec and noexec which contain a mirror of the directories used (as needed), but one has the files that are marked as executable and the other has the rest.  I have proven that splitting a rootfs this way provides a functional rootfs in the end, but I have only done this as a preprocessed step, so it cannot handle properly adding or removing files or changing permissions.  So, what if proot intercepted attempts to create a file or change a files permissions with code that made sure that the file is created or moved to the correct binding and that the link in the rootfs points at the correct one. 

3) I have released something that relies on proot that you might get a kick out of.  There are other things involved, but proot is a big part of it.  https://play.google.com/store/apps/details?id=champion.gnuroot

Thanks,
Corbin

[Cédric VINCENT]

Hello Corbin,

> 1) It seems that many times copy is a better replacement for a hard
> link than symbolic link. For example, in managing the group.lock
> file, they create a link of group.lock (turned into a symlink) to
> another temporary file group.#### . When they delete the temporary
> file group.###, group.lock is left as a hanging symlink and this
> causes things to fail later on that test for the existence of and
> contents of the group.lock file.
>
> A) Can you think of a more robust way to emulate a hard link,
> because neither a copy nor a symlink is a great answer?
> A binding is actually a good answer, but it would be a binding
> from within the rootfs to another file within the rootfs. Maybe
> these could be created when a hard link is created and get
> deleted when the file (that is now a binding) is removed. The
> list of these could be written to a file, for launching in the
> future. This feels like it could work, but maybe is a little
> clunky. Any better ideas? Currently my initial rootfs have no
> hard links (I replace them with copies initially)

As you said, hardlinks and symlinks are not semantically equivalent:

* for the hardlink "A <-> B", A and B points to the same content.
If one of them is removed, the other one is still valid.

* for the symlink "A -> B ", A points to B. If the latter is
removed, the former will be invalid.

To keep the original semantic, an intermediate file could be used: the
hardlink "A <-> B" can be translated into two symlinks ""A -> C" and
"B -> C", where C is the original content. What do you think about
this solution?

> B) There is no system call to do a copy by itself, so if I wanted a
> hard link to be a copy, how do I replace the one system call
> with multiple, or just drop it and then do some code to perform
> the copy?

Syscall chaining is a new feature that will be included in PRoot v3.2:

https://github.com/cedric-vincent/PRoot/commit/7cd0c919545a867f4df77446750ab5ebfd9b7055

To cancel a syscall use "set_sysnum(tracee, PR_void)" in the sysenter
stage.

> 2) This is something that proot could do well, would help Android
> users a lot, but might be outside the scope of what you want to do.
> If you take a normal linux rootfs, say debian, 90% of the files are
> non-executable and 10% are executable. Android often has a small
> partition that supports symlinks and executables and a larger
> partition (sometimes external) that is mounted noexec and may be
> formatted such that it does not support symlinks. So, what if the
> rootfs is split into 3 pieces... the rootfs itself has every file
> replaced with links to one of two bindings exec and noexec which
> contain a mirror of the directories used (as needed), but one has
> the files that are marked as executable and the other has the rest.
> I have proven that splitting a rootfs this way provides a
> functional rootfs in the end, but I have only done this as a
> preprocessed step, so it cannot handle properly adding or removing
> files or changing permissions. So, what if proot intercepted
> attempts to create a file or change a files permissions with code
> that made sure that the file is created or moved to the correct
> binding and that the link in the rootfs points at the correct one.

This could be done in a dedicated extension. For instance, my
teammates write and use such kind of extensions. I could help you to
write your own extension if you wish. Do you have some pseudo-code to
start with?

> 3) I have released something that relies on proot that you might get a
> kick out of. There are other things involved, but proot is a big
> part of it.
> https://play.google.com/store/apps/details?id=champion.gnuroot

It looks awesome! I don't own an Android device but I'm really proud
you based this project on PRoot! I took a quick look at the sources,
please could you remind me why you use QEMU?

Cédric.

[Corbin Champion]

Cédric,

I think this is a much better solution.  Please see my pseudo code with a few comments at the end of this reply. 

>>    B) There is no system call to do a copy by itself, so if I wanted a
>>       hard link to be a copy, how do I replace the one system call
>>       with multiple, or just drop it and then do some code to perform
>>       the copy?

>Syscall chaining is a new feature that will be included in PRoot v3.2:
>
>    https://github.com/cedric-vincent/PRoot/commit/7cd0c919545a867f4df77446750ab5ebfd9b7055
>
>To cancel a syscall use "set_sysnum(tracee, PR_void)" in the sysenter
>stage.

Great on both accounts!

>> 2) This is something that proot could do well, would help Android
>>    users a lot, but might be outside the scope of what you want to do.
>>    If you take a normal linux rootfs, say debian, 90% of the files are
>>    non-executable and 10% are executable.  Android often has a small
>>    partition that supports symlinks and executables and a larger
>>    partition (sometimes external) that is mounted noexec and may be
>>    formatted such that it does not support symlinks.  So, what if the
>>    rootfs is split into 3 pieces...  the rootfs itself has every file
>>    replaced with links to one of two bindings exec and noexec which
>>    contain a mirror of the directories used (as needed), but one has
>>    the files that are marked as executable and the other has the rest.
>>    I have proven that splitting a rootfs this way provides a
>>    functional rootfs in the end, but I have only done this as a
>>    preprocessed step, so it cannot handle properly adding or removing
>>    files or changing permissions.  So, what if proot intercepted
>>    attempts to create a file or change a files permissions with code
>>    that made sure that the file is created or moved to the correct
>>    binding and that the link in the rootfs points at the correct one.

>This could be done in a dedicated extension.  For instance, my
teammates write and use such kind of extensions.  I could help you to
write your own extension if you wish.  Do you have some pseudo-code to
start with?

Yes please.  I have included some pseudo code at the end of this reply.  Please take a look.  Any work you do is wonderful, anything you leave to me is fine, but please help me create a skeleton and show how I could chain together a couple of syscalls to do what I am trying to do in a few cases.  Also provide feedback if you think there is a better way to go about this. 

>> 3) I have released something that relies on proot that you might get a
>>    kick out of.  There are other things involved, but proot is a big
>>    part of it.
>>    https://play.google.com/store/apps/details?id=champion.gnuroot

>It looks awesome!  I don't own an Android device but I'm really proud
>you based this project on PRoot!  I took a quick look at the sources,
>please could you remind me why you use QEMU?

Glad you like it!  I am also the one who ported Octave to Android (which could benefit from all of this as well).  Sorry, github is out of sync with my local version.  There was an issue that caused QEMU to be used, but it has been resolved.  My source code is not greatly different, but this has changed. 

Here is the pseudo code:

static FilteredSysnum filtered_sysnums[] = {
        { PR_creat,                FILTER_SYSEXIT },  //can open or openat create an executable file??
        { PR_link,                 FILTER_SYSEXIT },
        { PR_linkat,               FILTER_SYSEXIT },
        { PR_unlink,               FILTER_SYSEXIT },
        { PR_unlinkat,             FILTER_SYSEXIT },       
        { PR_chmod,                FILTER_SYSEXIT }, 
        { PR_rename,               FILTER_SYSEXIT },
        { PR_renameat,             FILTER_SYSEXIT },
        { PR_mkdir,                FILTER_SYSEXIT },
        { PR_mkdirat               FILTER_SYSEXIT },
        { PR_rmdir,                FILTER_SYSEXIT },
        { PR_fchmod,               FILTER_SYSEXIT },
        { PR_fchmodat,             FILTER_SYSEXIT },
        FILTERED_SYSNUM_END,  //what am I missing?
};


case PR_creat:  //possibly open and openat need to be handled the same
   if (mode & ~umask) has any executable bit set
      actually create file at same path but with /exec binding path added to the front.
      create symlink from original path to path with /exec binding path added to the front.
   else
      actually create file at same path but with /noexec binding path added to the front.
      create symlink from original path to path with /noexec binding path added to the front.
   break;

case PR_link:  //handle linkat similarly
   if the target already points at a file that starts with .hard_link_
      create another symlink to that .hard_link_ file
   else
      move the target file to the same directory with the same name, but with .hard_link_ added to the front
      modify the symlink that used to point at the target file to point at .hard_link_ added to the front of its path
      create another symlink to that .hard_link_ file
   break;

case PR_unlink:  //handle unlinkat similarly
   delete the symlink at the given location
   unlink the file that is pointed to by the symlink (maybe under /exec or /noexec) if it doesn't start with .hard_link_
   break;
   //yes this leaves the .hard_link_ file hanging there, but there is very limited use of these and it is probably ok. 
   //otherwise would have to keep track of how many files point to the .hard_link_ file (maybe in the name .hard_link_4_
   //which would get changed to .hard_link_3_ when one link is deleted)

case PR_chmod:  //fchmod and fchmodat will be handled very similarly
   if mode has any executable bit set
      move the file to exec
      modify link to point at file under exec
   else
      move the file to noexec
      modify link to point at file under noexec
   break;

case PR_mkdir:
   mkdir in normal location, under exec and under noexec
   break;

case PR_rmdir:
   rmdir in normal location, under exec and under noexec
   break;

case PR_rename:
   rename the file that is under exec or under noexec
   modify symlink to have the new name, at the new location and point to the new file under exec or noexec.
   break;

Thank you again!
Corbin

[Cédric VINCENT]

On Wed, Oct 16, 2013 at 9:57 PM, Corbin Champion <corb...@gmail.com> wrote:
>>To keep the original semantic, an intermediate file could be used: the
>>hardlink "A <-> B" can be translated into two symlinks ""A -> C" and
>>"B -> C", where C is the original content. What do you think about
>>this solution?
>
> I think this is a much better solution.

Here is a proof-of-concept, not well tested:

https://github.com/cedric-vincent/PRoot/commit/bccc0c888aa8568a38096dc3064862bf566d7c1f

Have a look at the commit message.

> I have included some pseudo code at the end of this reply. Please
> take a look. Any work you do is wonderful, anything you leave to me
> is fine, but please help me create a skeleton and show how I could
> chain together a couple of syscalls to do what I am trying to do in
> a few cases. Also provide feedback if you think there is a better
> way to go about this.

[...]

> Here is the pseudo code:

[...]

OK, I'll try to convert your pseudo-code into PRoot statements next
week.

Cédric.

[Cédric VINCENT]

On Wed, Oct 16, 2013 at 10:22 PM, Cédric VINCENT

> Here is a proof-of-concept, not well tested:

Proof it is no well tested:

https://github.com/cedric-vincent/PRoot/commit/bce3100384148e6f9ac113e95f9e336a3abab6bd

[Corbin Champion]

This is great.  I will give it a whirl.

[Corbin Champion]

Here is a little more than pseudo-code for the management of /noexec and /exec.  I figure I should just learn out to create and maintain extensions and not just hassle you.  I am only part way into it, but I have some questions embedded in the code.  Can you take a look, when you get a chance because I am pretty sure they are all issues you have handled before.    

    int status;

    switch (event) {
    case INITIALIZATION: {
        /* List of syscalls handled by this extensions.  */

        static FilteredSysnum filtered_sysnums[] = {
            { PR_creat, FILTER_SYSEXIT },

            { PR_open, FILTER_SYSEXIT },
            { PR_openat, FILTER_SYSEXIT },

            { PR_link, FILTER_SYSEXIT },
            { PR_linkat, FILTER_SYSEXIT },
            { PR_unlink, FILTER_SYSEXIT },
            { PR_unlinkat, FILTER_SYSEXIT },      
            { PR_chmod, FILTER_SYSEXIT },
            { PR_rename, FILTER_SYSEXIT },
            { PR_renameat, FILTER_SYSEXIT },
            { PR_mkdir, FILTER_SYSEXIT },

            { PR_mkdirat, FILTER_SYSEXIT },

            { PR_rmdir, FILTER_SYSEXIT },
            { PR_fchmod, FILTER_SYSEXIT },
            { PR_fchmodat, FILTER_SYSEXIT },
            FILTERED_SYSNUM_END,

        };
        extension->filtered_sysnums = filtered_sysnums;
        return 0;
    }

    case SYSCALL_ENTER_END: {

        Tracee *tracee = TRACEE(extension);

        switch (get_sysnum(tracee, ORIGINAL)) {
        case PR_creat:
            mode_t currUmask;
            mode_t rqstMode;
                mode_t modMode;
            char origPath[PATH_MAX];
            char modPath[PATH_MAX];

            //are paths already interpretted by a previous step to handle bindings?

            //only need to do something if the file is being created within this rootfs, not some binding elsewhere
            //how do i check if the file being created is under this rootfs?
            //what variable has been set with the base of this rootfs?

            //figure out mode modified by umask
            currUmask = umask();
            rqstMode = peek_reg(tracee, CURRENT, SYSARG_2);
            modMode = rqstMode & ~currUmask;
            if (modMode & (S_IXUSR | S_IXGRP | S_IXOTH)) {  //if any executable bit will be set by this
                strcpy(modPath,"/exec");
            } else {
                strcpy(modPath,"/noexec");
            }
            //need to create a version of the original path that has the path to the base of the root removed from it
            //again need to understand how to get the base of the root
            //after I have this I need to concatinate it to the modPath
            strcat(modPath,????);  //so if path was /baseOfRoot/test/test.txt, the modPath should be /exec/test/test.txt
            //create symlink pointing at that will be created
            symlink(modPath,peek_reg(tracee, CURRENT, SYSARG_1));
            //replace /noexec or /exec at beginning of the string with what that actually means after binding
            //how do I do this??
            //then create file by modifying system call
            poke_reg(tracee, SYSARG_1, ????);
            break;
        case PR_open:  //i see two version of this with different number of arguments in the definition of open(2)
                   //how do you handle different number of arguments?
                   //int open(const char *pathname, int flags);
                   //int open(const char *pathname,int flags, mode_t mode);
            break;

Thanks,
Corbin

[Cédric VINCENT]

On Thu, Oct 17, 2013 at 8:27 PM, Corbin Champion <corb...@gmail.com> wrote:
> Can you take a look,
> when you get a chance because I am pretty sure they are all issues
> you have handled before.

One preliminary comment: I think the mmap-noexec extension would be a
good solution in your case too. Do you think you could give it a try?

> are paths already interpretted by a previous step to handle
> bindings?

Yes, in the SYSCALL_ENTER_END stage, all paths are said
"canonicalized" (see "man 7 path_resolution" for the definition).

> how do i check if the file being created is under this rootfs?

belongs_to_guestfs(tracee, path)

> what variable has been set with the base of this rootfs?

I don't understand that question :)

> again need to understand how to get the base of the root

get_root(tracee)


> strcat(modPath,????);

Always use "join_paths(...)" instead of standard string concatenation,
this will save you a lot of trouble!


> what that actually means after binding how do I do this??

Hum, not sure to understand. As of today, creating a binding on the
fly is not tested at all, however this will be done in a near future
for another purpose.


> poke_reg(tracee, SYSARG_1, ????);

Use set_sysarg_path(tracee, path, SYSARG_1) to push the path into
tracee's memory and to make the sysarg (register) points to it.

> i see two version of this with different number of arguments in the
> definition of open(2)

Actually, only the definition with 3 arguments exists, the other one
is just a wrapper implemented by the glibc.


Regards,
Cédric

[Corbin Champion]

> One preliminary comment: I think the mmap-noexec extension would be a
> good solution in your case too.  Do you think you could give it a try?

That one seemed good in intent (move almost entire rootfs to noexec region), but I had some problems.
1) Need to automatically determine which files, such as ld-linux.so.*, ld-uclib.so.* and statically linked exectuables to put on internal storage.  If this is not automatically handled, the rootfs has issues when users try to modify it.  Users might do anything. 
2) Performance.  It seemed slow and you had comments to that effect.  Sounded like you thought it might get better over time, but I wasn't sure how much better.
Can these two issues be handled?  I am not opposed to this extension, just wasn't sure if these two problems could be solved.

The extension I am proposing should have better performance (except when files are created or have their permissions modified) and should automatically handle users adding and removing files that must not be in a partition mounted as noexec.  This extension I am proposing moves 90% of a normal rootfs to the noexec partition and my belief is that is enough to make the amount remaining on internal storage as a non-issue for most users and devices. 

Do you think this is a fair comparison of the two ideas?

>> what variable has been set with the base of this rootfs?
>I don't understand that question :)

get_root(tracee)   is the answer.

>> what that actually means after binding how do I do this??
>Hum, not sure to understand.  As of today, creating a binding on the
>fly is not tested at all, however this will be done in a near future
>for another purpose.

I am trying to understand how to get the path at which /exec will be after binding.  This is because I would make a symlink to /exec and later on this will be interpreted by bindings to be something like /sdcard/noexec or something else (whatever I told it).  I want to know the path that a particular binding points at.

>> i see two version of this with different number of arguments in the
>> definition of open(2)
>Actually, only the definition with 3 arguments exists, the other one
>is just a wrapper implemented by the glibc.

How do I determine which ones actually exist?

This is all very helpful!

Thanks again!
Corbin

[Cédric VINCENT]

On Fri, Oct 18, 2013 at 1:44 AM, Corbin Champion <corb...@gmail.com> wrote:
> 1) Need to automatically determine which files, such as
> ld-linux.so.*, ld-uclib.so.* and statically linked exectuables to
> put on internal storage.  If this is not automatically handled, the
> rootfs has issues when users try to modify it.  Users might do
> anything.

You can detect it either statically:

     readelf -l /bin/sh | grep interpreter

or dynamically:

    https://github.com/cedric-vincent/PRoot/blob/v3.1/src/execve/interp.c#L186

> 2) Performance.  It seemed slow and you had comments to that effect.
> Sounded like you thought it might get better over time, but I wasn't
> sure how much better.

The trick used to bypasss the noexec restriction impacts the program
startup time only.  The figures I provided previously were against
programs that perform a lot of mmap-exec (emacs & xterm), hence the
poor performance results.  However, most of the programs run by
GNURoot users (ssh, telnet, bc, links, ...) should not suffer too much
since they don't perform as many mmap-exec.  For example, on my
Slackware64-14.0 workstation:

    proot -b /lib64/ld-linux-x86-64.so.2 -r /mnt/no-exec <program>

    =======  ============
    program  startup time
    =======  ============
    ssh              0.6s
    telnet           0.3s
    bc               0.3s
    links2           0.9s
    =======  ============

However, the startup cost becomes very high if you run a lot of small
programs sequentially, as it's typically the case with ./configure
scripts.

> The extension I am proposing should have better performance (except
> when files are created or have their permissions modified) and
> should automatically handle users adding and removing files that
> must not be in a partition mounted as noexec.  This extension I am
> proposing moves 90% of a normal rootfs to the noexec partition and
> my belief is that is enough to make the amount remaining on internal
> storage as a non-issue for most users and devices.
>
> Do you think this is a fair comparison of the two ideas?

I agree with you :) Your extension should be faster, although I'd like
to know if the mmap-noexec extension is an acceptable solution until
your extension is finished.  I think it is worth it as the mmap-noexec
extension may be way easier to maintain and validate (and it is almost
already done).

>>> what that actually means after binding how do I do this??  Hum,
>>not sure to understand.  As of today, creating a binding on the fly
>>is not tested at all, however this will be done in a near future for
>>another purpose.
>
> I am trying to understand how to get the path at which /exec will be
> after binding.  This is because I would make a symlink to /exec and
> later on this will be interpreted by bindings to be something like
> /sdcard/noexec or something else (whatever I told it).  I want to
> know the path that a particular binding points at.
>
>
>>> i see two version of this with different number of arguments in
>>> the definition of open(2) >>Actually, only the definition with 3
>>> arguments exists, the other one >>is just a wrapper implemented by
>>> the glibc.
>
> How do I determine which ones actually exist?

From kernel sources:

    $ cd linux
    $ git grep "SYSCALL_DEFINE.*\bopen\b" fs/
    fs/compat.c:COMPAT_SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, umode_t, mode)
    fs/open.c:SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, umode_t, mode)

> This is all very helpful!
>
> Thanks again!

You're welcome.  Actually I really appreciate what you are doing with
PRoot :) Sadly, I don't have as much free time as I would like in
order to contribute to your project.


Cédric.

[Corbin Champion]

Cédric,

>You can detect it either statically:
>
>     readelf -l /bin/sh | grep interpreter
>
>or dynamically:
>
>    https://github.com/cedric-vincent/PRoot/blob/v3.1/src/execve/interp.c#L186

Ok, but I would need to do this on the fly.  Maybe when they try to execute a file, I could test it then and then move it onto internal memory if needed.

>The trick used to bypasss the noexec restriction impacts the program
>startup time only.  The figures I provided previously were against
>programs that perform a lot of mmap-exec (emacs & xterm), hence the
>poor performance results.  However, most of the programs run by
>GNURoot users (ssh, telnet, bc, links, ...) should not suffer too much
>since they don't perform as many mmap-exec.

Hmm.. I understand what you are saying.  I will have to think how bad this really is for a user. 

>I agree with you :) Your extension should be faster, although I'd like
>to know if the mmap-noexec extension is an acceptable solution until
>your extension is finished.  I think it is worth it as the mmap-noexec
>extension may be way easier to maintain and validate (and it is almost
>already done).

I don't need to rush.  I don't need to put in one solution to only replace it a couple of weeks later. 

There is one other issue with mmap-noexec that I forgot to point out.  I still have an additional issue with putting the rootfs fully on external storage.  The external storage can be formatted such that symbolic links are not supported.  I think hhm pointed this out, but didn't push further on it. 

>From kernel sources:
>
>    $ cd linux
>    $ git grep "SYSCALL_DEFINE.*\bopen\b" fs/
>    fs/compat.c:COMPAT_SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, umode_t, mode)
>    fs/open.c:SYSCALL_DEFINE3(open, const char __user *, filename, int, flags, umode_t, mode)

Ah, ok.  Thanks.

>You're welcome.  Actually I really appreciate what you are doing with
>PRoot :) Sadly, I don't have as much free time as I would like in
>order to contribute to your project.

Free time is the issue here as well, otherwise I would have many projects in a much better place, but have to pay bills.  I appreciate what time you do put towards this.

Thanks,
Corbin

[Corbin Champion]

I am trying to build and test the new version of link2symlink.  It compiles alright as before, but when I launch, I get 

proot: ./syscall/heap.c:116: translate_brk_exit Assertion `0' failed. 

proot warning: signal 6 received from process 29466

and then it doesn't enter the specified root.

I am not having the same trouble with one built of x86.  I pulled the latest and then the link2symlink branch.  Is there something new untested for arm?  

Thanks,

Corbin 

[Cédric VINCENT]

As of my understanding, you merged the "link2symlink" branch and the
"next" branch. This latter introduces a new feature: the heap
emulation. Although, a couple of fixes for this new featrue are
pending, I'll push them publicly in two weeks. Some of these fixes
are specific to the ARM port.

Cédric.

[Corbin Champion]

I didn't fetch the "next" branch.  I just have the fetched the "master" branch and "link2symlink".  I will make sure I haven't fetched anything else.

Corbin

[Cédric VINCENT]

Oops, I've pushed the "heap feature" in the master branch on github by
mistake... You are right.

Could you just use the "link2symlink" branch until the next update?

[Corbin Champion]

Ok, so I got a little time last night.  Yeah, I just worked directly off of link2symlink and was fine.  I have made a pull request: https://github.com/cedric-vincent/PRoot/pull/28

Please take a look.  This makes link2symlink better.

Thanks,
Corbin

[Cédric VINCENT]

Smart patch! I added a couple of comments inlined in the pull
request, let me know if you want to take them into account before I
merge your work.

Regards,
Cédric.

[Corbin Champion]

I responded to some of your comments, which are appreciated.  Questions included.  I will finish cleaning it up based on the responses to those few questions.  :) 

Thanks,

Corbin

[Corbin Champion]

Looks like I next need to handle attempts to stat, or fstat and set the st_nlink accordingly.  Have you done similar?  Any example of overriding part of the stat struct returned by these system calls?up

Also, please see my couple of questions based on your code review.

Thanks,

Corbin

[Cédric VINCENT]

Hello Corbin,

Sorry for the delay, I was busy with the v3.2 release.

> Looks like I next need to handle attempts to stat, or fstat and set
> the st_nlink accordingly.

Right.

> Have you done similar?  Any example of overriding part of the stat
> struct returned by these system calls?

Yes, look at the PR_*stat cases in in
src/extension/fake_id0/fake_id0.c.

> Also, please see my couple of questions based on your code review.

I'll be busy for a couple of days due to the public release of CARE, a
new tool based on PRoot.  I'll try to answer you next week.

Regards,
Cédric.

[Cédric VINCENT]

Hello Corbin,

On Fri, Nov 8, 2013 at 6:37 PM, Cédric VINCENT <cedric....@gmail.com> wrote:

> Looks like I next need to handle attempts to stat, or fstat and set
> the st_nlink accordingly.

Right.

The field "st_ino" has to be emulated too.  I wrote a small test script (attached) to check for [missing] features and regressions, feel free to extend it and add it to the "link2symlink" branch.

Cédric.

[Corbin Champion]

That is ok.  CARE sounds like a great idea/project.  How do you get ST Micro to let you work on these fun projects?  How do they fit in with their products and plans?  I see your example of working with the stat struct and modifying its contents.  Looks easy enough.  Thanks!

[Cédric VINCENT]

> That is ok. CARE sounds like a great idea/project. How do you get
> ST Micro to let you work on these fun projects? How do they fit in
> with their products and plans?

The initial motivations for PRoot was to make the use of QEMU
user-mode both easy ("no setup") and possible on compute farms ("no
privileges"). My teammates use [a modified version of] QEMU user-mode
on compute farms [1] to automatically tune GCC (like in [2]) in order
to improve the performance of ST embedded software. This project --
named ATOS for "Auto-Tuning Optimization System" -- will be publicly
announced in a couple of weeks [3].

Also, PRoot is more generally used in ST as a lightweight virtualizer
in order to run/build/validate some Linux packages in a "guest" distro
different from from the "host" one (with or without QEMU user-mode).
By "lightweight", I mean the guest and host distros share the same
system configuration, like the network (ex. [4]) or the hardware
(/dev, /sys).

Since PRoot can monitor and manipulate Linux processes, our team
decided to re-use it to develop a couple of tools that would be really
helpful to ST, like CARE. This tool targets our support team and
customers, that's why it has to be public.

Despite being fun-looking projects, we manage them seriously and we do
our best to provide industrial strength products.

> I see your example of working with the stat struct and modifying its
> contents. Looks easy enough.

Great. Also, I replied to your comments in the pull request.

Regards,
Cédric.

[0] https://github.com/cedric-vincent/PRoot/blob/v3.2/doc/articles/extending_qemu.txt

[1] http://www.hipeac.net/system/files/pseudo_accurate_emulation.pdf

[2] http://gcc.gnu.org/wiki/cauldron2013?action=AttachFile&do=get&target=Using_TACT_for_GCC_Development.pdf

[3] http://compil13.cri.mines-paristech.fr/

[4] http://unix.stackexchange.com/questions/73203/how-to-create-rootfs-for-user-mode-linux-on-fedora-18#97315

[Corbin Champion]

I am back.  :)  Was working on a major release of another one of my apps.  There are a few bugs after the release, but I should be able to finish working on this extension soon over the holidays.  

[Cédric VINCENT]

On Wed, Dec 18, 2013 at 7:27 AM, Corbin Champion <corb...@gmail.com> wrote:
> I am back. :) Was working on a major release of another one of my apps.

I'm curious, which one ?

> There are a few bugs after the release, but I should be able to finish
> working on this extension soon over the holidays.

Great :) !

[Corbin Champion]

It was Octave (android port of GNU Octave).  It is my most popular app and the one I have spent the most time on.

https://play.google.com/store/apps/details?id=com.octave

It was a difficult update as I rebuilt everything.  Also, people are required to pay (at least a dollar) after 10 uses now.  The former change ran into an issue with some devices, which was fixed last night.  The later caused some people to get very upset, given that the app was formerly free and Octave is generally free. 

Getting this release out, allows me to focus (with my little free time) back on GNURoot, which will be more useful and more popular when complete. 

Corbin

[Corbin Champion]

Ok.  A few questions...

1) Some of the stat calls provide the path to the file, but others only provide the file descriptor (like fstat).  I was writing my own similar function, but found readlink_proc_pid_fd().  Is there any gotchas/dangers in using this?  

2) What is oldlstat and oldstat?  Why are they not in the list of things to handle in handle_sysexit_end()?

3) Lastly, I correctly determine when a stat related routine is being called and I correctly build a new struct stat that should satisfy the known issues with ino and nlink, but now I want to replace the whole struct stat in memory, as opposed to a field or two.  I have the the struct stat I want to use, I have the address of the old struct stat in memory (same as how fake_id0 gets the address) that I want to replace, but how do I perform the replacement in memory?

Thanks!

Corbin

  

[Cédric VINCENT]

Hello Corbin,

> 1) Some of the stat calls provide the path to the file, but others
> only provide the file descriptor (like fstat). I was writing my
> own similar function, but found readlink_proc_pid_fd(). Is there
> any gotchas/dangers in using this?

You can use this function. Although, it supports paths up to PATH_MAX
bytes only, that is, 4kB. There are similar limitations in PRoot
elsewhere actually (my bad).

> 2) What is oldlstat and oldstat?

Here is a quote from stat(2) man-page, slightly modified for the sake
of readability:

Over time, increases in the size of the stat structure have led to
three successive versions of stat(): __NR_oldstat, __NR_stat, and
__NR_stat64 (new in kernel 2.4). The glibc stat() wrapper function
hides these details from applications, invoking the most recent
version of the system call provided by the kernel, and repacking the
returned information if required for old binaries. Similar remarks
apply for fstat() and lstat().

> Why are they not in the list of things to handle in
> handle_sysexit_end()?

Because we never encountered this syscall and we don't know how to
test it.

> 3) Lastly, I correctly determine when a stat related routine is being
> called and I correctly build a new struct stat that should satisfy
> the known issues with ino and nlink, but now I want to replace the
> whole struct stat in memory, as opposed to a field or two. I have
> the the struct stat I want to use, I have the address of the old
> struct stat in memory (same as how fake_id0 gets the address) that
> I want to replace, but how do I perform the replacement in memory?

I guess you want to use "write_data", as in:

https://github.com/cedric-vincent/PRoot/blob/v3.2.1/src/syscall/exit.c#L390


Regards,
Cédric.

[Corbin Champion]

> You can use this function.  Although, it supports paths up to PATH_MAX 
> bytes only, that is, 4kB.  There are similar limitations in PRoot 
> elsewhere actually (my bad). 

This seems fine.  

>Because we never encountered this syscall and we don't know how to 
>test it.

Ok.  I won't worry about it. 

> I want to use "write_data", as in: 
>  https://github.com/cedric-vincent/PRoot/blob/v3.2.1/src/syscall/exit.c#L390 

That worked.  Thanks!  Some testing to do, but seems to be working nicely.  

I am not a git expert.  I am having trouble rolling the link2symlink branch forward to contain all of the new work on the main branch.  I have it working locally, but want to make sure the branch represents exactly what I am building from, especially now that I am using functions only found in fairly recent versions of main.  Could you merge link2symlink to the latest main branch and tell me what you did to do so?  Then I can commit and push my change to link2symlink.c?

Thanks,

Corbin

[Corbin Champion]

> I am not a git expert.  I am having trouble rolling the link2symlink branch forward to contain all of the new work on the main branch.  I have it working locally, but want to make sure the branch represents exactly what I am building from, especially now that I am using functions only found in fairly recent versions of main.  Could you merge link2symlink to the latest main branch and tell me what you did to do so?  Then I can commit and push my change to link2symlink.c?

I have figured it out.  Though it took some doing. 

https://github.com/cedric-vincent/PRoot/pull/28

Please look over my changes.  Also, I ran the test you sent earlier.  It works (where should I add it in the repo?).  I need to test fstat, but otherwise this should work.

Happy New Year!

Corbin

[Corbin Champion]

Looks like when I call fstat from C code it is not working.  Looks like the symbolic link is being resolved before I am seeing the system call, so it is already pointing at the .l2s.*.# file.  This should be easy to detect, but it is time for sleep.

Corbin

[Cédric VINCENT]

Hello Corbin,

Happy new year :)

> https://github.com/cedric-vincent/PRoot/pull/28
> Please look over my changes.

OK, I'll have a look soon.

> Also, I ran the test you sent earlier. It works (where should I add
> it in the repo?).

Since this test is specific to this extenstion, I suggest you to add
it under "src/extension/link2symlink/tests/".

Did you progress in this regard?

[Corbin Champion]

>> Looks like when I call fstat from C code it is not working.  Looks
>> like the symbolic link is being resolved before I am seeing the
>> system call, so it is already pointing at the .l2s.*.# file.  This
>> should be easy to detect, but it is time for sleep.

>Did you progress in this regard?

Yeah, I fixed some corner cases and added proper support of fstat in my commit last night. 

Corbin

[Corbin Champion]

Looks like I have some debugging to do.  On a real device (newer arm based device), my path is not being resolved, have to type "/bin/ls"  instead of "ls" and if I type "ls withsomepath" I get a lot of "Bad address" responses.  This didn't occur before I merged in the latest version of the main branch and made latest changes for link2symlink (not sure which is the culprit), but I needed one function from that and it is probably the right choice in the end.  How well tested in the latest main branch code?  Does this particular issue ring any bells?  I will try to narrow down the issue.

Thanks,

Corbin   

[Corbin Champion]

Looks like it is caused by a change in the link2symlink extension.  Not sure why it only shows up when I am on an arm device.  Not sure why it only shows up with my new changes to that extension.  Digging. 

Corbin  

[Corbin Champion]

    309                         if (sysnum == PR_fstatat64 || sysnum == PR_newfstatat)

    310                                 sysarg_path = SYSARG_2;

    311                         else

    312                                 sysarg_path = SYSARG_1;

    313                         size = read_string(tracee, original, peek_reg(tracee, CURRENT, sysarg_path), PATH_MAX);

The call to read_string is always returning a negative value.  Which is causing all use of stat related syscalls to fail.  Am I doing something obviously wrong?

Thanks,

Corbin

[Corbin Champion]

By the way, there is no problem when doing similar under x86 or when using qemu-arm-static.  Only see this on real device.  Root is in a different location too, but I don't think that is the issue.

Corbin

[Corbin Champion]

Lastly, read_string works in general.  It is used in move_and_symlink_path and elsewhere without issue, even on my android device.

Corbin

[Corbin Champion]

Ok, last update for the night.  Whether running qemu-arm-static or on the real device it is stat64 that is being called multiple times.  Under qemu-arm-static the size returned from the read_string is always some positive number.  It is -14 on the arm device.

Thanks,

Corbin

[Corbin Champion]

Interesting... If I google stat64 arm, one of the first results (#2) is a bug where the FUSE was not working on arm devices when stat64 was called, but was working on other types of devices.  This does not mean I haven't done something stupid, but maybe this is related??

Corbin

[Corbin Champion]

Ok it is my fault.  This is what I figured.  I used CURRENT and should have used ORIGINAL.  I recognized this by comparing my code to fake_id0, which also does a read_string as part of SYSCALL_EXIT_END.  I get the idea from the examples and the names, but can you provide a little explanation about which one should be used in which case?  I see CURRENT used for reading arguments in SYSCALL_ENTER_END, but CURRENT is only used for checking status in SYSCALL_EXIT_END and ORIGINAL is used instead for reading arguments or modifying the result.  

Thanks,

Corbin

[Corbin Champion]

Lastly, why did it work in some cases but not on this device? Is it a matter of multithreading?

[Corbin Champion]

Ok.  I guess MODIFIED is what you want if you want to see the paths already made to be absolute paths.

[Cédric VINCENT]

Hello Corbin,

Sorry for the delay.

> Ok it is my fault. This is what I figured. I used CURRENT and should
> have used ORIGINAL. I recognized this by comparing my code to
> fake_id0, which also does a read_string as part of SYSCALL_EXIT_END.
> I get the idea from the examples and the names, but can you provide a
> little explanation about which one should be used in which case? I
> see CURRENT used for reading arguments in SYSCALL_ENTER_END, but
> CURRENT is only used for checking status in SYSCALL_EXIT_END and
> ORIGINAL is used instead for reading arguments or modifying the
> result.

I confirm that all the issues you had encountered previsouly may be
due to a misuse of ORIGINAL/MODIFIED/CURRENT. Technically, PRoot
maintains three versions of tracees' registers:

- ORIGINAL: values passed from the application, before PRoot has made
any modifications (saved before SYSCALL_ENTER_START).

- MODIFIED: values passed to the kernel, once PRoot has made
modifications (saved after SYSCALL_ENTER_END).

- CURRENT: current values, they can be freely modified with
poke_reg().

This is required because ORIGINAL or MODIFIED values of tracee's
registers are required after these latters were overwritten by the
kernel or by PRoot itself.

> Lastly, why did it work in some cases but not on this device? Is it a
> matter of multithreading?

This kind of issue depends on the ABI. For instance on ARM EABI,
SYSARG_RESULT is the same register as SYSARG_1 (cf. src/tracee/reg.c),
that means the initial value of SYSARG_1 in the sysexit stage was
modified by the kernel.

Regards,
Cédric.

[Corbin Champion]

I got this working and fixed a couple of corner cases.  I will push code soon.  I released a newer version of GNURoot based on this.  This solves many issues for the different distributions I have provided.  People can now install pretty much anything on android without root now.  I even have xterms and a graphical desktop working.  Very cool.

Corbin

[Cédric VINCENT]

Really good news!

Did you made an announce about your new version somewhere? (I'd like
to share it in G+)

[Corbin Champion]

No big announcement, just the rollout of the new version to users on Google Play.  I want to add one new feature and fix one bug and then I will make a big announcement and can share it with you then.  I have on G+ as well.  

Corbin