Same command in different directories

[Greg Warner]

I'm using fabricate to, among the more usual software build activities, download and compile an embedded development toolchain.  Consequently, there are several instances where I os.chdir into a directory and run('make').  Unfortunately, fabricate doesn't seem to distinguish a 'make' run in one directory (binutils) from a make run in another (gcc) and doesn't run the latter.  

First, does fabricate have any features that allow me to do what I'm trying to do? or am I don't something wrong?

Looking at the .deps file (which, oddly enough, it seems that fabricate dumps in the cwd the first time run() is executed, which could be brittle if you os.chdir much), it seems that the commands aren't distinguished at all based on which directory they're run in.  So I did a quick hack to rectify this and modified two lines of code in fabricate.py so that the command key is now composed of the 'os.cwd() + command'.  

So second, is this hack on the right track?  Another, perhaps better approach, would be to have a .deps file in each working directory.

Thanks,

Greg

[Lex Trotman]

On 27 October 2013 10:04, Greg Warner <gdwa...@gmail.com> wrote:

I'm using fabricate to, among the more usual software build activities, download and compile an embedded development toolchain.  Consequently, there are several instances where I os.chdir into a directory and run('make').  Unfortunately, fabricate doesn't seem to distinguish a 'make' run in one directory (binutils) from a make run in another (gcc) and doesn't run the latter.  

First, does fabricate have any features that allow me to do what I'm trying to do? or am I don't something wrong?

You should not use os.chdir in the script.  Fabricate can't track it and doesn't know you did that.  You should do a cd inside run eg "run('cd xxx; make')" so fabricate can track it.

  

Looking at the .deps file (which, oddly enough, it seems that fabricate dumps in the cwd the first time run() is executed, which could be brittle if you os.chdir much), it seems that the commands aren't distinguished at all based on which directory they're run in.  So I did a quick hack to rectify this and modified two lines of code in fabricate.py so that the command key is now composed of the 'os.cwd() + command'.  

So second, is this hack on the right track?  Another, perhaps better approach, would be to have a .deps file in each working directory.

The .deps file is placed in the directory where the command is run.  Fabricate is not a recursive builder, it runs from the top level and uses only one .deps file for a build.  Whilst it is possible, my experience with another build tool is that its extremely complicated to use recursive or nested build records correctly.

Cheers

Lex

 

Thanks,

Greg

--
You received this message because you are subscribed to the Google Groups "fabricate users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fabricate-use...@googlegroups.com.
To post to this group, send email to fabrica...@googlegroups.com.
Visit this group at http://groups.google.com/group/fabricate-users.
For more options, visit https://groups.google.com/groups/opt_out.

[Lex Trotman]

[...]

Looking at the .deps file (which, oddly enough, it seems that fabricate dumps in the cwd the first time run() is executed, which could be brittle if you os.chdir much), it seems that the commands aren't distinguished at all based on which directory they're run in.  So I did a quick hack to rectify this and modified two lines of code in fabricate.py so that the command key is now composed of the 'os.cwd() + command'.  

So second, is this hack on the right track?  

oops forgot to say this is completely wrong, the command recorded in the .deps no longer matches the command in the run() call so fabricate will *always* re-run the command. 

Cheers

Lex

[Greg Warner]

I tried to cover all my bases in that regard.  I had it alter the command both when recording the deps, as well as when comparing them.  But something isn't working right.  

Thanks for letting me know I could do run("cd dir; command...").  I didn't know that.  I naively tried run('cd','dir') which didn't work (and doesn't make much sense anyway), which is why I resorted to os.chdir.

[Greg Warner]

On Sat, Oct 26, 2013 at 4:58 PM, Lex Trotman <ele...@gmail.com> wrote:

The .deps file is placed in the directory where the command is run.  Fabricate is not a recursive builder, it runs from the top level and uses only one .deps file for a build.

Yeah, it was putting my .deps file in a weird location because I had os.chdir'd before running my first command.  I now know better.

[Lex Trotman]

On 27 October 2013 14:47, Greg Warner <gdwa...@gmail.com> wrote:

On Sat, Oct 26, 2013 at 5:06 PM, Lex Trotman <ele...@gmail.com> wrote:

[...]

Looking at the .deps file (which, oddly enough, it seems that fabricate dumps in the cwd the first time run() is executed, which could be brittle if you os.chdir much), it seems that the commands aren't distinguished at all based on which directory they're run in.  So I did a quick hack to rectify this and modified two lines of code in fabricate.py so that the command key is now composed of the 'os.cwd() + command'.  

So second, is this hack on the right track?  

oops forgot to say this is completely wrong, the command recorded in the .deps no longer matches the command in the run() call so fabricate will *always* re-run the command. 

Cheers

Lex

I tried to cover all my bases in that regard.  I had it alter the command both when recording the deps, as well as when comparing them.  But something isn't working right.   

Thanks for letting me know I could do run("cd dir; command...").  I didn't know that.  

Actually I forgot the command is run by strace, not a shell, so you need to run it in a shell I think, eg run('bash -c "cd xxx;make"')

Cheers

Lex

 

I naively tried run('cd','dir') which didn't work (and doesn't make much sense anyway), which is why I resorted to os.chdir.

--

[Greg Warner]

On Sat, Oct 26, 2013 at 9:29 PM, Lex Trotman <ele...@gmail.com> wrote:

Actually I forgot the command is run by strace, not a shell, so you need to run it in a shell I think, eg run('bash -c "cd xxx;make"')

yuck.

[Lex Trotman]

On 27 October 2013 15:36, Greg Warner <gdwa...@gmail.com> wrote:

On Sat, Oct 26, 2013 at 9:29 PM, Lex Trotman <ele...@gmail.com> wrote:

Actually I forgot the command is run by strace, not a shell, so you need to run it in a shell I think, eg run('bash -c "cd xxx;make"')

I presume you tried the make option that tells it to cd to a directory first?

Cheers

Lex

 

yuck.

[Berwyn Hoyt]

On Sat, Oct 26, 2013 at 9:29 PM, Lex Trotman <ele...@gmail.com> wrote:

Actually I forgot the command is run by strace, not a shell, so you need to run it in a shell I think, eg run('bash -c "cd xxx;make"')

If you wanted to do that, wouldn't you use: run("cd xxx; make", shell=True)

[Lex Trotman]

I think that only runs the strace command in a shell, not the command that strace runs.  

If I read the code right, the arguments to run are appended to an strace command at fabricate.py:546 call to the function shell(), so the shell=True will apply to the strace, not the arguments to strace.

Cheers

Lex

[Berwyn Hoyt]

Good point.  I don't think that was the original intent of shell= so it could probably be fixed.

[Lex Trotman]

On 28 October 2013 19:11, Berwyn Hoyt <ber...@brush.co.nz> wrote:

Good point.  I don't think that was the original intent of shell= so it could probably be fixed.

Could be tricky since we want strace to monitor the command, and it reads that from its command line, luckily I think Gregs problem will be fixed by the -C option to make

Cheers

Lex

 

Actually I forgot the command is run by strace, not a shell, so you need to run it in a shell I think, eg run('bash -c "cd xxx;make"')

If you wanted to do that, wouldn't you use: run("cd xxx; make", shell=True)

I think that only runs the strace command in a shell, not the command that strace runs.  

If I read the code right, the arguments to run are appended to an strace command at fabricate.py:546 call to the function shell(), so the shell=True will apply to the strace, not the arguments to strace.

[Greg Warner]

Prior to running make, I also run 'configure'. I need to see if it has a similar switch.

You received this message because you are subscribed to a topic in the Google Groups "fabricate users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/fabricate-users/zbKeCZKz_BA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to fabricate-use...@googlegroups.com.

[Simon Alford]

I am hoping to address this issue at some point. If I understand correctly it would be solved by implementing a fix for Issue 44 https://code.google.com/p/fabricate/issues/detail?id=44

Simon.

[Lex Trotman]

On 29 October 2013 19:53, Simon Alford <simon....@gmail.com> wrote:

I am hoping to address this issue at some point. If I understand correctly it would be solved by implementing a fix for Issue 44 https://code.google.com/p/fabricate/issues/detail?id=44

Running the command in another directory is simple, subprocess.popen() takes a cwd= option, so it can just be passed through with all the unknown kwargs from run.

But the _do_strace() function will also have to decode the cwd option and prepend the appropriate prefix to all relative pathnames that strace discovers before they are returned, thats a bit more work.

Cheers

Lex

[Greg Warner]

In my first email, I described a hack where in a couple of places I changed 'command' to 'os.cwd() + command'.  It actually works really well.  (I also had to disable the dep caching because those aren't path specific either.)  The only downside is that the dep keys in the .deps file are a lot longer!  On upside to the cwd= approach is I like being able to change directories using the 'context manager' approach (example below using 'temp_chdir', defined elsewhere).   

This is a great stop-gap measure until issue 44 is fixed.

def toolchain():

    # binutils

    mkdir('downloads')

    with temp_chdir('downloads'):

        run('wget','http://ftp.gnu.org/gnu/binutils/binutils-{}.tar.gz'.format(BINUTILS))

        run('tar','xf','binutils-{}.tar.gz'.format(BINUTILS))

        with temp_chdir('binutils-{}'.format(BINUTILS)):

            mkdir('build')

            with temp_chdir('build'):

                run('../configure','--prefix={}'.format(BIN_DIR),'--target=arm-none-eabi','--enable-languages=c','--disable-libssp')

                run('make')

                run('make','install')