Is it possible to create a generated header which contains the git hash?

[Gábor Márton]

Hi,

I am trying to make a header which depends on the actual git hash. It would be also nice to indicate whether the repo is dirty or not. 

(This would be handy because we could generate a log entry which contains our deployed process' version. We might have more prcocesses with different versions deployed.)

So far I've got the following:

strace -e trace=open git rev-parse HEAD

...

open(".git/HEAD", O_RDONLY)             = 3

open(".git/refs/heads/gitHash", O_RDONLY) = 3

open(".git/packed-refs", O_RDONLY)      = 3

...

Therefore I tried with this Tupfile:

include_rules

: $(PROJECT_ROOT)/.git/HEAD $(PROJECT_ROOT)/.git/refs/heads/* $(PROJECT_ROOT)/.git/packed-refs |> ./gitstuff.sh > %o |> gitHash.h

PROJECT_ROOT is defined in the upper most Tupfile. gitstuff is the script which wraps git rev-parse.

But tup upd is not willing to handle any hidden files. :(

So my next idea was to create a symlink of .git (ln -s .git git) and replace '.git' with 'git' in the Tupfile. However I've got the following error:

tup error: Expected node 'refs' to be in directory 'git', but it is not there.

tup error: Failed to find directory ID for dir '../git/refs/heads/*' relative to 518

Any idea how to move on? Is it possible at all to do such thing with tup?

Many Thanks,

Gábor

[Pat Pannuto]

You can come pretty close without pointing at the actual git objects:

$ cat Tupfile

: foo.c |> gcc -DREV=\"`git rev-parse HEAD`\" %f -o foo |> %B

$ cat foo.c

#include <stdio.h>

#ifndef REV

#define REV "No Rev!"

#endif

int main() {

printf("Rev: %s\n", REV);

return 0;

}

$ tup && ./foo

[ tup ] [0.000s] Scanning filesystem...

[ tup ] [0.000s] Reading in new environment variables...

[ tup ] [0.000s] No Tupfiles to parse.

[ tup ] [0.000s] No files to delete.

[ tup ] [0.000s] No commands to execute.

[ tup ] [0.000s] Updated.

Rev: 2acfd44823c503c2fde71acb5ef0a16abc5cb26f

Now, with this approach, if something in git changes that isn’t in the dependency graph, the built output won’t get updated. At the same time, I’m not convinced that really matters. Using either commit would (by definition) create the same resulting executable. If the executable really in some way depends on something then it should be listed as an order-only input to tup, which will cause tup to rebuild whenever it changes.

HTH,

-Pat

--
--
tup-users mailing list
email: tup-...@googlegroups.com
unsubscribe: tup-users+...@googlegroups.com
options: http://groups.google.com/group/tup-users?hl=en
---
You received this message because you are subscribed to the Google Groups "tup-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tup-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gábor Márton]

Hi, 

Thanks for your reply. I've tried you suggestion and it resulted a valid Tupfile.

BUT, when I commit something into git and rerun tup upd than tup does not updates the target. 

Simply the dependency is not tracked.

So we are again at the beginning, I think the only way to indicate the dependency from the actual git version is via .git/... hidden files.

Is there any particular reason why tup forbids hidden files as input dependencies?

Thanks again,

Gábor

You received this message because you are subscribed to a topic in the Google Groups "tup-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tup-users/e_1ynmm1GA8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tup-users+...@googlegroups.com.

[Pat Pannuto]

Hmm.. good point. My next best suggestion for a workaround would be to install a post-commit hook that writes the hash to a header file.

HTH,

-Pat

[Ben Boeckel]

On Wed, May 14, 2014 at 13:03:38 -0400, Pat Pannuto wrote:
> Hmm.. good point. My next best suggestion for a workaround would be to install
> a post-commit hook that writes the hash to a header file.

That's not going to work if you want a flag whether the tree is dirty as
well. I think the solution is to have the ability to tell Tup "this
command should *always* run; ignore anything it reads" because that's
the only sane thing to do for something like this.

Also, what do you do for tarballs where git isn't available? Usually you
use a .gitattributes to have git archive embed the hash and then use
that (if it's available).

--Ben

[Matthias Vegh]

I'm a colleague of the original poster, and have spent the majority of today trying to get this work. It can be done, but requires a gruesome hack.

My first attempt was to add an environment variable to the mix, one that always changes, and would influence the rule that generated the git hash. Seeing as various shells provide such a variable-like thing ($RANDOM) I started there. It turns out, that $RANDOM is not exactly an environment variable (I'm not sure how exactly, but tup couldn't export it).

My second attempt was to add a symbolic link to one of the files in the .git folder, which would be updated when a new commit is made. The file I chose for this purpose was .git/index. I'm not sure if this is a correct choice, but making a commit, and then resetting updated the times in it. However, once I added the symbolic link as an input file to a rule, I realized, that even though the file pointed to by the link was updated, the link itself was not, so tup was unaware of the changes. (This solution is also not portable, as the .git folder can be renamed, in which case the link would be broken.)

My third attempt was to try to create a rule that would always need to be executed. I achieved this by introducing a circular dependency.

To achieve this, have a script, say alwaysRun.sh that you would like to always run. Add a rule for this script to generate whatever output files you need, then add the following or something similiar to alwaysRun.sh. Please note, that I am not at work atm, so I'm not sure if this will work verbatim.

function cdToActualProjectRoot() {  
    while [[ $PWD != / ] && [[ $PWD != '.tup' ] ; do cd ..; done
    cd ..
}

cdToActualProjectRoot
touch path/to/alwaysRun.sh

The reason this works is that tup detects the script as changed, and so needs to rerun it, however this is also a circular dependency, as even after running a full tup upd, there is still potential work to be done. Tup however does not complain about this, as the write to the script occured once we broke out of the fuse repository, and so tup thinks that a user has modified the script.

If you use this, I recommend using the ^o flag in your rule, so if the output of your alwaysRun.sh is the same as before, the build steps that it influences do not have to be run again.

I'm sure you will all agree that this is a dirty hack, however I didn't want to opt for the post commit hook, as AFAIK, hooks themselves cannot be added to the git repo, so if a new user clones our code, they would not have this functionality. An ideal solution would be an always run rule, or to be able to add hidden files as dependencies.

Hope that helps,

Matt

[Ben Boeckel]

On Fri, May 16, 2014 at 12:16:31 -0700, Matthias Vegh wrote:
> My first attempt was to add an environment variable to the mix, one that always
> changes, and would influence the rule that generated the git hash. Seeing as
> various shells provide such a variable-like thing ($RANDOM) I started there. It
> turns out, that $RANDOM is not exactly an environment variable (I'm not sure
> how exactly, but tup couldn't export it).

It's a shell builtin.

> My second attempt was to add a symbolic link to one of the files in the .git
> folder, which would be updated when a new commit is made. The file I chose for
> this purpose was .git/index. I'm not sure if this is a correct choice, but
> making a commit, and then resetting updated the times in it. However, once I
> added the symbolic link as an input file to a rule, I realized, that even
> though the file pointed to by the link was updated, the link itself was not, so
> tup was unaware of the changes. (This solution is also not portable, as the
> .git folder can be renamed, in which case the link would be broken.)

Symlinks are also notoriously fickle on Windows (needing Administrator
rights).

> I'm sure you will all agree that this is a dirty hack, however I didn't want to
> opt for the post commit hook, as AFAIK, hooks themselves cannot be added to the
> git repo, so if a new user clones our code, they would not have this
> functionality. An ideal solution would be an always run rule, or to be able to
> add hidden files as dependencies.

I solve this by having a build rule clone the hooks branch into
.git/hooks. This won't work in Tup either though :/ .

--Ben

[Andrei Gaponenko]

On Fri, 16 May 2014, Matthias Vegh wrote:

> My second attempt was to add a symbolic link to one of the files in the
> .git folder, which would be updated when a new commit is made. The file I
> chose for this purpose was .git/index. I'm not sure if this is a correct
> choice, but making a commit, and then resetting updated the times in it.
> However, once I added the symbolic link as an input file to a rule, I
> realized, that even though the file pointed to by the link was updated, the
> link itself was not, so tup was unaware of the changes.

mv .git git # and use files under git for tup dependencies
ln -s git .git # keep git working

is a possible workaround.

Andrei

[Freddie Chopin]

On the other hand - I guess that this
tup-doesn't-like-dot-directories-and-files is meant for ignoring .tup
and maybe build directories in fuse filesystem, so maybe it would be
possible to change tup's behavior to ignore only these directories
explicitly, instead of ALL hidden directories?

Regards,
FCh

[Lee Winter]

On Fri, May 16, 2014 at 3:16 PM, Matthias Vegh <matya...@gmail.com> wrote:

I'm a colleague of the original poster, and have spent the majority of today trying to get this work. It can be done, but requires a gruesome hack.

My first attempt was to add an environment variable to the mix, one that always changes, and would influence the rule that generated the git hash. Seeing as various shells provide such a variable-like thing ($RANDOM) I started there. It turns out, that $RANDOM is not exactly an environment variable.

(I'm not sure how exactly, but tup couldn't export it).

Can you assign $RANDOM to MYRANDOM and export that?

Something about all programming problems can be simplified by the addition of an extra level of indirection.

-- Lee

[Mátyás Végh]

I tried that, and that seemed to work, however the logic of say aliasing tup to MYRANDOM=$RANDOM tup means that if someone clones the git repo from scratch, and doesn't know about this, they won't have this functionality.
The same problem is present with a git hook. If someone clones the repo and doesn't apply the extra step to make things work, the build they produce might not have the current hash.

/Matt