Hi,

I'm using waf to create a build script for web projects (github link<https://github.com/mihai-rotaru/bricka>
).

This is what I intend it to do:
- by parsing an HTML file, detect referenced JavaScript files
- for each of these scripts, create a 'minify_js' task
- after all the minification tasks have been performed, create an
  'update_html' task, which would generate an HTML file referencing the
  new, minified JavaScript files.
- after the HTML file was updated, generate a 'compress_html' task.

I already implemented this functionality, in two different waf tools:
minfier.py and htmlcompressor.py. Everything works if I only load one of the
tools; but if I load both, only the last one executes. I mapped both task
generators to html using @extension.

If I try to use @feature, I get an error saying the task generating
functions
need 2 arguments, and only one is provided. By looking at the source code,
it
doesn't seem like feature and extension have different signatures, but I
guess
my Python skills are not enough to grasp these issues.

I attached a file which showcases the issue.

Any pointers would be very much appreciated.

Mihai

wscript.py 1K Download

Yet, the main problem is that it makes little sense to first copy the
file (1) before making changes to it (2). If (2) fails, for example,
if the program is interrupted, then (1) almost certainly has to be
performed again. The best thing is to have it done at once, and to
wrap the code executing the js compresor to prepare its output file:

class minify_base(Task):
   run_str = '${JSCOMPRESS} ${TGT}'
   vars = ... # additional conf.env.XYZ variables to depend on

class minify(minify_base):
   def run(self):
    self.outputs[0].write(self.inputs[0].read(flags='rb'), flags='wb')
    return minify_base.run(self)

@feature('html')
def html_feature(self):
   for node in self.source_list:
     output = node.get_bld()
     # use this when inputs == outputs in
     # the same virtual folder
     if not os.path.abspath(output.abspath()):
         output.sig = None
         output.parent.mkdir()

     inputs = [node]
     tsk = self.create_task('minify', inputs, output)

# user code in the wscript file
bld(features='html', source_list=[node])

Thomas

Hi Thomas, thank you for taking the time to write an extensive reply.

About using copy, it was only for illustrative purposes; the task really
looks like
this:

class minify_js( Task ):
    run_str = 'java -jar ${closure_compiler} ${jsminifier_options} --js
${SRC} --js_output_file ${TGT}'

I ended up using @feature and setting source_list, but ran into another
problem: "Deadlock detected: check the build order for the tasks", as shown
in
the attached screenshot.

Task order is set as such:

class minify( Task ):
    ...

class update( Task ):
    after = [ 'minify' ]
    ...

class compress( Task ):
    after = [ 'update' ]
    ...

@feature( 'html' )
@after_method( 'generate_minification_tasks' )
def generate_html_compression_tasks( self ):
    ...

By looking at the task id's in the error message, (screenshot attached),  
'compress' is
set to run after 'update', which is expected, but also 'update' is set to
run
after the 'compress' task. I tried manually removing the compress task from
update task's 'after' list, but apparently 'after' lists are populated
after the
@feature methods run, sometime during the build phase.

Why is this behaviour occurring, and how can I prevent it ? Is there a
better
way of piping nodes produced by a tool to another tool, while having the
tools
decoupled ? minifier_tool' needs to work regardless of whether
compressor_tool
is loaded or not.

In Chapter 10 of the waf book, a similar scenario is described - the
`source`
attribute is `extend`ed to contain the outputs of a task. But `source` is
available on task generators created with `bld.program`, which is not the
case
here.

Pseudocode of the approach I used:

minifier_tool:
    parse HTML and produce minification tasks
    if 'compressor_tool' is loaded:
        generate build/<name>.tmp.html
    else
        generate build/<name>.html

compressor_tool:
    if 'minifier_tool' is loaded:
        look for build/<name>.tmp.html
        generate build/<name>.html from build/<name>.tmp.html
    else:
        generate build/<name>.html from <name>.html

I'm also a bit confused about the `html_feature` function in your reply.
1) The `if`'s condition will always evaluate to `true`, because
`os.path.abspath()` alwas seems to return a non-empty string, even when the
parameter is an invalid path or a non-existing file. I'm guessing the
purpose
of this `if` is to check whether the file exists, so it should be
`os.path.exists(...)` ?
2) The `if` block itself - if I understand correctly, signatures are set on
output nodes when a task is successfully executed. Setting it to None should
allow an output node to become an input node for another task ? It doesn't
seem
to work in this case.

I struggled with this deadlock for quite a while, but couldn't figure it
out;
any pointers would be appreciated.

Mihai

wscript.py 2K Download

error.png 48K Download

You have a cycle here.

If you really want to write all this manually (it will not make the
incremental builds any faster):
* specify all the files as node objects, even the temporary ones
* have your tasks declare the exact inputs/outputs
* remove the after/before constraints (implied by the input/output nodes)

If you do not want that "if" block, and perhaps use no build folder
(top=out='.'), have your files written to a sub-directory.

Thomas