drake skipping steps with missing output when timecheck:false

19 views
Skip to first unread message

Koert Kuipers

unread,
May 6, 2015, 12:08:44 PM5/6/15
to drake-w...@googlegroups.com
this is my Drakefile:
!hdfs:///user/koert/step2 <- !hdfs:///user/koert/step1 [protocol:shell]
  hadoop fs -cp $INPUT $OUTPUT

!hdfs:///user/koert/step3 <- !hdfs:///user/koert/step2 [protocol:shell]
  hadoop fs -cp $INPUT $OUTPUT

on hdfs i only created the directory step1 with some files inside.
drake behaves as expected:
$ drake
The following steps will be run, in order:
  1: hdfs:///user/koert/step2 <- hdfs:///user/koert/step1 [missing output]
  2: hdfs:///user/koert/step3 <- hdfs:///user/koert/step2 [projected timestamped]
Confirm? [y/n]

now i change my Drakefile to this:
!hdfs:///user/koert/step2 <- !hdfs:///user/koert/step1 [protocol:shell timecheck:false]
  hadoop fs -cp $INPUT $OUTPUT

!hdfs:///user/koert/step3 <- !hdfs:///user/koert/step2 [protocol:shell timecheck:false]
  hadoop fs -cp $INPUT $OUTPUT

and i get this:
$ drake
The following steps will be run, in order:
  1: hdfs:///user/koert/step2 <- hdfs:///user/koert/step1 [missing output]

why is it no longer running step2? what did i do wrong?

thanks! koert
Reply all
Reply to author
Forward
0 new messages