this is my Drakefile:
!hdfs:///user/koert/step2 <- !hdfs:///user/koert/step1 [protocol:shell]
hadoop fs -cp $INPUT $OUTPUT
!hdfs:///user/koert/step3 <- !hdfs:///user/koert/step2 [protocol:shell]
hadoop fs -cp $INPUT $OUTPUT
on hdfs i only created the directory step1 with some files inside.
drake behaves as expected:
$ drake
The following steps will be run, in order:
1: hdfs:///user/koert/step2 <- hdfs:///user/koert/step1 [missing output]
2: hdfs:///user/koert/step3 <- hdfs:///user/koert/step2 [projected timestamped]
Confirm? [y/n]
now i change my Drakefile to this:
!hdfs:///user/koert/step2 <- !hdfs:///user/koert/step1 [protocol:shell timecheck:false]
hadoop fs -cp $INPUT $OUTPUT
!hdfs:///user/koert/step3 <- !hdfs:///user/koert/step2 [protocol:shell timecheck:false]
hadoop fs -cp $INPUT $OUTPUT
and i get this:
$ drake
The following steps will be run, in order:
1: hdfs:///user/koert/step2 <- hdfs:///user/koert/step1 [missing output]
why is it no longer running step2? what did i do wrong?
thanks! koert