Problem in overwriting existing empty directory

[Sandip Gaikwad]

Hi,

There seems to be some problem in overwriting existing empty directory.

Here is  how you can recreate it:

1. Create empty output directory  with only _SUCCESS file in it "tempout/_SUCCESS"
2. Now try to overwrite same directory with some data. Overwrite creates output file "tempout/ch9-13/out-m-00000" instead of "tempout/out-m-00000"

object Temp extends ScoobiApp {

  def run = {

    //create empty directory
    val temp1 = DList[Int]()
    persist(temp1.toTextFile("tempout", overwrite = true))

   //overwrite it
    val temp2 = DList(1, 2, 3, 4)
    persist(temp2.toTextFile("tempout", overwrite = true))

  }
}


Thanks,
Sandip

[Eric Torreborre]

Can you try the latest 0.9.0-SNAPSHOT? Also which exact version are you using now?

Eric.

[Sandip Gaikwad]

Hi Eric,

Thanks for the reply.  I am getting this error with 0.9.0-SNAPSHOT version.
repository/com/nicta/scoobi_2.10/0.9.0-SNAPSHOT/scoobi_2.10-0.9.0-SNAPSHOT.jar

Thanks,
Sandip.

[Eric Torreborre]

Can you please try to remove this version from your repo and download the latest one?

Thanks.

Eric.

[Matthieu Sprunck]

Hi Eric & Sandip,

I have the same issue. I opened the following issue on github :

https://github.com/NICTA/scoobi/issues/351

Is it currently fixed ?

Thanks,

Matthieu