Not able to start on windows

[sabari...@gmail.com]

I am trying to start the CDAP standalone SDK on my development laptop running Windows. I am seeing the following error in the cdap-process.log

Failed to start Standalone CDAP. com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: F:/cdap-sdk-3.2.2/data/explore/tmp on HDFS should be writable. Current permissions are: rw-rw-rw-

Exception in thread "ExploreExecutorService STARTING" com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: F:/cdap-sdk-3.2.2/data/explore/tmp on HDFS should be writable. Current permissions are: rw-rw-rw-

at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015)

at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001)

at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220)

at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106)

at co.cask.cdap.explore.executor.ExploreExecutorService.startUp(ExploreExecutorService.java:91)

at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43)

at java.lang.Thread.run(Thread.java:745)

I tried using my git and winutils to change the permissions of F:/cdap-sdk-3.2.2/data/explore/tmp to 777 but it still doesn't help.

Any pointers on what should be done to make this work?

Regards

Sab

[Andreas Neumann]

Hi Sab,

this looks intriguing. This message, afaict comes directly from your Windows JDK, and I am not sure how to interpret it, because it contradicts itself... It complains that the directory is not writable, then shows the current permissions, which imply it is writable. 

Is it possible that you are lacking permissions at some other level? Can you check that this directory  F:/cdap-sdk-3.2.2/data/explore/tmp exists and that you can access all of its parent directories?

Thanks -Andreas.

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/9e695496-ef0f-429d-90a4-a7ff805d179b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[sabari...@gmail.com]

Hi Andreas

Yeah I see that directory. I can open it and all, via command line and the explorer or even via git shell.

Yes, the message is contradictory. It's coming from Hive. This is the code in Hive

FsPermission writableHDFSDirPermission = new FsPermission((short)00733);
    FileSystem fs = rootHDFSDirPath.getFileSystem(conf);
    if (!fs.exists(rootHDFSDirPath)) {
      Utilities.createDirsWithPermission(conf, rootHDFSDirPath, writableHDFSDirPermission, true);
    }
    FsPermission currentHDFSDirPermission = fs.getFileStatus(rootHDFSDirPath).getPermission();
    LOG.debug("HDFS root scratch dir: " + rootHDFSDirPath + ", permission: "
        + currentHDFSDirPermission);
    // If the root HDFS scratch dir already exists, make sure it is writeable.
    if (!((currentHDFSDirPermission.toShort() & writableHDFSDirPermission
        .toShort()) == writableHDFSDirPermission.toShort())) {
      throw new RuntimeException("The root scratch dir: " + rootHDFSDirPath
          + " on HDFS should be writable. Current permissions are: " + currentHDFSDirPermission);
    }

Regards

Sab

[Andreas Neumann]

HI Sab,

it looks like it is missing the execute flag (it has rw-rw-rw which is 3 times 6). But it wants the 1 bit in all positions. You are saying that setting the permissions to 777 does not help? What happens if you delete that directory and let Hive recreate it with the permissions that it expects? This line should create it with the correct permissions:

    if (!fs.exists(rootHDFSDirPath)) {
      Utilities.createDirsWithPermission(conf, rootHDFSDirPath, writableHDFSDirPermission, true);
    }

If that does not help, is there anything in your Windows that would prevent setting the x bits? Can you try it on other directories?

Best -Andreas.

To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/c69a626a-8b71-45d0-bc12-9819b851784b%40googlegroups.com.