building a cube failed

[abhijit deka]

Hi,

I am new to Kylin and was trying to build my first cube.
I am using CHD 5.1.0 quickstart locale VM.

Please let me know if i am doing anything wrong in below steps
1.Did my ETL in PIG.
2.Stored the data in HBase (fact and dimension tables)
3.Created external table in Hive to use those through Hive/Kylin.
4.Started Kylin and followed instruction on how to create the Cube.
5.Started building the Cube.

But soon after the second step the build failed

#3 Step Name: Build Dimension Dictionary
Duration: 0.01 mins

Below is the Log

pool-7-thread-3]:[2015-05-07 05:35:15,117][ERROR][org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:65)] - error execute HadoopShellExecutable{id=53b85f66-da5f-46fb-9f87-6709e2d0f098-02, name=Build Dimension Dictionary, state=RUNNING}
java.io.FileNotFoundException: File hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor does not exist.
    at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:654)
    at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:102)
    at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:712)
    at org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSystem.java:708)
    at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
    at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:708)
    at org.apache.kylin.dict.lookup.HiveTable.findOnlyFile(HiveTable.java:116)
    at org.apache.kylin.dict.lookup.HiveTable.computeHDFSLocation(HiveTable.java:107)
    at org.apache.kylin.dict.lookup.HiveTable.getHDFSLocation(HiveTable.java:83)
    at org.apache.kylin.dict.lookup.HiveTable.getFileTable(HiveTable.java:76)
    at org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:71)
    at org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.java:164)
    at org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:154)
    at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:53)
    at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:42)
    at org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJob.java:53)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
    at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Then I Checked if the file exists or not and it seems like a permission issue.(Not sure though)


drwxrwxrwx   - hive     hive                0 2015-05-07 04:22 hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
drwxrwxrwx   - hive     hive                0 2015-05-07 04:20 hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_cal_dt
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupings
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales

How do I solve this issue?
Please help.

Thanks
Abhijit

[Akash Mishra]

The Cube is expecting "hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor" which is not present in the list

drwxrwxrwx   - hive     hive                0 2015-05-07 04:22 hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
drwxrwxrwx   - hive     hive                0 2015-05-07 04:20 hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:

8020/user/hive/warehouse/kylin_cal_dt
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupings
drwxrwxrwx   - cloudera supergroup          0 2015-05-04 07:18 hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales

Please check if this particular file is present on HDFS?

 

[Luke Han]

Please post question to Apache Kylin dev mailing list for further support.

Thanks.