Alluxio Fuse Connector

Amran Chen

unread,

Oct 2, 2016, 10:53:14 AM10/2/16

to Alluxio Users

Guys,

Could you give me concept and function of Alluxio Fuse Connector..

I already read official web:

http://www.alluxio.org/docs/master/en/Mounting-Alluxio-FS-with-FUSE.html

I still don't understand what's for Alluxio Fuse, I want to try to build it..

Thank you Guys.

Amran Chen

unread,

Oct 3, 2016, 10:48:48 AM10/3/16

to Alluxio Users

Hello any one can help please...

Amran Chen於 2016年10月2日星期日 UTC+8下午10時53分14秒寫道：

Amran Chen

unread,

Oct 3, 2016, 11:04:15 AM10/3/16

to Alluxio Users

Dear Guys,

I found an error when I install JDK 1.8 version. For last time I used 1.7 was ok.

But in Alluxio Fuse, based on official web said it must use 1.8.

I installed JDK 1.8 my alluxio web became like this:

HTTP ERROR 500

Problem accessing /home. Reason:

    Server Error

Caused by:

org.apache.jasper.JasperException: PWC6033: Error in Javac compilation for JSP

PWC6199: Generated servlet error:
The type java.io.ObjectInputStream cannot be resolved. It is indirectly referenced from required .class files


	at org.apache.jasper.compiler.DefaultErrorHandler.javacError(DefaultErrorHandler.java:126)
	at org.apache.jasper.compiler.ErrorDispatcher.javacError(ErrorDispatcher.java:296)
	at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:372)
	at org.apache.jasper.compiler.Compiler.compile(Compiler.java:433)
	at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:608)
	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:476)
	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:366)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:445)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:574)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1044)

official web is written to use alluxio GIT

git clone git://github.com/alluxio/alluxio.git

How about I use download alluxio wget http://downloads.alluxio.org/downloads/files/1.2.0/alluxio-1.2.0-hadoop2.7-bin.tar.gz

because my old alluxio system with this one, I want to add Alluxio Fuse...

Any solution?

Amran Chen於 2016年10月3日星期一 UTC+8下午10時48分48秒寫道：

Jiří Šimša

unread,

Oct 3, 2016, 11:40:48 AM10/3/16

to Amran Chen, Alluxio Users

Hi Amran,

The issue you are running into is this one. The solution is to downgrade your Java version or use Alluxio 1.3.0 (RC1), in which the issue is fixed.

Best,

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Jiří Šimša

Alluxio, Inc.

Amran Chen

unread,

Oct 4, 2016, 1:50:02 AM10/4/16

to Alluxio Users, amran...@gmail.com

Hi Jiri,

I tried to use Alluxio 1.3.0 RC1, Alluxio web page can be displayed but I tried to run runTests, it showed me this error:

2016-10-04 13:46:51,362 ERROR type (MetricsConfig.java:loadConfigFile) - Error oading metrics configuration file.

2016-10-04 13:46:51,364 INFO type (MetricsSystem.java:startSinksFromConfig) - tarting sinks with config: {}.

2016-10-04 13:46:51,423 INFO type (AbstractClient.java:connect) - Alluxio cliet (version 1.3.0-RC1) is trying to connect with FileSystemMasterClient master @localhost/127.0.0.1:19998

2016-10-04 13:46:51,435 INFO type (AbstractClient.java:connect) - Client regisered with FileSystemMasterClient master @ localhost/127.0.0.1:19998

runTest Basic CACHE_PROMOTE MUST_CACHE

2016-10-04 13:46:51,521 ERROR type (MetricsConfig.java:loadConfigFile) - Error oading metrics configuration file.

2016-10-04 13:46:51,521 WARN type (MetricsSystem.java:startSinksFromConfig) - inks have already been started.

2016-10-04 13:46:51,569 INFO type (AbstractClient.java:connect) - Alluxio cliet (version 1.3.0-RC1) is trying to connect with BlockMasterClient master @ locahost/127.0.0.1:19998

2016-10-04 13:46:51,571 INFO type (AbstractClient.java:connect) - Client regisered with BlockMasterClient master @ localhost/127.0.0.1:19998

2016-10-04 13:46:51,586 ERROR type (CliUtils.java:runExample) - Exception runnig test: alluxio.examples.BasicOperations@6483f5ae

java.lang.RuntimeException: No available Alluxio worker found

at alluxio.client.block.AlluxioBlockStore.getOutStream(AlluxioBlockStor.java:185)

at alluxio.client.file.FileOutStream.getNextBlock(FileOutStream.java:33)

at alluxio.client.file.FileOutStream.write(FileOutStream.java:301)

at alluxio.client.file.FileOutStream.write(FileOutStream.java:286)

at alluxio.examples.BasicOperations.writeFile(BasicOperations.java:87)

at alluxio.examples.BasicOperations.call(BasicOperations.java:73)

at alluxio.examples.BasicOperations.call(BasicOperations.java:43)

at alluxio.cli.CliUtils.runExample(CliUtils.java:51)

at alluxio.cli.TestRunner.runTest(TestRunner.java:149)

at alluxio.cli.TestRunner.runTests(TestRunner.java:121)

at alluxio.cli.TestRunner.main(TestRunner.java:98)

Failed the test!

Jiří Šimša於 2016年10月3日星期一 UTC+8下午11時40分48秒寫道：

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-user...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Amran Chen

unread,

Oct 4, 2016, 2:02:36 AM10/4/16

to Alluxio Users, amran...@gmail.com

I have two workers, they run well:

userceph@master:~/alluxio-1.3.0-RC1$ jps

1971 SecondaryNameNode

6148 AlluxioMaster

2152 ResourceManager

1706 NameNode

6314 Jps

userceph@worker1:~$ jps

6610 Jps

6564 AlluxioWorker

1770 NodeManager

1631 DataNode

userceph@worker2:~$ jps

6228 AlluxioWorker

1783 NodeManager

6296 Jps

1643 DataNode

The Error :

./bin/alluxio runTest Basic CACHE THROUGH

2016-10-04 14:00:38,850 ERROR type (MetricsConfig.java:loadConfigFile) - Error loading metrics configuration file.

2016-10-04 14:00:38,852 INFO type (MetricsSystem.java:startSinksFromConfig) - Starting sinks with config: {}.

2016-10-04 14:00:38,914 INFO type (AbstractClient.java:connect) - Alluxio client (version 1.3.0-RC1) is trying to connect with FileSystemMasterClient master @ localhost/127.0.0.1:19998

2016-10-04 14:00:38,932 INFO type (AbstractClient.java:connect) - Client registered with FileSystemMasterClient master @ localhost/127.0.0.1:19998

2016-10-04 14:00:39,001 ERROR type (MetricsConfig.java:loadConfigFile) - Error loading metrics configuration file.

2016-10-04 14:00:39,002 WARN type (MetricsSystem.java:startSinksFromConfig) - Sinks have already been started.

2016-10-04 14:00:39,055 INFO type (AbstractClient.java:connect) - Alluxio client (version 1.3.0-RC1) is trying to connect with BlockMasterClient master @ localhost/127.0.0.1:19998

2016-10-04 14:00:39,057 INFO type (AbstractClient.java:connect) - Client registered with BlockMasterClient master @ localhost/127.0.0.1:19998

2016-10-04 14:00:39,071 ERROR type (CliUtils.java:runExample) - Exception running test: alluxio.examples.BasicOperations@103f852

java.io.IOException: No available Alluxio worker found

at alluxio.client.file.FileSystemContext.getWorkerAddresses(FileSystemContext.java:149)

at alluxio.client.file.FileSystemContext.createWorkerClient(FileSystemContext.java:88)

at alluxio.client.file.FileOutStream.<init>(FileOutStream.java:132)

at alluxio.client.file.FileOutStream.<init>(FileOutStream.java:98)

at alluxio.client.file.BaseFileSystem.createFile(BaseFileSystem.java:103)

at alluxio.examples.BasicOperations.writeFile(BasicOperations.java:86)

at alluxio.examples.BasicOperations.call(BasicOperations.java:73)

at alluxio.examples.BasicOperations.call(BasicOperations.java:43)

at alluxio.cli.CliUtils.runExample(CliUtils.java:51)

at alluxio.cli.TestRunner.runTest(TestRunner.java:149)

at alluxio.cli.TestRunner.main(TestRunner.java:103)

Failed the test!

Amran Chen於 2016年10月4日星期二 UTC+8下午1時50分02秒寫道：

Jiří Šimša

unread,

Oct 4, 2016, 10:49:22 AM10/4/16

to Amran Chen, Alluxio Users

Hi Amran,

It looks like the workers cannot register with the master. Can you print what is the the worker logs? (Can be found in ${ALLUXIO_HOME}/logs/worker.log on the worker machines).

Best,

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Amran Chen

unread,

Oct 4, 2016, 11:24:41 PM10/4/16

to Alluxio Users, amran...@gmail.com

Hi Jiri,

I have seen master.log and worker.log, there is no log in both of them.

I could copy file from local to alluxio. I can run hadoop mapreduce from hdfs source to hdfs destination.

But I tried to run hadoop mapreduce source from alluxio and destination to alluxio as below:

hadoop jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount -libjars /home/userceph/alluxio-1.3.0-RC1/core/client/target/original-alluxio-core-client-1.3.0-RC1-jar-with-dependencies.jar alluxio://192.168.225.134:19998/wordcount/myfile alluxio://192.168.225.134:19998/wordcount/output

it showed me this error for Alluxio-1.3.0-RC1 :

userceph@hadoop:~$ hadoop jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount -libjars /home/userceph/alluxio-1.3.0-RC1/core/client/target/original-alluxio-core-client-1.3.0-RC1-jar-with-dependencies.jar alluxio://192.168.225.134:19998/wordcount/myfile alluxio://192.168.225.134:19998/wordcount/output

16/10/05 11:21:57 INFO logger.type: initialize(alluxio://192.168.225.134:19998/wordcount/myfile, Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml). Connecting to Alluxio: alluxio://192.168.225.134:19998/wordcount/myfile

16/10/05 11:21:57 INFO logger.type: alluxio://192.168.225.134:19998 alluxio://192.168.225.134:19998

16/10/05 11:21:57 INFO logger.type: Loading Alluxio properties from Hadoop configuration: {}

16/10/05 11:21:57 ERROR logger.type: Error loading metrics configuration file.

16/10/05 11:21:57 INFO logger.type: Starting sinks with config: {}.

16/10/05 11:21:57 ERROR logger.type: Error loading metrics configuration file.

16/10/05 11:21:57 WARN logger.type: Sinks have already been started.

16/10/05 11:21:57 INFO logger.type: getWorkingDirectory: /

16/10/05 11:21:57 INFO client.RMProxy: Connecting to ResourceManager at hadoop/192.168.225.134:8032

16/10/05 11:21:58 INFO logger.type: getFileStatus(alluxio://192.168.225.134:19998/wordcount/output)

16/10/05 11:21:58 INFO logger.type: Alluxio client (version 1.3.0-RC1) is trying to connect with FileSystemMasterClient master @ /192.168.225.134:19998

java.io.IOException: Failed to login: No Alluxio User is found.

at alluxio.security.LoginUser.login(LoginUser.java:99)

at alluxio.security.LoginUser.get(LoginUser.java:57)

at alluxio.security.authentication.PlainSaslTransportProvider.getClientTransport(PlainSaslTransportProvider.java:53)

at alluxio.AbstractClient.connect(AbstractClient.java:173)

at alluxio.AbstractClient.retryRPC(AbstractClient.java:339)

at alluxio.client.file.FileSystemMasterClient.getStatus(FileSystemMasterClient.java:183)

at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:176)

at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:168)

at alluxio.hadoop.AbstractFileSystem.getFileStatus(AbstractFileSystem.java:293)

at alluxio.hadoop.FileSystem.getFileStatus(FileSystem.java:25)

at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)

at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:145)

at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266)

at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)

at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)

at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)

at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)

at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)

at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)

at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: javax.security.auth.login.LoginException: No Alluxio User is found.

at alluxio.security.LoginUser.login(LoginUser.java:92)

... 34 more

Any solution?

Jiří Šimša於 2016年10月4日星期二 UTC+8下午10時49分22秒寫道：

Amran Chen

unread,

Oct 4, 2016, 11:34:38 PM10/4/16

to Alluxio Users, amran...@gmail.com

Is this one Alluxio-1.3.0-RC1 bugs error?

Because for the previous version Alluxio 1-2.0 I could run it well.

I need to Install Alluxio Fuse, it asked me to use Oracle JDK 1.8, but Alluxio-1.2.0 is not support for JDK 1.8. Only Alluxio-1.3.0

Any solution?

Amran Chen於 2016年10月5日星期三 UTC+8上午11時24分41秒寫道：

Amran Chen

unread,

Oct 5, 2016, 12:02:40 AM10/5/16

to Alluxio Users, amran...@gmail.com

My alluxio-env.sh configuration as below:

export JAVA="${JAVA_HOME}/bin/java"

ALLUXIO_MASTER_HOSTNAME=${ALLUXIO_MASTER_HOSTNAME:-"localhost"}

ALLUXIO_WORKER_MEMORY_SIZE=${ALLUXIO_WORKER_MEMORY_SIZE:-"1322MB"}

ALLUXIO_RAM_FOLDER=${ALLUXIO_RAM_FOLDER:-"/mnt/ramdisk"}

#ALLUXIO_UNDERFS_ADDRESS=${ALLUXIO_UNDERFS_ADDRESS:-"${ALLUXIO_HOME}/underFSStorage/"}

ALLUXIO_UNDERFS_ADDRESS="hdfs://hadoop:9000"

export ALLUXIO_JAVA_OPTS+="

-Dalluxio.user.file.writetype.default=CACHE_THROUGH

-Dlog4j.configuration=file:${CONF_DIR}/log4j.properties

-Dalluxio.worker.tieredstore.levels=1

-Dalluxio.worker.tieredstore.level0.alias=MEM

-Dalluxio.worker.tieredstore.level0.dirs.path=${ALLUXIO_RAM_FOLDER}

-Dalluxio.worker.tieredstore.level0.dirs.quota=${ALLUXIO_WORKER_MEMORY_SIZE}

-Dalluxio.underfs.address=${ALLUXIO_UNDERFS_ADDRESS}

-Dalluxio.worker.memory.size=${ALLUXIO_WORKER_MEMORY_SIZE}

-Dalluxio.master.hostname=${ALLUXIO_MASTER_ADDRESS}

-Dorg.apache.jasper.compiler.disablejsr199=true

-Djava.net.preferIPv4Stack=true

"

export ALLUXIO_MASTER_JAVA_OPTS="${ALLUXIO_JAVA_OPTS}"

export ALLUXIO_WORKER_JAVA_OPTS="${ALLUXIO_JAVA_OPTS}"

Amran Chen於 2016年10月5日星期三 UTC+8上午11時34分38秒寫道：

Amran Chen

unread,

Oct 5, 2016, 8:45:56 AM10/5/16

to Alluxio Users, amran...@gmail.com

I just tried Alluxio-1.2.0 I could run hadoop mapreduce smoothly, the source file from alluxio to alluxio

But Alluxio-1.3.0 It was error. Is this bug error from new version of Alluxio-1.3.0 ?

Amran Chen於 2016年10月5日星期三 UTC+8下午12時02分40秒寫道：

Chaomin Yu

unread,

Nov 3, 2016, 4:14:12 PM11/3/16

to Amran Chen, Alluxio Users

Hi Amran,

Can you please try to add the Alluxio client jar in $HADOOP_CLASSPATH ? e.g. change the following in "hadoop-env.sh". This is a new requirement in Alluxio 1.3.0 according to the Alluxio 1.3.0 Running Hadoop MapReduce on Alluxio docs.

$ export HADOOP_CLASSPATH=/<PATH_TO_ALLUXIO>/core/client/target/alluxio-core-client-1.3.0-jar-with-dependencies.jar:${HADOOP_CLASSPATH}

Hope this helps,

Chaomin

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Cheers,

Chaomin

Alluxio Inc

Reply all

Reply to author

Forward