Error alluxio

[Amran Chen]

Hi All,

I found this error :

 ERROR logger.type: Error loading metrics configuration file

what does it mean?

Tq

[Pei Sun]

Hi Amran,

   Which version are you using? This error usually means property alluxio.metrics.conf.file points to a missing file. We have removed the logging in branch-1.3.

Pei

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Pei Sun

[Amran Chen]

Pei Sun,

I was using Alluxio 1.3. I need to use JDK 1.8

Any help?

Pei Sun於 2016年10月6日星期四 UTC+8上午12時52分00秒寫道：

Hi Amran,

   Which version are you using? This error usually means property alluxio.metrics.conf.file points to a missing file. We have removed the logging in branch-1.3.

Pei

On Wed, Oct 5, 2016 at 8:42 AM, Amran Chen <amran...@gmail.com> wrote:

Hi All,

I found this error :

 ERROR logger.type: Error loading metrics configuration file

what does it mean?

Tq

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-user...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pei Sun

[Pei Sun]

Hi Amran,

    I am not very clear what are you trying to achieve. As I said, this ERROR message just says that alluxio.metrics.conf.file points to a metrics config file that cannot be correctly parsed or opened (say the file is missing). You can safely ignore it if you don't want to setup any external metrics sink. If you want to stop seeing this error in your log, you can set alluxio.metrics.conf.file to point to some config file or just set it to empty string("").

Pei

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pei Sun

[Pei Sun]

Hi Amran,

    I forgot to mention, you can also wait for 1.3-RC2 or build from branch-1.3 which removes this logging when the file is missing.

Pei

--

Pei Sun

[Amran Chen]

Pei Sun,
I am using for running map reduce that data source and destination from Alluxio 1.3.0

bin/hadoop jar hadoop-examples-1.3.0-RC1...jar wordcount -libjars /<PATH_TO_ALLUXIO>/core/client/target/alluxio-core-client-1.3.0-...-with-dependencies.jar alluxio://localhost:19998/wordcount/input.txt alluxio://localhost:19998/wordcount/output

it retrieves me the error..the next step i will use it for fuse connector that need jdk 1.8 based on aluxio official web instruction..

[Pei Sun]

Amran,

OK, it doesn't look like you are setting up the metrics system. You can just ignore the error msg. 

Pei

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Pei Sun

[Amran Chen]

Hi Pei,

I ignored it, it return for me this error when I run hadoop:

userceph@hadoop:~$ hadoop jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount -libjars alluxio-1.3.0-RC1/core/client/target/alluxio-core-client-1.3.0-RC1-jar-with-dependencies.jar alluxio://192.168.225.136:19998/user/userceph/myfile alluxio://192.168.225.136:19998/user/userceph/output

16/10/06 10:33:24 INFO logger.type: initialize(alluxio://192.168.225.136:19998/user/userceph/myfile, Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml). Connecting to Alluxio: alluxio://192.168.225.136:19998/user/userceph/myfile

16/10/06 10:33:24 INFO logger.type: alluxio://192.168.225.136:19998 alluxio://192.168.225.136:19998

16/10/06 10:33:24 INFO logger.type: Loading Alluxio properties from Hadoop configuration: {}

16/10/06 10:33:25 ERROR logger.type: Error loading metrics configuration file.

16/10/06 10:33:25 INFO logger.type: Starting sinks with config: {}.

16/10/06 10:33:25 ERROR logger.type: Error loading metrics configuration file.

16/10/06 10:33:25 WARN logger.type: Sinks have already been started.

16/10/06 10:33:25 INFO logger.type: getWorkingDirectory: /

16/10/06 10:33:25 INFO logger.type: getWorkingDirectory: /

16/10/06 10:33:25 INFO client.RMProxy: Connecting to ResourceManager at hadoop/192.168.225.136:8032

16/10/06 10:33:32 INFO logger.type: getFileStatus(alluxio://192.168.225.136:19998/user/userceph/output)

16/10/06 10:33:33 INFO logger.type: Alluxio client (version 1.3.0-RC1) is trying to connect with FileSystemMasterClient master @ /192.168.225.136:19998

java.io.IOException: Failed to login: No Alluxio User is found.

        at alluxio.security.LoginUser.login(LoginUser.java:99)

        at alluxio.security.LoginUser.get(LoginUser.java:57)

        at alluxio.security.authentication.PlainSaslTransportProvider.getClientTransport(PlainSaslTransportProvider.java:53)

        at alluxio.AbstractClient.connect(AbstractClient.java:173)

        at alluxio.AbstractClient.retryRPC(AbstractClient.java:339)

        at alluxio.client.file.FileSystemMasterClient.getStatus(FileSystemMasterClient.java:183)

        at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:176)

        at alluxio.client.file.BaseFileSystem.getStatus(BaseFileSystem.java:168)

        at alluxio.hadoop.AbstractFileSystem.getFileStatus(AbstractFileSystem.java:293)

        at alluxio.hadoop.FileSystem.getFileStatus(FileSystem.java:25)

        at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1426)

        at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:145)

        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:266)

        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:139)

        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)

        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:422)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)

        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)

        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1308)

        at org.apache.hadoop.examples.WordCount.main(WordCount.java:87)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)

        at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)

        at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:498)

        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: javax.security.auth.login.LoginException: No Alluxio User is found.

        at alluxio.security.LoginUser.login(LoginUser.java:92)

        ... 34 more

What does "java.io.IOException: Failed to login: No Alluxio User is found." mean? 

Thank you Pei

Pei Sun於 2016年10月6日星期四 UTC+8上午1時40分27秒寫道：

Amran,

OK, it doesn't look like you are setting up the metrics system. You can just ignore the error msg. 

Pei

On Wed, Oct 5, 2016 at 10:37 AM, Amran Chen <amran...@gmail.com> wrote:

Pei Sun,
I am using for running map reduce that data source and destination from Alluxio 1.3.0

bin/hadoop jar hadoop-examples-1.3.0-RC1...jar wordcount -libjars /<PATH_TO_ALLUXIO>/core/client/target/alluxio-core-client-1.3.0-...-with-dependencies.jar alluxio://localhost:19998/wordcount/input.txt alluxio://localhost:19998/wordcount/output

it retrieves me the error..the next step i will use it for fuse connector that need jdk 1.8 based on aluxio official web instruction..

--
You received this message because you are subscribed to the Google Groups "Alluxio Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-user...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pei Sun

[Chaomin Yu]

Hi Amran, can you please check whether you have the alluxio client jar added in HADOOP_CLASSPATH? per the doc here.

In order for the Alluxio client jar to be available to the JobClient, you can modify HADOOP_CLASSPATH by changing hadoop-env.sh to:

$ export HADOOP_CLASSPATH=/<PATH_TO_ALLUXIO>/core/client/target/alluxio-core-client-1.3.0-RC1-jar-with-dependencies.jar:${HADOOP_CLASSPATH}

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Cheers,

Chaomin

Alluxio Inc

[Amran Chen]

Dear Chaomin,

Thank you for your good response.

I had set HADOOP_CLASSPATH before in Hadoop as below:

export JAVA_HOME=/usr/lib/jvm/jdk/

# The jsvc implementation to use. Jsvc is required to run secure datanodes

# that bind to privileged ports to provide authentication of data transfer

# protocol.  Jsvc is not required if SASL is configured for authentication of

# data transfer protocol using non-privileged ports.

#export JSVC_HOME=${JSVC_HOME}

export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}

# Extra Java CLASSPATH elements.  Automatically insert capacity-scheduler.

for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do

  if [ "$HADOOP_CLASSPATH" ]; then

    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/userceph/alluxio/core/client/target/alluxio-core-client-1.3.0-SNAPSHOT-jar-with-dependencies.jar

  else

    export HADOOP_CLASSPATH=$f

  fi

done

For core-site.xml

<configuration>

 <property>

   <name>fs.default.name</name>

   <value>hdfs://hadoop:9000</value>

 </property>

  <property>

  <name>fs.alluxio.impl</name>

  <value>alluxio.hadoop.FileSystem</value>

  <description>The Alluxio FileSystem (Hadoop 1.x and 2.x)</description>

</property>

<property>

  <name>fs.alluxio-ft.impl</name>

  <value>alluxio.hadoop.FaultTolerantFileSystem</value>

  <description>The Alluxio FileSystem (Hadoop 1.x and 2.x) with fault tolerant support</description>

</property>

<property>

  <name>fs.AbstractFileSystem.alluxio.impl</name>

  <value>alluxio.hadoop.AlluxioFileSystem</value>

  <description>The Alluxio AbstractFileSystem (Hadoop 2.x)</description>

</property>

</configuration>

For Alluxio workers:

hadoop  #hostname

For alluxio-env.sh:

export JAVA="${JAVA_HOME}/bin/java"

ALLUXIO_MASTER_HOSTNAME=${ALLUXIO_MASTER_HOSTNAME:-"localhost"}

ALLUXIO_WORKER_MEMORY_SIZE=${ALLUXIO_WORKER_MEMORY_SIZE:-"1GB"}

ALLUXIO_RAM_FOLDER=${ALLUXIO_RAM_FOLDER:-"/mnt/ramdisk"}

ALLUXIO_UNDERFS_ADDRESS=hdfs://hadoop:9000

export ALLUXIO_JAVA_OPTS+="

  -Dalluxio.user.file.writetype.default=CACHE_THROUGH

  -Dlog4j.configuration=file:${CONF_DIR}/log4j.properties

  -Dalluxio.worker.tieredstore.levels=1

  -Dalluxio.worker.tieredstore.level0.alias=MEM

  -Dalluxio.worker.tieredstore.level0.dirs.path=${ALLUXIO_RAM_FOLDER}

  -Dalluxio.worker.tieredstore.level0.dirs.quota=${ALLUXIO_WORKER_MEMORY_SIZE}

  -Dalluxio.underfs.address=${ALLUXIO_UNDERFS_ADDRESS}

  -Dalluxio.worker.memory.size=${ALLUXIO_WORKER_MEMORY_SIZE}

  -Dalluxio.master.hostname=${ALLUXIO_MASTER_ADDRESS}

  -Dorg.apache.jasper.compiler.disablejsr199=true

  -Djava.net.preferIPv4Stack=true

"

export ALLUXIO_MASTER_JAVA_OPTS="${ALLUXIO_JAVA_OPTS}"

export ALLUXIO_WORKER_JAVA_OPTS="${ALLUXIO_JAVA_OPTS}"

JPS:

userceph@hadoop:~/alluxio/conf$ jps

8545 DataNode

9314 NodeManager

9602 Jps

8787 SecondaryNameNode

4360 AlluxioMaster

8969 ResourceManager

4442 AlluxioWorker

8350 NameNode

I still have error if i run hadoop mapreduce as below:

userceph@hadoop:~$ hadoop jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount -libjars alluxio/core/client/target/alluxio-core-client-1.3.0-SNAPSHOT-jar-with-dependencies.jar alluxio://192.168.225.137:19998/user/userceph/myfile alluxio://192.168.225.137:19998/user/userceph/output

16/10/06 12:38:54 INFO logger.type: initialize(alluxio://192.168.225.137:19998/user/userceph/myfile, Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, hdfs-default.xml, hdfs-site.xml). Connecting to Alluxio: alluxio://192.168.225.137:19998/user/userceph/myfile

16/10/06 12:38:54 INFO logger.type: alluxio://192.168.225.137:19998 alluxio://192.168.225.137:19998

16/10/06 12:38:54 INFO logger.type: Loading Alluxio properties from Hadoop configuration: {}

16/10/06 12:38:54 INFO logger.type: Starting sinks with config: {}.

16/10/06 12:38:54 WARN logger.type: Sinks have already been started.

16/10/06 12:38:54 INFO logger.type: getWorkingDirectory: /

16/10/06 12:38:54 INFO logger.type: getWorkingDirectory: /

16/10/06 12:38:54 INFO client.RMProxy: Connecting to ResourceManager at hadoop/192.168.225.137:8032

16/10/06 12:38:54 INFO logger.type: getFileStatus(alluxio://192.168.225.137:19998/user/userceph/output)

16/10/06 12:38:54 INFO logger.type: Alluxio client (version 1.3.0-SNAPSHOT) is trying to connect with FileSystemMasterClient master @ /192.168.225.137:19998

userceph@hadoop:~$

I don't know what's problem this one on Alluxio-1.3.0 RC1 or Alluxio 1.3.0 Git Master.

But I run on Alluxio-1.2.0 so far no problem.

But I need to run Alluxio Fuse, Alluxio Official Web requirement is Oracle JDK 1.8 so I have to upgrade to Alluxio 1.3.0

Do you have solution? I have tried several days to google search, I still don't find any fix solution...

Thank you Chaomin and Pei

Chaomin Yu於 2016年10月6日星期四 UTC+8下午12時16分30秒寫道：

Cheers,

Chaomin

Alluxio Inc

[Amran Chen]

No Alluxio user is found means should I set userlogin aluxio?

alluxio.security.login.username ?

Amran Chen於 2016年10月6日星期四 UTC+8下午12時42分07秒寫道：

[Chaomin Yu]

Hi Amran,

Thank you for providing the detailed configuration.

A major different between 1.3 and 1.2 is that security is enabled by default in 1.3. That's why you did not see this in 1.2. Also, if you don't have Alluxio client jar in HADOOP_CLASSPATH in 1.2.0 it was okay, but it's now required in 1.3.0.

I encountered this similar error before during 1.3 testing. Adding the Alluxio client jar to HADOOP_CLASSPATH correctly will fix this problem.

I wonder in your following script, is it possible that HADOOP_CLASSPATH end up being "f" without alluxio jar?

Would it make sense to move the "export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/userceph/alluxio/core/client/target/alluxio-core-client-1.3.0-SNAPSHOT-jar-with-dependencies.jar" out of for loop so that Alluxio jar is guaranteed to be added in HADOOP_CLASSPATH?

Hope this helps,

Chaomin

-------Your snippet -------

# Extra Java CLASSPATH elements.  Automatically insert capacity-scheduler.

for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do

  if [ "$HADOOP_CLASSPATH" ]; then

    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/userceph/alluxio/core/client/target/alluxio-core-client-1.3.0-SNAPSHOT-jar-with-dependencies.jar

  else

    export HADOOP_CLASSPATH=$f

  fi

done

---------------------------------

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Amran Chen]

Chaomin ...!

Thank you very much, problem has been solve ^^

Warm Regards,

Amran

Chaomin Yu於 2016年10月7日星期五 UTC+8上午12時52分59秒寫道：

[Chaomin Yu]

Glad to hear the problem is resolved.

Best,

Chaomin

To unsubscribe from this group and stop receiving emails from it, send an email to alluxio-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.