Tachyon on Hadoop 2.7.1 build failure

[Hawin Jiang]

Hi  All

This is my first to play Tachyon.  I got build failure. 

Can someone help me to fix it?

Thanks. 

Here are my steps:

1. git clone git://github.com/amplab/tachyon.git

2. cd tachyon

3. Set “<hadoop.version>2.7.1</hadoop.version>” instead of “<hadoop.version>2.6.0</hadoop.version>”

4. mvn -Dhadoop.version=2.7.1 clean package

I have received some errors below:

Tests run: 11, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.135 sec <<< FAILURE!

changeLocalFilePermissionTest(tachyon.util.io.FileUtilsTest)  Time elapsed: 0.049 sec  <<< FAILURE!

java.lang.AssertionError: null

        at org.junit.Assert.fail(Assert.java:86)

        at org.junit.Assert.assertTrue(Assert.java:41)

        at org.junit.Assert.assertFalse(Assert.java:64)

        at org.junit.Assert.assertFalse(Assert.java:74)

        at tachyon.util.io.FileUtilsTest.changeLocalFilePermissionTest(FileUtilsTest.java:51)

changeLocalDirPermissionTests(tachyon.util.io.FileUtilsTest)  Time elapsed: 0.009 sec  <<< FAILURE!

java.lang.AssertionError: null

        at org.junit.Assert.fail(Assert.java:86)

        at org.junit.Assert.assertTrue(Assert.java:41)

        at org.junit.Assert.assertFalse(Assert.java:64)

        at org.junit.Assert.assertFalse(Assert.java:74)

        at tachyon.util.io.FileUtilsTest.changeLocalDirPermissionTests(FileUtilsTest.java:74)

Running tachyon.util.io.PathUtilsTest

Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec Running tachyon.util.io.BufferUtilsTest Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.77 sec Running tachyon.resource.ResourcePoolTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.138 sec Running tachyon.conf.TachyonConfTest Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec Results : Failed tests: FileUtilsTest.changeLocalFilePermissionTest:51 null FileUtilsTest.changeLocalDirPermissionTests:74 null Tests run: 157, Failures: 2, Errors: 0, Skipped: 4

Tests run: 157, Failures: 2, Errors: 0, Skipped: 4 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Tachyon Parent ..................................... SUCCESS [ 4.658 s] [INFO] Tachyon Common ..................................... FAILURE [01:29 min] [INFO] Tachyon Under File System .......................... SKIPPED [INFO] Tachyon Under File System - Local FS ............... SKIPPED [INFO] Tachyon Under File System - HDFS ................... SKIPPED

[INFO] Tachyon Under File System - Gluster FS ............. SKIPPED

[INFO] Tachyon Under File System - Swift .................. SKIPPED

[INFO] Tachyon Under File System - S3 ..................... SKIPPED

[INFO] Tachyon Clients .................................... SKIPPED

[INFO] Tachyon Clients - Implementation ................... SKIPPED

[INFO] Tachyon Clients - Distribution ..................... SKIPPED

[INFO] Tachyon Servers .................................... SKIPPED

[INFO] Tachyon MiniCluster ................................ SKIPPED

[INFO] Tachyon Shell ...................................... SKIPPED

[INFO] Tachyon Examples ................................... SKIPPED

[INFO] Tachyon Integration Tests .......................... SKIPPED

[INFO] Tachyon Integration ................................ SKIPPED

[INFO] Tachyon Assemblies ................................. SKIPPED

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 01:34 min

[INFO] Finished at: 2015-11-09T03:51:54-05:00

[INFO] Final Memory: 59M/921M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on project tachyon-common: There are test failures.

[ERROR]

[ERROR] Please refer to /data/opt/tachyon/common/target/surefire-reports for the individual test results.

[ERROR] -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[ERROR]

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :tachyon-common

[and...@tachyonnexus.com]

Hi Hawin, thanks for reporting this issue.

It looks like the failing tests are using chmod to change file permissions, then failing when the permissions don't actually change. What OS and Java versions are you running?

Best,

Andrew

[Hawin Jiang]

Hi  Andrew

Here is the details:

OS:

Linux 3.10.0-229.14.1.el7.x86_64 #1 SMP Tue Sep 15 15:05:51 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Java:

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

Would you please tell me how to pass this first step?  We are evaluating apache tachyon in our new system now. we are using Hadoop 2.7.1. 

Please let me know if you need more inputs from my side.

Thanks.

Best regards

Hawin

[Andrew Audibert]

Hi Hawin,

Can post the log at common/target/logs/tests.log? There might be some hints in there about why the permissions aren't changing.

In the short term to unblock yourself, you could run maven with -DskipTests.

Best,

Andrew

On Monday, November 9, 2015 at 1:02:35 AM UTC-8, Hawin Jiang wrote:

[Hawin Jiang]

Hi  Andrew

Please check common/target/logs/tests.log below. 

java.lang.NullPointerException

        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

2015-11-09 14:15:51,292 ERROR transport.TSaslTransport (TSaslTransport.java:open) - SASL negotiation failure

javax.security.sasl.SaslException: Plain authentication failed: User authentication fails [Caused by javax.security.sasl.AuthenticationException: User authentication fails]

        at tachyon.security.authentication.PlainSaslServer.evaluateResponse(PlainSaslServer.java:121)

        at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)

        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)

        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)

        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: javax.security.sasl.AuthenticationException: User authentication fails

        at tachyon.security.authentication.AuthenticationUtilsTest$ExactlyMatchAuthenticationProvider.authenticate(AuthenticationUtilsTest.java:360)

        at tachyon.security.authentication.CustomAuthenticationProviderImpl.authenticate(CustomAuthenticationProviderImpl.java:57)

        at tachyon.security.authentication.PlainSaslServer$PlainServerCallbackHandler.handle(PlainSaslServer.java:204)

        at tachyon.security.authentication.PlainSaslServer.evaluateResponse(PlainSaslServer.java:112)

        ... 8 more

2015-11-09 14:15:51,293 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run) - Error occurred during processing of message.

java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Plain authentication failed: User authentication fails

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)

        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.thrift.transport.TTransportException: Plain authentication failed: User authentication fails

        at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)

        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)

        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)

        ... 4 more

Best regards

Hawin

[Hawin Jiang]

Hi  All

Can someone take a look at this issue?

Thanks

Best regards

Hawin

[Andrew Audibert]

Hawin,

There doesn't seem to be anything in the log related to chmod. I can't reproduce the problem locally so it's difficult to say what is happening. In the first failed test, chmod works the first two times we call it, then fails without reporting an error the third time.

I've created https://tachyon.atlassian.net/browse/TACHYON-1252 to track this issue - it would be great if you could investigate the failure and submit a PR.

Best,

Andrew

[Hawin Jiang]

Hi  Andrew

I am seeing that Tachyon is asking password.  Do you know how to config username and password?

BTW, I have run 'chmod -R 777 tachyon/'  I am still having some issues below. 

Thanks. 

2015-11-09 20:25:21,747 ERROR transport.TSaslTransport (TSaslTransport.java:open) - SASL negotiation failure

javax.security.sasl.SaslException: Plain authentication failed: No password provided [Caused by java.lang.IllegalStateException: No password provided]

        at tachyon.security.authentication.PlainSaslServer.evaluateResponse(PlainSaslServer.java:121)

        at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539)

        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283)

        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)

        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.IllegalStateException: No password provided

        at com.google.common.base.Preconditions.checkState(Preconditions.java:149)

        at tachyon.security.authentication.PlainSaslServer.evaluateResponse(PlainSaslServer.java:96)

        ... 8 more

2015-11-09 20:25:21,748 ERROR server.TThreadPoolServer (TThreadPoolServer.java:run) - Error occurred during processing of message.

java.lang.RuntimeException: org.apache.thrift.transport.TTransportException: Plain authentication failed: No password provided

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)

        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:268)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.thrift.transport.TTransportException: Plain authentication failed: No password provided

        at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)

        at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)

        at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)

        at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)

        ... 4 more

2015-11-09 20:25:21,828 WARN   (UnderFileSystemRegistry.java:find) - No Under File System Factory implementation supports the path /test/path

2015-11-09 20:25:21,829 WARN   (UnderFileSystemRegistry.java:find) - No Under File System Factory implementation supports the path file:///test/path

2015-11-09 20:25:23,386 INFO   (FileUtils.java:createStorageDirPath) - Folder /tmp/junit7834920697236222850/tmp was created!

Best regards

Hawin

[Andrew Audibert]

Those logs are the combined logs from running all of the tests - the errors you see in the log are expected and happen during different tests. The first failing test is FileUtilsTest.changeLocalFilePermissionTest. It looks like

  @Test

  public void changeLocalFilePermissionTest() throws IOException {

    File tempFile = mTestFolder.newFile("perm.txt");

    FileUtils.changeLocalFilePermission(tempFile.getAbsolutePath(), "000");

    Assert.assertFalse(tempFile.canRead() && tempFile.canWrite() && tempFile.canExecute());

    FileUtils.changeLocalFilePermission(tempFile.getAbsolutePath(), "777");

    Assert.assertTrue(tempFile.canRead() && tempFile.canWrite() && tempFile.canExecute());

    // File deletion should fail, because we don't have write permissions

    FileUtils.changeLocalFilePermission(tempFile.getAbsolutePath(), "444");

    Assert.assertTrue(tempFile.canRead());

    Assert.assertFalse(tempFile.canWrite()); <--- This is the line that fails

    Assert.assertFalse(tempFile.canExecute());

    // expect a file permission error when we open it for writing

    mException.expect(IOException.class);

    @SuppressWarnings({"unused", "resource"})

    FileWriter fw = new FileWriter(tempFile);

    Assert.fail("opening a read-only file for writing should have failed");

  }

Line 51, the Assert.assertFalse(tempFile.canWrite()) after setting the permissions to "444" is the line that fails. changeLocalFilePermission is essentially a wrapper around chmod. You can see it here.

- Andrew

[Hawin Jiang]

Hi  Andrew

Thanks for your information.  I think /bin/chmod is not working fine from command line.  

Probably, we need to point the full path for /bin/chmod or other ways to change file permissions. 

Yesterday, I also tried to mvn Tachyon in Eclipse. I received 'build success'. 

[Haoyuan Li]

Thanks for the updates, Hawin!

Haoyuan

--
You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Haoyuan Li

CEO, Tachyon Nexus

[Dinglong Li]

I encountered the same problem today. 

I built success but failed most of the tests.
In my ubuntu 14.04 

chmod works fine. But it said "failed to chamod 777" and the common/target/logs/tests.log said No password provided, but it has never asked me for passwd.

However, in my arch linux, it passed all the tests. 

在 2015年11月10日星期二 UTC+8上午2:16:38，and...@tachyonnexus.com写道：

[Andrew Audibert]

Hi Dinglong,

I think the "No password provided" in the logs might be coming from a different test (logs from all tests in the common module get written to common/target/logs/tests.log).O n your ubuntu, are you able to run /bin/chmod 777 from the terminal? The test is creating a temporary file and then running /bin/chmod 777 /path/to/file. Can you post the stacktrace so that we can see which line of the test is failing?

Thanks,

Andrew