tachyon performance test

[Sup Mom]

Hi,

I am trying to run the performance.java test, which works just fine when running a write test case, however running with a read test case gives the following exception
'ERROR : WHAT? 16777216 1'

Can any one please help regarding this?

[Haoyuan Li]

Could you provide a bit more log? e.g. stack trace?

Best,

Haoyuan

--
You received this message because you are subscribed to the Google Groups "Tachyon Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tachyon-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Haoyuan Li

AMPLab, EECS, UC Berkeley

http://www.cs.berkeley.edu/~haoyuan/

[Sup Mom]

Hi Haoyuan,

This is the output that I got:

14/09/10 09:00:16 INFO : TachyonFilesReadTest Threads 1 FilesPerThread 1 TotalFiles 1 BLOCK_SIZE_KB 4 BLOCKS_PER_FILE 512 FILE_SIZE_MB 2 Tachyon_WRITE_BUFFER_SIZE_KB 1024 BaseFileNumber 0 :
14/09/10 09:00:16 INFO : Verifying the reading data...
14/09/10 09:00:16 INFO : Trying to connect master @ /192.168.1.37:19998
14/09/10 09:00:16 INFO : User registered at the master r-sw-hulk11.vbench.com/192.168.1.37:19998 got UserId 84
14/09/10 09:00:16 INFO : Trying to get local worker host : r-sw-hulk11.vbench.com
14/09/10 09:00:16 INFO : Connecting local worker @ r-sw-hulk11.vbench.com/192.168.1.37:29998
14/09/10 09:00:16 ERROR : WHAT? 16777216 1
java.lang.RuntimeException: WHAT? 16777216 1
        at tachyon.util.CommonUtils.runtimeException(CommonUtils.java:479)
        at tachyon.examples.Performance$TachyonReadWorker.readPartition(Performance.java:240)
        at tachyon.examples.Performance$TachyonReadWorker.run(Performance.java:291)
Exception in thread "Thread-0" java.lang.RuntimeException: java.lang.RuntimeException: WHAT? 16777216 1
        at tachyon.util.CommonUtils.runtimeException(CommonUtils.java:475)
        at tachyon.examples.Performance$TachyonReadWorker.run(Performance.java:293)
Caused by: java.lang.RuntimeException: WHAT? 16777216 1
        at tachyon.util.CommonUtils.runtimeException(CommonUtils.java:479)
        at tachyon.examples.Performance$TachyonReadWorker.readPartition(Performance.java:240)
        at tachyon.examples.Performance$TachyonReadWorker.run(Performance.java:291)
14/09/10 09:00:16 INFO : 11.0 Mb/sec. TachyonFilesReadTest Threads 1 FilesPerThread 1 TotalFiles 1 BLOCK_SIZE_KB 4 BLOCKS_PER_FILE 512 FILE_SIZE_MB 2 Tachyon_WRITE_BUFFER_SIZE_KB 1024 BaseFileNumber 0 : Entire Read  Took 178 ms. Current System Time: 1410328816826
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[顾荣]

Hey Sup Mom,

Actually, I think this a bug. We have fixed it and it works well with our modification. We have submitted a PR (https://github.com/amplab/tachyon/pull/426) now. You can run again when the PR is merged.

By the way, if you are interested in conducting performance test for Tachyon on your cluster, there is a project called Tachyon-Perf (https://github.com/PasaLab/tachyon-perf) which you can try. It is a general performance test framework for Tachyon and now contains some test suites. People can also add customized test suites into it an easy way.


Best,
Rong

[Pengfei Xuan]

Hi Rong,

Does Tachyon-Perf support cluster mode? Is there any user manual or tutorial for this test suit? Thanks!

Best,

Pengfei  

[顾荣]

Hey Pengfei,

Yeah, Tachyon-Perf supports multi-node multi-thread mode. Currently, we have put the mannual on the ReadMe of the project https://github.com/PasaLab/tachyon-perf .
The details of each Test suit is also described on their wiki page respectively https://github.com/PasaLab/tachyon-perf/wiki/Benchmarks .

Best,
Rong

[Pengfei Xuan]

Cool Stuff! I'll definitely give a try. 

BTW, is it possible for you to add random read and write features on current test suits? Something like the options introduced in TestDFSIO. 

Usage: TestDFSIO [genericOptions] -read [-random | -backward | -skip [-skipSize Size]] | -write | -append | -clean [-compression codecClassName] [-nrFiles N] [-size Size[B|KB|MB|GB|TB]] [-resFile resultFileName] [-bufferSize Bytes]  

I actually integrated TestDFSIO atop of LocalTachyonCluster recently. But it's only used for unit tests, rather than performance benchmark.

https://github.com/amplab/tachyon/pull/391

[顾荣]

Hey Pengfei,

We have support random/sequential read, it can be configured by the mode parameter here https://github.com/PasaLab/tachyon-perf/wiki/The-Read-Test .
BTW, we did a survery of performance systems,including TestDFSIO, before we started to build Tachyon-Perf. The survey is attached in this email.

Best,
Rong

[Pengfei Xuan]

How about skip read? For example, 

Tachyon-Perf -read -skip -skipSize 1MB -bufferSize 4MB

Thanks!