Google Groups

'native-lzo library not available' issue

unionx Jul 23, 2012 7:37 PM
Posted in group: elephantbird-dev
I have installed and configured lzo on my cluseter. I can use a script to compress my data on HDFS now.

I have configured codecs in core-site.xml and mapred-site.xml, and I have added JAVA_LIBRARY_PATH and HADOOP_CLASSPATH in

But there are some error. When I try to use Pig or Java to load lzo files, there is 'native-lzo library not available' error:

2012-07-24 10:22:13,113 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2012-07-24 10:22:13,122 [main] ERROR - ERROR 2999: Unexpected internal error. native-lzo library not available
2012-07-24 10:22:13,122 [main] ERROR - java.lang.RuntimeException: native-lzo library not available
    at com.hadoop.compression.lzo.LzopCodec.createDecompressor(
    at com.hadoop.compression.lzo.LzopCodec.createInputStream(
    at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(
    at org.apache.hadoop.mapred.MapTask.runNewMapper(

But when the job is intializating, it can intialize lzo native library successfully:

2012-07-24 10:18:31,016 [Thread-12] INFO  com.hadoop.compression.lzo.GPLNativeCodeLoader - Loaded native gpl library
2012-07-24 10:18:31,021 [Thread-12] INFO  com.hadoop.compression.lzo.LzoCodec - Successfully loaded & initialized native-lzo library [hadoop-lzo rev 6bb1b7f8b9044d8df9b4d2b6641db7658aab3cf8]

Is there something I missed?

‘(hello world)