Error while running Camus on maprfs

[Bipp Tej]

Hi,

    I am trying to run Camus job on maprfs with Schema Registry URL. Below is my Command 

./bin/camus-run -D schema.registry.url=http://0.0.0.0:8081 -P etc/camus/camus.properties

I have been able to connect to Schema Registry using the command "curl -X GET -i http://0.0.0.0:8081/subjects" and see that my topic is registered there. When i run the job, i am getting ConnectionRefused error .I am not sure what is happening. I have read in this group that another person got the same error but that was because of not providing the schema url.

Below is the stack trace: 

java.io.IOException: com.linkedin.camus.coders.MessageDecoderException: Error deserializing Avro message

at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.getWrappedRecord(EtlRecordReader.java:125)

at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.nextKeyValue(EtlRecordReader.java:265)

at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:565)

at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)

at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:796)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)

at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

Caused by: com.linkedin.camus.coders.MessageDecoderException: Error deserializing Avro message

at io.confluent.camus.etl.kafka.coders.AvroMessageDecoder.deserialize(AvroMessageDecoder.java:151)

at io.confluent.camus.etl.kafka.coders.AvroMessageDecoder.decode(AvroMessageDecoder.java:158)

at io.confluent.camus.etl.kafka.coders.AvroMessageDecoder.decode(AvroMessageDecoder.java:41)

at com.linkedin.camus.etl.kafka.mapred.EtlRecordReader.getWrappedRecord(EtlRecordReader.java:122)

... 12 more

Caused by: java.net.ConnectException: Connection refused

at java.net.PlainSocketImpl.socketConnect(Native Method)

at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

at java.net.Socket.connect(Socket.java:589)

at java.net.Socket.connect(Socket.java:538)

at sun.net.NetworkClient.doConnect(NetworkClient.java:180)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)

at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)

at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)

at sun.net.www.http.HttpClient.New(HttpClient.java:308)

at sun.net.www.http.HttpClient.New(HttpClient.java:326)

at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1168)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1104)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:998)

at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:932)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1512)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)

at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)

at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:147)

at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:181)

at io.confluent.kafka.schemaregistry.client.rest.RestService.getId(RestService.java:317)

at io.confluent.kafka.schemaregistry.client.rest.RestService.getId(RestService.java:310)

at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getSchemaByIdFromRegistry(CachedSchemaRegistryClient.java:61)

at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getByID(CachedSchemaRegistryClient.java:100)

at io.confluent.camus.etl.kafka.coders.AvroMessageDecoder.deserialize(AvroMessageDecoder.java:110)

... 15 more

   Can some one help me out with this issue

[Bipp Tej]

Figured out the issue. I had to use public IP address instead of localhost (0.0.0.0) in this case to connect to schema registry