proto3, com.google.protobuf.CodedInputStream.readStringRequireUtf8()Ljava/lang/String;

377 views
Skip to first unread message

Thinkhard.J Park

unread,
Dec 17, 2016, 8:51:53 PM12/17/16
to Protocol Buffers
Hi, 

I'm using protocol Buffer(proto3) from my latest project. But I encounter below problem.
An encoded message generated from python and I'm trying to decode it from scala.(I'm using SPARK.)
You can see the actual message from "value" and its length 227 as described in console.
The error field just typical string type field. I also have tried to decode string field as "UTF-8" but it is not worked. 
It looks encode/decode problem. Are there some other guys who encounter similar problems? Would you please help me?


Console output
...
value: 
U
$a5a9c2bb-efd2-4ea3-ae0e-7479925a7807"kafka_producer.py*2016-12-18 10:37:25.075614*
data{"url": "some url 0", "title": "some title", "_id": "5815a37c43cfd44120e50538", "content": "some content", "tags": ["football"]}
value length: 227
Exception in thread "streaming-job-executor-0" java.lang.NoSuchMethodError: com.google.protobuf.CodedInputStream.readStringRequireUtf8()Ljava/lang/String;
at org.libero.messages.Messages$Event.<init>(Messages.java:160)
at org.libero.messages.Messages$Event.<init>(Messages.java:117)
at org.libero.messages.Messages$Event$1.parsePartialFrom(Messages.java:1564)
at org.libero.messages.Messages$Event$1.parsePartialFrom(Messages.java:1559)
at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
at org.libero.messages.Messages$EventDataProduced.<init>(Messages.java:1742)
at org.libero.messages.Messages$EventDataProduced.<init>(Messages.java:1697)
at org.libero.messages.Messages$EventDataProduced$1.parsePartialFrom(Me
...

Code fragments.
....
 nonEmptyMsgs.foreachRDD { msgsRDD =>
      println("Trying to print each RDD: " + msgsRDD)
      println("Count: " + msgsRDD.count())
      val elems = msgsRDD.collect()
      println("Elems: " + elems)
      for (v <- elems) {
        println("key: " + v._1)
        println("value: " + v._2)
        println("value length: " + v._2.length())
        val bytes = new String(v._2).getBytes()
        val event = EventDataProduced.parseFrom(ByteString.copyFrom(bytes))
        println("event: " + event)
      }
    }
...



Reply all
Reply to author
Forward
0 new messages