Unable to get parseFrom as an available method from generated proto

26 views
Skip to first unread message

Mayank Jha

unread,
Mar 6, 2019, 6:36:11 AM3/6/19
to ScalaPB
My build.sbt is 


PB.targets in Compile := Seq(

  PB.gens.java -> (sourceManaged in Compile).value,

  scalapb.gen(javaConversions = true) -> (sourceManaged in Compile).value

)

libraryDependencies += "com.thesamet.scalapb" %% "scalapb-runtime" % scalapb.compiler.Version.scalapbVersion % "protobuf"



My scalapb.sbt is this.


addSbtPlugin("com.thesamet" % "sbt-protoc" % "0.99.19")


libraryDependencies += "com.thesamet.scalapb" %% "compilerplugin" % "0.8.1"

Nadav Samet

unread,
Mar 6, 2019, 12:13:50 PM3/6/19
to Mayank Jha, ScalaPB
You seem to have everything set up correctly. Maybe you are looking for a parseFrom method at the wrong place? Can you provide the protos you have and code that you expect to compile?

--
You received this message because you are subscribed to the Google Groups "ScalaPB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalapb+u...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalapb/5410f109-24a5-4e64-93a2-f78f366b0f7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
-Nadav

Mayank Jha

unread,
Mar 6, 2019, 8:27:52 PM3/6/19
to ScalaPB
syntax = "proto3";
package cats_proto;
option java_package = "org.carousell.protos";
message EventMessage {
    enum EventType {
        UNKNOWN = 0;
        IMPRESSION = 1;
        CLICK = 2;
        VIEWABLE = 3;
    }
    EventType event_type = 2;
    string tracking_id = 3;
    string user_id = 4;
    string promotion_id = 5;
The above is the proto I am trying to compile.

Mayank Jha

unread,
Mar 6, 2019, 8:40:56 PM3/6/19
to ScalaPB
The below is the generated proto 

// Generated by the Scala Plugin for the Protocol Buffer Compiler.
// Do not edit!
//
// Protofile syntax: PROTO3


package org.carousell.protos.cats_analytics


@SerialVersionUID(0L)
final case class EventMessage(
    eventType
: org.carousell.protos.cats_analytics.EventMessage.EventType = org.carousell.protos.cats_analytics.EventMessage.EventType.UNKNOWN,
    trackingId
: _root_.scala.Predef.String = "",
    userId
: _root_.scala.Predef.String = "",
    promotionId
: _root_.scala.Predef.String = ""
   
) extends scalapb.GeneratedMessage with scalapb.Message[EventMessage] with scalapb.lenses.Updatable[EventMessage] {
   
@transient
   
private[this] var __serializedSizeCachedValue: _root_.scala.Int = 0
   
private[this] def __computeSerializedValue(): _root_.scala.Int = {
     
var __size = 0
     
     
{
        val __value
= eventType
       
if (__value != org.carousell.protos.cats_analytics.EventMessage.EventType.UNKNOWN) {
          __size
+= _root_.com.google.protobuf.CodedOutputStream.computeEnumSize(2, __value.value)
       
}
     
};
     
     
{
        val __value
= trackingId
       
if (__value != "") {
          __size
+= _root_.com.google.protobuf.CodedOutputStream.computeStringSize(3, __value)
       
}
     
};
     
     
{
        val __value
= userId
       
if (__value != "") {
          __size
+= _root_.com.google.protobuf.CodedOutputStream.computeStringSize(4, __value)
       
}
     
};
     
     
{
        val __value
= promotionId
       
if (__value != "") {
          __size
+= _root_.com.google.protobuf.CodedOutputStream.computeStringSize(5, __value)
       
}
     
};
      __size
   
}
   
final override def serializedSize: _root_.scala.Int = {
     
var read = __serializedSizeCachedValue
     
if (read == 0) {
        read
= __computeSerializedValue()
        __serializedSizeCachedValue
= read
     
}
      read
   
}
   
def writeTo(`_output__`: _root_.com.google.protobuf.CodedOutputStream): _root_.scala.Unit = {
     
{
        val __v
= eventType
       
if (__v != org.carousell.protos.cats_analytics.EventMessage.EventType.UNKNOWN) {
          _output__
.writeEnum(2, __v.value)
       
}
     
};
     
{
        val __v
= trackingId
       
if (__v != "") {
          _output__
.writeString(3, __v)
       
}
     
};
     
{
        val __v
= userId
       
if (__v != "") {
          _output__
.writeString(4, __v)
       
}
     
};
     
{
        val __v
= promotionId
       
if (__v != "") {
          _output__
.writeString(5, __v)
       
}
     
};
   
}
   
def mergeFrom(`_input__`: _root_.com.google.protobuf.CodedInputStream): org.carousell.protos.cats_analytics.EventMessage = {
     
var __eventType = this.eventType
     
var __trackingId = this.trackingId
     
var __userId = this.userId
     
var __promotionId = this.promotionId
     
var _done__ = false
     
while (!_done__) {
        val _tag__
= _input__.readTag()
        _tag__ match
{
         
case 0 => _done__ = true
         
case 16 =>
            __eventType
= org.carousell.protos.cats_analytics.EventMessage.EventType.fromValue(_input__.readEnum())
         
case 26 =>
            __trackingId
= _input__.readString()
         
case 34 =>
            __userId
= _input__.readString()
         
case 42 =>
            __promotionId
= _input__.readString()
         
case tag => _input__.skipField(tag)
       
}
     
}
      org
.carousell.protos.cats_analytics.EventMessage(
          eventType
= __eventType,
          trackingId
= __trackingId,
          userId
= __userId,
          promotionId
= __promotionId
     
)
   
}
   
def withEventType(__v: org.carousell.protos.cats_analytics.EventMessage.EventType): EventMessage = copy(eventType = __v)
   
def withTrackingId(__v: _root_.scala.Predef.String): EventMessage = copy(trackingId = __v)
   
def withUserId(__v: _root_.scala.Predef.String): EventMessage = copy(userId = __v)
   
def withPromotionId(__v: _root_.scala.Predef.String): EventMessage = copy(promotionId = __v)
   
def getFieldByNumber(__fieldNumber: _root_.scala.Int): _root_.scala.Any = {
     
(__fieldNumber: @_root_.scala.unchecked) match {
       
case 2 => {
          val __t
= eventType.javaValueDescriptor
         
if (__t.getNumber() != 0) __t else null
       
}
       
case 3 => {
          val __t
= trackingId
         
if (__t != "") __t else null
       
}
       
case 4 => {
          val __t
= userId
         
if (__t != "") __t else null
       
}
       
case 5 => {
          val __t
= promotionId
         
if (__t != "") __t else null
       
}
     
}
   
}
   
def getField(__field: _root_.scalapb.descriptors.FieldDescriptor): _root_.scalapb.descriptors.PValue = {
      _root_
.scala.Predef.require(__field.containingMessage eq companion.scalaDescriptor)
     
(__field.number: @_root_.scala.unchecked) match {
       
case 2 => _root_.scalapb.descriptors.PEnum(eventType.scalaValueDescriptor)
       
case 3 => _root_.scalapb.descriptors.PString(trackingId)
       
case 4 => _root_.scalapb.descriptors.PString(userId)
       
case 5 => _root_.scalapb.descriptors.PString(promotionId)
     
}
   
}
   
def toProtoString: _root_.scala.Predef.String = _root_.scalapb.TextFormat.printToUnicodeString(this)
   
def companion = org.carousell.protos.cats_analytics.EventMessage
}


object EventMessage extends scalapb.GeneratedMessageCompanion[org.carousell.protos.cats_analytics.EventMessage] {
 
implicit def messageCompanion: scalapb.GeneratedMessageCompanion[org.carousell.protos.cats_analytics.EventMessage] = this
 
def fromFieldsMap(__fieldsMap: scala.collection.immutable.Map[_root_.com.google.protobuf.Descriptors.FieldDescriptor, _root_.scala.Any]): org.carousell.protos.cats_analytics.EventMessage = {
    _root_
.scala.Predef.require(__fieldsMap.keys.forall(_.getContainingType() == javaDescriptor), "FieldDescriptor does not match message type.")
    val __fields
= javaDescriptor.getFields
    org
.carousell.protos.cats_analytics.EventMessage(
      org
.carousell.protos.cats_analytics.EventMessage.EventType.fromValue(__fieldsMap.getOrElse(__fields.get(0), org.carousell.protos.cats_analytics.EventMessage.EventType.UNKNOWN.javaValueDescriptor).asInstanceOf[_root_.com.google.protobuf.Descriptors.EnumValueDescriptor].getNumber),
      __fieldsMap
.getOrElse(__fields.get(1), "").asInstanceOf[_root_.scala.Predef.String],
      __fieldsMap
.getOrElse(__fields.get(2), "").asInstanceOf[_root_.scala.Predef.String],
      __fieldsMap
.getOrElse(__fields.get(3), "").asInstanceOf[_root_.scala.Predef.String]
   
)
 
}
 
implicit def messageReads: _root_.scalapb.descriptors.Reads[org.carousell.protos.cats_analytics.EventMessage] = _root_.scalapb.descriptors.Reads{
   
case _root_.scalapb.descriptors.PMessage(__fieldsMap) =>
      _root_
.scala.Predef.require(__fieldsMap.keys.forall(_.containingMessage == scalaDescriptor), "FieldDescriptor does not match message type.")
      org
.carousell.protos.cats_analytics.EventMessage(
        org
.carousell.protos.cats_analytics.EventMessage.EventType.fromValue(__fieldsMap.get(scalaDescriptor.findFieldByNumber(2).get).map(_.as[_root_.scalapb.descriptors.EnumValueDescriptor]).getOrElse(org.carousell.protos.cats_analytics.EventMessage.EventType.UNKNOWN.scalaValueDescriptor).number),
        __fieldsMap
.get(scalaDescriptor.findFieldByNumber(3).get).map(_.as[_root_.scala.Predef.String]).getOrElse(""),
        __fieldsMap
.get(scalaDescriptor.findFieldByNumber(4).get).map(_.as[_root_.scala.Predef.String]).getOrElse(""),
        __fieldsMap
.get(scalaDescriptor.findFieldByNumber(5).get).map(_.as[_root_.scala.Predef.String]).getOrElse("")
     
)
   
case _ => throw new RuntimeException("Expected PMessage")
 
}
 
def javaDescriptor: _root_.com.google.protobuf.Descriptors.Descriptor = CatsAnalyticsProto.javaDescriptor.getMessageTypes.get(0)
 
def scalaDescriptor: _root_.scalapb.descriptors.Descriptor = CatsAnalyticsProto.scalaDescriptor.messages(0)
 
def messageCompanionForFieldNumber(__number: _root_.scala.Int): _root_.scalapb.GeneratedMessageCompanion[_] = throw new MatchError(__number)
  lazy val nestedMessagesCompanions
: Seq[_root_.scalapb.GeneratedMessageCompanion[_]] = Seq.empty
 
def enumCompanionForFieldNumber(__fieldNumber: _root_.scala.Int): _root_.scalapb.GeneratedEnumCompanion[_] = {
   
(__fieldNumber: @_root_.scala.unchecked) match {
     
case 2 => org.carousell.protos.cats_analytics.EventMessage.EventType
   
}
 
}
  lazy val defaultInstance
= org.carousell.protos.cats_analytics.EventMessage(
 
)
 
sealed trait EventType extends _root_.scalapb.GeneratedEnum {
    type
EnumType = EventType
   
def isUnknown: _root_.scala.Boolean = false
   
def isImpression: _root_.scala.Boolean = false
   
def isClick: _root_.scala.Boolean = false
   
def isViewable: _root_.scala.Boolean = false
   
def companion: _root_.scalapb.GeneratedEnumCompanion[EventType] = org.carousell.protos.cats_analytics.EventMessage.EventType
 
}
 
 
object EventType extends _root_.scalapb.GeneratedEnumCompanion[EventType] {
   
implicit def enumCompanion: _root_.scalapb.GeneratedEnumCompanion[EventType] = this
   
@SerialVersionUID(0L)
   
case object UNKNOWN extends EventType {
      val value
= 0
      val index
= 0
      val name
= "UNKNOWN"
     
override def isUnknown: _root_.scala.Boolean = true
   
}
   
   
@SerialVersionUID(0L)
   
case object IMPRESSION extends EventType {
      val value
= 1
      val index
= 1
      val name
= "IMPRESSION"
     
override def isImpression: _root_.scala.Boolean = true
   
}
   
   
@SerialVersionUID(0L)
   
case object CLICK extends EventType {
      val value
= 2
      val index
= 2
      val name
= "CLICK"
     
override def isClick: _root_.scala.Boolean = true
   
}
   
   
@SerialVersionUID(0L)
   
case object VIEWABLE extends EventType {
      val value
= 3
      val index
= 3
      val name
= "VIEWABLE"
     
override def isViewable: _root_.scala.Boolean = true
   
}
   
   
@SerialVersionUID(0L)
   
final case class Unrecognized(value: _root_.scala.Int) extends EventType with _root_.scalapb.UnrecognizedEnum
   
    lazy val values
= scala.collection.Seq(UNKNOWN, IMPRESSION, CLICK, VIEWABLE)
   
def fromValue(value: _root_.scala.Int): EventType = value match {
     
case 0 => UNKNOWN
     
case 1 => IMPRESSION
     
case 2 => CLICK
     
case 3 => VIEWABLE
     
case __other => Unrecognized(__other)
   
}
   
def javaDescriptor: _root_.com.google.protobuf.Descriptors.EnumDescriptor = org.carousell.protos.cats_analytics.EventMessage.javaDescriptor.getEnumTypes.get(0)
   
def scalaDescriptor: _root_.scalapb.descriptors.EnumDescriptor = org.carousell.protos.cats_analytics.EventMessage.scalaDescriptor.enums(0)
 
}
 
implicit class EventMessageLens[UpperPB](_l: _root_.scalapb.lenses.Lens[UpperPB, org.carousell.protos.cats_analytics.EventMessage]) extends _root_.scalapb.lenses.ObjectLens[UpperPB, org.carousell.protos.cats_analytics.EventMessage](_l) {
   
def eventType: _root_.scalapb.lenses.Lens[UpperPB, org.carousell.protos.cats_analytics.EventMessage.EventType] = field(_.eventType)((c_, f_) => c_.copy(eventType = f_))
   
def trackingId: _root_.scalapb.lenses.Lens[UpperPB, _root_.scala.Predef.String] = field(_.trackingId)((c_, f_) => c_.copy(trackingId = f_))
   
def userId: _root_.scalapb.lenses.Lens[UpperPB, _root_.scala.Predef.String] = field(_.userId)((c_, f_) => c_.copy(userId = f_))
   
def promotionId: _root_.scalapb.lenses.Lens[UpperPB, _root_.scala.Predef.String] = field(_.promotionId)((c_, f_) => c_.copy(promotionId = f_))
 
}
 
final val EVENT_TYPE_FIELD_NUMBER = 2
 
final val TRACKING_ID_FIELD_NUMBER = 3
 
final val USER_ID_FIELD_NUMBER = 4
 
final val PROMOTION_ID_FIELD_NUMBER = 5
}


Nadav Samet

unread,
Mar 6, 2019, 10:14:27 PM3/6/19
to Mayank Jha, ScalaPB
Ok, see the EventMessage companion object? It inherits from GeneratedMessageCompanion which provides the parseFrom method you are looking for.


For more options, visit https://groups.google.com/d/optout.


--
-Nadav

Mayank Jha

unread,
Mar 6, 2019, 10:22:34 PM3/6/19
to ScalaPB
When I try to access companion, it says it is not a member of the message. Could there be a possible problem with protoc bridge version ?

Mayank Jha

unread,
Mar 6, 2019, 10:25:50 PM3/6/19
to ScalaPB
I tried this, 
      val em = new EventMessage()

 

      em
.companion.parseFrom()


but it doesn't seem to work

Nadav Samet

unread,
Mar 6, 2019, 10:30:36 PM3/6/19
to Mayank Jha, ScalaPB
In Scala, a method on the companion object is like a static method in Java. You call it like this: EventMessage.parseFrom(bytes), and it would return an instance of an EventMessage.


For more options, visit https://groups.google.com/d/optout.


--
-Nadav

Mayank Jha

unread,
Mar 6, 2019, 10:47:03 PM3/6/19
to ScalaPB
EventMessage.parseFrom(bytes)

does not work for me.

Mayank Jha

unread,
Mar 6, 2019, 10:48:50 PM3/6/19
to ScalaPB
Should I use the java pb files instead ?

   
def withUserId(__v: _root_.scala.Predef.String): EventMessage =<span style="color:rgb(0

Nadav Samet

unread,
Mar 6, 2019, 10:54:05 PM3/6/19
to Mayank Jha, ScalaPB
Can you be more specific how "EventMessage.parseFrom" does not work for you? Is there an error message you are seeing? Are you sure you are importing the Scala companion object and not the Java one?

If you can share a minimal example project you have difficulties it would be easier to help further. A zip file attachment, or a project on github would do.

--
You received this message because you are subscribed to the Google Groups "ScalaPB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalapb+u...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.


--
-Nadav

Mayank Jha

unread,
Mar 6, 2019, 11:10:10 PM3/6/19
to ScalaPB
I get the error value parseFrom is not member of the object EventMessage 

Nadav Samet

unread,
Mar 6, 2019, 11:13:25 PM3/6/19
to Mayank Jha, ScalaPB
Very strange. Most likely explanation is that your code is referring to some other EventMessage, not the one generated by ScalaPB. Can you share a minimal project so I can easily reproduce it locally? 


For more options, visit https://groups.google.com/d/optout.


--
-Nadav

Mayank Jha

unread,
Mar 7, 2019, 1:51:52 AM3/7/19
to ScalaPB
My bad, sorry. I had missed out on giving the complete build.sbt 


libraryDependencies ++= Seq(

 

 
"org.scala-lang" % "scala-library" % scalaVersion.value % "provided",

 
"org.apache.spark" %% "spark-core" % "2.3.3" % "provided",

 
"org.apache.spark" %% "spark-streaming" % "2.3.3" % "provided",

 
"org.apache.spark" %% "spark-streaming-kafka-0-10" % "2.3.3",

 
"org.apache.kafka" %% "kafka" % "0.10.0.0",

 
"org.apache.spark" %% "spark-streaming-kafka-0-10-assembly" % "2.3.3"  

)


 

libraryDependencies
+= "org.scalatest" %% "scalatest" % "3.0.1" % "test"


 

// test suite settings

fork
in Test := true

javaOptions
++= Seq("-Xms512M", "-Xmx2048M", "-XX:MaxPermSize=2048M", "-XX:+CMSClassUnloadingEnabled")

// Show runtime of tests

testOptions
in Test += Tests.Argument(TestFrameworks.ScalaTest, "-oD")


 

// JAR file settings


 

// don't include Scala in the JAR file

//assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = false)


 

// Add the JAR file naming conventions described here: https://github.com/MrPowers/spark-style-guide#jar-files

// You can add the JAR file naming conventions by running the shell script

assemblyMergeStrategy
in assembly := {

 
case m if m.toLowerCase.endsWith("manifest.mf")          => MergeStrategy.discard

 
case m if m.toLowerCase.matches("meta-inf.*\\.sf$")      => MergeStrategy.discard

 
case "log4j.properties"                                  => MergeStrategy.discard

 
case m if m.toLowerCase.startsWith("meta-inf/services/") => MergeStrategy.filterDistinctLines

 
case "reference.conf"                                    => MergeStrategy.concat

 
case _                                                   => MergeStrategy.first

}


So, what I did was I removed the above section, and then did a compilation it started working, and then I put it back, and recompiled, it still works. I am really puzzled how it fixed now :'( 

Mayank Jha

unread,
Mar 7, 2019, 2:23:04 AM3/7/19
to ScalaPB
here is the minimised version of the code.


For more options, visit https://groups.google.com/d/optout.


--

Nadav Samet

unread,
Mar 7, 2019, 2:36:13 AM3/7/19
to Mayank Jha, ScalaPB
The code you does not compile since you are calling parseFrom() without passing it a byte array or an input stream:

[error] /tmp/foo/spa/spa/src/main/scala/org/carousell/nvr/CatsMetrics.scala:7: overloaded method value parseFrom with alternatives:
[error]   (s: Array[Byte])org.carousell.protos.cats_analytics.EventMessage <and>
[error]   (input: java.io.InputStream)org.carousell.protos.cats_analytics.EventMessage <and>
[error]   (input: com.google.protobuf.CodedInputStream)org.carousell.protos.cats_analytics.EventMessage
[error]  cannot be applied to ()
[error]      EventMessage.parseFrom()
[error]                   ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed



For more options, visit https://groups.google.com/d/optout.


--
-Nadav
Reply all
Reply to author
Forward
0 new messages