Scalapb generates class file that exceeds JVM code size limits

David Greenberg

unread,

Jul 5, 2019, 2:47:56 PM7/5/19

to ScalaPB

Hello, I am trying to generate a protobuf which has 200 fields, all of which are repeated doubles. However, scalapb isn't able to write this, because the class file is too large.

Is there a way to reduce the code size with flags or params to ScalaPB, or do I just need to split this message into multiple messages? Is there anything I could implement to make the codegen more efficient for this less common use case?

Thank you,

David

Nadav Samet

unread,

Jul 5, 2019, 4:48:25 PM7/5/19

to David Greenberg, ScalaPB

Hi David,

It looks like for each field we generate `withX` and `addX` which calls the case class's `copy()`. Scala's compiler expands the copy() calls to something that is proportional in size to the number of fields. I am not seeing any method that is larger than 64k though so I am not sure what precisely is triggering this error.

However, we can add a flag that makes ScalaPB not generate the various withX, addX, clearX methods - feel free to file an issue if that helps. In any case, I'd advise splitting this large message into multiple messages.

--
You received this message because you are subscribed to the Google Groups "ScalaPB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalapb+u...@googlegroups.com.
To post to this group, send email to sca...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalapb/34c755ae-f18d-474b-8727-d9324a341729%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

-Nadav

Nadav Samet

unread,

Jul 5, 2019, 5:02:59 PM7/5/19

to David Greenberg, ScalaPB

Actually, which version of Scala are you using? I can reproduce this the problem you are seeing in 2.12.4, but not in 2.12.8 and 2.13.0.

--

-Nadav

David Greenberg

unread,

Jul 5, 2019, 5:14:12 PM7/5/19

to ScalaPB

I am using Scala 2.12.5. I've written up the stack trace & an example file in this issue here: https://github.com/scalapb/ScalaPB/issues/613

I'll be happy to close if we can figure out what the root is; hopefully the exact proto file (modulo censored field names) is helpful.

On Friday, July 5, 2019 at 5:02:59 PM UTC-4, Nadav Samet wrote:

Actually, which version of Scala are you using? I can reproduce this the problem you are seeing in 2.12.4, but not in 2.12.8 and 2.13.0.

On Fri, Jul 5, 2019 at 1:47 PM Nadav Samet <thes...@gmail.com> wrote:

Hi David,

It looks like for each field we generate `withX` and `addX` which calls the case class's `copy()`. Scala's compiler expands the copy() calls to something that is proportional in size to the number of fields. I am not seeing any method that is larger than 64k though so I am not sure what precisely is triggering this error.

However, we can add a flag that makes ScalaPB not generate the various withX, addX, clearX methods - feel free to file an issue if that helps. In any case, I'd advise splitting this large message into multiple messages.

On Fri, Jul 5, 2019 at 11:47 AM David Greenberg <dsg123...@gmail.com> wrote:

Hello, I am trying to generate a protobuf which has 200 fields, all of which are repeated doubles. However, scalapb isn't able to write this, because the class file is too large.

Is there a way to reduce the code size with flags or params to ScalaPB, or do I just need to split this message into multiple messages? Is there anything I could implement to make the codegen more efficient for this less common use case?

Thank you,
David

--
You received this message because you are subscribed to the Google Groups "ScalaPB" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sca...@googlegroups.com.

To post to this group, send email to sca...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalapb/34c755ae-f18d-474b-8727-d9324a341729%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
-Nadav

--
-Nadav

Nadav Samet

unread,

Jul 5, 2019, 5:23:58 PM7/5/19

to David Greenberg, ScalaPB

Can you upgrade to Scala 2.12.8 and report if it works without any change there? It might be worth to implement this flag anyway to reduce class size, but at least you'll not be blocked on this.

To unsubscribe from this group and stop receiving emails from it, send an email to scalapb+u...@googlegroups.com.

To post to this group, send email to sca...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scalapb/b6fbcf1a-5da8-4442-b10e-396643088f4c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-Nadav

David Greenberg

unread,

Jul 5, 2019, 5:43:16 PM7/5/19

to ScalaPB

Upgrading to scala 2.12.8 indeed fixes this issue. I'm going to close the github issue, because I don't think that's something I would ever want, except to get my builds to work.

Thank you for your awesome work on ScalaPB!

To view this discussion on the web visit https://groups.google.com/d/msgid/scalapb/b6fbcf1a-5da8-4442-b10e-396643088f4c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
-Nadav

Reply all

Reply to author

Forward