optimizer generating incorrect bytecode (SI-9403)

[Jeff Olson]

I opened a bug report about this a few days ago, but I thought I would mention it here in case any interested parties didn't see it.

I tried upgrading a large scala 2.10 project to 2.11.7 last week and immediately ran into issues with failing test cases. These test cases all involve a value class (wrapping a Long) with many methods marked @inline. I was able to isolate the cause of one of the failing tests cases and filed the above bug report. I can easily tweak the code to make things work for that case, but I still have a dozen other failing tests. These tests all pass if I leave off -optimize. It's extremely difficult to isolate these bugs, and since it is likely they all have the same root cause, I don't want to spend a lot of time tracking them down until the above issue is understood and fixed.

This is a show-stopper bug for me. I can't upgrade my project to 2.11 until it is fixed.

Thanks,

Jeff

[Lukas Rytz]

I will take a look at the issue this or next week. 2.11.8 is planned for the end of September.

Thanks for the report and for minimizing the issue!

Lukas

--
You received this message because you are subscribed to the Google Groups "scala-internals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-interna...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jeff Olson]

Thanks Lukas! Once you have a fix let me know and I can run through the rest of my test suite to see if it fixes my other issues, or if there are remaining problems. It would be great to have everything fixed in 2.11.8.

-Jeff

[Jeff Olson]

Well, Lukas, found and fixed the above problem in quick order (Thanks!!). Sadly, it didn't fix all of my test failures. Using the latest nightly build, I spent another three hours this afternoon tracking down the next optimizer failure: SI-9422.

With such simple code failures, I'm beginning to wonder: does anyone actually use the optimizer? It seems like these kind of bugs should have surfaced long before now.

In particular, is the scala library itself compiled with -optimise?

-Jeff  

[Rex Kerr]

It is, but the library's usage of code patterns has characteristics that are not always representative of all usage patterns.  And most people don't bother, because in practice it has, especially historically, only rarely helped.

I am still surprised to see such a simple error.

  --Rex

[Jason Zaugg]

Thanks for trying out the fix and for finding the next bug.

Yes, we do compile the library with -optimise. We also run all of the functional tests for the compiler with the optimizer during nightly builds and PR validation.

But in spite of this, being a non-default option, the optimizer gets far less real world testing than the rest of the compiler and isn’t of the quality that we’d like. We are in the process of replacing the optimizer in Scala 2.12 and have invested a bit more in unit testing and in integration testing (by enabling the optimizer in the builds of all the open source projects we track in our community build, so we hope to have a marked improvement in quality.

Regarding this bug, I’ve found the source of the regression and a workaround and noted these on the ticket.

​

-jason

[Som Snytt]

It might be time to rename it -optimistic.

In other news, they've discovered how OSS projects succeed:

http://www.pbs.org/newshour/rundown/ants-move-really-big-stuff/

It turns out that the newcomer ants know which direction to move in.

Apparently, the ants who have been at it awhile are disillusioned, depressed and disoriented. But the new ant says, Hey, this way to the nest! for about a second until they, too, grow despondent over the immensity of the task and the distance to their goal.

They also explained why good people leave the project after a year or two when they realize their efforts are no longer effective:

<<Their “captain ant” hypothesis went out the window, too. For most of the 98 journeys that the team recorded, they did spot one ant that clung to the object for the whole journey, but its orientation rarely matched the direction of team’s movement.>>

That's such a coincidence, because with the release of "Ant Man", I'd come this close to nominating paulp as "Captain Ant", except that "Captain Sbt" is closer to the mark.

By another remarkable coincidence, the researchers discovered, using tiny microphones mounted on the backs of the insects, that "Sbt" is what the ants call themselves in their ant language. A new ant arrives at the project and announces, Yo, sbt!

So it's absolutely true after all that Sbt is not an initialism, but is pronounced more like Spit or Zbit. Spit is how ants transfer the chemical signal that is their lingua franca. It puts an entirely new spin on the nerd practice of spitting in beer.

[Som Snytt]

This is on a more technical level, but I forgot to mention that

> How do ants synchronize to move really big stuff?

they don't synchronize but use a non-blocking Future.

Sometimes there ensues a rather vicious struggle among the worker ants whether to use Actors.

[Jeff Olson]

Thanks for the info guys. It's good to know that the optimizer is getting exercised somewhat. And thanks for the very quick bug fix! I appreciate the hard work. I look forward to 2.11.8.

I ran through the rest of my test suite with the latest nightly and -Yconst-opt:false, and all my tests now pass so I shouldn't have any more nasty surprises for you. I'll check again with tomorrow's nightly build and the constant optimization re-enabled to see that Jason's patch did indeed fix the problem.

It might be time to rename it -optimistic.

Only if you add a -pessimistic flag for those of us disillusioned, depressed, and disoriented old ants who've be at this far too long.

-Jeff