Scala 2.9.x's problems with the DalvikVM

Showing 1-61 of 61 messages
Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 9/13/11 1:39 PM
Hi all,

while I had a somewhat workable solution to create some simple Android
apps with Scala, I've now come to the point (as well) where Dalvik
fails either while dexopt'ing a proguarded Scala program or later at
runtime when it tries to load a class (basically reported in [1]). I'm
starting this thread to gather some more information to find out how
this problem can be solved by Scala, Android, or at all.

The basic problem lies in in dalvik's custom memory allocator,
LinearAlloc. Without going into details, it seems like that allocator
is used similar to the PermGen in HotSpot to allocate class data, like
vtables etc. The problem here is that this allocation region is
statically fixed to 5 MB [2] per classloader.

For a program that fails consider the single line from SI-4620:

   class ObservableTest extends ArrayBuffer[Int]

Even if proguarded, ArrayBuffer alone has such a trail of dependencies
that those 5MB are not enough.

I patched dexopt to log for what purpose and what class it is
allocating memory, here's the output [3].

(The line 'D/dalvikvm(14166): CLASS: linking 'Ljava/lang/Class;'...'
always comes in between the corresponding entries for the class)

As you can see at the end there are classes which need big amounts of
memory, I think that's because the huge number of methods in those
classes coming from several interfaces.

As a starter, here are some suggestions how to go further from here:

 * Find out which classes exactly cause the problem, and how big the
memory would have to be to support Scala
 * Try to use multiple class loaders to have multiple allocation regions of 5 MB
 * Work with the Android team to find a better general allocator scheme
 * Try to reduce the footprint (of the most common parts) of the Scala library
 * Find out why proguard can't shrink Scala's library further into
better consumable bits
 * Create a more lightweight Scala library for Android development

More to come...

--
Johannes

[1] https://issues.scala-lang.org/browse/SI-4620
[2] LinearAlloc.c : #define DEFAULT_MAX_LENGTH  (5*1024*1024)
[3] https://gist.github.com/1214972

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Simon Ochsenreither 9/13/11 4:34 PM
Hi,
> [3]https://gist.github.com/1214972
I wonder why scalac generates an anonymous class for every single method
in TraversableOnce/StreamViewLike ...

Bye,


Simon

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Paul Butcher 9/14/11 7:20 AM
Very many thanks for your work on this, Johannes.

I certainly hope that at some point these restrictions will be lifted in Android (I know that the Clojure community is butting up against the same limitations at the moment). But having said that:

On 13 Sep 2011, at 21:39, Johannes Rudolph wrote:
> * Try to reduce the footprint (of the most common parts) of the Scala library
> * Find out why proguard can't shrink Scala's library further into
> better consumable bits
> * Create a more lightweight Scala library for Android development


One or more of these options, it strikes me, definitely has to be part of the solution. Resource constrained devices are going to be with us for some time, and if Scala can't sensibly address them, that's going to be a very significant chink in its armour.

We currently use Scala very heavily on Android, but at the moment only for test code, not production. I was hoping that we were getting close to the point where that could change, but unfortunately the opposite seems to be true :-(

Anyway, thanks again for your efforts!

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Stefan Langer 9/14/11 11:16 AM
One thing we should also try to find out is a summary of bugs that are
already opened against Android, that address these issues and if we
find some that are not adressed we should open the bug.

-
Stefan

2011/9/14 Paul Butcher <pa...@paulbutcher.com>:

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Jeff Axelrod 9/21/11 2:24 PM
Unfortunately, somehow I saw this problem when using Scala 2.8.  It's a moderately sized application, and I got the problem when adding a single Scala class to the project.  Has anyone seen this problem in Scala 2.8?
Re: Scala 2.9.x's problems with the DalvikVM James Moore 10/19/11 9:33 AM
On Sep 14, 7:20 am, Paul Butcher <p...@paulbutcher.com> wrote:
> I certainly hope that at some point these restrictions will be lifted inAndroid(I know that the Clojure community is butting up against the same limitations at the moment)....
> One or more of these options, it strikes me, definitely has to be part of the solution. Resource constrained devices are going to be with us for some time, and if Scala can't sensibly address them, that's going to be a very significant chink in its armour.

I think that it's unlikely to be usefully fixed on the Android side in
any kind of reasonable timeframe.  Since it didn't happen for ICS last
night (at least as far as I can tell), that means you're probably
stuck with the limit for at least another year for brand-new devices.
And 4.0 devices will be around for a while, so if you want any sort of
market coverage, you're looking at multiple years before any Android
change makes a difference.

But like you said, if Scala just can't do small devices that's a real
problem.  6 megs of libraries for anything past 'hello world' isn't
really workable.

What kind of priority does fixing this have in the scala dev world?
I'd like to start doing Scala + Android (and just found the
https://groups.google.com/forum/#!forum/scala-on-android google
group), but it's unclear what kind of priority these issues have.
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Rex Kerr 10/19/11 9:46 AM
This could probably be fixed--albeit with a performance penalty--fairly quickly by compiling the library with specialization turned off.
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM martin odersky 10/19/11 9:50 AM


On Wed, Oct 19, 2011 at 6:46 PM, Rex Kerr <ich...@gmail.com> wrote:
This could probably be fixed--albeit with a performance penalty--fairly quickly by compiling the library with specialization turned off.

I'd be interested in an experiment to see if it was true. If yes, we might want to ship with an alternate "slim" library for Android.

Cheers

 -- Martin



--
Martin Odersky
Prof., EPFL and Chairman, Typesafe
PSED, 1015 Lausanne, Switzerland
Tel. EPFL: +41 21 693 6863
Tel. Typesafe: +41 21 691 4967

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Paul Butcher 10/19/11 10:03 AM
On 19 Oct 2011, at 17:50, martin odersky wrote:
On Wed, Oct 19, 2011 at 6:46 PM, Rex Kerr <ich...@gmail.com> wrote:
This could probably be fixed--albeit with a performance penalty--fairly quickly by compiling the library with specialization turned off.

I'd be interested in an experiment to see if it was true. If yes, we might want to ship with an alternate "slim" library for Android.

This would be great if it was a quick win :-)

Any feeling for how severe the performance penalty might be?

--
paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 10/19/11 10:05 AM
On Wed, Oct 19, 2011 at 6:46 PM, Rex Kerr <ich...@gmail.com> wrote:
> This could probably be fixed--albeit with a performance penalty--fairly
> quickly by compiling the library with specialization turned off.

What makes you think so? Looking at the dump from my original post, my
greatest hopes right now are that (if there's a "quick fix") the
proposed changes to the view hierarchy could improve things quite a
lot.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Rex Kerr 10/19/11 3:53 PM
It was a guess based upon specialization being responsible for much of the increase in size of the library, but those classes seem to have been fairly effectively removed from the set included in the jar.  What I don't know is whether there is any / enough of a static size penalty--even if the specialized classes aren't there--to make a difference.  I have anecdotal evidence for both "no" and "yes", and haven't tested carefully.

  --Rex
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Miguel Garcia 12/4/11 6:01 AM

Just in case this helps, compiling the scala lib sources with -no-specialization and -sourcepath results in jar size close to 6MB instead of the usual 8MB resulting from using specialization (the default) and -optimize (which inlines methods and thus in general increases size).

A further reduction should be possible by leaving out all parallel collection classes, which don't seem necessary on Android (if I remember correctly, Scala+GWT has a source-level conversion utility based on jdk2ikvm [1] to perform just that). Details in the Scala+GWT google group (I guess).

Another round of size reduction could result from "modularizing the Scala library".
  (a) In the Java world, a big jar file can be broken up into several smaller ones using tools like [2].
  (b) Two modularization efforts:
         https://github.com/scala-incubator/modularisation
         and
         https://github.com/paulp/scala-modular

Miguel
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/


References

[1] http://lamp.epfl.ch/~magarcia/jdk2ikvm/

[2] http://www.kirkk.com/main/Main/JarAnalyzer

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Patrick Logan 12/4/11 10:03 AM
"leaving out all parallel collection classes, which don't seem necessary on Android"

Except by 2013 when these devices will have a half dozen ARMs or double that.

The Scala compiler folks should just avoid targeting Android devices for production apps in 2012. Yes, shrink the footprint, etc. But prioritize your overall roadmap, and realize that Android devices will meet you more than half way by the time you are production-ready.

You cannot boil the ocean. The ocean's temperature is increasing... it will rise up to meet you.

Oh. Bad analogy.

-Patrick

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Simon Ochsenreither 12/4/11 12:04 PM
Hi Miguel,

Did you try a recent version?
The file size of the standard jar is now at 6.8 MB here (2.10.0.r26094-b20111201023257).

How much reduction is still needed?

Thanks and bye,

Simon
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Miguel Garcia 12/4/11 12:40 PM
Simon,

The jar comparison I made wasn't very scientific (I just compared lib/scala-library.jar with some version I compiled). In any case, my suggestion was mostly about trying in order (1) -no-specialization, (2) no parallel collections, (3) breaking the library into manageable chunks; before trying extending the compiler to target Dalvik (which can also be done).

Miguel
http://lamp.epfl.ch/~magarcia/ScalaCompilerCornerReloaded/


Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM James Moore 12/4/11 12:44 PM
On Sun, Dec 4, 2011 at 12:04 PM, Simon Ochsenreither
<simon.och...@googlemail.com> wrote:
> Hi Miguel,
>
> Did you try a recent version?
> The file size of the standard jar is now at 6.8 MB here
> (2.10.0.r26094-b20111201023257).
>
> How much reduction is still needed?

The critical metric isn't size, it's call sites.  No more than 64k
(method definitions) + (calls to methods) is allowed.

Smaller is better, obviously, but I don't think size is the blocking
problem right now.

--
James Moore
ja...@restphone.com
http://jamesmoorecode.blogspot.com/

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Paul Butcher 12/4/11 3:23 PM
I wish that that was true. Unfortunately I fear that the limit is built into the fabric of the Dalvik VM as it currently stands and is unlikely to go away any time soon :-(

--


paul.butcher->msgCount++

Snetterton, Castle Combe, Cadwell Park...
Who says I have a one track mind?

http://www.paulbutcher.com/
LinkedIn: http://www.linkedin.com/in/paulbutcher
MSN: pa...@paulbutcher.com
AIM: paulrabutcher
Skype: paulrabutcher

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM Simon Ochsenreither 12/5/11 4:01 AM
Is there a bug filed in Android's bug tracker?

Would make sense to make some noise there. Probably together with other languages which might have the same problem...
Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/5/11 5:22 AM
With more and more traffic on this topic, here's a follow-up on my
findings on some of the points:

On Tue, Sep 13, 2011 at 10:39 PM, Johannes Rudolph
<johannes...@googlemail.com> wrote:
>  * Find out which classes exactly cause the problem, and how big the
> memory would have to be to support Scala

To find out how much memory would be needed I compiled a dalvik
version with increased LinearAlloc size. The example from the ticket
[1] would need a LinearAlloc size of about 10 MB (twice the current
size).

Then I checked what exactly needs those MBs of memory and I think I've
pretty much identified the current problem with the DalvikVM. See this
snippet (https://gist.github.com/1426684), which collects all of the
interfaces a class implements in a naive way:

scala> import collection.JavaConversions._
import collection.JavaConversions._

scala> def allIf(clazz: Class[_]): Seq[String] =
clazz.getInterfaces.map(_.toString) ++
clazz.getInterfaces.flatMap(allIf)
allIf: (clazz: Class[_])Seq[String]

scala> val theClazz =
Class.forName("scala.collection.immutable.StreamViewLike$$anon$10")
theClazz: Class[_] = class scala.collection.immutable.StreamViewLike$$anon$10

scala> val res0 = { allIf(theClazz) }
res0: Seq[String] = ...
scala> res0.size
res16: Int = 9223

scala> res0.distinct.size
res17: Int = 49

9223 ?!? What you see is multiple interface inheritance at work.
Unfortunately, the `allIf` function roughly corresponds to the way the
dalvik vm currently collects the full set of interfaces a
class/interface implements. The problem is that the DalvikVM doesn't
remove the duplicate interfaces in a final step (there was code to do
that, but is currently commented for some problems with other
functionality). Each interface needs at least 8 bytes of memory and
this is only one of the problematic classes. I guess this was never
better optimized because an interface hierarchy as complicated as this
is unusual in Java. I attached the hierarchy of the given class
(without ScalaObject) for illustration.

The question is if there is anything on the Scala side which would
improve the situation. The change of providing abstract classes for
some of the classes in the hierarchy didn't do much good here
unfortunately (because overall the hierarchy itself didn't change
much).

But there is something simple which could be done: There are a lot of
useless interface declarations which are already covered by super
interfaces. Most obvious example is ScalaObject which is a super
interface of all those interfaces. Removing all those interfaces would
mean a reduction to approx. 2000 interfaces for this example (but less
for other less spectacular cases).

>  * Try to use multiple class loaders to have multiple allocation regions of 5 MB

That won't work, there is some code which looks as if each ClassLoader
would be getting its own region of 5MB but in reality this is not
(yet?) implemented in dalvik.

>  * Find out why proguard can't shrink Scala's library further into better consumable bits

If you think about how shrinking works, you can quite easily see that
finding the minimal set of classes/methods needed for execution is
undecidable in theory (if you can't say for sure that a program halts,
neither you can say  - for some points in the program - that those
points will be reached, so you can't prune from there on). In practice
you would have to optimistically include this code but this may be not
the biggest problem (because common programs may be sufficiently
simple). What is really hard, is handling virtual calls. As long as
you can't be sure what exactly the receiver of a virtual call is you
would have to include all possible subclasses. Now think about what
this means if you use Scala's Function classes. I think his explains
why proguard has problems with the highly inter-dependent collection
classes. There's academic literature on this topic, so maybe there is
a good enough solution which just no one has implemented for Java
bytecode.

--
Johannes

[1] https://github.com/humcycles/dexfailure

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: Scala 2.9.x's problems with the DalvikVM Simon Ochsenreither 12/5/11 6:14 AM
Imho there seem to be ways to make it less worse than it currently is, imho the Android team should really fix this problem.

Did you create an issue/comment somewhere?
Re: Scala 2.9.x's problems with the DalvikVM Jeff Axelrod 12/5/11 6:44 AM
There was an Android bug filed in March 2010.  The present status is "working as intended."  Star it and add your feedback.
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/5/11 6:50 AM
That's not the same as the current bug.

On Mon, Dec 5, 2011 at 3:44 PM, Glenview Jeff <glenvi...@gmail.com> wrote:
> There was an Android bug filed in March 2010.  The present status is
> "working as intended."  Star it and add your feedback.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Scala 2.9.x's problems with the DalvikVM James Moore 12/5/11 8:14 AM

I don't know a bug number, but they're aware of the issue.  When I
talked to some Dalvik guys at Google IO this year, they knew it was an
issue for scala, and thought that scala was an indicator that other
people would be hitting it too ("canary in the coal mine" was the
phrase used).  They wanted to change it, but didn't have any
dates/releases for when that might happen.

--
James Moore
ja...@restphone.com
http://jamesmoorecode.blogspot.com/

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 12/5/11 8:20 AM
On Mon, Dec 5, 2011 at 5:22 AM, Johannes Rudolph
<johannes...@googlemail.com> wrote:
> scala> val res0 = { allIf(theClazz) }
> res0: Seq[String] = ...
> scala> res0.size
> res16: Int = 9223
>
> scala> res0.distinct.size
> res17: Int = 49
>
> 9223 ?!? What you see is multiple interface inheritance at work.

The interface minimization is checked in; the above now produces

scala> res0.size
res2: Int = 2351

scala> res0.distinct.size
res3: Int = 49

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 12/5/11 8:22 AM
Admittedly 2351 is still rather a large number.  Here's the breakdown.

(interface scala.ScalaObject,634)
(interface scala.collection.GenTraversableOnce,263)
(interface scala.collection.Parallelizable,233)
(interface scala.collection.GenTraversableLike,233)
(interface scala.collection.generic.HasNewBuilder,156)
(interface scala.collection.generic.GenericTraversableTemplate,126)
(interface scala.collection.GenTraversable,126)
(interface scala.collection.GenIterableLike,77)
(interface scala.collection.GenTraversableViewLike,58)
(interface scala.collection.GenIterable,50)
(interface scala.collection.GenTraversableView,49)
(interface scala.collection.TraversableLike,30)
(interface scala.collection.generic.FilterMonadic,30)
(interface scala.collection.TraversableOnce,30)
(interface scala.collection.GenIterableViewLike,29)
(interface scala.Equals,27)
(interface scala.collection.GenIterableView,23)
(interface scala.collection.Traversable,18)
(interface scala.collection.GenSeqLike,15)
(interface scala.collection.GenSeq,12)
(interface scala.collection.IterableLike,12)
(interface scala.collection.GenTraversableViewLike$Transformed,11)
(interface scala.collection.Iterable,9)
(interface scala.collection.GenSeqViewLike,9)
(interface scala.collection.TraversableViewLike,9)
(interface scala.collection.TraversableView,9)
(interface scala.collection.ViewMkString,9)
(interface scala.collection.GenIterableViewLike$Transformed,8)
(interface scala.collection.IterableViewLike,6)
(interface scala.collection.GenSeqView,6)
(interface scala.collection.IterableView,6)
(interface scala.collection.GenSeqViewLike$Transformed,3)
(interface scala.Function1,3)
(interface scala.collection.SeqView,3)
(interface scala.collection.Seq,3)
(interface scala.collection.IterableViewLike$Transformed,3)
(interface scala.collection.SeqLike,3)
(interface scala.collection.TraversableViewLike$Transformed,3)
(interface scala.collection.SeqViewLike,3)
(interface scala.PartialFunction,3)
(interface scala.collection.SeqViewLike$Transformed,2)
(interface scala.collection.GenIterableViewLike$ZippedAll,2)
(interface scala.collection.immutable.StreamViewLike,1)
(interface scala.collection.immutable.StreamViewLike$ZippedAll,1)
(interface scala.collection.SeqViewLike$ZippedAll,1)
(interface scala.collection.GenSeqViewLike$ZippedAll,1)
(interface scala.collection.IterableViewLike$ZippedAll,1)
(interface scala.collection.immutable.StreamView,1)
(interface scala.collection.immutable.StreamViewLike$Transformed,1)

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 12/5/11 8:26 AM
Attached is the bytecode diff before and after that change, if
anyone's interested.
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Jeff Axelrod 12/5/11 8:42 AM
Unless I'm misunderstanding something here, in the bug report, danfuzz (presumably at Google) wrote:

I'm marking this as "working as intended," but that's really not the whole story:

It turns out that you have made the first report we've seen of anyone running into Dalvik's existing limit of
65536 method references per dex file.

In the short term, we are going to fix the error message to be less opaque. In the long term, we will do a rev of
the format to allow for "jumbo" method references (and field references, while we're at it). We knew we were
seeing string counts approaching the 16-bit limit before we shipped 1.0, and so we introduced the const-
string/jumbo variant of the const-string opcode. However, we didn't think we'd see method counts hit the
limit for quite a while. You have made us rethink our timeline!

As you noticed, by splitting your output into multiple dex files, you are neatly working around the problem.
You will of course have a bit more to do to make multiple dex files work as part of an Android application, but
all the right Android/Dalvik library support should be there for you to do that.

Thanks for the report, and I'm sorry that I can't promise a quick solution to the problem, as an executable

format rev is no small matter.


On Monday, December 5, 2011 8:50:58 AM UTC-6, Johannes Rudolph wrote:
That's not the same as the current bug.

On Mon, Dec 5, 2011 at 3:44 PM, Glenview Jeff <glenvi...@gmail.com> wrote:
> There was an Android bug filed in March 2010.  The present status is
> "working as intended."  Star it and add your feedback.

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/5/11 9:06 AM
On Mon, Dec 5, 2011 at 5:42 PM, Glenview Jeff <glenvi...@gmail.com> wrote:
> Unless I'm misunderstanding something here, in the bug report, danfuzz
> (presumably at Google) wrote:

Oh, I didn't want to dispute that this may be still an issue, it's
just not the current blocker. At least not the one I'm seeing here. Is
anyone else still blocked by the bug from the issue?

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Cédric Beust ♔ 12/5/11 10:10 AM

On Mon, Dec 5, 2011 at 8:42 AM, Glenview Jeff <glenvi...@gmail.com> wrote:
danfuzz (presumably at Google)

Dan is the creator and main architect of Dalvik.

-- 
Cédric

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Jeff Axelrod 12/5/11 11:00 AM

On Monday, December 5, 2011 12:10:17 PM UTC-6, Cédric Beust ♔ wrote:

Dan is the creator and main architect of Dalvik.


 If anybody is interested, he said he no longer works at Google or on Android.  He responded to my inquiry today about issue 7147.

I don't know: I left Google and stopped working on Android a couple months ago...You've always been able to load additional dex files into an app. I don't deny that it's extra effort in the build system and to put together the app bootstrap code, but it only has to be set up once for the entire Scala community.

 


Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/5/11 1:06 PM
For your delight (and awaiting your votes), here's the Android bug report

http://code.google.com/p/android/issues/detail?id=22586

Actually, the Android code once contained a fix for this issue which
was commented out later on. However, I think the reason why the old
fix had to be inactivated is invalid and then I went on and fixed the
issue blocking this one as well, so we might have good luck and have
this problem fixed in dalvik before 2015.


--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/5/11 1:13 PM
And BTW: you may find it interesting to know that you can produce the
same kind of problem with a 50-line Java file:

https://gist.github.com/1434551#file_src%2Ftest.java

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Simon Ochsenreither 12/5/11 1:49 PM
Thank you very much for your work!
unk...@googlegroups.com 12/6/11 6:00 AM <This message has been deleted.>
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 12/17/11 8:51 AM
BTW, thanks for the effort here, it's much appreciated. I did one
superficial test with a SNAPSHOT version last week, and unfortunately
it still isn't working. I had no time yet to rerun with my customized
dalvik version to check how much things have improved exactly and
what's still missing. (Not every day is the right day to get your
mobile phone out of the continuous reboot loop caused by a broken
dalvikvm...)

On Mon, Dec 5, 2011 at 5:20 PM, Paul Phillips <pa...@improving.org> wrote:
> On Mon, Dec 5, 2011 at 5:22 AM, Johannes Rudolph
> <johannes...@googlemail.com> wrote:
>> scala> val res0 = { allIf(theClazz) }
>> res0: Seq[String] = ...
>> scala> res0.size
>> res16: Int = 9223
>>
>> scala> res0.distinct.size
>> res17: Int = 49
>>
>> 9223 ?!? What you see is multiple interface inheritance at work.
>
> The interface minimization is checked in; the above now produces
>
> scala> res0.size
> res2: Int = 2351
>
> scala> res0.distinct.size
> res3: Int = 49

--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 11:33 AM
Is there any progress with issue? I am greatly interested in it. I modified createramdisks scripts from github.com/jberkel/android-sdk-scala so that able to preload scala-library (any version and other java libraries) to emulator withing seconds. This is significant reduce build process. Libraries load to /data and security descriptors to /system/etc/permission. Rooted phone is unnecessary.
Also that way allow to get error only when library used, not at boot stage. But unfortunately there is always sporadically with any scala versions, 2.8 too :-(

E/dalvikvm(  787): LinearAlloc exceeded capacity (5242880), last=19092
E/dalvikvm(  787): VM aborting
I/DEBUG   (   31): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
I/DEBUG   (   31): Build fingerprint: 'generic/sdk/generic/:2.2/FRF91/43546:eng/test-keys'
I/DEBUG   (   31): pid: 787, tid: 787  >>> org.digimead.digi.inetd.sshd <<<
or something like this for any significant code. So sad.
I am sure that the only right way is to run scala-test suite at android emulator. If scala-test compiller suite pass without any errors than scala ready for tablet devices. Maybe to write android activity, that allow to run scala-test suite in adb shell? But without support from scala core developers – it will be useless.

PS “As of December 2011 there are over 700,000 Android devices activated every day.” It is very disappoint that scala developers ignore such part of market and build compiler in unportable way, that allow to use cross platform language only at limited number of targets.

PPS I may share my scripts that allow preload java(scala) libraries at emulator. Also I am ready to help with it, but I am afraid that my knowledge a bit limited.

King regards,
Alexey
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM √iktor Klang 1/5/12 11:40 AM
Have you've checked the latest Scala version from master?--
Viktor Klang

Akka Tech Lead
Typesafe - Enterprise-Grade Scala from the Experts

Twitter: @viktorklang

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 11:44 AM


On Thu, Jan 5, 2012 at 11:33 AM, Alexey Aksenov <alexe...@gmail.com> wrote:
If scala-test compiller suite pass without any errors than scala ready for tablet devices. Maybe to write android activity, that allow to run scala-test suite in adb shell? But without support from scala core developers – it will be useless.

What kind of support do you have in mind? I can't do the work, but I support the goal.
 
PS “As of December 2011 there are over 700,000 Android devices activated every day.” It is very disappoint that scala developers ignore such part of market and build compiler in unportable way, that allow to use cross platform language only at limited number of targets.

Personally I'm not ignoring it, but I have to choose my battles.  Take a look at the bug database sometime and then try to tell me I can work on android-specific matters in good conscience.

Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 12:28 PM
Thank you all for answer. I was afraid that there is not activity and
interested in solve.

Ok. I did test. What's next?

download latest with git clone https://github.com/scala/scala.git
compile with and fastdist
upload
Scala code runner version 2.10.0.rdev-4118-2012-01-05-gbe46e48 --
Copyright 2002-2011, LAMP/EPFL

Using Scala  in /home/ezh/.sbt/boot/scala-2.10.0-SNAPSHOT

Generating scala-library.jar...

Generating scala-collection.jar...

Generating scala-immutable.jar...

Generating scala-mutable.jar...

Generating scala-actors.jar...

Converting scala-library.jar into a dex file...

Converting scala-collection.jar into a dex file...

Converting scala-immutable.jar into a dex file...

Converting scala-mutable.jar into a dex file...

Converting scala-actors.jar into a dex file...

Dex files were successfully generated (configs/framework)

recompile my project with sbt

and :-( other error, scala broken at thunk? Libraries loaded
successful.
Log strings like «D/o.d.d.i.s*.SSHDActivity(  282): alive
» generated with "org.slf4j" % "slf4j-android" % "1.6.1-RC1" FYI
Proguard disabled. Only plain preloaded libraries... How may I fix
this issue? I may connect debugger from eclipse within one moment. But
where to look?

org.digimead.digi.inetd.sshd.SSHDActivity.onResume(SSHDActivity.scala:
94)
 is
  override def onResume() {

    log.debug("onResume(...)")

    super.onResume()

    if (sshdService.get() == null)


94!!!!:      future { bindService(new Intent(this,
classOf[SSHDService]), sshdConnection, Context.BIND_AUTO_CREATE) }


    registerReceiver(receiver, new
IntentFilter(SSHDService.updateIntent))

  }

I/ActivityManager(   59): Starting activity: Intent
{ act=android.intent.action.MAIN
cat=[android.intent.category.LAUNCHER] flg=0x10200000
cmp=org.digimead.digi.inetd.sshd/.SSHDActivity }

I/ActivityManager(   59): Start proc org.digimead.digi.inetd.sshd for
activity org.digimead.digi.inetd.sshd/.SSHDActivity: pid=282 uid=10036
gids={}

I/ARMAssembler(   59): generated
scanline__00000077:03545404_00000004_00000000 [ 47 ipp] (67 ins) at
[0x359058:0x359164] in 264230 ns

I/AndroidLoggerFactory(  282): Logger name 'o.d.d.i.sshd.SSHDActivity'
exceeds maximum length of 23 characters, using
'o.d.d.i.s*.SSHDActivity' instead.

D/o.d.d.i.s*.SSHDActivity(  282): alive

D/o.d.d.i.s*.SSHDActivity(  282): onCreate(...)

I/AndroidLoggerFactory(  282): Logger name
'o.d.d.i.sshd.service.TabActivity' exceeds maximum length of 23
characters, using 'o.d.d.i.s*.s*.TabActiv*' instead.

D/dalvikvm(  282): GC_FOR_MALLOC freed 3891 objects / 370824 bytes in
34ms

D/dalvikvm(  282): GC_FOR_MALLOC freed 3838 objects / 444456 bytes in
39ms

D/dalvikvm(  282): GC_EXTERNAL_ALLOC freed 80 objects / 5008 bytes in
30ms

I/AndroidLoggerFactory(  282): Logger name
'o.d.d.i.sshd.info.TabActivity' exceeds maximum length of 23
characters, using 'o.d.d.i.s*.i*.TabActiv*' instead.

D/o.d.d.i.s*.SSHDActivity(  282): onStart(...)

D/o.d.d.i.s*.SSHDActivity(  282): onResume(...)

I/dalvikvm(  282): Could not find method
scala.actors.IScheduler.getClass, referenced from method
scala.actors.scheduler.DaemonScheduler$.makeNewScheduler

W/dalvikvm(  282): VFY: unable to resolve interface method 1527:
Lscala/actors/IScheduler;.getClass ()Ljava/lang/Class;

D/dalvikvm(  282): VFY: replacing opcode 0x72 at 0x0032

D/dalvikvm(  282): VFY: dead code 0x0035-0047 in Lscala/actors/
scheduler/DaemonScheduler$;.makeNewScheduler ()Lscala/actors/
IScheduler;

I/dalvikvm(  282): Could not find method
scala.actors.IScheduler.getClass, referenced from method
scala.actors.Reactor$$anon$2.makeNewScheduler

W/dalvikvm(  282): VFY: unable to resolve interface method 1527:
Lscala/actors/IScheduler;.getClass ()Ljava/lang/Class;

D/dalvikvm(  282): VFY: replacing opcode 0x72 at 0x003f

D/dalvikvm(  282): VFY: dead code 0x0042-0054 in Lscala/actors/Reactor$
$anon$2;.makeNewScheduler ()Lscala/actors/IScheduler;

I/dalvikvm(  282): Could not find method scala.actors.Actor.wait,
referenced from method scala.actors.Actor$class.liftedTree1$1

W/dalvikvm(  282): VFY: unable to resolve interface method 1021:
Lscala/actors/Actor;.wait ()V

D/dalvikvm(  282): VFY: replacing opcode 0x72 at 0x0000

I/dalvikvm(  282): Could not find method scala.actors.Actor.notify,
referenced from method scala.actors.Actor$class.scala$actors$Actor$
$resumeActor

W/dalvikvm(  282): VFY: unable to resolve interface method 982: Lscala/
actors/Actor;.notify ()V

D/dalvikvm(  282): VFY: replacing opcode 0x72 at 0x0004

D/dalvikvm(  282): VFY: dead code 0x0007-0007 in Lscala/actors/Actor
$class;.scala$actors$Actor$$resumeActor (Lscala/actors/Actor;)V

D/AndroidRuntime(  282): Shutting down VM

W/dalvikvm(  282): threadid=1: thread exiting with uncaught exception
(group=0x4001d800)

E/AndroidRuntime(  282): FATAL EXCEPTION: main

E/AndroidRuntime(  282): java.lang.NoSuchMethodError:
scala.actors.IScheduler.getClass

E/AndroidRuntime(  282):         at scala.actors.scheduler.DaemonScheduler
$.makeNewScheduler(DaemonScheduler.scala:29)

E/AndroidRuntime(  282):         at scala.actors.scheduler.DelegatingScheduler
$class.newActor(DelegatingScheduler.scala:63)

E/AndroidRuntime(  282):         at scala.actors.scheduler.DaemonScheduler
$.newActor(DaemonScheduler.scala:17)

E/AndroidRuntime(  282):         at scala.actors.Reactor
$class.dostart(Reactor.scala:220)

E/AndroidRuntime(  282):         at scala.actors.FutureActor.scala$actors
$Actor$$super$dostart(Future.scala:47)

E/AndroidRuntime(  282):         at scala.actors.Actor
$class.dostart(Actor.scala:666)

E/AndroidRuntime(  282):         at
scala.actors.FutureActor.dostart(Future.scala:47)

E/AndroidRuntime(  282):         at scala.actors.Reactor
$class.start(Reactor.scala:229)

E/AndroidRuntime(  282):         at scala.actors.FutureActor.scala$actors
$Actor$$super$start(Future.scala:47)

E/AndroidRuntime(  282):         at scala.actors.Actor
$class.start(Actor.scala:670)

E/AndroidRuntime(  282):         at
scala.actors.FutureActor.start(Future.scala:47)

E/AndroidRuntime(  282):         at scala.actors.Futures$.future(Future.scala:
122)

E/AndroidRuntime(  282):         at
org.digimead.digi.inetd.sshd.SSHDActivity.onResume(SSHDActivity.scala:
94)

E/AndroidRuntime(  282):         at
android.app.Instrumentation.callActivityOnResume(Instrumentation.java:
1149)

E/AndroidRuntime(  282):         at
android.app.Activity.performResume(Activity.java:3823)

E/AndroidRuntime(  282):         at
android.app.ActivityThread.performResumeActivity(ActivityThread.java:
3118)

E/AndroidRuntime(  282):         at
android.app.ActivityThread.handleResumeActivity(ActivityThread.java:
3143)

E/AndroidRuntime(  282):         at
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:
2684)

E/AndroidRuntime(  282):         at android.app.ActivityThread.access
$2300(ActivityThread.java:125)

E/AndroidRuntime(  282):         at android.app.ActivityThread
$H.handleMessage(ActivityThread.java:2033)

E/AndroidRuntime(  282):         at
android.os.Handler.dispatchMessage(Handler.java:99)

E/AndroidRuntime(  282):         at android.os.Looper.loop(Looper.java:123)

E/AndroidRuntime(  282):         at
android.app.ActivityThread.main(ActivityThread.java:4627)

E/AndroidRuntime(  282):         at
java.lang.reflect.Method.invokeNative(Native Method)

E/AndroidRuntime(  282):         at
java.lang.reflect.Method.invoke(Method.java:521)

E/AndroidRuntime(  282):         at com.android.internal.os.ZygoteInit
$MethodAndArgsCaller.run(ZygoteInit.java:868)

E/AndroidRuntime(  282):         at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626)

E/AndroidRuntime(  282):         at dalvik.system.NativeStart.main(Native
Method)


On 5 янв, 23:40, √iktor Ҡlang <viktor.kl...@gmail.com> wrote:
> Have you've checked the latest Scala version from master?
>
...
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 12:46 PM


On Thu, Jan 5, 2012 at 12:28 PM, Alexey Aksenov <alexe...@gmail.com> wrote:
I/dalvikvm(  282): Could not find method
scala.actors.IScheduler.getClass, referenced from method
scala.actors.scheduler.DaemonScheduler$.makeNewScheduler

W/dalvikvm(  282): VFY: unable to resolve interface method 1527:
Lscala/actors/IScheduler;.getClass ()Ljava/lang/Class;

That is a bug, we are apparently generating invokeinterface instructions for java.lang.Object methods when the target is an interface.  I don't think they're supposed to choke on it (don't interfaces inherit from Object?) but we should fix it regardless.
Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 1:10 PM
Ok, sure. I have git code. I may recompile scala library. I may run
and check it. But how may I help to fix this issue?

My own goal - successful launch simple as hello world program. What is
the next step? Open new issue? Or is it possible to solve this in this
mail thread? Please, help.

That kind of support I have in mind. There is a problem and I cannot
solve it alone :-(

I absolutely have no idea how handle:

> That is a bug, we are apparently generating invokeinterface instructions
> for java.lang.Object methods when the target is an interface.  I don't
> think they're supposed to choke on it (don't interfaces inherit from
> Object?) but we should fix it regardless.

How "should I fix it regardless?" :-) My ability to knowledge
everything is a bit limited. I don't know internal mechanic of scala
compiler. But I may regardless if needed with help of someone.
So is it possible to fix in some test branch?
Could you provide me some tips with source file name, than probably
contain bug. I try to read a code.

> Paul Phillips
> On Thu, Jan 5, 2012 at 11:33 AM, Alexey Aksenov <alexey....@gmail.com>wrote:
>
>> If scala-test compiller suite pass without any errors than scala ready for
>> tablet devices. Maybe to write android activity, that allow to run
>> scala-test suite in adb shell? But without support from scala core
>> developers – it will be useless.
>
> What kind of support do you have in mind? I can't do the work, but I
> support the goal.


On 6 янв, 00:46, Paul Phillips <pa...@improving.org> wrote:
Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 1:13 PM
I am sorry

But I may _regardless if needed with help of someone.

actually

But I may _investigate if needed with help of someone.
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 1:16 PM


On Thu, Jan 5, 2012 at 1:10 PM, Alexey Aksenov <alexe...@gmail.com> wrote:
How "should I fix it regardless?" :-) My ability to knowledge
everything is a bit limited. I don't know internal mechanic of scala
compiler. But I may regardless if needed with help of someone.

I already fixed it, but for reference, the files most likely to be relevant to bytecode generation issues are:


The change in this case:

diff --git a/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala b/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala
index 241163885d..c5624a86aa 100644
--- a/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala
+++ b/src/compiler/scala/tools/nsc/backend/jvm/GenJVM.scala
@@ -1199,7 +1199,7 @@ abstract class GenJVM extends SubComponent with GenJVMUtil with GenAndroid with
       hostSymbol.info ; methodOwner.info
 
       def isInterfaceCall(sym: Symbol) = (
-           sym.isInterface
+           sym.isInterface && methodOwner != ObjectClass
         || sym.isJavaDefined && sym.isNonBottomSubClass(ClassfileAnnotationClass)
       )
       // whether to reference the type of the receiver or

Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 2:04 PM
Unsuccessful. There was 2 attempt (I want to check that results are
the same)

Only difference is
unable to resolve interface method 1522 before
vs
unable to resolve interface method 1527 after

other numbers are same

I suspect that interface methods quantity changed: -5

Anything else?

PS Compiler code is too heavy for me. I think that I must spent a lot
of time (more than I have, I am afraid) to rich acceptable level of
knowledge for solve such type problems alone :-(

D/o.d.d.i.s*.SSHDActivity(  315): onResume(...)
I/dalvikvm(  315): Could not find method
scala.actors.IScheduler.getClass, referenced from method
scala.actors.scheduler.DaemonScheduler$.makeNewScheduler
W/dalvikvm(  315): VFY: unable to resolve interface method 1522:
Lscala/actors/IScheduler;.getClass ()Ljava/lang/Class;
D/dalvikvm(  315): VFY: replacing opcode 0x72 at 0x0032
D/dalvikvm(  315): VFY: dead code 0x0035-0047 in Lscala/actors/
scheduler/DaemonScheduler$;.makeNewScheduler ()Lscala/actors/
IScheduler;
I/dalvikvm(  315): Could not find method
scala.actors.IScheduler.getClass, referenced from method
scala.actors.Reactor$$anon$2.makeNewScheduler
W/dalvikvm(  315): VFY: unable to resolve interface method 1522:
Lscala/actors/IScheduler;.getClass ()Ljava/lang/Class;
D/dalvikvm(  315): VFY: replacing opcode 0x72 at 0x003f
D/dalvikvm(  315): VFY: dead code 0x0042-0054 in Lscala/actors/Reactor$
$anon$2;.makeNewScheduler ()Lscala/actors/IScheduler;
I/dalvikvm(  315): Could not find method scala.actors.Actor.wait,
referenced from method scala.actors.Actor$class.liftedTree1$1
W/dalvikvm(  315): VFY: unable to resolve interface method 1021:
Lscala/actors/Actor;.wait ()V
D/dalvikvm(  315): VFY: replacing opcode 0x72 at 0x0000
I/dalvikvm(  315): Could not find method scala.actors.Actor.notify,
referenced from method scala.actors.Actor$class.scala$actors$Actor$
$resumeActor
W/dalvikvm(  315): VFY: unable to resolve interface method 982: Lscala/
actors/Actor;.notify ()V
D/dalvikvm(  315): VFY: replacing opcode 0x72 at 0x0004
D/dalvikvm(  315): VFY: dead code 0x0007-0007 in Lscala/actors/Actor
$class;.scala$actors$Actor$$resumeActor (Lscala/actors/Actor;)V
D/AndroidRuntime(  315): Shutting down VM
W/dalvikvm(  315): threadid=1: thread exiting with uncaught exception
(group=0x4001d800)
E/AndroidRuntime(  315): FATAL EXCEPTION: main
E/AndroidRuntime(  315): java.lang.NoSuchMethodError:
scala.actors.IScheduler.getClass
E/AndroidRuntime(  315):         at scala.actors.scheduler.DaemonScheduler
$.makeNewScheduler(DaemonScheduler.scala:29)
E/AndroidRuntime(  315):         at scala.actors.scheduler.DelegatingScheduler
$class.newActor(DelegatingScheduler.scala:63)
E/AndroidRuntime(  315):         at scala.actors.scheduler.DaemonScheduler
$.newActor(DaemonScheduler.scala:17)
E/AndroidRuntime(  315):         at scala.actors.Reactor
$class.dostart(Reactor.scala:220)
E/AndroidRuntime(  315):         at scala.actors.FutureActor.scala$actors
$Actor$$super$dostart(Future.scala:47)
E/AndroidRuntime(  315):         at scala.actors.Actor
$class.dostart(Actor.scala:666)
E/AndroidRuntime(  315):         at
scala.actors.FutureActor.dostart(Future.scala:47)
E/AndroidRuntime(  315):         at scala.actors.Reactor
$class.start(Reactor.scala:229)
E/AndroidRuntime(  315):         at scala.actors.FutureActor.scala$actors
$Actor$$super$start(Future.scala:47)
E/AndroidRuntime(  315):         at scala.actors.Actor
$class.start(Actor.scala:670)
E/AndroidRuntime(  315):         at
scala.actors.FutureActor.start(Future.scala:47)
E/AndroidRuntime(  315):         at scala.actors.Futures$.future(Future.scala:
122)
E/AndroidRuntime(  315):         at
org.digimead.digi.inetd.sshd.SSHDActivity.onResume(SSHDActivity.scala:
94)
E/AndroidRuntime(  315):         at
android.app.Instrumentation.callActivityOnResume(Instrumentation.java:
1149)
E/AndroidRuntime(  315):         at
android.app.Activity.performResume(Activity.java:3823)
E/AndroidRuntime(  315):         at
android.app.ActivityThread.performResumeActivity(ActivityThread.java:
3118)
E/AndroidRuntime(  315):         at
android.app.ActivityThread.handleResumeActivity(ActivityThread.java:
3143)
E/AndroidRuntime(  315):         at
android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:
2684)
E/AndroidRuntime(  315):         at android.app.ActivityThread.access
$2300(ActivityThread.java:125)
E/AndroidRuntime(  315):         at android.app.ActivityThread
$H.handleMessage(ActivityThread.java:2033)
E/AndroidRuntime(  315):         at
android.os.Handler.dispatchMessage(Handler.java:99)
E/AndroidRuntime(  315):         at android.os.Looper.loop(Looper.java:123)
E/AndroidRuntime(  315):         at
android.app.ActivityThread.main(ActivityThread.java:4627)
E/AndroidRuntime(  315):         at
java.lang.reflect.Method.invokeNative(Native Method)
E/AndroidRuntime(  315):         at
java.lang.reflect.Method.invoke(Method.java:521)
E/AndroidRuntime(  315):         at com.android.internal.os.ZygoteInit
$MethodAndArgsCaller.run(ZygoteInit.java:868)
E/AndroidRuntime(  315):         at
com.android.internal.os.ZygoteInit.main(ZygoteInit.java:626)
E/AndroidRuntime(  315):         at dalvik.system.NativeStart.main(Native
Method)

On 6 янв, 01:16, Paul Phillips <pa...@improving.org> wrote:
> On Thu, Jan 5, 2012 at 1:10 PM, Alexey Aksenov <alexey....@gmail.com> wrote:
> > How "should I fix it regardless?" :-) My ability to knowledge
> > everything is a bit limited. I don't know internal mechanic of scala
> > compiler. But I may regardless if needed with help of someone.
>
> I already fixed it, but for reference, the files most likely to be relevant
> to bytecode generation issues are:
>
> http://github.com/scala/scala/tree/master/src/compiler/scala/tools/ns...http://github.com/scala/scala/tree/master/src/compiler/scala/tools/ns...
Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 2:11 PM
Try to use reflection a bit with

classOf[scala.actors.IScheduler].getMethods().foreach(m =>
log.error(m.toString))

E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.execute(java.lang.Runnable)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.execute(scala.Function0)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.executeFromActor(java.lang.Runnable)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract boolean
scala.actors.IScheduler.isActive()
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.managedBlock(scala.concurrent.ManagedBlocker)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.newActor(scala.actors.Reactor)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.onTerminate(scala.actors.Reactor,scala.Function0)
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.shutdown()
E/o.d.d.i.s*.SSHDActivity(  338): public abstract void
scala.actors.IScheduler.terminated(scala.actors.Reactor)
> >http://github.com/scala/scala/tree/master/src/compiler/scala/tools/ns......
Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 2:28 PM
under my linux system in repl output of
scala> classOf[scala.actors.IScheduler].getMethods().foreach(m
=>println(m.toString))
public abstract void scala.actors.IScheduler.shutdown()
public abstract void scala.actors.IScheduler.execute(scala.Function0)
public abstract void
scala.actors.IScheduler.execute(java.lang.Runnable)
public abstract boolean scala.actors.IScheduler.isActive()
public abstract void
scala.actors.IScheduler.terminated(scala.actors.Reactor)
public abstract void
scala.actors.IScheduler.executeFromActor(java.lang.Runnable)
public abstract void
scala.actors.IScheduler.newActor(scala.actors.Reactor)
public abstract void
scala.actors.IScheduler.onTerminate(scala.actors.Reactor,scala.Function0)
public abstract void
scala.actors.IScheduler.managedBlock(scala.concurrent.ManagedBlocker)

scala>

same. There isn't getClass method, but it is interface

Error generated by ?silly? string. Why it call getClass on interface?
looks bad

    Debug.info(this+": starting new "+sched+" ["+sched.getClass+"]")
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 2:52 PM

On Thu, Jan 5, 2012 at 2:28 PM, Alexey Aksenov <alexe...@gmail.com> wrote:
> > Unsuccessful. There was 2 attempt (I want to check that results are
> > the same)

I can see it not generating those invokeinterface calls, so my guess is
you are not testing what you think you are. For one thing you will have
to build the compiler, and then rebuild the distribution in its entirety
with the new compiler, if you want to be working with the relevant
bytecode.

> Why it call getClass on interface? looks bad

What was it that you thought I just fixed?

Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 3:11 PM
Thank you, for attempt to fix it Paul.

If I remove Debug.info strings - my application work like with scala
2.8.2

But I got old error as expected. I am sure that the only way is run
scala-test suit under android. (at least)
PS google team may be not interested at this at all (for example
because of high priority tasks)
I may help you or maybe someone else to automate testing android. All
my checks I did at standard SDK emulator under gentoo linux x64.
There is no any problems to write few scripts that allow automate
process.
IMHO The problem is to find someone that would clear bugs from scala
code. If there someone from scala development team I will help him
with great pleasure.

That's why it look like
> PS “As of December 2011 there are over 700,000 Android devices activated
> every day.” It is very disappoint that scala developers ignore such part of
> market and build compiler in unportable way, that allow to use cross
> platform language only at limited number of targets.

D/dalvikvm(  243): GC_FOR_MALLOC freed 5222 objects / 401392 bytes in
39ms
D/o.d.d.i.s*.s*.FilterAc*(  243): onCreate(...)
E/dalvikvm(  243): LinearAlloc exceeded capacity (5242880), last=104
E/dalvikvm(  243): VM aborting
I/DEBUG   (   31): *** *** *** *** *** *** *** *** *** *** *** *** ***
*** *** ***
I/DEBUG   (   31): Build fingerprint: 'generic/sdk/generic/:2.2/
FRF91/43546:eng/test-keys'


Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/5/12 3:40 PM
> I can see it not generating those invokeinterface calls, so my guess is
> you are not testing what you think you are. For one thing you will have
> to build the compiler, and then rebuild the distribution in its entirety
> with the new compiler, if you want to be working with the relevant
> bytecode.

Seems so, Paul
Let me ask, latest git copy build properly at your environment if you
replace lib/scala* with latest?
Maybe you are right, but when I replace lib/scala*.jar with files from
dist, I got

init:
     [echo]        build time: 6 January 2012, 03:34:36
     [echo]      java version: OpenJDK 64-Bit Server VM 1.6.0_22
     [echo]         java args: -Xms1536M -Xmx1536M -Xss1M -
XX:MaxPermSize=192M -XX:+UseParallelGC
     [echo]        javac args:
     [echo]       scalac args:
     [echo]      build number: 2.10.0.rdev-4118-2012-01-06-gbe46e48
    [mkdir] Created dir: /home/ezh/0projectsAndroid/junk/scala/lib/
extra

locker.start:

locker.pre-lib:

locker.lib:
    [mkdir] Created dir: /home/ezh/0projectsAndroid/junk/scala/build/
locker/classes/library
    [javac] Compiling 25 source files to /home/ezh/0projectsAndroid/
junk/scala/build/locker/classes/library
[scalacfork] Compiling 687 files to /home/ezh/0projectsAndroid/junk/
scala/build/locker/classes/library
[scalacfork] /home/ezh/0projectsAndroid/junk/scala/src/library/scala/
collection/parallel/Tasks.scala:15: error: object forkjoin is not a
member of package concurrent
[scalacfork] import scala.concurrent.forkjoin._
[scalacfork]                         ^
[scalacfork] /home/ezh/0projectsAndroid/junk/scala/src/library/scala/
collection/parallel/Tasks.scala:459: error: value fork is not a member
of ForkJoinTasks.this.TaskImpl[R,Tp]
[scalacfork]       fjtask.fork
[scalacfork]              ^
[scalacfork] /home/ezh/0projectsAndroid/junk/scala/src/library/scala/
collection/parallel/Tasks.scala:482: error: value fork is not a member
of ForkJoinTasks.this.TaskImpl[R,Tp]
[scalacfork]       fjtask.fork
[scalacfork]              ^
[scalacfork] three errors found

BUILD FAILED
/home/ezh/0projectsAndroid/junk/scala/build.xml:341:
java.lang.RuntimeException: Compilation failed because of an internal
compiler error; see the error output for details.
        at scala.sys.package$.error(package.scala:27)
        at scala.tools.ant.sabbus.ScalacFork.execute(ScalacFork.scala:153)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:
291)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:
106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:390)
        at org.apache.tools.ant.Target.performTasks(Target.java:411)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:
1397)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
        at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:
41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
        at org.apache.tools.ant.Main.runBuild(Main.java:801)
        at org.apache.tools.ant.Main.startAnt(Main.java:218)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

Total time: 23 seconds


On 6 янв, 02:52, Paul Phillips <pa...@improving.org> wrote:
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 3:51 PM


On Thu, Jan 5, 2012 at 3:40 PM, Alexey Aksenov <alexe...@gmail.com> wrote:
Let me ask, latest git copy build properly at your environment if you
replace lib/scala* with latest?

No, that's not how to do it.

  ant all.clean build

(I'm not sure of the state of the sbt build, so ant is the safe way.)

Then use the jars in ./build/pack/lib, which will have been built by a compiler built from the contents of src/.

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/5/12 7:13 PM


On Thu, Jan 5, 2012 at 3:11 PM, Alexey Aksenov <alexe...@gmail.com> wrote:
IMHO The problem is to find someone that would clear bugs from scala
code. If there someone from scala development team I will help him
with great pleasure.

You can't say the raised issues haven't been fixed pretty speedily thus far.  (I'm assuming you will eventually find I did indeed fix the invokeinterface issue.) If you want to create an automated test rig for scala on android, I think it's a lovely idea - I'll help as much as I can, which isn't much.  If you can find someone else, all the better, but someone elses are hard to come by. I think I'm pretty good at prioritizing issues raised by people who are moving the ball a little ways on their own, so give it a shot.  What's the worst that can happen.

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 1/9/12 2:42 AM
Hi Paul,

I finally found time to do another proper test and it seems that at
least the example code from SI-4620 now works with the latest snapshot
(scala-compiler-2.10.0-20120105.025731-245.jar).

I checked the resource consumption with my logging dalvikvm and the
test app now seems to use only 4MB of LinearAlloc memory which is
under the current limit of 5MB. In newer versions of Android (> 4)
this limit has been increased to 8 MB so there should be some buffer
space for the next time.

I assume it was your fix which decreased resource consumption to this
level. So thanks again!

Johannes

Re: Scala 2.9.x's problems with the DalvikVM Nathan Moos 1/11/12 5:27 AM
Is LinearAlloc per-classloader or per-application? Pardon my
ignorance ...

Nathan Moos

On Jan 9, 3:42 am, Johannes Rudolph <johannes.rudo...@googlemail.com>
wrote:
> Hi Paul,
>
> I finally found time to do another proper test and it seems that at
> least the example code from SI-4620 now works with the latest snapshot
> (scala-compiler-2.10.0-20120105.025731-245.jar).
>
> I checked the resource consumption with my logging dalvikvm and the
> test app now seems to use only 4MB of LinearAlloc memory which is
> under the current limit of 5MB. In newer versions of Android (> 4)
> this limit has been increased to 8 MB so there should be some buffer
> space for the next time.
>
> I assume it was your fix which decreased resource consumption to this
> level. So thanks again!
>
> Johannes
>
> On Sat, Dec 17, 2011 at 5:51 PM, Johannes Rudolph
>
>
>
>
>
> <johannes.rudo...@googlemail.com> wrote:
> > BTW, thanks for the effort here, it's much appreciated. I did one
> > superficial test with a SNAPSHOT version last week, and unfortunately
> > it still isn't working. I had no time yet to rerun with my customized
> > dalvik version to check how much things have improved exactly and
> > what's still missing. (Not every day is the right day to get your
> > mobile phone out of the continuous reboot loop caused by a broken
> > dalvikvm...)
>
> > On Mon, Dec 5, 2011 at 5:20 PM, Paul Phillips <pa...@improving.org> wrote:
> >> On Mon, Dec 5, 2011 at 5:22 AM, Johannes Rudolph
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 1/11/12 5:33 AM
It's per application, or more exactly per VM, see [1]. No need to
apologize, I only found out by reading the sources as well.

Here's the explaining comment from the sources:
/*
 * Someday, retrieve the linear alloc struct associated with a particular
 * class loader.  For now, always use the boostrap loader's instance.
 */

[1] https://github.com/android/platform_dalvik/blob/master/vm/LinearAlloc.cpp#L87

Re: Scala 2.9.x's problems with the DalvikVM Nathan Moos 1/11/12 11:11 AM
It was mentioned earlier on this forum that a Google representative
said that multiple DEX files would solve the problem, is that invalid?

Nathan Moos

On Jan 11, 6:33 am, Johannes Rudolph <johannes.rudo...@googlemail.com>
wrote:
> It's per application, or more exactly per VM, see [1]. No need to
> apologize, I only found out by reading the sources as well.
>
> Here's the explaining comment from the sources:
> /*
>  * Someday, retrieve the linear alloc struct associated with a particular
>  * class loader.  For now, always use the boostrap loader's instance.
>  */
>
> [1]https://github.com/android/platform_dalvik/blob/master/vm/LinearAlloc...
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 1/11/12 10:56 PM
I think we have at least three different issues here:

1.) The dex-file binary format can only handle 65k methods. This is
what could be handled by multiple dex files. This is currently worked
around by using proguard or by pre-installing the Scala libraries to
the developer phone.

http://code.google.com/p/android/issues/detail?id=20814

2.) A deep interface hierarchy can lead to excessive LinearAlloc use
in dalvik vm. Scala's collection library has quite a "deep interface
hierarchy" and triggered this problem. The situation will be improved
on the Scala side by omitting redundant interface declarations (Scala
trunk version is already much better in this regard). Maybe this will
improved on the Dalvik side if the proposed fix will be accepted.

http://code.google.com/p/android/issues/detail?id=22586

3.) The problem Alexey reported earlier in this thread where in a
former Scala 2.10-SNAPSHOT version an `invokeinterface` bytecode is
generated for calls to methods defined in `java.lang.Object` for some
cases
which works in Hotspot but fails in Dalvik. This is now fixed in Scala.

Johannes

Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/14/12 9:41 AM
IMHO I find one more problem place.

I/dalvikvm(  618): Could not find method
sun.misc.Unsafe.throwException, referenced from method
scala.concurrent.forkjoin.ForkJoinTask.rethrowException
W/dalvikvm(  618): VFY: unable to resolve virtual method 26641: Lsun/
misc/Unsafe;.throwException (Ljava/lang/Throwable;)V
D/dalvikvm(  618): VFY: replacing opcode 0x6e at 0x0004

Has dalvik proper support of sun.misc.Unsafe?

Best regards,
Alexey
Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Paul Phillips 1/14/12 10:36 AM


On Sat, Jan 14, 2012 at 9:41 AM, Alexey Aksenov <alexe...@gmail.com> wrote:
IMHO I find one more problem place.

I/dalvikvm(  618): Could not find method
sun.misc.Unsafe.throwException, referenced from method
scala.concurrent.forkjoin.ForkJoinTask.rethrowException
W/dalvikvm(  618): VFY: unable to resolve virtual method 26641: Lsun/
misc/Unsafe;.throwException (Ljava/lang/Throwable;)V
D/dalvikvm(  618): VFY: replacing opcode 0x6e at 0x0004

Try setting system property actors.enableForkJoin=false .  (Not suggesting as a long term answer.)

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 1/16/12 5:25 AM
There's another possible solution:

https://github.com/jrudolph/scala-android-libs/issues/3

Can you try that and report back?

Johannes

--

Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: [scala-debate] Re: Scala 2.9.x's problems with the DalvikVM Johannes Rudolph 1/17/12 12:58 AM
On Sat, Jan 14, 2012 at 7:36 PM, Paul Phillips <pa...@improving.org>
wrote:> On Sat, Jan 14, 2012 at 9:41 AM, Alexey Aksenov
<alexe...@gmail.com>>> IMHO I find one more problem place.>>>>

I/dalvikvm(  618): Could not find method>>
sun.misc.Unsafe.throwException, referenced from method>>
scala.concurrent.forkjoin.ForkJoinTask.rethrowException>> W/dalvikvm(
618): VFY: unable to resolve virtual method 26641: Lsun/>>
misc/Unsafe;.throwException (Ljava/lang/Throwable;)V>> D/dalvikvm(
618): VFY: replacing opcode 0x6e at 0x0004>>> Try setting system
property actors.enableForkJoin=false .  (Not suggesting> as a long
term answer.)

Did you test this solution, Alexey?

I don't think it will work since it looks like this being a
verification problem. Apache Harmony seems to be missing
Unsafe.throwException altogether. If setting the property alone isn't
working can you try removing the complete "scala.concurrent.forkjoin"
package from your Scala jar?


--
Johannes

-----------------------------------------------
Johannes Rudolph
http://virtual-void.net

Re: Scala 2.9.x's problems with the DalvikVM Alexey Aksenov 1/17/12 4:44 AM
Hello, Johannes

setting system property actors.enableForkJoin=false inappropriate. I
tried different combinations.

For example application activity singleton (that loading first) as
simple as:

package org.digimead.digi.inetd.sshd

import org.slf4j.LoggerFactory

class SSHDApplication extends android.app.Application {
  protected val log =
LoggerFactory.getLogger(getClass.getName().replaceFirst("org.digimead.digi.inetd",
"o.d.d.i"))
  System.setProperty("actors.enableForkJoin", "false")
  log.debug("alive")
}

logcat:

I/ActivityManager(   78): Start proc org.digimead.digi.inetd.sshd for
activity org.digimead.digi.inetd.sshd/.SSHDActivity: pid=621 uid=10040
gids={3003}
I/dalvikvm(  621): Turning on JNI app bug workarounds for target SDK
version 7...
D/dalvikvm(   37): GC_EXPLICIT freed 37K, 4% free 9030K/9347K, paused
2ms+17ms
W/NetworkManagementSocketTagger(   78): setKernelCountSet(10040, 1)
failed with errno -2
D/dalvikvm(   37): GC_EXPLICIT freed <1K, 4% free 9031K/9347K, paused
2ms+78ms
I/AndroidLoggerFactory(  621): Logger name
'o.d.d.i.sshd.SSHDApplication' exceeds maximum length of 23
characters, using 'o.d.d.i.s*.SSHDApplica*' instead.

<-- set actors.enableForkJoin HERE
D/o.d.d.i.s*.SSHDApplica*(  621): alive
D/dalvikvm(   37): GC_EXPLICIT freed <1K, 4% free 9031K/9347K, paused
2ms+3ms
I/AndroidLoggerFactory(  621): Logger name 'o.d.d.i.sshd.SSHDActivity'
exceeds maximum length of 23 characters, using
'o.d.d.i.s*.SSHDActivity' instead.
D/dalvikvm(  621): GC_CONCURRENT freed 331K, 5% free 9161K/9607K,
paused 3ms+3ms
D/dalvikvm(  155): GC_CONCURRENT freed 384K, 6% free 9529K/10119K,
paused 3ms+3ms
<-- set actors.enableForkJoin HERE too
D/o.d.d.i.s*.SSHDActivity(  621): alive
D/o.d.d.i.s*.SSHDActivity(  621): onCreate(...)
I/dalvikvm(  621): Could not find method
sun.misc.Unsafe.throwException, referenced from method
scala.concurrent.forkjoin.ForkJoinTask.rethrowException
W/dalvikvm(  621): VFY: unable to resolve virtual method 26641: Lsun/
misc/Unsafe;.throwException (Ljava/lang/Throwable;)V
D/dalvikvm(  621): VFY: replacing opcode 0x6e at 0x0004
D/dalvikvm(  621): GC_CONCURRENT freed 295K, 5% free 9339K/9735K,
paused 3ms+3ms

> I don't think it will work since it looks like this being a
> verification problem

looks like this

> can you try removing the complete "scala.concurrent.forkjoin"
package from your Scala jar?

I am sorry. Too busy for a few days. Maybe a bit later, I'll do it
ASAP

> https://github.com/jrudolph/scala-android-libs/issues/3

This is different situation. Actually actors work. I just said that
there might be a problem.
Also I am not loading libraries in emulator at boot time. Scala
libraries loading when application launched. This way avoid crush with
dxopt at boot time with scala 2.9.1 for example.

On Jan 17, 12:58 pm, Johannes Rudolph
<johannes.rudo...@googlemail.com> wrote:
> On Sat, Jan 14, 2012 at 7:36 PM, Paul Phillips <pa...@improving.org>
> wrote:> On Sat, Jan 14, 2012 at 9:41 AM, Alexey Aksenov
> <alexey....@gmail.com>>> IMHO I find one more problem place.>>>>
More topics »