How to load a class file to bytes with a class loader and make an instance of it?

313 views
Skip to first unread message

Dave

unread,
Apr 17, 2012, 8:21:22 PM4/17/12
to scala-user
I wonder why I get "java.lang.ClassCastException: Foo cannot be cast
to Foo" in the line
with val foo =
clsloader.findClass("Foo").newInstance.asInstanceOf[Foo] when I do

val b = loadClassData(name)
defineClass(name, b, 0, b.length).asInstanceOf[Class[Foo]]

but it works (printing M) when I do

Class.forName("Foo").asInstanceOf[Class[Foo]]


Am I missing some triviality or is it a bug?


foo.scala
=========
class Foo { def m() = "M"}


main.scala
==========
class FileClassLoader extends ClassLoader {
override def findClass(name: String) = {
val b = loadClassData(name)
defineClass(name, b, 0, b.length).asInstanceOf[Class[Foo]] //
java.lang.ClassCastException: Foo cannot be cast to Foo

//with Class.forName works
//Class.forName("Foo").asInstanceOf[Class[Foo]]
}

def loadClassData(name: String): Array[Byte] = {
val is = new java.io.FileInputStream(name + ".class")
val bis = new java.io.BufferedInputStream(is)
val b = Stream.continually(bis.read).takeWhile(-1 !
=).map(_.toByte).toArray
//println(b.length)
//println(b.mkString(","))
b
}
}
object Main {
def main(args: Array[String]) {
val clsloader = new FileClassLoader
val foo = clsloader.findClass("Foo").newInstance.asInstanceOf[Foo]
println(foo.m) // prints M
}
}


C:\scala-2.10.0-M2\footest>scala Main
java.lang.ClassCastException: Foo cannot be cast to Foo
at Main$.main(main.scala:47)
at Main.main(main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at scala.tools.nsc.util.ScalaClassLoader$$anonfun$run
$1.apply(ScalaClassLoader.scala:90)
at scala.tools.nsc.util.ScalaClassLoader
$class.asContext(ScalaClassLoader.scala:38)
at scala.tools.nsc.util.ScalaClassLoader
$URLClassLoader.asContext(ScalaClassLoader.scala:159)
at scala.tools.nsc.util.ScalaClassLoader
$class.run(ScalaClassLoader.scala:90)
at scala.tools.nsc.util.ScalaClassLoader
$URLClassLoader.run(ScalaClassLoader.scala:159)
at scala.tools.nsc.CommonRunner$class.run(ObjectRunner.scala:
28)
at scala.tools.nsc.ObjectRunner$.run(ObjectRunner.scala:45)
at scala.tools.nsc.CommonRunner
$class.runAndCatch(ObjectRunner.scala:35)

at scala.tools.nsc.ObjectRunner
$.runAndCatch(ObjectRunner.scala:45)
at scala.tools.nsc.MainGenericRunner.runTarget
$1(MainGenericRunner.scala:70)
at
scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:92)

at scala.tools.nsc.MainGenericRunner
$.main(MainGenericRunner.scala:101)
at
scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

Som Snytt

unread,
Apr 18, 2012, 4:54:26 AM4/18/12
to Dave, scala-user
On Tue, Apr 17, 2012 at 5:21 PM, Dave <dave.mah...@hotmail.com> wrote:
I wonder why I get "java.lang.ClassCastException: Foo cannot be cast
to Foo" in the line
[snip]

Am I missing some triviality or is it a bug?


The non-triviality is the difference between the Scala classloader and the custom classloader.

Compare classOf[Foo].getClassLoader.

This works:
java -classpath ".:scala-library.jar" Main

 

Dave Mahabiersing

unread,
Apr 18, 2012, 6:04:44 AM4/18/12
to scala...@googlegroups.com
Okay thanks Jan and Som.
 

 

Date: Wed, 18 Apr 2012 09:57:16 +0200
Subject: Re: [scala-user] How to load a class file to bytes with a class loader and make an instance of it?
From:
To:

Hi Dave,

when 2 class-loaders define/load the same class file the resulting 2 classes are considered different by the JVM.

The class Foo referenced in you code (say class Main) e.g. in asInstanceOf[Foo] is loaded by the class-loader which loaded the class Main. The second Foo is the one defined by the FileClassLoader. Those 2 are unrelated from the JVM point of view.

Regards,
Jan

Dave

unread,
Apr 18, 2012, 6:13:42 AM4/18/12
to scala-user

>
> This works:
> java -classpath ".:scala-library.jar" Main

For me not. It's the same:

C:\scala-2.10.0-M2\footest>java -classpath ".;..\lib\scala-
library.jar" Main
Exception in thread "main" java.lang.ClassCastException: Foo cannot be
cast to F
oo
at Main$.main(main.scala:47)
at Main.main(main.scala)

Maybe you still had the Class.forName("Foo").asInstanceOf[Class[Foo]]
class files?

Dave

unread,
Apr 18, 2012, 6:24:41 AM4/18/12
to scala-user
Another problem I had is renaming the file Foo.class to Foo2.class in
which the class Foo is defined.

The class loader doesn't like it that the filename has another name
than the class name.

My main plan is to merge a second class in a byte array so that is one
class Foo

foo2.scala
==========
class Foo { def n() = "N"}

this compiles too Foo.class as well
rename Foo.class to Foo2.class

and load Foo.class and Foo2.class with a class loader into a byte
array and create a class Foo.

Then I would have binary partial classes which I could pack in
seperate jar files and load them as one class in to memory.

But there are obviously some non-trivial obstacles due to the jvm.

Rex Kerr

unread,
Apr 18, 2012, 8:35:04 AM4/18/12
to Dave, scala-user
Try giving the full path, not the relative path.  Sometimes Java doesn't see the relative path the way you think it ought to (and it doesn't complain if it can't find a jar that you specify).  (Still not sure it will work for you, but there could be a very simple reason why what you're trying doesn't show any improvement: java isn't actually finding the jar!)

  --Rex

Dave

unread,
Apr 18, 2012, 8:51:35 AM4/18/12
to scala-user
No it is not working
C:\scala-2.10.0-M2\footest>java -classpath ".;C:\scala-2.10.0-M2\lib
\scala-libra
ry.jar" Main
Exception in thread "main" java.lang.ClassCastException: Foo cannot be
cast to F
oo
at Main$.main(main.scala:47)
at Main.main(main.scala)

But the finding and loading the class was never the problem (it finds
the class and loads every byte), but the casting to Foo ("Foo cannot
be cast to Foo").
> > class files?- Tekst uit oorspronkelijk bericht niet weergeven -
>
> - Tekst uit oorspronkelijk bericht weergeven -

Dave

unread,
Apr 18, 2012, 3:58:42 PM4/18/12
to scala-user
If I change line:

val foo = clsloader.findClass("Foo").newInstance.asInstanceOf[Foo]

into

val foo = clsloader.loadClass("Foo").newInstance.asInstanceOf[Foo]

and run with:

C:\scala-2.10.0-M2\footest>java -cp ".;..\lib\scala-library.jar" Main
M

then it works.

So it better not to run with scala Main

On 18 apr, 10:54, Som Snytt <som.sn...@gmail.com> wrote:

Dave

unread,
Apr 18, 2012, 4:07:39 PM4/18/12
to scala-user
Ahum
It works because findClass isn't called ...
> > java -classpath ".:scala-library.jar" Main- Tekst uit oorspronkelijk bericht niet weergeven -

Dave

unread,
Apr 18, 2012, 5:31:29 PM4/18/12
to scala-user
I know what the cause is:
It tries to look for the file scala.ScalaObject.class inside
defineClass which raises an java.io.FileNotFoundException.
Probably it has to be scala/ScalaObject.class


main.scala
==========
class FileClassLoader extends ClassLoader {
override def loadClass(name: String) : Class[_] = loadClass(name,
false)

override def loadClass(name: String, resolve: Boolean) : Class[_] =
{
val b = loadClassData(name)
var cls : Class[_] = null
try {
cls = defineClass(name, b, 0, b.length)
} catch {
case (ioe : java.io.IOException) => println(ioe)
}
if(cls==null) {
println("Then try to load " + name + " the normal way")
cls = findSystemClass(name)
}
if(resolve && cls!=null)
println("Resolve " + name)
resolveClass(cls)
cls
}

def hex(buf: Array[Byte]): String = buf.map("%02x" format
_).mkString

def loadClassData(name: String): Array[Byte] = {
val is = new java.io.FileInputStream(name + ".class")
val bis = new java.io.BufferedInputStream(is)
val b = Stream.continually(bis.read).takeWhile(-1 !
=).map(_.toByte).toArray
println(b.length)
println(hex(b))
b
}
}

object Main {
def main(args: Array[String]) {
val clsloader = new FileClassLoader
val foo = clsloader.loadClass("Foo").newInstance.asInstanceOf[Foo]
println(foo.m) // prints M
}
}


Output
======
C:\scala-2.10.0-M2\footest>java -cp ".;..\lib\scala-library.jar" Main
572
cafebabe00000031001801000a536f7572636546696c65010009666f6f2e7363616c610100016d01
001428294c6a6176612f6c616e672f537472696e673b010004436f64650100014d08000601000f4c
696e654e756d6265725461626c650100063c696e69743e0100032829560100106a6176612f6c616e
672f4f626a65637407000b0c0009000a0a000c000d0100085363616c6153696701001e4c7363616c
612f7265666c6563742f5363616c615369676e61747572653b01000562797465730100b606017d31
41210102010b091961695c380b030d09710150336e61524c68680101140701316102050208193509
014203020a150521412e19386815055911010236626d064c21210405030d3d13272e5a3275210979
21234401111505091221423a6443320c1742410a11052d19362d1937621f0a54576d193b090b5501
4111010c020d714a672e1b3b3f2905390243010d011b05110122020e010903591221413715037101
2261420f0a0579412141422a75652674770d01001952756e74696d6556697369626c65416e6e6f74
6174696f6e73010003466f6f0700140100117363616c612f5363616c614f626a6563740700160021
0015000c0001001700000002000100030004000100050000001b00010001000000031207b0000000
0100080000000600010000000100010009000a000100050000001d00010001000000052ab7000eb1
0000000100080000000600010000000100030001000000020002000f000000030500000013000000
0b0001001000010011730012
java.io.FileNotFoundException: scala.ScalaObject.class (Het systeem
kan het opge
geven bestand niet vinden)
Then try to load Foo the normal way
M
> > - Tekst uit oorspronkelijk bericht weergeven -- Tekst uit oorspronkelijk bericht niet weergeven -

Som Snytt

unread,
Apr 18, 2012, 8:11:42 PM4/18/12
to Dave, scala-user
On Wed, Apr 18, 2012 at 5:51 AM, Dave <dave.mah...@hotmail.com> wrote:
No it is not working

Sorry I misled you; it was my midnight bungle; that was an intermediate result that was wrong.

As penance, here is what I wound up with.  As you can see, ScalaClassLoader is not magical, qua classloader, but it (SCL) is instructive in other ways.  To recap, this test gets Foo loaded once by your FileClassLoader.  As you pointed out, your custom findClass is called (only) if the parent.loadClass comes up empty.

usage: scalac loadclass.scala ; mv loadclass/Foo.class loadclass/Testee.class hidden/loadclass ; scala loadclass.Test

package loadclass

// put me in "hidden"

class Foo { def m() = "M"}

// put me in "hidden"
class Testee {
  def test() = println(getClass.getClassLoader.loadClass("loadclass.Foo").newInstance.asInstanceOf[Foo].m)
}

import tools.nsc.util.ScalaClassLoader
import java.io._

class FileClassLoader(f: File, p: ClassLoader) extends ClassLoader(p) with ScalaClassLoader {
  override protected def trace: Boolean = true

  override def findClass(name: String) = {
    val result = customFindClass(name)
    classLoaderLog("findClass(%s) = %s".format(name, result))
    result
  }
  private def customFindClass(name: String) = {

    val b = loadClassData(name)
    defineClass(name, b, 0, b.length)
  }
  private def loadClassData(name: String): Array[Byte] = {
    val what = new File(f, name.replace('.','/') +  ".class")
    if (!what.exists) throw new ClassNotFoundException(name)
    val is = new java.io.FileInputStream(what)

    val bis = new java.io.BufferedInputStream(is)
    val b = Stream.continually(bis.read).takeWhile(-1 !=).map(_.toByte).toArray
    //println(b.length)
    //println(b.mkString(","))
    b
  }
}

//scalac loadclass.scala ; mv loadclass/Foo.class loadclass/Testee.class hidden/loadclass ; scala loadclass.Test
object Test {

  def main(args: Array[String]) {
    val clsloader = new FileClassLoader(new File("hidden"), getClass.getClassLoader)
    val cls = clsloader.loadClass("loadclass.Testee")
    val sut = cls.newInstance
    cls.getDeclaredMethod("test").invoke(sut)
  }
}




Dave

unread,
Apr 19, 2012, 7:15:27 AM4/19/12
to scala-user
Thanks!
With your example I begin a little bit to understand how the
classloader works.
And I didn't know about ScalaClassLoader.

The basic ingredients of the recipe are:
1. Create a custom classloader . Override the findClass method in the
custom classloader with the custom implementation.
2. Instantiate this custom classloader and associate a folder (e.g.
"hidden" with an on call site created classloader (e.g.
getClass.getClassLoader). Somehow the current folder "." doesn't work.
3. Use a proxy class (e.g. classload.Testee) to call the real target
class (e.g. loadclass.Foo). The proxy and target must reside in a
package (e.g. loadclass). Somehow packageless and directly calling the
target class doesn't work.
4. The proxy and the target class has to be moved in the classloader
associated folder (that is "hidden")
5. Then you can call loadclass which in turn calls custom classloader
findClass



main.scala
==========
package loadclass
import tools.nsc.util.ScalaClassLoader
import java.io._

class Foo { def m() = "M"}

class Testee {
def test() =
println(getClass.getClassLoader.loadClass("loadclass.Foo").newInstance.asInstanceOf[Foo].m)
}

class FileClassLoader(f: File, p: ClassLoader) extends ClassLoader(p)
with ScalaClassLoader {
override protected def trace: Boolean = true

override def findClass(name: String) = {
val b = loadClassData(name)
val cls = defineClass(name, b, 0, b.length)
cls
}

def hex(buf: Array[Byte]): String = buf.map("%02x" format
_).mkString

def loadClassData(name: String): Array[Byte] = {
val what = new File(f, name.replace('.','/') + ".class")
if (!what.exists) throw new ClassNotFoundException(name)
val is = new java.io.FileInputStream(what)
val bis = new java.io.BufferedInputStream(is)
val b = Stream.continually(bis.read).takeWhile(-1 !
=).map(_.toByte).toArray
println(b.length)
println(hex(b))
b
}
}

object Main {
def main(args: Array[String]) {
val clsloader = new FileClassLoader(new
File("hidden"),getClass.getClassLoader)
val cls = clsloader.loadClass("loadclass.Testee")
val sut = cls.newInstance
cls.getDeclaredMethod("test").invoke(sut)
}
}



C:\scala-2.10.0-M2\footest>scala loadclass.Main
1022
cafebabe00000031003b01000a536f7572636546696c6501000a6d61696e2e7363616c6101000474
657374010003282956010004436f646501000d7363616c612f507265646566240700060100074d4f
44554c452401000f4c7363616c612f507265646566243b0c00080009090007000a01000f4c696e65
4e756d6265725461626c650100106a6176612f6c616e672f4f626a65637407000d01000867657443
6c61737301001328294c6a6176612f6c616e672f436c6173733b0c000f00100a000e001101000f6a
6176612f6c616e672f436c61737307001301000e676574436c6173734c6f6164657201001928294c
6a6176612f6c616e672f436c6173734c6f616465723b0c001500160a0014001701000d6c6f616463
6c6173732e466f6f0800190100156a6176612f6c616e672f436c6173734c6f6164657207001b0100
096c6f6164436c617373010025284c6a6176612f6c616e672f537472696e673b294c6a6176612f6c
616e672f436c6173733b0c001d001e0a001c001f01000b6e6577496e7374616e636501001428294c
6a6176612f6c616e672f4f626a6563743b0c002100220a0014002301000d6c6f6164636c6173732f
466f6f0700250100016d01001428294c6a6176612f6c616e672f537472696e673b0c002700280a00
2600290100077072696e746c6e010015284c6a6176612f6c616e672f4f626a6563743b29560c002b
002c0a0007002d0100063c696e69743e0c002f00040a000e00300100085363616c6153696701001e
4c7363616c612f7265666c6563742f5363616c615369676e61747572653b01000562797465730100
bd06017d3141210102010b0931412b5a3a754b1654116141010a593e0c476d193762674e1c016145
02010d3901226102070e032151212103060209310c676e1a0602170521212e193c62130969014201
0450452a2c370d1e09031f49691101050602230529316f59316d432611310305020c270e0c472e59
286355161c470f430316011105612301043d533a4c474f100b022f41110104410702052129210401
4301370521412f5a3a75290561024341081e13097102430103565d26240801001952756e74696d65
56697369626c65416e6e6f746174696f6e730100106c6f6164636c6173732f546573746565070037
0100117363616c612f5363616c614f626a65637407003900210038000e0001003a00000002000100
0300040001000500000034000300010000001cb2000b2ab60012b60018121ab60020b60024c00026
b6002ab6002eb100000001000c000000060001000000080001002f0004000100050000001d000100
01000000052ab70031b100000001000c000000060001000000070003000100000002000200320000
000305000000360000000b0001003300010034730035
[Cl#17933220] loadClass(scala.ScalaObject, false) = interface
scala.ScalaObject
[Cl#17933220] loadClass(java.lang.Object, false) = class
java.lang.Object
[Cl#17933220] loadClass(loadclass.Testee, false) = class
loadclass.Testee
[Cl#17933220] loadClass(scala.Predef$, false) = class scala.Predef$
[Cl#17933220] loadClass(java.lang.Class, false) = class
java.lang.Class
[Cl#17933220] loadClass(java.lang.ClassLoader, false) = class
java.lang.ClassLoa
der
585
cafebabe00000031001801000a536f7572636546696c6501000a6d61696e2e7363616c610100016d
01001428294c6a6176612f6c616e672f537472696e673b010004436f64650100014d08000601000f
4c696e654e756d6265725461626c650100063c696e69743e0100032829560100106a6176612f6c61
6e672f4f626a65637407000b0c0009000a0a000c000d0100085363616c6153696701001e4c736361
6c612f7265666c6563742f5363616c615369676e61747572653b01000562797465730100b806017d
3141210102010b091961695c380b030d0911025c3862490e64176d5d3a04014d190141020811051d
61512201050b05255111010237625d1e541161430105550634182d03020e110931714a1936664752
042261040a0e0341511121450106670e0c472e590503274111316255326259067c254d5b33646922
2951030143012d053141281b386a697a221261060903310169114101050635012109614701025b52
0941040502083b25116104030207275214184e5c3401001952756e74696d6556697369626c65416e
6e6f746174696f6e7301000d6c6f6164636c6173732f466f6f0700140100117363616c612f536361
6c614f626a65637407001600210015000c0001001700000002000100030004000100050000001b00
010001000000031207b00000000100080000000600010000000500010009000a000100050000001d
00010001000000052ab7000eb1000000010008000000060001000000050003000100000002000200
0f0000000305000000130000000b0001001000010011730012
[Cl#17933220] loadClass(loadclass.Foo, false) = class loadclass.Foo
M


On 19 apr, 02:11, Som Snytt <som.sn...@gmail.com> wrote:
> On Wed, Apr 18, 2012 at 5:51 AM, Dave <dave.mahabiers...@hotmail.com> wrote:
> > No it is not working
>
> Sorry I misled you; it was my midnight bungle; that was an intermediate
> result that was wrong.
>
> As penance, here is what I wound up with.  As you can see, ScalaClassLoader
> is not magical, qua classloader, but it (SCL) is instructive in other
> ways.  To recap, this test gets Foo loaded once by your FileClassLoader.
> As you pointed out, your custom findClass is called (only) if the
> parent.loadClass comes up empty.
>
> usage: scalac loadclass.scala ; mv loadclass/Foo.class
> loadclass/Testee.class hidden/loadclass ; scala loadclass.Test
>
> package loadclass
>
> // put me in "hidden"
> class Foo { def m() = "M"}
>
> // put me in "hidden"
> class Testee {
>   def test() =
> println(getClass.getClassLoader.loadClass("loadclass.Foo").newInstance.asIn­stanceOf[Foo].m)

Dave Mahabiersing

unread,
Apr 19, 2012, 8:40:36 AM4/19/12
to scala...@googlegroups.com
>Hi Dave,
>2 remarks. You can use URLClassLoader instead of your FileClassLoader, it provides what you need.
>Second, it seems you are trying to somehow dynamically enhance the class Foo.
 
Actually it is the opposite:
I like to dynamically and binary shrink the class Foo to relax lifecycle removal of classes and at the same time maintain the binary backward compatibility with the direct previous version.
 
Lifecycle step 1 (start):
class Foo {
  def m() = "M"
  def n() = "N"
}
 
Lifecycle step 2 (source level/compile time deprecation with @deprecated warning):
class Foo {
def m() = "M"
@deprecated def n() = "N"
}
 
Lifecycle step 3 (object level deprecation maybe with keyword deprecated, classed are in different source files, binary classes are packed in different jars):
class Foo { def m() = "M"} // goes into the main jar
deprecated class Foo { @deprecated def n() = "N"} //goes into delta jar
Lifecycle step 4 final removal:
class Foo { def m() = "M"}
 
 
So what is still missing is class overloading (or partial classes) and a way to cast (i.e. asInstanceOf[Foo]) it to the aggregate of classes of Foo.
I think with mixin traits it is not possible because it changes the signature.
 
 
>You could define a trait Enhance which is defined close to class Foo (and loaded by the same class loader), and potentially class Foo could have a mutable reference to the >Enhance. Then you could have implementations of the Enhance somewhere else (also loaded by different class-loader if needed). The advantage is obvious: when calling Enhance >methods from Foo you don't need reflection.

>Regards,
>Jan

Dave

unread,
Apr 19, 2012, 8:44:53 AM4/19/12
to scala-user
More readable
Reply all
Reply to author
Forward
0 new messages