Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

How to include an external (binary) file into a java program at compile (!) time ???

1,232 views
Skip to first unread message

Tobias Merler

unread,
Sep 30, 2004, 6:14:54 PM9/30/04
to
I have a java class and a second file which contains some binary data (assume a zip archive or a gif picture).

At the moment when I compile the class I would like to include this file into a variable of this class.
I think of a variable declaration like:

byte[] mybinaryfile = <include dummy.zip>

How can I implement this idea?
Remember again: I don't want to load this file at runtime but this file should be
internal part of the one and only java class.

Tobias

Ankur Bulsara

unread,
Sep 30, 2004, 8:07:11 PM9/30/04
to
One way to archieve resource bundling at compile time is to package the
resources in a jar. Then at runtime, you read in the resource using the
classloader.

Here is a file that gets a resource from the classloader:
http://www.koders.com/java/fid92BF1D1C6E5D4FF57B19FE8B65A49A98E0E77FF1.aspx?s=getresourceasstream

-Ankur


"Tobias Merler" <mer...@gmx.net> wrote in message
news:cji0gu$fpt$00$1...@news.t-online.com...

Dave Glasser

unread,
Sep 30, 2004, 11:47:39 PM9/30/04
to
mer...@gmx.net (Tobias Merler) wrote on Fri, 1 Oct 2004 00:14:54 +0200
in comp.lang.java.programmer:


The only way I can think of is to declare the file contents as an
array of bytes right in your Java class file, like this:

byte[] mybinary file = {0x65, 0x66, ... };

You could easily write a Java program (or Perl script or whatever)
that would read the binary file and generate the snippet of code for
that declaration.


--
Check out QueryForm, a free, open source, Java/Swing-based
front end for relational databases.

http://qform.sourceforge.net

If you're a musician, check out RPitch Relative Pitch
Ear Training Software.

http://rpitch.sourceforge.net

Jacob

unread,
Oct 1, 2004, 1:01:00 AM10/1/04
to
Tobias Merler wrote:

There is really no difference in what you are requesting (textually
import the content of the file into the .java file) and actually
load the file at runtime, other than the latter being more elegant
and better supported by the tools you have available.

If you insist on the former approach you will need to do a
preprocessing step (perl, m4, etc.) as Dave G. suggests.

Otherwise, put your binary file in the classpath (i.e. along
with your class files) and refer to it from you java source like this:

String fileName = "myfile.zip";
String packageName = getClass().getPackage().getName();
String packageLocation = packageName.replace ('.', '/');
String filePath = "/" + packageLocation + "/" + fileName;

InputStream stream = getClass().getResourceAsStream (filePath);

At this point you have a handle to your (yet unloaded) file.
You can then load it into a byte[] using the stream.read() methods.

Remember: By this approach the file is *not* accessed through the
client file system (which I guess is your concern), but from within
the application deployment unit.

Tor Iver Wilhelmsen

unread,
Oct 1, 2004, 3:04:20 AM10/1/04
to
Dave Glasser <dgla...@pobox.com> writes:

> byte[] mybinary file = {0x65, 0x66, ... };

This is immensely inefficient sompared to reading it as a resource
from the classpath. It's the same as

byte[] mybinaryFile = new byte[...];
mybinaryFile[0] = 0x65;
mybinaryFile[1] = 0x66;
...

Tor Iver Wilhelmsen

unread,
Oct 1, 2004, 3:07:26 AM10/1/04
to
mer...@gmx.net (Tobias Merler) writes:

> At the moment when I compile the class I would like to include this
> file into a variable of this class.

No, you really would not like that.

> I think of a variable declaration like:
>
> byte[] mybinaryfile = <include dummy.zip>
>
> How can I implement this idea?

You can use a preprocessor to make the Java source which you then
compile. Others have suggested Perl, which is perhaps reasonable.

> Remember again: I don't want to load this file at runtime but this
> file should be internal part of the one and only java class.

Well, I hope you know the file then cannot be larger than approx. 20
kilobytes - probably even less.

Michael Borgwardt

unread,
Oct 1, 2004, 3:55:43 AM10/1/04
to
Tor Iver Wilhelmsen wrote:

> Dave Glasser <dgla...@pobox.com> writes:
>
>
>>byte[] mybinary file = {0x65, 0x66, ... };
>
>
> This is immensely inefficient sompared to reading it as a resource
> from the classpath.

Most importantly, it will only work for pretty small files because of the 64k
bytecode limitation on method size.

Jean-Francois Briere

unread,
Oct 1, 2004, 5:03:45 AM10/1/04
to
You could download Jatha, which is a preprocessor that generates plain
JAVA sources from JAVA sources with macros.
You can find it in here: http://www.cybercom.net/~kimbly/jatha/

The idea is to transform a JAVA file like the follwing (note the
.jatha extension) into a plain JAVA file without any macro.

File IncludeTest.java.jatha:
----------------------------------------------------------------------
public class IncludeTest {
private static int[] mybinaryfile = @INCLUDE_STREAM(8, dummy.zip);

public static void main(String[] args) {
for (int i = 0; i < mybinaryfile.length; i++) {
System.out.print(mybinaryfile[i] + " ");
if ((i % 20) == 0)
System.out.println();
}
}
}
----------------------------------------------------------------------

In this file there is a macro I created: INCLUDE_STREAM, which is a
custom JAVA class
based on the Jatha library, and which does all the work on reading the
provided binary file
to write its content appropriately in the generated JAVA source:

File macros/INCLUDE_STREAM.java:
----------------------------------------------------------------------
package macros;

import java.io.*;
import jatha.Macro;
import jatha.Expander;

public class INCLUDE_STREAM extends Macro {
static final String LS = System.getProperty("line.separator");

public void expand(String[] args, Writer out, Expander expander) {
if (args.length != 2)
expander.error("INCLUDE_STREAM macro takes two arguments:
startColumn and filename.");

String startColumn = args[0];
String fileName = args[1];
BufferedInputStream bis = null;

try {
int col = Integer.parseInt(startColumn);
byte[] buffer = new byte[20];
int len;

bis = new BufferedInputStream(new
FileInputStream(fileName));
out.write("{" + LS);

while ((len = bis.read(buffer)) != -1) {
tab(out, col);

for (int i = 0; i < len; i++)
out.write(format(buffer[i]) + ", ");

out.write(LS);
}

tab(out, col);
out.write("}");
}
catch (NumberFormatException e) {
expander.error("The startColumn element (" + startColumn +
") is not an int.");
}
catch (IOException e) {
expander.error("IO error while reading file '" + fileName
+ "': " + e.getMessage());
}
finally {
try { bis.close(); }
catch (Exception e) {}
}
}

void tab(Writer out, int col) throws IOException {
for (int i = 0; i < col; i++)
out.write(" ");
}

String format(int n) {
String nStr = Integer.toHexString(n & 0xff);

if (nStr.length() == 1)
nStr = "0" + nStr;

return "0x" + nStr;
}
}
----------------------------------------------------------------------

You simply build this macro JAVA class,
then execute Jatha from the folder where the .jatha file is located:

> java jatha.Main

And this will convert IncludeTest.java.jatha into IncludeTest.java:

File IncludeTest.java:
----------------------------------------------------------------------
public class IncludeTest {
private static int[] mybinaryfile = {
0x50, 0x4b, 0x03, 0x04, 0x14, 0x00, 0x00, 0x00, 0x08, 0x00,
0x78, 0x1d, 0x41, 0x31, 0xb0, 0x9a, 0xc9, 0xbb, 0x27, 0x00,
0x00, 0x00, 0x00, 0x00, 0x64, 0x75, 0x6d, 0x6d, 0x79, 0x2e,
0x74, 0x78, 0x74, 0x50, 0x4b, 0x05, 0x06, 0x00, 0x00, 0x00,
0x00, 0x01, 0x00, 0x01, 0x00, 0x37, 0x00, 0x00, 0x00, 0x4e,
0x00, 0x00, 0x00, 0x00, 0x00,
};

public static void main(String[] args) {
for (int i = 0; i < mybinaryfile.length; i++) {
System.out.print(mybinaryfile[i] + " ");
if ((i % 20) == 0)
System.out.println();
}
}
}
----------------------------------------------------------------------

Cheers

Jacob

unread,
Oct 1, 2004, 5:33:27 AM10/1/04
to
Jean-Francois Briere wrote:

> You could download Jatha, which is a preprocessor that generates plain
> JAVA sources from JAVA sources with macros.

But why?

Compared to accessing this runtime from the deployment unit
(.jar say) this approach seems unnecessary complex.
This goes for the OP problem in particular (as it contains
binary data), but also in general.

Dave Glasser

unread,
Oct 1, 2004, 7:09:52 PM10/1/04
to
Tor Iver Wilhelmsen <tor.iver....@broadpark.no> wrote on 01 Oct
2004 09:04:20 +0200 in comp.lang.java.programmer:

A couple of points: First, I'm well aware of how to use ClassLoader's
getResource() or getResourceAsStream() methods, but I decided not to
question the OP's reasons when he stated:

> Remember again: I don't want to load this file at
> runtime but this file should be internal part of the
> one and only java class.

and instead give him an answer that, AFAICT, satisfied his
requirement.

Second, although I wouldn't bet money on it, I don't think you're
correct when you say:

>It's the same as
>
>byte[] mybinaryFile = new byte[...];
>mybinaryFile[0] = 0x65;
>mybinaryFile[1] = 0x66;

Since my code snippet (which should have said "mybinaryfile" instead
of "mybinary file") declared the array and initialized it using an
array literal, I believe the array would be created and initialized
entirely at compile time. It would do exactly what the OP wanted, it
just wouldn't do it in the syntactically clean and simple way he was
looking for.

Third, even if you're correct that my code would be equivalent to:

>byte[] mybinaryFile = new byte[...];
>mybinaryFile[0] = 0x65;
>mybinaryFile[1] = 0x66;

it wouldn't be all that expensive compared to reading the same number
of bytes into the same array from a file; in fact it might be less
expensive. Even though populating it from a file may not assign each
array element one at a time (although it may), you wouldn't have all
the buffering and the loading of extra java.io classes and the disk
I/O going on under the covers, using up resources and CPU cycles.

Dave Glasser

unread,
Oct 1, 2004, 7:11:44 PM10/1/04
to
Michael Borgwardt <bra...@brazils-animeland.de> wrote on Fri, 01 Oct
2004 09:55:43 +0200 in comp.lang.java.programmer:

That's only if the OP decided to declare it as a local variable,
rather than an instance or static variable.

Carl Howells

unread,
Oct 1, 2004, 7:21:42 PM10/1/04
to
Dave Glasser wrote:
>
> That's only if the OP decided to declare it as a local variable,
> rather than an instance or static variable.

Nope. And that's because you're wrong about the other point. Decompile
a .class file sometime using javap -c, and look at how array constants
are initialized. It's with a series of assignments. Each of those
assignments will be a few bytes of code, too... So you're actually
looking at significantly under 64k as the maximum size for a byte array
declared anywhere in java code.

Carl Howells

unread,
Oct 1, 2004, 7:27:39 PM10/1/04
to

Ugh. I hate speaking unclearly. ... the maximum size for a byte array
declared *using an array literal* anywhere in java code.

Hal Rosser

unread,
Oct 1, 2004, 7:58:26 PM10/1/04
to

"Tobias Merler" <mer...@gmx.net> wrote in message
news:cji0gu$fpt$00$1...@news.t-online.com...
How about including the file in a jar file with the class?
-


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.770 / Virus Database: 517 - Release Date: 9/27/2004


Dave Glasser

unread,
Oct 1, 2004, 8:09:42 PM10/1/04
to
Carl Howells <chow...@janrain.com> wrote on Fri, 01 Oct 2004 16:21:42
-0700 in comp.lang.java.programmer:

>Dave Glasser wrote:
>>
>> That's only if the OP decided to declare it as a local variable,
>> rather than an instance or static variable.
>
>Nope. And that's because you're wrong about the other point. Decompile
>a .class file sometime using javap -c, and look at how array constants
>are initialized. It's with a series of assignments.

I did, and you're correct. Live and learn.

Tor Iver Wilhelmsen

unread,
Oct 2, 2004, 3:43:00 AM10/2/04
to
Dave Glasser <dgla...@pobox.com> writes:

> I did, and you're correct. Live and learn.

You probably made the assumption that class files have a data segment
like binary code formats (COFF/PE), but they don't - except for string
literals, which end up in a "string pool".

Jean-Francois Briere

unread,
Oct 2, 2004, 4:06:52 PM10/2/04
to
By the way guys, those will eventually produce an error:

byte[] mybinaryfile = {0x65, 0x66, ... };

-or-

byte[] mybinaryFile = new byte[...];
mybinaryFile[0] = 0x65;
mybinaryFile[1] = 0x66;
...

Because in any binary file, you will have some bytes from 0x80 to 0xFF.

The correct code is:

int[] mybinaryfile = {0x65, 0x66, ... };

-or-

int[] mybinaryFile = new int[...];

Yakov

unread,
Oct 2, 2004, 11:59:33 PM10/2/04
to
> How can I implement this idea?
> Remember again: I don't want to load this file at runtime but this file should be
> internal part of the one and only java class.
>
> Tobias

Since it was not explained why the file must be a part of the class,
it's hard to give a proper advice, but here's one of the solutions:

Write a program that reads this file into a variable and then
serializes the class into a file using writeObject(this). Now the
serialized version of this class includes the file. Now you any
program can recreate the instance of this class (that includes the
file) using readObject().

Regards,
Yakov Fain

Ann

unread,
Oct 3, 2004, 1:00:11 AM10/3/04
to
but OP wants to not have any code at runtime to do it

"Yakov" <yf...@hotmail.com> wrote in message
news:b04e2170.04100...@posting.google.com...

Yakov

unread,
Oct 3, 2004, 7:35:34 AM10/3/04
to
"Ann" <An...@nospam.invalid> wrote in message news:<vHL7d.310134$Fg5.58253@attbi_s53>...

> but OP wants to not have any code at runtime to do it

The method that reads this file will be called only once during the
very first run only and never again

Regards,
Yakov Fain

Paul Lutus

unread,
Sep 30, 2004, 7:10:34 PM9/30/04
to
Tobias Merler wrote:

To be included in your Java source file, the data must be representable in
text form. So read the binary file and create a text list of values
expressed in hex or decimal, sepaated by commas, and add this to your
source as part of the declaration of a byte array.

--
Paul Lutus
http://www.arachnoid.com

0 new messages