What's the best practice for packing assets into an APK and reading natively?

[Robert Green]

Hi Guys,

What's the least painful way to package assets with the APK and read
them natively? I read everything I could find on the topic and
opinions seem to vary greatly.

I saw an option about storing them uncompressed in assets then
retrieving the file descriptor, offset and length and passing those to
native to read - seems ok but annoying if you have lots of proprietary
extensions you have to set up aapt , but that's just a one-time deal
so maybe not too bad..

I saw another option about having java read them and passing the bytes
through JNI. That seems like it might be slow if loading megabytes of
resources, but then again if you just read into a direct byte buffer,
that's not so bad so long as no individual resource is too big. The
biggest one I've ever had is 500KB, which wouldn't be a problem.

Then another person said that on first run, they have a bit of java
code just copy all the assets out to the SD card where the native code
can open them directly. I like this because it gives native code
direct access, but it doubles the data size and also causes a delay
when first running the game, not to mention requires an SD card is
installed (which I'm sure some will complain about and give bad
ratings for reason they won't mention)

So what's the consensus? Did I miss any options? Also does anyone
have code examples for either of the first two?

Currently my vote is option 2, since it only requires a native to java
method to read a file, then you've got a pointer to the entire
contents.

Thanks!

[charles]

Here is a Java method I wrote which copies files from the /assets in
APK to the application's internal storage on the SDK card.

It is imperfect and likely to change in the future -- but it works for
now.

---------------------------------------------------------------------------------

public void install() {

new Thread(new Runnable() {
public void run() {

try {
/* Instantiate an AM and store a list files under /assets/bin
in an array. */
AssetManager assetManager = getAssets();
String[] sourceFiles = assetManager.list("bin");

for (String src : sourceFiles) {
File targetFile = new File(DATA_PATH+"/bin", src);

/* If the target file does not exist, create a copy from the
bundled asset. */
if (!targetFile.exists()) {
BufferedOutputStream out = new BufferedOutputStream(new
FileOutputStream(targetFile));
BufferedInputStream in = new
BufferedInputStream(assetManager.open("bin/"+src, 2));

int len;
byte[] buf = new byte[1024];

while((len = in.read(buf)) != -1) {
out.write(buf, 0, len);
}

Log.i(MSG_TAG, "Wrote file "+targetFile.toString());
in.close();
out.close();
} else {
Log.i(MSG_TAG, src+": File already exists. Nothing to be
done.");
}
}

} catch (IOException io) {
Log.e(MSG_TAG, "Error! Install failed.");
Log.e(MSG_TAG, io.getMessage());
} catch (Exception ex) {
Log.e(MSG_TAG, "Error! Install failed.");
Log.e(MSG_TAG, ex.getMessage());
}
}
}).start();
isInstalled = true;
displayToastMessage("MeshApp has successfully been installed!");
}

----------------------------------------------------------------------------------------------

You can see the whole source file here:
http://wlan-lj.net/browser/trunk/meshapp/android/meshapp/src/net/wlanlj/meshapp/MainActivity.java

Hope this helps.

- Charles

[fulanito]

Here is a detailed example of how to read a file located in the apk,
from within native code, using an AssetFileDescriptor:

The file name is myFile.mp3 (I use the .mp3 suffix to avoid file
compression), and it is located in the res/raw/ directory.
--------------------------------------------
--------------------------------------------
In the Java side:
package com.myApp;
public final class myClass{
public static int init(MyActivity activity)
{
AssetFileDescriptor afd =
activity.getApplicationContext().getResources().openRawResourceFd(R.raw.myF
ile);
int res = 0;
if (afd != null)
{
FileDescriptor fd = afd.getFileDescriptor();
int off = (int) afd.getStartOffset();
int len = (int) afd.getLength();
res = initNative(fd, off, len);
}
return res;
}
public static native int initNative(FileDescriptor fd, long
off, long
len);
}

-------------------------------------------
-------------------------------------------
In C++ side:
in the .h file:
----------------
#include <stdio.h>
#include <jni.h>
extern "C" {
JNIEXPORT jint JNICALL Java_com_myApp_myClass_initNative
(JNIEnv * env, jclass thiz, jobject fd_sys, jlong off, jlong len);
}

in the .cpp file:
--------------------
#define
SUCCESS
0
#define ERROR_CODE_CANNOT_OPEN_MYFILE 100
#define ERROR_CODE_CANNOT_GET_DESCRIPTOR_FIELD 101
#define ERROR_CODE_CANNOT_GET_FILE_DESCRIPTOR_CLASS 102
JNIEXPORT jint JNICALL Java_com_myApp_myClass_initNative
(JNIEnv * env, jclass thiz, jobject fd_sys, jlong off, jlong len)
{
jclass fdClass = env->FindClass("java/io/FileDescriptor");
if (fdClass != NULL){
jfieldID fdClassDescriptorFieldID = env-
>GetFieldID(fdClass,
"descriptor", "I");
if (fdClassDescriptorFieldID != NULL && fd_sys != NULL)
{
jint fd = env->GetIntField(fd_sys,
fdClassDescriptorFieldID);
int myfd = dup(fd);
FILE* myFile = fdopen(myfd, "rb");
if (myFile){
fseek(myFile, off, SEEK_SET);
/
***************************************************
here myFile is a regular FILE*
pointing to
the file I want to read.
I use the fscanf function:

***************************************************/
fscanf(fp, "%d%d", &var1, &var2);
fscanf(fp, "%d%d", &var3, &var4);
...
...
...
return (jint)SUCCESS;
}
else {
return (jint)
ERROR_CODE_CANNOT_OPEN_MYFILE;
}
}
else {
return
(jint)ERROR_CODE_CANNOT_GET_DESCRIPTOR_FIELD;
}
}
else {
return
(jint)ERROR_CODE_CANNOT_GET_FILE_DESCRIPTOR_CLASS;
}
}

Just a short warning, quoting fadden (http://groups.google.com/group/
android-ndk/browse_thread/thread/a69084018e87a5a8):
>"descriptor" is a package-private field inside FileDescriptor, and as
>such isn't part of the API, is subject to change, etc.

"descriptor" field is used in the C++ code, so pay attention that the
above code is using non API fields, and may crash in the future.
I hope that until this happens, the NDK team will come up with an
official solution to reading assets in native code :-).

Hope this helps,
Fulanito.

> You can see the whole source file here:http://wlan-lj.net/browser/trunk/meshapp/android/meshapp/src/net/wlan...

[Angus Lees]

I have examples of both of your first 2 options in ScummVM:

 http://scummvm.svn.sourceforge.net/viewvc/scummvm/scummvm/trunk/backends/platform/android/asset-archive.cpp?view=markup

Complete with a seek() implementation that gets expensive if you seek backwards while using InputStream (It's a pity ACCESS_RANDOM is basically useless).  Also note that there is no way to tell whether an AssetManager.list() result is a subdirectory or a file - I guess based on whether the filename contains a '.' (which works for my use case).  (I note these problems don't exist in the Android private AssetManager API - unfortunately they got lost in the Java wrapper)

(GPL, as is the rest of ScummVM.  So if your project is also under a GPL compatible license, feel free to just snarf this code directly)

For my use, I much prefer an AssetFileDescriptor since it makes seeking cheap and I do a lot of seeking.  This requires the asset to be added to the apk uncompressed (note that zip file assets are not always added uncompressed despite what the aapt documentation says - you can force it with 'aapt package -0 zip')

Another option (and one I've used previously) is to pass the pathname of the apk down to your JNI code and just use your own native unzip code directly - effectively bypassing AssetManager entirely.  Nice and fast, with full control over what information is available and most zip libraries give a more natural native API.  I guess it is theoretically possible that an Android implementation could recode your apk into something no longer a zip file on package install, so bypassing AssetManager would get you into trouble - seems pretty unlikely to me though.  The additional unzip code will also bloat your native library further.

 - Gus

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To post to this group, send email to andro...@googlegroups.com.
To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.

[Kelly]

I prefer to use libzip to read all the files out in native. It seems
the most straightforward to me.

First I made a wrapper class to open the apk in whatever scope I want:
--------
class ApkManager
{
public:
zip* m_pArchive;
ApkManager()
{
if (gApkLoc.length() > 0)
{
m_pArchive = zip_open(gApkLoc.c_str(), 0, NULL);
if (m_pArchive == NULL)
{
LOGE("Error loading APK %s", gApkLoc.c_str());
return;
}
}
}

~ApkManager()
{
if (m_pArchive)
{
zip_close(m_pArchive);
m_pArchive = NULL;
}
}
};
------------

Then I run through to get the names of all the files (all in the apk):
------------
int numFiles = zip_get_num_files(pArchive);
//Loop through all files
for (int i = 0; i < numFiles; ++i)
{
const char* name = zip_get_name(pArchive, i, 0);
------------

And open the file I want:
------------
zip_file* file = zip_fopen(pArchive, name, 0);
// Copy into buffer
zip_fread(file, data, maxsize);
------------

Now do what you want with the data.


Cheers.

[Robert Green]

Kelly,

I really like the way you do it. Do you have a really clean way to
get the path to the APK? I found this code snippet in the group:

String apkFilePath = null;
ApplicationInfo appInfo = null;
PackageManager packMgmr = mActivity.getPackageManager();
try
{
appInfo = packMgmr.getApplicationInfo( "com.abc.YourPackageName", 0 );
apkFilePath = appInfo.sourceDir;
}
catch( NameNotFoundException e){
}

Does that seem right to you? Also did libzip require any
modifications to build?

Cheers

[Kelly]

Yup, that's the good stuff.

I used the libzip and some example code from someone opening pngs from
the APK. Lost/deleted the bookmark though :-(
Might not be too hard to find if you google it.

[Robert Green]

Kelly,

Which libzip did you use? I'm looking for one with a super small
footprint. I don't care if it can't compress - I just want small and
easy to build.

Thanks

[Robert Green]

Nevermind, I found it - http://www.anddev.org/ndk_opengl_-_loading_resources_and_assets_from_native_code-t11978.html

:)

On Aug 3, 1:49 pm, Kelly <senor...@gmail.com> wrote:

[Danny Backx]

Is there a guarantee that the APK remains available after installation
of an application ?

Danny

On Aug 5, 5:36 am, Robert Green <rbgrn....@gmail.com> wrote:
> Nevermind, I found it -http://www.anddev.org/ndk_opengl_-_loading_resources_and_assets_from_...
>

[Robert Green]

I got zlib and libzip building for Android but I didn't like the 500KB
footprint from that so I went the original direction and ended up
writing some C->Java invocation code to read an asset in its entirety
and return the byte array from it to native for use. Now I can load
assets in Win32 and Android. w00t!

[Dianne Hackborn]

On Thu, Aug 5, 2010 at 9:00 AM, Danny Backx <da...@backx.info> wrote:

Is there a guarantee that the APK remains available after installation
of an application ?

Not formally, no.  I can't say we have a short-term plan to change this, but can't absolutely promise that it won't change some day in the future.

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails.  All such questions should be posted on public forums, where I and others can see and answer them.