Build OsmAnd from the command line (Was: New fork: Osmand Touch (for developers))

[Harry van der Wolf]

This came from the thread about " New fork: Osmand Touch (for developers)", but I don't want to hijack that thread so I made a new one. Not because it is so important.

In reply on Roger's post: It was quite simple so I wrote a new page on the OsmAnd wiki under Technical Articles being Build OsmAnd from the command line

It is currently for Linux/Unix/BSDs/Mac OS X. My intention is to expand it to Windows but that's untested (at least by me. I don't even have a windows machine) and I'm not even sure whether the build tools will build on windows.

Please let me know your feedback.

I will expand the article on how to build OsmAndMapCreator.

Harry

2014-08-05 10:34 GMT+02:00 Harry van der Wolf <hvd...@gmail.com>:

OK,

I will write a longer version. I will split up the original part from "my" very small addition and rewrite it as a complete new article like "Install terminal commandline build environment on Linux/OS X" (or something like that. The title is not so catchy).

It should be there next weekend. Probably Wednesday evening (Dutch time zone :) )

Harry

2014-08-05 9:32 GMT+02:00 RogerE <roge...@gmail.com>:

Harry,
I tried your command line guide and after some fiddlin' the code does indeed compile and run, but it don't seem to include the native code (tested on a Galaxy S5). The apk file is also smaller than the nightly build, 20.7MB vs. 27MB (OsmAnd-master-nb-2014-08-01.apk). Maybe the guide really is too short then...

Roger

On Monday, 28 July 2014 19:57:15 UTC+2, Harry van der Wolf wrote:

I updated the wiki specifically for this reason to add the "Terminal based compilation on Linux or Mac OS X".

Is that too short? (no pun intended)

2014-07-28 19:23 GMT+02:00 RogerE <roge...@gmail.com>:

The biggest problem right now for potential new contributors is that it is very difficult to set up the development environment and build the code. The development environment How-To Wiki page is out-datet and is not of any help for the current code. It used to be reasonably easy up until about a year ago - just import the code into Eclipse, build, download and run, but not any more.

Even if the code structure is not optimal as is, it will be a really useful step forward if someone with the knowledge would take the time to write a step-by-step guide on how to set up the development environment and build the current code (or even better update the Wiki How-To).

Roger

--
You received this message because you are subscribed to the Google Groups "Osmand" group.

To unsubscribe from this group and stop receiving emails from it, send an email to osmand+un...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Osmand" group.
To unsubscribe from this group and stop receiving emails from it, send an email to osmand+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Harry van der Wolf]

Also added the section on how to build OsmAndMapCreator.

Harry

[Saikrishna Arcot]

Hi,

I wrote the steps that I used to build OsmAnd without using the repo tool here. You might want to consider it for inclusion as a separate section in your wiki page.

[Saikrishna Arcot]

You might also want to mention that, for the NDK download, for any Androids before 2014 (and possibly any Androids during the first half of 2014), the 32-bit target NDK will be needed, and not to use the 64-bit one.

[Harry van der Wolf]

Hi Saikrishna,

Thanks for your additions.

You wrote:

1. Run "git clone https://github.com/osmandapp/Osmand.git android"
2. Run "git clone -b legacy_core https://github.com/osmandapp/OsmAnd-core.git core"
3. Run "git clone https://github.com/osmandapp/OsmAnd-resources.git resources"
4. Run "ANDROID_SDK=<path_to_SDK> ANDROID_NDK=<path_to_NDK> android/OsmAnd/old-ndk-build.sh". Replace <path_to_SDK> and <path_to_NDK> with appropriate values. This will build the native libraries.
5. Change your current directory to android/OsmAnd, and run "ant -file build.xml -Dndk.dir=<path_to_NDK> -Dsdk.dir=<path_to_SDK> -Dbuild.version=<version_code> -Dnativeoff=false "-DversionFeatures=+play_market +gps_status" "-DFEATURES=+play_market +gps_status" debug". Again, replace <path_to_SDK> and <path_to_NDK> with appropriate values. The <version_code> is what will get displayed as the version. This will build the app itself, and will include the native libraries built in the previous step.
   
6. To install it onto your device, run "adb install bin/OsmAnd-debug.apk".

I will not use the git clone steps as it is very prone to errors. I also started that way 2 years ago but due to the complex structure of the repositories the repo tool is THE swiss-army knife to bind them all together ("One Tool to rule them all, One Tool to find them, One Tool to bring them all and in the darkness bind them" ) as some writer clearly described, with a minor change.
If changes occur in repository names or they get deleted (they have) or created (they have), you have to find out all for yourself. The repo tool does that all automatically for you.

For example: your git commands miss the new "help" repository that has been added recently as of 1.8.3. Repo picked that up automatically.

I didn't mention the (version)Features parameters on purpose. These parameters are already defaults for over a year (I know they are still mentioned in the build jobs on the build server, but these jobs are very old). Currently it is just the opposite: if you don't want to use, for example, the gps_status "plugin" you have to specify "-DFEATURES=-gps_status". Anyway, I will add them to the article as your list of parameters isn't complete.

The version addition (step 5) is a good one. I never use it myself and I only used it once when I created a one-time fork myself which I released, but it is indeed required to mention.

The adb trick (step 6) is a nice one as well. Most users will simply copy the apk to their phone and then install it, but it can be done shorter.

On windows however you need admin right to be able to connect to your phone with adb. I will mention that.

Harry

 

[Harry van der Wolf]

2014-08-10 1:28 GMT+02:00 RogerE <roge...@gmail.com>:

Thanks Harry, the command line guide is easy to follow now, but the native code still don't seem to be included correctly. The "old-ndk-build.sh" command ends with one of the expected lines and the apk contains the expected "libgnustl_shared.so" file. The nightly build apk does however include a file named "libosmand.so", wich is missing in the new build - so something isn't quite right. I use sdk_r23.0.2-linux and ndk-r9d.

Please try with the ndk-r8e. About 6-8 months ago there was an issue with the r9 and you really needed to build with r8. I had forgotten that. I just built with the r9 and for me it works. It seems even slighlty faster.

 

I did also initially run into another problem: the "old-ndk-build.sh" command ended with one of the expected lines indicating successful completion, but when looking trough the log it contained a fault message: "..../functions.sh: line 107: svn: command not found". Apparently, svn is also needed and when installed everything built without errors. I guess the guide should also include a statement about svn.

Yes, you are right. Some of the dependencies are downloaded via svn.  I removed everything and did a clean install myself but hadn't thought of svn and hadn't removed it.

I just added it to the wiki.

Harry

[Harry van der Wolf]

2014-08-10 10:39 GMT+02:00 Harry van der Wolf <hvd...@gmail.com>:

2014-08-10 1:28 GMT+02:00 RogerE <roge...@gmail.com>:

Thanks Harry, the command line guide is easy to follow now, but the native code still don't seem to be included correctly. The "old-ndk-build.sh" command ends with one of the expected lines and the apk contains the expected "libgnustl_shared.so" file. The nightly build apk does however include a file named "libosmand.so", wich is missing in the new build - so something isn't quite right. I use sdk_r23.0.2-linux and ndk-r9d.

Please try with the ndk-r8e. About 6-8 months ago there was an issue with the r9 and you really needed to build with r8. I had forgotten that. I just built with the r9 and for me it works. It seems even slighlty faster.

Please stick to the ndk-8 releases. I had not done a clean build with r9. The ndk-r8 is also the ndk release used in the project.

Harry

[Max]

Please stick to the ndk-8 releases. I had not done a clean build with r9.

Since several month I use r9d without any problems.
I didn't try new r10 (July 2014) yet.

The ndk-r8 is also the ndk release used in the project.

On the build server, a special patched r8 version is used, because of a NDK bug (which was never fixed by Google) with Android 2.1 or 2.2 (native lib does not run).

Regards,
Max

[Harry van der Wolf]

Thanks for your addition Max. Did some quick builds this morning. I will again remove everything and try both ndk-r9 and ndk-r10
I will specify ndk-8 and ndk-r9 in the wiki for the time being.

Harry

--

[Max]

because of a NDK bug (which was never fixed by Google) with Android 2.1 or 2.2 (native lib does not run).

After searching for a while I think I found it again:
https://code.google.com/p/android/issues/detail?id=43819

Regards,
Max

[Max]

Just did a build with NDK r10 (android-ndk32-r10-linux-x86_64.tar.bz2) without problems.

Regards,
Max

[Harry van der Wolf]

2014-08-10 12:54 GMT+02:00 Max <openstre...@nurfuerspam.de>:

Just did a build with NDK r10 (android-ndk32-r10-linux-x86_64.tar.bz2) without problems.

Regards,
Max

So did I. No problems indeed. I added it to the wiki.

Harry

[Max]

Why not just use

ant -file tools/OsmAndMapCreator/build.xml clean compile build

?

This creates build.zip with whole map creator inside.

Regards,
Max

[Harry van der Wolf]

2014-08-11 16:02 GMT+02:00 Max <openstre...@nurfuerspam.de>:

Why not just use

ant -file tools/OsmAndMapCreator/build.xml clean compile build

?

This creates build.zip with whole map creator inside.

Regards,
Max

I had already forgotten that one. I have used that one also some 1-1½ year ago and abandoned it.

The fact is that the "clean compile build" option drags in a whole lot of dependencies that get compiled and result in a mega size build folder.

My route "clean compile jar" leads to a 480MB folder size inside my code base folder (osmandtools in the wiki) as it only builds OsmandMapCreator. And with a few commands (or a simple shell script) you build your package.

Your "clean compile build" leads to a 7 GB !!! folder resulting in the same OsmAndMapCreator (be it automatically packed into a zip).

It drags the entire QT and IOS dependencies in on linux (and you can't even build the IOS stuff as you need to be on Mac OS X with XCode).

(Next to that: the current version build for "clean compile build" breaks on line 185 in the build.xml)

Anyway, I don't want that complete waste of disk space of 7GB versus 480MB, but for completeness I will definitely add it to the wiki.

And I will add a simple zip command to the end of my cp part to make a zip.

Harry

[Max]

Your "clean compile build" leads to a 7 GB !!! folder resulting in the same OsmAndMapCreator (be it automatically packed into a zip).

It drags the entire QT and IOS dependencies in on linux (and you can't even build the IOS stuff as you need to be on Mac OS X with XCode).

Why do you use the Qt build?
I use legacy core, like the build server does:
https://github.com/osmandapp/OsmAnd-core/tree/legacy_core

Also OsmAnd uses legacy core.
 

Anyway, I don't want that complete waste of disk space of 7GB versus 480MB

If you use legacy core, it is much smaller than 7 GB.

Regards,
Max

[Harry van der Wolf]

Then what do you specify?

If I use "repo init -u git://github.com/osmandapp/OsmAnd-manifest.git -m jenkins_tools.xml",
I do get the legacy core. It is even the default.

And if I then specify "ant -file tools/OsmAndMapCreator/build.xml clean compile  build", it still drags in all the QT based stuff.

This was 1 year ago also the same.

What else do you specify to prevent that?

Harry

--

[Max]

Then what do you specify?

-m rendering.xml

Takes 296 MB to build.
 

If I use "repo init -u git://github.com/osmandapp/OsmAnd-manifest.git -m jenkins_tools.xml",

I do get the legacy core. It is even the default.

jenkins_tools.xml does not use legacy_core (anymore):
https://github.com/osmandapp/OsmAnd-manifest/blob/master/jenkins_tools.xml

Regards,
Max

[Harry van der Wolf]

I used "repo init -u git://github.com/osmandapp/OsmAnd-manifest.git -m rendering.xml"

Then "ant -file tools/OsmAndMapCreator/build.xml clean compile build"

I get a 1.2GB build folder and the build fails.

I don't get it.

--