Porting hugin and panotools to Android

[Guillaume Lesniak]

Hey everyone,

As you might or might not know, I've released a component called "PicSphere" in the new CyanogenMod camera app Focal. This component is based on PanoTools and Hugin libraries.

I've been mostly hacking around to make it work, and it does work, but there are still a few kinks I need to iron out.

The repositories are available at https://github.com/cyanogenmod/android_external_Focal

- First, cpfind is just crashing and I cannot get it to work. I don't have the stacktrace anymore, as I ended up using autopano-sift-c for the alignment and cp detection, but I read here and there that cpfind works better (as in, faster) than autopano is most situations.

- I had to hack around a bit that wouldn't build in vigra, see https://github.com/CyanogenMod/android_external_Focal/blob/cm-10.2/vigra/include/vigra/impexbase.hxx#L154 - it doens't seem to affect the final result, but I would like to know if there's a better way. Basically, the compiler cannot find Accessor::ElementAccessor

- Another big problem is that it's slow. Stitching a correct panorama with 640x480 shots takes about a minute, where it takes 3 seconds on my home PC. I know smartphones aren't as powerful as PCs, but going from 3 seconds to 1 minutes is quite a leap (for a quadcore 1.5GHz, 2GB RAM device). Is there any plans for NEON/ARM enhancements, or at least could you give me some pointers on where should I focus optimizing (adding NEON ASM code) to make the whole process faster.

Here's the flow of the rendering process (from https://github.com/CyanogenMod/android_packages_apps_Focal/blob/cm-10.2/src/org/cyanogenmod/focal/picsphere/PicSphere.java#L183) :

1. autopano --align --bottom-is-left --generate-horizon 2 --ransac on --maxmatches 30 --keep-unrefinable off --projection 2,HFOV
2. ptclean --- I disabled it, as it seems to hang for panoramas above 180°. I tracked the bug down to some function pointer call that never returns, but didn't go any further.
3. autooptimiser -v HFOV -p
   
4. pano_modify -o project.pto --center --canvas=3000x1500
   
5. nona -o project -m TIFF_m
   
6. enblend *.tif

Note: On step 4, I use a fixed 3000x1500 canvas for the simple reason that Google PhotoSphere XMP metadata requires 2:1 ratio pictures. I plan to hack the sources further and enforce a 2:1 ratio after calculating the crop size, to keep the real image size.

autooptimiser alignment also seem to fail if there is any unconnected image.

What do you think of this flow? Could it be better?

My primary concern remains the speed of the rendering, so if anyone would like to help me or take the task, I'd be more than happy.

Thanks for your help!

[Caleb Anderson]

I think this is awesome. Is there a way to try it without using CM?

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/02ac38de-cb02-49d2-9d5b-edd6a47af26a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Jim Watters]

Guillaume,

Great news!
I've watched a video demo of Focal but have not tried it out yet.

PanoStitch, available on Google Play ( there is a free demo and a paid full version)
https://play.google.com/store/apps/details?id=com.myboyfriendisageek.panostitch.demo&hl=en
Also uses Panotools and Hugin but I don't notice any credit or link to the source code they are using.
It will give you a chance to compare speeds and see if they managed to improve it.

Developers,

What are the GLP requirements for using Panotools and Hugin in an Android app?

Jim Watters

-- 
Jim Watters
http://photocreations.ca

[Guillaume Lesniak]

There is no official way to try it without CM, but I might try to build it externally in some time. I'll post it here if I have time.

If PanoStitch uses Hugin, then it's an illegal app. The SIFT algorithms are patented by the University of British Columbia, and everything is GPL, which means the app must be open-sourced too, especially the changes made to it (and Hugin is far from compiling on Android without changes). PanoStitch is not only a paid app, but also not releasing AT LEAST their changes, making it a 100% illegal app. You might want to bring that to the GPL-Violations list.

That's why I put Focal as GPL too, and that's why I was excited to work on it: because I could use basically everything I want without worrying about licensing.

I'll eventually check and compare the speed, but if they made no changes, it'll likely be the same as I have. I'll unpack/decompile the APK for further licensing concerns.

[Guillaume Lesniak]

Confirmed: it uses Hugin GPL code in libpano.so:

 # strings libpano.so | grep "jni" 

jni/prebuilts/include/boost/exception/detail/exception_ptr.hpp

jni/hugin-2011.4.0/src/hugin_base/panodata/Panorama.h

jni/hugin-2011.4.0/src/hugin_base/hugin_utils/stl_utils.h

jni/hugin-2011.4.0/src/hugin_base/nona/Stitcher.h

jni/hugin-2011.4.0/src/hugin_base/nona/Stitcher.cpp

jni/hugin-2011.4.0/src/hugin_base/panodata/Panorama.cpp

jni/hugin-2011.4.0/src/hugin_base/hugin_utils/utils.h

jni/hugin-2011.4.0/src/hugin_base/panodata/image_variables.h

jni/hugin-2011.4.0/src/hugin_base/panodata/PanoramaOptions.cpp

jni/hugin-2011.4.0/src/hugin_base/panodata/PTScriptParsing.cpp

jni/hugin-2011.4.0/src/hugin_base/panodata/SrcPanoImage.cpp

jni/hugin-2011.4.0/src/hugin_base/panodata/ImageVariableGroup.cpp

jni/hugin-2011.4.0/src/hugin_base/panodata/StandardImageVariableGroups.cpp

jni/hugin-2011.4.0/src/hugin_base/panotools/PanoToolsInterface.cpp

[Matthew Petroff]

I've sent an email to the developer requesting the PanoStitch source code. We'll see if he complies.

-Matthew

[Guillaume Lesniak]

I've sent one too yesterday actually. He said he's currently on vacations and he will send it next week.

Anyway, back on topic :) Is there any known issue that would cause ptclean or autooptimiser to hang under certain conditions?

Guillaume

[Matthew Petroff]

They replied, providing the source code at this link:
https://www.dropbox.com/s/adqbtaej8gud6wn/PanoStitch-source.7z

-Matthew

[Guillaume Lesniak]

Small heads-up.

After more checking, and thanks to PanoStitch source, I was able to find out why cpfind/autooptimiser/... were randomly freezing. Turns out I still don't know the exact cause, but it seems linked to the way Java Runtime is calling binaries. If I ran cpfind manually, it would go all the way through perfectly fine. If I ran it through the Runtime class of Java, it would freeze. I ended up writing a small helper class in JNI that calls C's popen function to call every binary, and it seems to work fine now (at least, no more random freeze).

My rendering chain is now like this:

1. pto_gen -p2
2. cpfind --multirow
3. ptclean
4. autooptimiser -a -l -m -s
5. pano_modify
6. nona
7. multiblend

I swapped enblend for multiblend, which is noticeably faster than enblend for the same quality.

After checking cpfind parameters and docs on PanoTools Wiki, it looks like there is a "--prealigned" mode, which matches images after a predefined order (and thus, much faster and in a more exact way). Since I'm using the gyroscope to give out a preview of the final sphere, there should be a way to store each image supposed position at the time of capture, and feed it to cpfind. Unless I understand it wrong, this should make cpfind much more precise and faster, right?

Unfortunately, I couldn't find any documentation or example showing how/what to write in the pto to pre-align images (or what pto_xxxx tool could be used for that). Could someone give me some hint about that?

[Bruno Postle]

On Sat 17-Aug-2013 at 14:38 -0700, Guillaume Lesniak wrote:
>
> After checking cpfind parameters and docs on PanoTools Wiki, it
> looks like there is a "--prealigned" mode, which matches images
> after a predefined order (and thus, much faster and in a more
> exact way). Since I'm using the gyroscope to give out a preview of
> the final sphere, there should be a way to store each image
> supposed position at the time of capture, and feed it to cpfind.
> Unless I understand it wrong, this should make cpfind much more
> precise and faster, right?
>
> Unfortunately, I couldn't find any documentation or example
> showing how/what to write in the pto to pre-align images (or what
> pto_xxxx tool could be used for that). Could someone give me some
> hint about that?

You can use pto_var (from either the default or 2013.0 branch) to
set the roll pitch and yaw of each photo in a .pto project.

--
Bruno

[Guillaume Lesniak]

Thanks, this is working pretty good in most situations :)

However, I'm still optimizing alignment using autooptimiser's -a parameter. While it works fine in most cases, tricky cases (such as skies or pictures without a lot of control points) makes it mess pictures orientation as it thinks the matches are in a different orientation from what's already in the PTO.

Is there some way to tell it to use the manual/existing angle if there is too much difference? So that, if we have two pictures of a clean blue sky, it would rely on the orientation read by the gyroscope, rather than some random matching? Or, if there is too much differences between the PTO angle and the optimized angle, keep the PTO angle? That would prevent this kind of behavior, as I'm mostly using autooptimiser's alignment to fix small shifts due to the imperfection of taking a 360x180 shot with a mobile phone without tripod.

Guillaume

 

[Aisyah Darojati]

i tried to open the link but they said that the file i looked for has been moved or deleted.

would you mind to send the file to me personally? my email is: darojat...@gmail.com

thank you for your kindness...

[yoonyou...@gmail.com]

I tried to open the link but they said that the file has been moved or deleted.

Would you mind to send the file to me? My email is: yoonyou...@gmail.com 

Thank you for your kindness.

[Ganidu Madapatha]

I tried to open the link but they said that the file has been moved or deleted.

Would you mind to send the file to me? My email is: ganidu.m...@gmail.com

Thank you for your kindness.