hugin installer for Windows Vista SVN 3975

[Ad Huikeshoven]

Hi all,

Please find a new hugin installer for Windows Vista based on SVN revision 3975 at http://hugin.huikeshoven.org/

Ad Huikeshoven

[hbl]

Ad, I installed this release on a Toshiba notebook running Windows
Vista sp1. The Data Execution Protection feature is closing the app.
This is common to all versions since 7.0. Are you or anyone else able
to test using a PC running native Windows Vista?

[Bart van Andel]

Maybe it's common to you, but I never had this issue personally, and I
still don't. My Sony laptop (Core2 Duo with Centrino2 technology) is
running Vista SP2 (no problems on the former SP1 either). I haven't
tweaked my system against DEP or anything like that either (if that's
even possible), it ran out of the box. Maybe there's an issue with
your configuration?

Best,
Bart

[hbl]

Bart, it is hard to say what the problem is. I've just installed SP2
with no change. On my laptop, DEP is closing Hugin with event name
BEX (buffer overflow exception). That is what is triggering DEP to
close the program.

Does your laptop have DEP installed? If so, are you able to create an
exception for Hugin? MS says installers should be created using
Visual Studio 8 to be compatible with DEP. I wonder if that is being
done here.

[Dedalus]

Thanks for trying to install and run my builds, and thanks for
reporting specific problems to this list. Personally I haven't seen
this type of error ever. The build process uses Visual Studio 9.0
2008. Is it MS suggestion to downgrade to VS8 for compatibility with
DEP? Is DEP a specific feature of Toshiba laptops? Is the Toshiba
notebook very old? I build and run on a Packard Bell laptop with Vista
Home Premium, and do regularly install as well on a quadcore desktop
PC.
I find it hard to figure out a way to solve your problem. Based on
what has been written - you're experiencing problems with many
versions since 0.7 on your Toshiba notebook with Vista, something
specific with your Toshiba notebook might cause the closing of the
app. So, a bit of a stupid question: have you tried to install the
software on another computer or laptop and experienced the same
problem?

Ad Huikeshoven
PS Weekly I built an installer running a build script I published on
this list, which actually formalizes the description of the
documentation of the build process by Yuval Levy and Guido Kohlmeyer.
My programming ability is limited to writing a simpleton.

[Yuval Levy]

hbl wrote:
> MS says installers should be created using
> Visual Studio 8 to be compatible with DEP.

MS says a lot of things to sell its stuff. You're the first user that
reported this issue and thousands have downloaded and used Hugin 0.8
snapshots on Windows XP and Vista without reporting any of this. If it
was a systemic problem, we'd read more such reports.

The current assumption is that the bug is well hidden and for whatever
reason it triggers only on your specific setup. The bug may trigger
another kind of error on other setups; or may not trigger at all.

BAD NEWS: This kind of spurious, non-reproducible bugs are the most
nasty ones to identify and fix.

The solution is not to change things for others. We need to analyze
thoroughly Hugin's behavior on your machine under controlled
circumstances so that we can infer from the difference in circumstances
where the problem is.

People here are willing to help, but it all depends on you because
nobody else has access to a machine that triggers the error.

First we need to establish a version of Hugin that works on your box. I
hope this will be one of the 0.7.0.XXXX installer.

Then you will get from SVN that XXXX version and build it against the
SDK that you have currently on your machine.

Best case scenario both work well, then we know that the bug is
somewhere between XXXX and the lastest TRUNK. The next step would be to
find the YYYY SVN version where YYYY+1 triggers the bug and YYYY does
not. Then the developers can look at the code between those two releases
and hopefully identify the bug and fix it.

If we are lucky, there is a difference in behavior between 0.7.0.XXXX
installed and SVN XXXX built with your SDK. Then we know that the
problem is in the SDK and we can start looking at the different SDK
components to identify which one it is. This is the most likely scenario.

Worse case scenario, also 0.7.0.XXXX does not work, and we have to dig
even further down.

At this point it is counter productive to introduce *any* change that
affects the general public. It is only on your setup. You'll need to be
patient and follow the advice that is given to you here on the list, and
not what MS says.

Yuv

[Bruno Postle]

On Wed 01-Jul-2009 at 18:08 -0400, Yuval Levy wrote:
>
>The current assumption is that the bug is well hidden and for whatever
>reason it triggers only on your specific setup. The bug may trigger
>another kind of error on other setups; or may not trigger at all.

It is possible this is the same problem encountered by RueiKe.

--
Bruno

[Yuval Levy]

Bruno Postle wrote:
> It is possible this is the same problem encountered by RueiKe.

I did not want to speculate. I think it is likely the same bug causing
the two problems, which would be an extra bonus as both Rick and Howard
can take a systematic approach to identify the bug from two different
spurious behaviors.

The big question is whether to release 0.8 before this bug is fixed, or
keep postponing 0.8. For users that are not affected by this bug, the
current snapshot is good enough for a release...

Yuv

[Bruno Postle]

On Wed 01-Jul-2009 at 18:32 -0400, Yuval Levy wrote:
>
>The big question is whether to release 0.8 before this bug is fixed, or
>keep postponing 0.8. For users that are not affected by this bug, the
>current snapshot is good enough for a release...

Yes, I'd like to go with the current release candidate, but I'm not
sure if these problems are in hugin, the windows build, windows
itself or some 3rd party windows software.

Does anyone else see these crashes on OS X or Linux? (note that
enblend or enfuse crashing during stitching isn't a hugin bug)

--
Bruno

[RueiKe]

For Windows Vista, there is an option under "System Properties" ->
"Performance Options" to change the settings of DEP. On my system, it
was set to be enable for essential Windows programs only. When I
change it to be active for all programs, I sitll don't see it raising
any DEP error for hugin. I am running 64bit, SP1, so maybe SP2 is
more sensitive.

Seems like I am the only one reporting major issues in not being able
to use anything newer than SVN3884 for large projects. Are there
others using the latest builds on windows for large projects with no
problems?

Regards,
Rick

[hbl]

Ad, my bad. It is Visual Studio 9 which includes Visual C++ 2008.

I agree it is difficult to isolate these issues. I have another
computer running XP that has no problems with hugin or gimp (and, I do
not intend to jostle anything less bad luck descend on that
computer :) It is DEP that is detecting a buffer overflow and is
closing Hugin. I am not sure if my XP machine has DEP installed.

At Yuval's suggestion, I have agreed to test build earlier versions to
try to isolate the problem area.

The Toshiba notebook I use was purchased in 2008 so it came with
Windows Vista and update has been activated.

DEP is a anti-malware defense that comes with Vista. Apparently it
has been available for some time on MS servers.

[hbl]

Yuval, what is the avenue of attack? What would you like me to do to
isolate the problem?

[hbl]

Rick, when I try that option, the OS refuses stating that Hugin must
run under DEP.

[Yuval Levy]

hbl wrote:
> Yuval, what is the avenue of attack? What would you like me to do to
> isolate the problem?

thanks for your help, Howard. The first thing we need is to find the
basement, i.e. how far back we need to go for the bug NOT to trigger.

try
<https://sourceforge.net/projects/hugin/files/hugin/hugin-0.7.0_win32-setup.exe>
cross fingers and report back.

good luck
Yuv

[ryan+...@sleevi.com]

Howard,

Would you mind checking http://support.microsoft.com/kb/912923 and
reporting back a bit more about the values from WMIC?

I'm hoping to identify:
1) Are you using hardware-enforced DEP?
2) What the current value is for the Windows enforcement of
it
3) Any possible hardware switches (Note the last line of
Method 1)

In addition to the information wmic will provide, can you also tell
me:
1) Which mobel Toshiba laptop (Trying to understand
processor architecture)
2) What version of Windows you're running (Full version
string of major/minor/build + 32 bit or 64-bit)

If I followed the thread correctly, I understand you can and have
compiled your own versions. If you are running on a 64-bit platform with a
64-bit OS, are you compiling 64-bit Windows binaries or the default 32-bit
ones? I'm suspecting based on your description that you're on 32-bit and
running 32-bit, I just wanted to make sure, since DEP can get weird when
there are mismatches.

To be fair, DEP has caused me more headache than it's worth, simply
because the sheer number of buggy programs out there. It's a trade-off
between hard crashes (because of the NX bit being triggered) or soft crahses
+ potential security holes (to be fair, not every buffer under/overrun
represents an exploitable security hole, and surprisingly some systems can
'gracefully' recover, even if they're still buggy in the first place).

While you're doing that, I'll flip on hardware DEP on my Vista x64
machine and see if I can reproduce with the native x64 binaries I've been
compiling for my own environment. I know it's probably not a 1:1 mapping of
your problem, but hopefully if it is some bug in Hugin, it'll surface there
as well.

Thanks,
Ryan

[hbl]

Thanks for checking in, Ryan. My notebook has the 32-bit version of
Vista Home Premium. I suspect hardware detection is on as I have
turned off sw detection using bcdedit.

On hardware enforcement, I get True, 3.

On Jul 1, 7:38 pm, <ryan+hu...@sleevi.com> wrote:
> Howard,
>
>         Would you mind checkinghttp://support.microsoft.com/kb/912923and

[Ryan Sleevi]

Howard,

I've enabled DEP on my system, and the programs I know to have
trouble do properly report BEX errors, so I know DEP is getting raised
appropriately. I've also confirmed with wmic that all the settings reflect
its enabled, with a value of True, 3, as you did.

I went ahead and built a binary based on the latest SVN (for native
x64), through the full build process, including the installer. I opted for
x64 since, as noted in the Microsoft KB article and the prior e-mail, always
good to match platform and application.

I've tried several scenarios, loading old projects, TIFFs and JPGs,
etc, and had no problem. Certainly, there are a lot of variables at play
here, so it may take some time to isolate. In the mean time, I'd suggest
following Yuval's directions of trying the last 'certified' version, in
attempt to find the basement on when this issue might have first been
introduced. This should be fairly easy and non-invasive.

What I'd like to try as a next step, as a means of hopefully getting
more information, is getting a crashdump from you when a DEP error has been
raised.

It's been a while since I've had to do it by hand (tend to use
API-based hooks to catch when an application crashes), so I hope you'll bear
with me if I direct you down the wrong path once or twice. Since you
mentioned Vista, the following KB article may help:
http://support.microsoft.com/kb/931673

Since I don't have the symbols Ad used for his installer, I'll be
mailing you off-list a 32-bit based version on the latest SVN that I'll
compile locally. This will ensure I have the necessary debug symbol files to
wind through the stack. If this is a concern for you, we can work off-list
on directing the necessary compilation steps, just be aware that it's not
uncommon for symbols to reach hundreds of megs for large projects.

Once you've gotten the new setup binary and everything is installed,
when you get an application crash (eg: due to DEP), see if you can follow
the directions of Method 1 as a means to intercept the crash. Once you have
that crash (located in the path specified), can you attach it and e-mail it
to me off-list? I'm not sure the default settings of Vista for creating
crashes, but a simple Crashdump of Notepad is around 50 megs. If it's too
large to e-mail, we can work off-list on a transfer mechanism. By all means,
compress compress compress! If you're unable to generate a crashdump once
it's raised a DEP error, let me know, and I'll see further if I can
investigate.

For those following on-list, especially Bruno, if a final 'formal'
Win32 build is released, it might help to keep the symbol files around.
Certainly users can build their own binaries, preserve the symbol files, and
debug themselves, but given how most Windows users aren't necessarily handy
with compiling and debugging their software, it may be helpful for some of
the weird bugs that remain post-release (as I recall, there are a few
mystery ones out there affecting Windows users)

Ryan

> -----Original Message-----
> From: hugi...@googlegroups.com [mailto:hugi...@googlegroups.com] On
> Behalf Of hbl
> Sent: Wednesday, July 01, 2009 9:02 PM
> To: hugin and other free panoramic software
> Subject: [hugin-ptx] Re: hugin installer for Windows Vista SVN 3975
>
>

[dex Otaku]

Just a note -

dl'd and am running on XP SP3 with no issues thus far. Rendering
seems a lot faster than 0.7.0.

D

[hbl]

All, I will try to get through the suggested testing scenarios by the
weekend. Will post results then.

--
Howard

[Yuval Levy]

RueiKe wrote:
> Seems like I am the only one reporting major issues in not being able
> to use anything newer than SVN3884 for large projects. Are there
> others using the latest builds on windows for large projects with no
> problems?

OK, I booted my workstation (X2 6000+, 8GB RAM) in Windows (XP SP2). I
completely removed any trace of Hugin from it, fetched Ad's latest
installer (SVN3975) and run my 294 pics project through it.

1. Load Pictures works well

2. Align triggers autopano-sift-c (I had to manually set the options to
"--maxdim 800 --maxmatches %p %o %s" like I have them on my Pentium M
1.6GHz, 2GB RAM Ubuntu notebook).

autopano-sift-c runs, the first couple of images are processed and I go
away. When I come back there is a wxExecute error window saying that it
could not run the autopano-sift-c command.

next I'll try the same with an earlier, 0.7 version.

Yuv

[Yuval Levy]

Hi Ryan,

ryan+...@sleevi.com wrote:
> To be fair, DEP has caused me more headache than it's worth, simply
> because the sheer number of buggy programs out there.

in this case DEP may prove to be a debugging tool :)

honestly I don't think that researching DEP is of any help in his
context. If DEP had not caught the buffer overflow, it may have gone
unnoticed for a longer while. We have an issue to take care of in the
Hugin codebase or in an SDK component. Once it will be fixed it won't be
relevant to DEP anymore.

Yuv

[Yuval Levy]

Hi again, Ryan,

Ryan Sleevi wrote:
> I went ahead and built a binary based on the latest SVN (for native
> x64), through the full build process, including the installer.

this means that you also built enblend-enfuse? autopano-sift-c? libpano?
wxwidgets? and all the other SDK components? for native x64? it would be
nice if you could contribute documentation of the process so that we can
release an x64 "official" version as well.

we had a case recently when enblend-enfuse built with debug symbols
would work while stripped of them it would crash - the debug symbols
were just enough "padding" for the overflow not to do too much damage. I
hope this is not the explanation for why your build does not crash.

On the other hand, it could be that the bug is in the updated SDK. Would
have to take the old SDK, re-build the old wxWidgets to support the
GLCanvas and try a recent Hugin SVN against it...

Yuv

[Harry van der Wolf]

Hi all,

I googled on "Toshiba Vista DEP" and found more hits on similar issues. It might be a Toshiba issue, not a Hugin issue.

Harry

2009/7/2 Yuval Levy <goo...@levy.ch>

[Ryan Sleevi]

I did update documentation of the process.

http://wiki.panotools.org/Hugin_SDK_%28MSVC_2008%29

and

http://wiki.panotools.org/Hugin_SDK_%28MSVC_2008%29_Patches


The "downside" to the current process is it requires two sets of
SDKs/wxWidgets/etc. The sheer impact of being able to get side-by-side
libraries (eg: ilmbase.x64.lib and ilmbase.x32.lib) and all the project
settings were way more headache and heartache than I wanted to invest during
the weekend that I worked on getting everything running, although that is
the logical next step.

I'm working on updating the patch set mentioned on the patch pages, as it
looks like there was one bit of patch trouble with libpano, although it was
resolved in < 5 minutes. Other than that, the directions are still valid and
can be done from scratch, as I went through everything again last night when
setting up the x32 build for Howard.

I've definitely seen benefits to the x64 version - Most notably on
autopano-sift-c, as I'm able to throw my larger projects at it and have it
go well into the 7-8 gig range on my system generating control points,
rather than having to manually break things into chunks. Of course, it
sounds like the future work on getting rows/columns setup and optimizing how
CP generation gets done will resolve that.

Lately I've been working on getting profile-guided versions of
enblend/enfuse/hdrmerge and the other 'heavy lifters' of the Hugin stack,
and have been seeing performance gains on an order of 10-25%. Quite handy on
the large projects.

In the enblend-enfuse case, are you talking about a debug binary or a
release binary with debug symbols? In the case of Windows, debug symbols are
stored in a separate file (.pdb), which makes it easy to ship "Production"
exes (with no debug code/memory address alterations/etc), but when crashes
occur, be able to load the symbols and correlate the lines/files/etc.
Optimizing compilers can naturally make this harder, but my experience with
.pdb's is that it's fairly accurate. Using the Intel Compiler chain, which
can perform much more rigorous (and intelligent) optimizations, it does get
harder to make both "screaming awesome" and "debuggable", but it's not
impossible. I was just proposing that the .PDBs be preserved (for the SDK
and for Hugin itself) when a release is made, possibly on the project file
store, and that way anyone with Visual Studio (including the Express
Editions, IIRC), would be able to load those up and step through the
crashes.

As for the SDK, like I mentioned, everything was built by hand from scratch
following the directions on that page, which AFAIK is how Ad is building
his, so we'll see. Hopefully Howard will have more information using the
build I provided him, and we'll be able to get to the bottom of this.

[Ryan Sleevi]

Only if you like debugging Windows ;)

What I meant by DEP being more headache is that there are any number of
reasons that DEP can be triggered independent of Hugin, so it may not be a
legitimate bug.

An example to illustrate this (with all of its frustration) is some recent
heartache I had with a certain laptop manufacturer that sounds like "Bell"
and the drivers they supplied for the built-in biometric device. The
"security functions" of the drivers ended up hooking into several Windows
API calls for filesystem access (yes, you can do this, and there are a
number of legitimate reasons - most popular being anti-virus/security
products). Unfortunately, these drivers had a bug (if we count "suck" as a
bug) and it had a huge negative impact on my system performance.

Compiling one of the projects I work on, which has thousands of files and
hundreds of thousands LOC took almost 100x longer than normal, and
eventually failed, because there was a leak in the drivers. A quick
uninstall, and my system was back to screaming performance.

Consider then, the situation if these drivers had a buffer underrun/overrun.
In Windows, the DEP exception would be triggered, and it would work its way
up to the stack, finally into the application that made the OS call (which
it didn't know was being hooked by the buggy app). That app would then
terminate, with the user being told it was that app that was buggy.

I wish I could say situations like this are uncommon, but I see regularly
applications that instill themselves into critical locations (shell
handlers, API hooks, control panel drivers) that are buggy, and because of
how the exception works its way up the chain, it ends up crashing a
perfectly benign application (as it cannot control for those crashes).

That's just the frame of mind I'm in with regards to researching this, is
that past experience, especially with DEP, by no means signifies a bug in
Hugin necessarily. While still worth investigating, I would not think this
bug (especially with the difficulty reproducing) should be considered a
'block' in any means.

Cheers,

Ryan

> -----Original Message-----
> From: hugi...@googlegroups.com [mailto:hugi...@googlegroups.com] On
> Behalf Of Yuval Levy
> Sent: Thursday, July 02, 2009 4:01 PM
> To: hugi...@googlegroups.com
> Subject: [hugin-ptx] Re: hugin installer for Windows Vista SVN 3975
>
>

[hbl]

Harry, I think Toshiba has less to do with the issue than the OS. DEP
depends on both hardware and software flags. Both are supplied by
parties outside Toshiba. I have no doubt a buffer overflow is being
detected. And, as I recall, this, many yarns ago, was the very first
attack on an OS: The password buffer was 256 bytes long. After that,
bogus passwords overwrote critical code space.

But, why now?

Looking at the history file, there have been unexplained closures of
more than hugin or GIMP; Windows Explorer also was in the list.
Presumably, closed by the hardware flag because I don't recall seeing
a message from DEP. Perhaps the latest Windows OS update improved the
software DEP with the notification message from the system tray. Just
conjecture...

We shall persist until a solution is identified. Thanks to you all
for your participation.

[finbref.2006]

> Please find a new hugin installer for Windows Vista based on SVN revision
> 3975 athttp://hugin.huikeshoven.org/

when the program loads, it says Version "3966", although the about
screen is correct.
thomas

[Yuval Levy]

Ryan Sleevi wrote:
> I did update documentation of the process.
>
> http://wiki.panotools.org/Hugin_SDK_%28MSVC_2008%29
>
> and
>
> http://wiki.panotools.org/Hugin_SDK_%28MSVC_2008%29_Patches

did I thank you? THANK YOU, Ryan!

I've linked the pages from
<http://wiki.panotools.org/Development_of_Open_Source_tools#Supported_Platforms>

and with you and your expertise around I allowed myself to note that
64bit Windows is now supported.

Will you contribute a 64bit installer of 0.8.0 final after Bruno
released it?

Yuv

[Ryan Sleevi]

> and with you and your expertise around I allowed myself to note that
> 64bit Windows is now supported.

Well, it's only supported if those two patch sets get merged upstream prior
to 0.8 (your prior e-mail asking for thoughts). My belief is they should be
very minor (the patches to Hugin itself, really Vigra, were straight from
upstream just backported).

The patches to the supplementary products are (more or less) just patching
the solution files, although other projects that use Vigra had a few changes
applied to them as well. I'll try to update the libpano .sln patch (part of
the second set), since I found I missed something with x64 on it.

> Will you contribute a 64bit installer of 0.8.0 final after Bruno
> released it?

With regards to the installer itself, my process has not been clean. I get
several errors with the .iss that I haven't bothered to look deeper into.
Mostly issues with matchnsift/matchpoint/perl files being missing. I haven't
bothered to look into this, because it hasn't affected my personal usage,
but it would be a hindrance towards a 'proper' release. The other part is
still my own personal concern regarding the patented code and compiling it -
being an American and all. The whole distinction of direct and indirect
infringement is a point of concern.

As to the steps/compilation themselves, the process can be fully
accomplished with Microsoft's free-as-in-beer toolchain. The x64 compilers,
while not native to the Visual Studio 2008 Express Edition, can be obtained
from the Windows Platform SDK 6.1 and native x64 binaries compiled (or
cross-compiled, for those on x32). Using the Express Edition requires one
extra step (as documented at
http://jenshuebel.wordpress.com/2009/02/12/visual-c-2008-express-edition-and
-64-bit-targets/ ), but otherwise, compilation should prove quite smooth.

[Bruno Postle]

On Tue 07-Jul-2009 at 14:38 -0400, Ryan Sleevi wrote:
>
>Well, it's only supported if those two patch sets get merged upstream prior
>to 0.8 (your prior e-mail asking for thoughts). My belief is they should be
>very minor (the patches to Hugin itself, really Vigra, were straight from
>upstream just backported).

Sorry these didn't get applied, I'm trying not to break anything
that is currently already working and 64bit Windows is a new feature
rather than a bugfix (this shouldn't stop anyone creating a 0.8.0
64bit binary installer, they will just have to patch first).

--
Bruno

[RueiKe]

I have spent a few days with Ryan's 64bit build of hugin, and I think
"wow!" is the best way to summarize the experience so far. Here are a
few observations:

I set the memory usage for enfuse and enblend to 7GB. Both went upto
the limit while stitching a large project with no issues. I also
noticed nona got upto about 5GB and hugin itself got to 3GB, even
though I had the image cache set to 500MB.

A 17k wide 360x180 blended/enfused equirectangular projection from
125, 70MB, 16bit TIFF files stitched in 72min! Previous projects took
about 5hrs.

I tried one autopano-sift-c test case for 15 images with --maxdim
4000 set and it found over 3000 control points. Typically, I have
found the previous versions have issues adding control points when --
maxdim is set to > ~1200. I also tried adding a Nadir with these
settings, and ~10 control points were found for all adjacent images,
which never happened in previous runs.

The optimizer really didn't seem much faster. Still only used about
12% of cpu resources.

Overall hugin interface was a bit perkier, especially when dealing
with control point editing and cropping.

Both preview windows were more responsive. I had issues with every
previous build where I would get a boost thread resource error about 1
in 5 times when adjusting the projection in the regular preview
screen. For this build, I could not make it crash!

Not a single crash since I started testing on large projects! At
times, I have had a project active over days, without closing or re-
starting.

Definitely a much improved user experience for large projects. It
looks like from the discussion threads that not many people in the
user community are dealing with such large projects, but that is all I
work with. This build certainly makes that work more pleasant! I
understand that it should not be critical path for 0.8.0, but it would
be a great enhancement for the next release.

Regards,
Rick