PTGui unusable, memory problem

[Bob]

Hi

I have just installed PTGui Pro 9.0.1 on OS X 10.6.5. I have a
panorama ready to create... as soon as I click 'create panorama' my
machine's free memory goes from over 9GB to nothing in just a few
seconds. At that point the OS starts swapping like crazy and the
whole machine becomes pretty much unusable. If I cancel the create I
get the memory back pretty much immediately, and then PTGui becomes
responsive once more around 30 seconds later.
At this point I'm ready to request a refund... (license purchased
yesterday), but before I do - any one have any ideas how to resolve
this issue?

thanks
Bob

[michael crane]

this is probably due to user ignorance of software it is trying to use.
Suggest reading all available documentations .
mick

[Erik Krause]

Am 13.01.2011 22:50, schrieb Bob:
> I have just installed PTGui Pro 9.0.1 on OS X 10.6.5. I have a
> panorama ready to create... as soon as I click 'create panorama' my
> machine's free memory goes from over 9GB to nothing in just a few
> seconds. At that point the OS starts swapping like crazy and the
> whole machine becomes pretty much unusable.

...because it is *used* of course. PTGui is designed to use all
available resources to speed up panorama creation. Just wait until it is
ready.

If you want to use your computer for other tasks while PTGui runs, go to
Tools->Options->Advanced, and restrict number of threads, RAM usage and
such. Go to Folders&Files and specify the fastest disk that you have as
temp drive and make sure there's enough space. Preferably this is *not*
the disk where your system swaps to.

--
Erik Krause
http://www.erik-krause.de

[Bob]

Hi Erik

Thank you for your reply, thanks for taking the time to make helpful
suggestions. I should have supplied more detail I think:

When seeing this issue CPU utilization is low, so limiting threads,
cores etc isn't going to help. The system is choking on IO. I
already have the temp folder pointed to a RAID 0 striped pair, each of
which is pretty fast in it's own right. This drive has a ton of space
free - it's purpose in life is purely fro scratch/temp use.

Next, In PTGui preferences I have changed the use/limit available RAM
option, this doesn't seem to have any effect.

Finally, I have also used PTGui 8.3.x - that ran very well and I have
built a few HDR and non HDR panos using it. So, while I wouldn't
claim to be an expert PTGui user, I at least have some idea what I am
doing.

Thanks!
Bob

PS - an update, I just tried again, to see what the CPU runs at.
Initially I see around 750% (8 core machine) until the memory runs
out, then it drops to about 30%. I adjusted the allow available
memory usage from auto (85% displayed greyed out) to 50% and re-ran
the pano. Now things are better - PTGui grabs a whole load of memory,
but I still get about 3GB left over. Initially CPU utilization is
again around 750% until all the memory is used, then it drops to only
around 30%-40%. The machine remains totally responsive. I think I
need to go back to 8.3 and see what that does, until now I had never
had any reason to look - it just worked.

[Hans]

On Jan 14, 12:35 am, Bob <b...@dampsquid.com> wrote:
> Hi Erik
>
> Thank you for your reply, thanks for taking the time to make helpful
> suggestions.  I should have supplied more detail I think:
>
> When seeing this issue CPU utilization is low, so limiting threads,
> cores etc isn't going to help.  The system is choking on IO.  

The reason your CPU utilization is low is that you are using to many
threads.
If you really have an 8 core machine you should probably use around 6
threads for best performance.

PTgui 9 should default as auto setting to the number of real cores.
This has been changed since 8.3.10 which actually set it to all
hyperthreads (16 threads for an 8 core machine) and that caused
exactly what you describe.

It also depends on what kind of panos you do. If the images needs a
lot of warping like fisheyes or wideangles for a spherical they use
much more power and the machine becomes unresponsive during warping
if you use to much of the machine.

For Gigapixel panos with low FOV you have very little warping and each
image warps up to 10 times faster and it can utilize more cores.
When tha stitch starts blending it just use one core so it drops to
just 100%.

Hans

[PTGui Support]

Hi Bob,

Could you send the output of Help - System Information to sup...@ptgui.com?

Also, are you running any other memory intensive applications? For
example if both Photoshop and PTGui are running and both configured to
use 50% of your RAM you're in for some endless swapping. Either quit all
other applications or configure PTGui to use less RAM.

Joost

[PTGui Support]

Mick, one more such reply and you'll be banned here as well.

Joost

[PTGui Support]

On 14-01-11 00:35, Bob wrote:
> Finally, I have also used PTGui 8.3.x - that ran very well and I have
> built a few HDR and non HDR panos using it. So, while I wouldn't
> claim to be an expert PTGui user, I at least have some idea what I am
> doing.

Hi Bob,

When exactly did you purchase a license for PTGui 8.3.x?

Kind regards,

New House Internet Services BV
Joost Nieuwenhuijse

-----------------------------------------------
PTGui - Photo Stitching Software
www.ptgui.com

For support, see: http://www.ptgui.com/faq/
-----------------------------------------------

[Bob Porter]

Hi Joost

If you'd like to contact me privately I'll give you the details on that...  

IM	bob.p...@usa.net (MSN) mrdampsquid (AIM & Yahoo) bobporter (Skype)
Email	bob at dampsquid dot com

--
You received this message because you are subscribed to the Google Groups "PTGui" group.
To post to this group, send email to pt...@googlegroups.com
To unsubscribe from this group, send email to ptgui+un...@googlegroups.com
Please do not add attachments to your posts; instead you may upload files at
http://groups.google.com/group/ptgui/files
For more options, visit this group at http://groups.google.com/group/ptgui

[Bob Porter]

Hi

I will try and get some more meaningful information over the weekend. I went back to 8.3.x with this particular project and I did see similar 'problems' to what I reported on 9.0.x. From that point of view, I think I gave the impression this was a new problem with 9.0.x - that does not appear to be the case, my apologies.

This project is a little different from previous ones:
- the output size is large-ish (14,000 x 7,000 pixels),
- the HDR set is nine rather than 3-5 I usually use,
- I setup the pano with just mid-range exposures (two per set) to get the control points and then added the rest of the exposures because with the whole set I was getting a messed up result (subject of other post)
- and of course the software is new (to me at least).

So in other words I have changed more than one thing at once.

From a 'what else is the Mac doing' point of view, yes there are other apps running, what exactly changes all the time. The machine is A Mac Pro 3,1 with 14GB RAM, fast drives, two of which are configured as a RAID0 pair just for temporary/scratch space. All in all this machine rarely has problems dealing with workloads.

As I said, I'll try to get more objective info over the weekend, in the meantime if you have any input on what to try specifically - please let me know :-)
Thank you!

Bob

[Hans]

On Jan 14, 4:17 pm, Bob Porter <b...@dampsquid.com> wrote:
> Hi
>
> I will try and get some more meaningful information over the weekend.  I went back to 8.3.x with this particular project and I did see similar 'problems' to what I reported on 9.0.x.  From that point of view, I think I gave the impression this was a new problem with 9.0.x - that does not appear to be the case, my apologies.

Remember one thing when comparing with 8.3
8.3 was not 64bit so you will never get close to use your memory.

The default memory usage PTGui sets is based on your RAM and even
just a few applications open will ruin that.
Even if you just open CS5 it takes around 340 mb and with a few other
opened in backgound you can easy block PTGui from using the amount it
is set for in the preferences.

I just did a test on my iMac i7 w 8GB ram. Default memory is set to
69% but I increased it to 80. Even with just Safari and mail and a
couple of small applications open PTGui was blocked for around 5
minutes on the first set of 3 images. I could see that it used
everything I had. Free memoiry was only 8mb. Its obvious that it then
"learns" to use less memory because it started at set 7-8-9 to speed
up.

I reduced the memory to the auto settings and this time it warped form
the beginning at full speed but in the end the stitch took the same
time.

Another thing. on the iMac the standard hardisc is much faster than
any external firewire external. I got my test reduced from 25 min to
18 by using the standard 7200 disc instead of an empty 7200 Firewire
800.


Hans

>
> This project is a little different from previous ones:
> - the output size is large-ish (14,000 x 7,000 pixels),
> - the HDR set is nine rather than 3-5 I usually use,
> - I setup the pano with just mid-range exposures (two per set) to get the control points and then added the rest of the exposures because with the whole set I was getting a messed up result (subject of other post)
> - and of course the software is new (to me at least).  
>
> So in other words I have changed more than one thing at once.
>
> From a 'what else is the Mac doing' point of view, yes there are other apps running, what exactly changes all the time.  The machine is  A Mac Pro 3,1 with 14GB RAM, fast drives, two of which are configured as a RAID0 pair just for temporary/scratch space.  All in all this machine rarely has problems dealing with workloads.
>
> As I said, I'll try to get more objective info over the weekend, in the meantime if you have any input on what to try specifically - please let me know :-)
> Thank you!
>
> Bob
>
> On 14 Jan 2011, at 4:31 AM, PTGui Support wrote:
>
>
>
> > Hi Bob,
>

> > Could you send the output of Help - System Information to supp...@ptgui.com?

[Bob Porter]

Hi Hans

Thanks that's useful information. It seems PTGui's memory settings are based on total physical RAM, not what's actually available when the app starts (which was my interpretation of 'available'. I did turn down it's RAM use manually, something like 60% which left me with 3GB free when a pano was running. The machine remained responsive in other apps, even though disk I/O was going a little crazy, so that's good.

I also left my 'problem' project running under my initial conditions (that lead me to write the first email on this). It did complete eventually!

As an aside I did a simple pano in 8.3 and 9 to see the difference in 32 bit vs 64 bit. In 8.3 from start to finish to 5 mins 50s. In 9 it was 2 mins 50s! So that was nice!

Bob

[PTGui Support]

Yes, 'available' might be a confusing term, the setting is indeed based
on total RAM.

Good to hear the problem is solved.

Joost

[Bob Porter]

I thought I'd just close this thread out with some observations I hope others find useful.  

When reading keep in mind the machine's configuration: Mac Pro, 14GB memory, 8 cores, multiple fast drives.

My initial complaint was that in 9.x when stitching, PTGui consumed all available memory resulting in the OS having to swap to disk (a lot!).  The result was the whole machine became unresponsive/unusable.  This remains the case, but is controllable now I know what's happening...

First of all, the key configuration detail is how much available memory PTGui is allocated (set in PTGui advanced preferences).  This figure is 'auto' or a percentage.  The key thing is the percentage relates to the machine's total physical RAM, not how much RAM is free when PTGui starts.  Knowing that, I configured PTGui to take a percentage such that I still had at least 1GB free during the stitch.  The result is the machine remains perfectly useable.  PTGui starts to use disk (a lot) but since the OS itself isn't swapping everything carries on as normal.

The next observation is that this 'problem' wasn't seen in version 8.x because that was 32 Bit, and so the max memory PTGui could ever get was way less than my machine had available/free.

Someone suggested I should decrease the number of threads from 8 to perhaps six.  This makes no difference, as when stitching a large panorama disk IO is the bottle neck, not CPU.  Even if not hitting the disk, decreasing this doesn't do anything as CPU isn't quite max'ed out (though it's nice to see PTGui does indeed make good use of the available horsepower!)

Lastly I wondered how big a panorama could I stitch without PTGui needing to hit the disk in a big way.  Here's what I found...  I start with 14GB RAM, with the OS, apps that are 'always on', plus maybe Safari or something else I can usefully use while stitching is happening, there's say around 10 GB left free.  Configure the percentage above to say around 65%, that leaves around 1GB still free even if PTGui calls on all memory it's allowed.  Then I stitched bigger and bigger panos until I saw a slowdown in the speed of PTGui and an increase in disk IO (watch with Activity Monitor).  I found I could get up to around 50 Megapixels and still have everything pretty much done in memory.  That means the stitch was done in only a minute or two.  Of course I could quit everything else to free up more memory, change that percentage, and so on... and I will if I need to... however I'm much more likely to make smaller test stitches until I'm happy with the results and then batch stitch full size overnight when the machine can take as long as it likes.  

In summary - the move from 32 to 64 bit introduced this 'problem' which really isn't a problem at all once you understand that key preference - available memory is based on physical total.

Hope that helps others.

Bob

[Joergen Geerds]

Bob,

Thank you for posting your experience... it is in line with what we
were posting/suggesting here on this forum for months now. I just
wanted to add that the same is true for Windows, since both OS work
quite similar in this regard.

joergen
newyorkpanorama.com

[Roy]

Hi Hans

Based on your comment, "... For Gigapixel panos with low FOV you have

very little warping and each image warps up to 10 times faster and it
can utilize more cores. When tha stitch starts blending it just use

one core so it drops to just 100%." I would kike to know if I correct
distortions (15mm with EOS 5D MarkII, for spherical pano) in Lightroom
before stiching in PTGui, does this help CPU usage/time taken to align/
stitch?

Thanks

Roy

[Hans]

On Jan 21, 6:16 pm, Roy <fotob...@gmail.com> wrote:
> Hi Hans
>
> Based on your comment, "... For Gigapixel panos with low FOV you have
> very little warping and each image warps up to 10 times faster and it
> can utilize more cores. When tha stitch starts blending it just use
> one core so it drops to just 100%." I would kike to know if I correct
> distortions (15mm with EOS 5D MarkII, for spherical pano) in Lightroom
> before stiching in PTGui, does this help CPU usage/time taken to align/
> stitch?

I never tried that and I do not believe it is a good idea. As far as I
know you will get the same as I do in CS5 Camera Raw and that means
you loose a lot of the vertical view.
Also as soon as you do sphericals you will always have large warping
on the images. Just try any rectangular and drag it down below the
horisont in the editor and you see what happens.

If you do cropped gigapixels you usually are not more than 40 degree
from the horisont and each image covers much less FOV.

I have to say I was amazed a couple of days ago when I stitched a 675
images 5 gigapixel 220x50 degrees. It took just 2 hours.
I was expecting at least 3 hours.

Hans

[Joergen Geerds]

On Jan 21, 12:16 pm, Roy <fotob...@gmail.com> wrote:
> I would kike to know if I correct
> distortions (15mm with EOS 5D MarkII, for spherical pano) in Lightroom
> before stiching in PTGui, does this help CPU usage/time taken to align/
> stitch?

No, you can't remove the distortions in LR to "help" PTGui, since
PTGui doesn't remove them, but rather warps them into a completely
different shape (as you can see when you look at the individual images
in the panorama window.)
The point Hans was quite rightfully making is that images from a
longer lens need less warping, or in other words the pixel size stays
relatively similar... fisheye images for a spherical pano can be
warped to 2x the size in a very peculiar way, and that simply takes
more time than the relatively simple key-stone warp for "regular" pano
tiles.

the easiest way is to be patient while rendering (after you optimized
the memory and system as mentioned in many other posts).

joergen
newyorkpanorama.com

[Roy]

Thanks to both Hans & Joergen for clarifying issue.

Roy