Memory Issues

[Greg Takacs]

I have an i7 920 with 12GB RAM, running PTGui Pro 8.2.2 on Windows 7
RTM 64bit. I am stitching a 360x180 HDR shot with a 12mm lens, 3
exposures per shot, 19 shots for a total of 57 6 megapixel jpegs are
the originals getting imported, target is 11K x 5.5K psd file if
exposure fused single layer.

I have two memory issues:

1) No matter what I set the memory usage percentage to (it defaults to
67%, I changed it to 90% still leaving 1.2GB for OS and other apps)
PTGui never seems to use more than 2.5GB and it writes to my raid0
swap partition instead which is slow as molasses compared to RAM. The
previous version 8.2.1 seemed to have used all the RAM I set it to, I
could stitch up to 10k by 5k HDR panos from 21, 10 megapixel fisheye
images in memory under 6 minutes. So this is a big issue for me.

2) I have made an 8GB RAM disk leaving 4GB for OS and set the RAM disk
as my preferred swap partition and set my RAID0 for my secondary swap
partition. When I do this, PTGui will use the preferred swap but there
are occasions where it runs out of storage on it and instead of
realizing that it's out of space in the middle of the warping and
starting to use my RAID0 it just crashes and gives an error mesage
saying that it failed to write to my RAM disk as it ran out of space.
Somehow it fails to realize that it's getting out of space and instead
of falling back on the secondary drive gracefully by the time it
realizes it's out of space it's too late and it crashes.

Neither of the two issues would be a killer by themselves, but the
combination of them is just a total party pooper.

The second issue is not that big of a deal for me at the moment, but
I'm getting some SSD drives and they will be my primary swaps and I'll
use an ordinary HDD for fall over secondary swap. Once that happens
I'd like this second issue to get fixed too as I might just run out of
disk space on the SSD RAID0 and need to gracefully fall over onto the
secondary swap.

[supertrogg]

Hi,
I just replied to your other post where I commented on my use of a RAM
drive. I too have seen this problem, I have even run out of space with
a 20Gb virtual drive! It is rather annoying but the only way I have
around this is to set the swap drive to my raid if the final pano will
be large. Other than that I could install the full 128Gb RAM that my
workstation can take!! But I suspect that even then I could run out of
space (Gigapan anyone?)!! One additional oddity, if I pre-load the
images (small panos only) to the virtual drive, at some point during
the stitch it will end with a BSOD!!

[Greg Takacs]

I have made some further tests and it seems that 8.2.1 and 8.2.2
handle memory the same way. The difference I have experienced was due
to running different sets of images through them.
On my first test, where I thought memory was handled correctly in
8.2.1 I have used an HDR panorama made from 21 10 megapixel fisheye 8
bit TIF images 3 exposures per shot. The memory still does not get
used as advertised. It will go up to 6GB but it never gets near the
10.5GB hard limit that I set. It's basically ignoring 4+ GB of my RAM.

The second test I ran was with 57 6 megapixel 12mm shots that aere
jpegs. With this set, the memory usage never went passed the 3.5GB
mark.

What this tells me is that PTGui will load one set of the images to be
warped into RAM if it can but it simply never assumed that it would
ever have more RAM than images to be processed concurrently and it
simply stops putting images in RAM and starts using the swap drive
instead for the warped output. The difference between the 3.5GB and
the 6GB of memory usage seems to be matching the difference between 6
and 10 megapixel source files respectively.

From this experiment I can conclude that until PTGui starts handling
large amounts of RAM, 6GB+ it still benefits from a RAM drive for
swap, however it will run into the second problem with crashes due to
running of out space on the preferred drive and not falling over to
the non-preferred secondary or third swap drives.

Can someone from PTGui confirm these bugs and hopefully fix them?

[Joergen Geerds]

hi greg,

I just did a test with a 16GB RAM disk (under OSX 10.6), and ptgui
8.22 does fill up the ram disk very gracefully, loading the overflow
onto the raid, never filling it to the brim.
the bug therefore seems to be windows specific, maybe the OS doesn't
report the free space properly back to ptgui, so ptgui writes more
data than it should, and boom, bsod. just a thought.

joergen

[Greg Takacs]

Joergen,

The swap partition memory allocation can very possibly be an OS
specific issue. I must also add that sometimes the render works fine
and PTGui falls over to the secondary swap gracefully. But other times
it simply dies. There was another thread about this very same issue
not too long ago: http://groups.google.com/group/ptgui/browse_thread/thread/982da7a375542d63

PTGui support claimed that it must be due to the system drive getting
full. I had to reiterate that this is simply not the case as I have no
system drive listed in my swap list, only a RAM disk and a RAID0 array
with nothing on either of them and I have the very same problem. It is
intermittent, does not happen on every render, but it has happened
enough now that I completely abandoned the RAM drive idea. Truth be
told had the memory get managed properly there would be no need for
the RAM drive at all. That is until my SSDs come in and I end up
running out of space on them (I am planning to use them as both system
drives as well as swaps).

BTW, how does memory behave on OSX in PTGui? Why would you use a RAM
drive instead of straight RAM? Do you have the exact same memory
management problems that I have outlined, rendering anything over a
certain amount essentially useless?

I have done a bit more testing. Using 8.2.2 with 12GB RAM, stitching 7
10 megapixel 8bit tiffs. Memory as follows:

before starting PTGui:
37% Used Physical Memory
In use: 4733MB (2048MB in RAM disk)
Modified: 110MB
Standby: 1360MB
Free: 6072MB

While Running the Stitch of 11536x5768 blended jpeg, PTGui set to use
90% of RAM:
62% Used Memory
In Use: 7800MB
Modified: 417MB
Standby:2535MB
Free:1891MB

Disk drives are going crazy reading and writing while I have over 5GB
or RAM truly available (total of Standby and Free). Actually the total
available keeps going up as the warping progresses, PTGui's memory
usage drops from the peak of about 4.8GB to under 3GB.

There is clearly something stopping PTGui from using all my RAM.

If I remove the 2GB ramdisk the only thing that changes is that now I
have more in standby and free memory and 2GB less in used. PTGui does
not use any more RAM at all.

[PTGui Support]

Hi Greg (and Joergen),

There are so many details to memory handling and stitching speed issues,
many of which are hardware dependent, I just cannot and don't want to go
into details. I have some ideas for improvements which will get
implemented in future versions. For the moment it sometimes helps if you
reduce the number of threads PTGui uses.

Just a few remarks:

Setting PTGui RAM usage to 90% probably only slows down your PC as you
don't leave enough memory available for the OS and other applications.
Applications will start fighting for memory, trying to swap each other
onto disk. This can completely stall your computer.

PTGui can only crash itself, it can never crash or BSOD your computer.
If this happens there is a hardware or driver problem.

There seems to be a bug on some systems where PTGui gets reported that
there is enough space but subsequently is unable to create the temporary
file of that size. I haven't been able to reproduce this and it might be
a hardware specific problem but PTGui can be made more tolerant to this
instead of failing right away.

Joost

[Greg Takacs]

> Setting PTGui RAM usage to 90% probably only slows down your PC as you
> don't leave enough memory available for the OS and other applications.
> Applications will start fighting for memory, trying to swap each other
> onto disk. This can completely stall your computer.

Joost,

90% of 12GB is vastly different than 90% of 4GB. My system has 12GB
RAM. I think I'm leaving plenty of RAM for my other applications to
run with 1.2GB RAM.

> PTGui can only crash itself, it can never crash or BSOD your computer.
> If this happens there is a hardware or driver problem.

I personally never experienced the BSOD, just the error message about
the hard drives getting full and not falling over to the next swap
drive.

Could you also advise why PTGui never uses all the RAM that is
allocated to it? On my system I have not seen it go above 4.8GB. I
think it is obvious that the memory management did not have 12GB RAM
in mind when it was written. It hurts me to see all that free RAM and
my hard drives thrashing........

[Dakota Virtual Tours]

I was drawn to this thread because I too an experiencing the need for
faster processing but from the stories and examples above I may have
to consider myself "lucky". I just have a 3 year old Toshiba laptop
with 2 gigs of RAM and a 100GB hard drive that is 90% full. I shoot 6
positions with 9 exposures each (54 pictures) with the final pano
image of 4000 by 2000 jpg. The hardrive light is on all the time but
I've never experienced any crashes. Typically I shoot 6 to 8 houses a
day (real estate) and never use the Gui, I just use a DOS batch
program to evoke the batch stitcher. Maybe that is what saves me from
having problems? When I get to my office at the end of the day only
half of the jobs are generally processed so I just let the computer
run all night and in the morning all is done and well. So up to 8
houses with 6 scenes average with 54 pictures per scene totaling
around 48 scenes with 2592 images shot. Geez, no wonder why I can't
get a camera to last more than a year! PTGUI works fine though. I
have version 8.0

[Greg Takacs]

Dakota Virtual Tours,

I think you don't experience these issues because you are using the
typical machine that PTGui had in mind when its memory and disk
management was written. You have 2GB RAM not 12GB so you never see the
memory not getting utilized. It only shows up when your first 7 images
that are getting warped (or however many cores/threads you have
running) are not filling out your memory. Considering that even with 6
megapixel images running at 8 threads I only utilize 2.8GB that is
still way over your 2GB limit and hence has no effect on you. Even at
half the thread count it's down to 1.4GB which is still pretty much
all you want to run on 2GB RAM anyway.

You also use a single hard drive as swap, which will not show you the
issue with running out of space on your primary swap area and
graciously falling over onto the second swap. In your case if there is
not enough room to do your panorama you simply get an error message
before any kind of stitching beings telling you that you don't have
enough swap room to do the stitch.

You're also outputting 4,000x2,000 panos for real estate which is
significantly less than the 11,000x5,500 panoramas that I am trying to
render. This cuts down on any kind of warping and blending requirement
by a factor of 7.5. That makes a huge difference.

I am still on the opinion that PTGui is just not managing large
amounts of RAM efficiently and the multiple swap location feature has
not been fully tested. I still don't think there is anything else out
there that would do a better/faster or even comparably good/fast job
than PTGui but there is definitely room for improvement with the way I/
O and memory is handled with today's fast multi core CPUs and large
amounts of RAM available.

[PTGui Support]

Hi Greg,

Greg Takacs wrote:
>
>
>> Setting PTGui RAM usage to 90% probably only slows down your PC as you
>> don't leave enough memory available for the OS and other applications.
>> Applications will start fighting for memory, trying to swap each other
>> onto disk. This can completely stall your computer.
>
> Joost,
>
> 90% of 12GB is vastly different than 90% of 4GB. My system has 12GB
> RAM. I think I'm leaving plenty of RAM for my other applications to
> run with 1.2GB RAM.

I wouldn't call that plenty. You might open Task Manager before you
launch PTGui and go to the Performance tab: how much memory is used by
other applications?

>
>> PTGui can only crash itself, it can never crash or BSOD your computer.
>> If this happens there is a hardware or driver problem.
>
> I personally never experienced the BSOD, just the error message about
> the hard drives getting full and not falling over to the next swap
> drive.
>
> Could you also advise why PTGui never uses all the RAM that is
> allocated to it? On my system I have not seen it go above 4.8GB. I
> think it is obvious that the memory management did not have 12GB RAM
> in mind when it was written. It hurts me to see all that free RAM and
> my hard drives thrashing........

First of all please see 2.18:
http://www.ptgui.com/support.html#2_18

Also, are you sure you are running the 64 bit version of PTGui Pro?

Is write caching is enabled on all your drives? For USB drives this is
not the case by default.

Joost

[Greg Takacs]

I have found one problem with my initial test, I didn't have Norton
anti virus turned off. Once I turned it off, memory usage during the
warping process was what I have expected and speed has improved.
However blending performance was still abysmal. See further below.

> First of all please see 2.18:http://www.ptgui.com/support.html#2_18

I have read that article and I can concur with the fact that PTGui
will use more RAM that what's visible from task manager. PTGui t ask
will eat up 10GB but the system will report mosto f that RAM shareable
with other tasks and will treat it as available.

> Also, are you sure you are running the 64 bit version of PTGui Pro?

Yes. It's PTGui Pro 8.2.2

>
> Is write caching is enabled on all your drives? For USB drives this is
> not the case by default.

I am not using USB drives. It was another member who posted issues
with running out of disk space erros who used an external USB drive.
I'm only using a single internal RAID-0 array on my on-board Intel
RAID controller as my temp partition.

I have performed the Gigapixel Speed test:
http://groups.google.com/group/ptgui/msg/f2d85afc2ecebb23

I have also performed the same test using Autopano Giga 2.0.3:
http://groups.google.com/group/ptgui/msg/d69d43262f63f808

And here is my final assessment of this issue:
http://groups.google.com/group/ptgui/msg/60439db98067c9ba

Bottom line is that there are two problems:
1) Severe I/O boundness of the blending process on fast CPU with tons
of RAM. It is SEVERE.
The warping seems to be OK, but blending is clearly doing something
wrong.
2) Failure to fall back onto a secondary temp drive if the first drive
is rather small such as an 8GB RAM drive partition. This has nothing
to do with system partitions. Here is the other relevant thread for
this issue:
http://groups.google.com/group/ptgui/browse_thread/thread/982da7a375542d63/560ef18281bff10d

Will either of these two issues be addressed/acknowledged/fixed?

[Greg Takacs]

Joost, I missed one of your questions:

> I wouldn't call that plenty. You might open Task Manager before you
> launch PTGui and go to the Performance tab: how much memory is used by
> other applications?

As it is right now, with FireFox MSN messenger and Pidgin IM running,
I'm using 1360MB of RAM. That's 1.32GB. A bit more than the 10% I have
reserved. However as you can see from my other posts, this is clearly
not the issue. If you'd like I can certainly re-run the test with the
advised 67% memory allocation to see if it gets any better.

Regards,

Greg

[PTGui Support]

Greg Takacs wrote:
> Bottom line is that there are two problems:
> 1) Severe I/O boundness of the blending process on fast CPU with tons
> of RAM. It is SEVERE.
> The warping seems to be OK, but blending is clearly doing something
> wrong.
> 2) Failure to fall back onto a secondary temp drive if the first drive
> is rather small such as an 8GB RAM drive partition. This has nothing
> to do with system partitions. Here is the other relevant thread for
> this issue:
> http://groups.google.com/group/ptgui/browse_thread/thread/982da7a375542d63/560ef18281bff10d
>
> Will either of these two issues be addressed/acknowledged/fixed?

Hi Greg,

I have acknowledged both problems already recently on this forum, they
are being investigated. I'm glad you managed to find the main problem
(Norton AV) yourself though.

In the current version it may help if you reduce the number of threads
that PTGui is configured to use (Options/Advanced). This reduces the
fragmentation of disk IO and has been reported to actually increase the
speed on certain systems.

Joost

[Greg Takacs]

Joost,

I apologize for not utilizing the search function and realizing that
the issues have been acknowledged and hopefully worked on. I've been
using PTGui since 5.4 and this is the first time I've really felt the
urge/need to ask for support on an otherwise great app. In the
meantime I will run another test with fewer cores enabled.

--Greg

On Sep 15, 10:46 am, PTGui Support <supp...@ptgui.com> wrote:
> Greg Takacs wrote:
> > Bottom line is that there are two problems:
> > 1) Severe I/O boundness of the blending process on fast CPU with tons
> > of RAM. It is SEVERE.
> > The warping seems to be OK, but blending is clearly doing something
> > wrong.
> > 2) Failure to fall back onto a secondary temp drive if the first drive
> > is rather small such as an 8GB RAM drive partition. This has nothing
> > to do with system partitions. Here is the other relevant thread for
> > this issue:

> >http://groups.google.com/group/ptgui/browse_thread/thread/982da7a3755...