no performance plus with dual xeon and 8 x 1tb HDD raid0

10 views
Skip to first unread message

Querdenker Ltd.

unread,
Dec 10, 2009, 8:42:06 AM12/10/09
to PTGui Support
Hi,

my old pc with vista 64bit (Quad core Q6600, 8GB Ram, 3 x 2TB HDD
simultaneously accessed from ptgui)
to stich my gigapixel panorama it took 3days

my new computer
vista 64bit (windows 7 64bit crashed every time before finishing the
stitching)
2 x Intel 5520 2.26 GHz Xeon Quad Core
24 GB Ram (up to 144 GB possible but with 11000€ way to expensive)
8 x 1tb Samsung Spinpoint F3 HDD as striping volume (raid card
configuration not windows configuration)
only vista (with all updates) and ptgui nothing more installed

virtual cache and the hole work is going on the raid system (but the
windows system is installed on an old 200GB HDD)

after the Ram is full the I/O operation is about 600 MB/s in the
middle
when it starts going forward with the warping it is only 200MB/s in
the middle

now to the problem
why is this new system not significantly faster than the old system?
where is the fault? where is the possibility to get the stitching
faster?

I want to stitch way bigger panoramas, but with this slow system it
would take way too long

it is possible to update the 8x1tb raid to 12x1tb raid (other hdds are
too expensive, the hole system was not really cheap) but the
performance plus wouldn't be significantly

with only using RAM the system is extremely fast
the speedtest results http://hdview.at/speedtest/results.html speak
for itself

Jeffrey Martin

unread,
Dec 11, 2009, 4:57:57 AM12/11/09
to PTGui Support
Hi,

Stitching is the LEAST of your worries if you want to make "really
huge" panos.

Just start the rendering, and have a good weekend. Who cares :) But
then you have to open it in photoshop, and actually do things to the
image. that's the bottleneck.

Of course i'm interested to hear what your problem is, but don't dwell
on stitching speed as your main issue.

Querdenker Ltd.

unread,
Dec 11, 2009, 6:35:10 AM12/11/09
to PTGui Support
photoshop itself is no problem at all

the trick to get a fast workflow with photoshop is a written action

just use a small picture of the panorama and write a special action

a 3 gigapixel panorama finishes in about 6h (the question is, how
complicated the action is)

stitching is the bottleneck for me
and especially that there are no performance acceleration with such a
big workstation is frustrating
why is there no benefit with so much power?

Querdenker Ltd.

unread,
Dec 11, 2009, 6:37:48 AM12/11/09
to PTGui Support
one good thing, with the 8.3.5 version of ptgui my windows 7 isn't
crashing anymore

Joergen Geerds

unread,
Dec 11, 2009, 8:41:19 AM12/11/09
to PTGui Support
hi,

here are a couple of thoughts where you might find bottlenecks (not
sure how big your gigapixel pano is, but 3 days rendering on your
machine sounds like a 20-40gpx).

-set your raid0 blocks to the smallest amount possible, probably
around 16-64KB. I do not know what blocksize ptgui 8.3.5 under win7 is
using, but based on previous experiences, it is still on the smaller
size, so optimizing your raid0 towards that might help.

-partition your drives: create outer and inner partitions and use them
for two raid0 sets, i.e. raid_faster and raid_fast

-turn off your memory paging file: with 24GB ram i believe it's
totally unnecessary to have a paging file, especially if you have an
older/slower 200GB bootdisk. i have a win7 install on a relatively
slow disk (i am a mac user), and was really unhappy about the UI
speed, until I turned the paging off completely, and win7 became
finally responsive.

-look into your raid card: 600MB/s for 8 disks is rather slow, you
should see 750-850MB/s peaks with 500mb/s average. maybe your card
isn't optimal for this task, and hits a pci-e brick-wall (which I have
seen mentioned at barefeats.com quite often). there seem to be only
very few raid cards that have 8 sata channels and can do 1200mb/s
(which is the next brick-wall). maybe splitting it up into two 4-
channel raid cards may help?

-restart your computer before you start the render, and don't start
any other application before/while rendering (you are using the 64bit
versions of win7 and ptgui and you have set ptgui to 90%, didn't
you?). just walk away and leave it alone as elton brown would say on
his foodnetwork show. if you want to do something else, grab your
laptop.

those are the only things I can think of right now.

good luck

joergen

Querdenker Ltd.

unread,
Dec 11, 2009, 2:46:51 PM12/11/09
to PTGui Support
it is only a small 3gp pano :/

tested it with 16kb and 64kb blocks - 64kb got it slightly faster

to partition and inner and outer one is not necessary, windows is
writing first on the fastest end

turning off the memory paging file would be unwise... ptgui is
shutting down with memory error

my windows 7 system is now working, so my page file is on a veloci
raptor

the raid card is an asus pike 1068e it should be fast enough
especially it is also designed for sas not only sata

in vista I had peaks about 850 or 900 mb/s
in windows 7 only peaks of 550 and average about 230 mb/s

the raid card is on a expansion slot of my motherboard, so that I can
use my other 8 sas/sata on board

the problem is not the waiting, the problem is, that my old computer
is as fast as this configuration and that shouldn't be possible, it
has to be much faster

Joergen Geerds

unread,
Dec 11, 2009, 2:56:50 PM12/11/09
to PTGui Support
I do not know enough about windows system to give a real qualified
answer to your problem.
The only thing I can see is maybe the win7 driver for your raid isn't
working properly yet, so you probably have to wait for an update from
asus.

good luck

joergen

Querdenker Ltd.

unread,
Dec 11, 2009, 3:03:13 PM12/11/09
to PTGui Support
I had tested the system with vista, but it was not faster :/

my head is spinning, everything is much faster and better than my old
system, but it cannot stitch faster, that is ridiculous, if it wasn't
that expensive I would throw this hole computer out of the window, I
tested so many possible weak point the last 3 weeks, but I don't find
an answer :(

Joergen Geerds

unread,
Dec 11, 2009, 3:54:32 PM12/11/09
to PTGui Support
I was in a similar situation 6 months ago, banging my head against the
wall and no idea why my computer wasn't performing with ptgui as it
was supposed to (about a comparable setup, only a mac). the recent
8.3.x updates made me happier in terms of processing speed, so I
stopped complaining as much. but really fine-tuning a system isn't
always possible, and with ptgui so much depends on the application,
and not so much on the system... as I said, I don't know enough about
win7 (other than I am really not big fan of MS) to make detailed
recommendations.

joergen

Querdenker Ltd.

unread,
Dec 11, 2009, 4:17:06 PM12/11/09
to PTGui Support
it is quite frustrating...

now I'm thinking of putting more ram into the system... but I'm
already 1000€ over my estimated budget :/

when I'm crazy enough, I will put 32gb more into my system

PTGui Support

unread,
Dec 13, 2009, 5:26:23 AM12/13/09
to pt...@googlegroups.com
Could you post the project file (just the .pts) to get an impression of
the size of the panorama?

And could you post the output of Help -> System Information

Are you actually running the 64 bit version of PTGui 8.3.5, not the 32
bit version or an older version?

Joost

On 10-Dec-09 14:42, Querdenker Ltd. wrote:
> Hi,
>
> my old pc with vista 64bit (Quad core Q6600, 8GB Ram, 3 x 2TB HDD
> simultaneously accessed from ptgui)
> to stich my gigapixel panorama it took 3days
>
> my new computer
> vista 64bit (windows 7 64bit crashed every time before finishing the
> stitching)
> 2 x Intel 5520 2.26 GHz Xeon Quad Core
> 24 GB Ram (up to 144 GB possible but with 11000� way to expensive)

Querdenker Ltd.

unread,
Dec 13, 2009, 8:44:40 AM12/13/09
to PTGui Support
uploaded pano.pts project file

yes I'm running the 64bit version of ptgui

here the output of Help -> System Information

CPU: GenuineIntel

CPUInfo:
0x00000000: 0x0000000b 0x756e6547 0x6c65746e 0x49656e69
0x00000001: 0x000106a5 0x02100800 0x009ce3bd 0xbfebfbff
0x00000002: 0x55035a01 0x00f0b2e4 0x00000000 0x09ca212c
0x00000003: 0x00000000 0x00000000 0x00000000 0x00000000
0x00000004: 0x00000000 0x00000000 0x00000000 0x00000000
0x00000005: 0x00000040 0x00000040 0x00000003 0x00001120
0x00000006: 0x00000003 0x00000002 0x00000001 0x00000000
0x00000007: 0x00000000 0x00000000 0x00000000 0x00000000
0x00000008: 0x00000000 0x00000000 0x00000000 0x00000000
0x00000009: 0x00000001 0x00000000 0x00000000 0x00000000
0x0000000a: 0x07300403 0x00000044 0x00000000 0x00000603
0x0000000b: 0x00000000 0x00000000 0x000000ff 0x00000002
0x80000000: 0x80000008 0x00000000 0x00000000 0x00000000
0x80000001: 0x00000000 0x00000000 0x00000001 0x28100800
0x80000002: 0x65746e49 0x2952286c 0x6f655820 0x2952286e
0x80000003: 0x55504320 0x20202020 0x20202020 0x45202020
0x80000004: 0x30323535 0x20402020 0x37322e32 0x007a4847
0x80000005: 0x00000000 0x00000000 0x00000000 0x00000000
0x80000006: 0x00000000 0x00000000 0x01006040 0x00000000
0x80000007: 0x00000000 0x00000000 0x00000000 0x00000100
0x80000008: 0x00003028 0x00000000 0x00000000 0x00000000

SIMD current: SSE2
, supported: SSE2
Platform: win64/win64
Number of CPUs: 16
Physical memory: 24567 MB
Virtual memory: 8388607 MB
MemoryStatus:
MemoryLoad: 10
TotalPhys: 24567,1 MB
AvailPhys: 21990 MB
TotalPageFile: 49132,4 MB
AvailPageFile: 46316,1 MB
TotalVirtual: 8,38861e+006 MB
AvailVirtual: 8,38845e+006 MB
PageFaultCount: 7380
WorkingSetSize: 24,375 MB (peak 25,1758 MB)
QuotaPagedPoolUsage: 0,260139 MB (peak 0,261703 MB)
QuotaNonPagedPoolUsage: 0,023468 MB (peak 0,0238876 MB)
PagefileUsage: 9,25391 MB (peak 9,66016 MB)
PrivateUsage: 9,25391 MB
Has default project: No

Config:
AllowOver4GB: 1
AutoMaxRamPercentage: 1
AutoReloadModifiedSourceImages: 0
CPGeneratorNumCPsPerImagePair: 10
CPTabSplitterHeight: 96
CheckCompatiblePano12: 0
CheckForBetaUpdates: 1
CheckPano12Dll: 0
ConfigVersion: 80305
FormPos/BatchStitcher/Height: 399
FormPos/BatchStitcher/Left: -816
FormPos/BatchStitcher/Top: 39
FormPos/BatchStitcher/Width: 731
FormPos/ControlPoints/Height: 300
FormPos/ControlPoints/Left: 760
FormPos/ControlPoints/Top: 430
FormPos/ControlPoints/Width: 400
FormPos/MainWindow/Height: 1036
FormPos/MainWindow/Left: 548
FormPos/MainWindow/Top: 53
FormPos/MainWindow/Width: 1185
FormPos/NumericalTransform/Left: 860
FormPos/NumericalTransform/Top: 570
FormPos/Options/Height: 600
FormPos/Options/Left: 406
FormPos/Options/Top: 386
FormPos/Options/Width: 600
FormPos/PanoramaEditor/Height: 300
FormPos/PanoramaEditor/Left: 119
FormPos/PanoramaEditor/Top: 103
FormPos/PanoramaEditor/Width: 400
GridLayouts/ImageParams/ColWidth0: 120
GridLayouts/ImageParams/ColWidth1: 120
GridLayouts/ImageParams/ColWidth10: 50
GridLayouts/ImageParams/ColWidth11: 50
GridLayouts/ImageParams/ColWidth12: 50
GridLayouts/ImageParams/ColWidth13: 50
GridLayouts/ImageParams/ColWidth14: 50
GridLayouts/ImageParams/ColWidth15: 50
GridLayouts/ImageParams/ColWidth16: 50
GridLayouts/ImageParams/ColWidth17: 50
GridLayouts/ImageParams/ColWidth18: 50
GridLayouts/ImageParams/ColWidth19: 50
GridLayouts/ImageParams/ColWidth2: 40
GridLayouts/ImageParams/ColWidth20: 50
GridLayouts/ImageParams/ColWidth21: 50
GridLayouts/ImageParams/ColWidth22: 50
GridLayouts/ImageParams/ColWidth23: 50
GridLayouts/ImageParams/ColWidth24: 80
GridLayouts/ImageParams/ColWidth25: 50
GridLayouts/ImageParams/ColWidth26: 50
GridLayouts/ImageParams/ColWidth27: 50
GridLayouts/ImageParams/ColWidth28: 50
GridLayouts/ImageParams/ColWidth29: 50
GridLayouts/ImageParams/ColWidth3: 50
GridLayouts/ImageParams/ColWidth4: 50
GridLayouts/ImageParams/ColWidth5: 50
GridLayouts/ImageParams/ColWidth6: 100
GridLayouts/ImageParams/ColWidth7: 50
GridLayouts/ImageParams/ColWidth8: 50
GridLayouts/ImageParams/ColWidth9: 50
GridLayouts/ImageParams/RowHeight: 100
GridLayouts/SourceImages/ColWidth0: 120
GridLayouts/SourceImages/ColWidth1: 400
GridLayouts/SourceImages/ColWidth2: 60
GridLayouts/SourceImages/ColWidth3: 60
GridLayouts/SourceImages/RowHeight: 100
LicenseAccepted/8.2.1.Pro: 1
LicenseAccepted/8.3.4.Pro: 1
LicenseAccepted/8.3.5.Pro: 1
MRUList0: C:\Users\Chef\Desktop\Gigapixel\pano.pts
MRUList1: C:\Users\Chef\Desktop\Gigapixel\ppano4.pts
MRUList2: C:\Users\Chef\Desktop\Gigapixel\pano3.pts
MRUList3: C:\Users\Chef\Desktop\Gigapixel\pano2.pts
MRUListSize: 4
MaxRamPercentage: 90
PTViewerAppQuickTime: 0
Pano12WarpPreDownSample: 0
PreviousAutoCheckForUpdates: 14588
RegNamePro: Daniel Richter
TempDir: H:\
TempDir2:
TempDir3:
TempDirPreferred: 1
TempDirPreferred2: 0
TempDirPreferred3: 0
WarnSendBatchStitcherCloseProject: 1

Global Config:

KSmith

unread,
Dec 13, 2009, 7:46:21 PM12/13/09
to PTGui Support
I don't know if this will help, but I hear your pain.

Regarding the paging file, I picked up something recently from working
with video:
Move the paging file from your boot drive to a fast drive that is not
being used for anything else (reading or writing) during stitching.
Hopefully you'll see some improvement. It did help me with Vegas on
32bit Vista.

Keith

PTGui Support

unread,
Dec 14, 2009, 6:32:47 AM12/14/09
to pt...@googlegroups.com
The system info looks perfectly fine.

Stitching your project will take a long time due to the fullframe
fisheye output projection. PTGui is optimized for rectilinearm,
cylindrical or equirectangular output. For all other projections no
cropped layers will be used internally.

Try switching to equirectangular output and it will save hours of
stitching time.

Joost

Querdenker Ltd.

unread,
Dec 14, 2009, 4:51:19 PM12/14/09
to PTGui Support
It got a little bit faster :)

in 3 days I have 72gb Ram, maybe this is enough for only using ram ^^
I'm quite curious how the performance react with so much ram
(still have empty ram slots but no money left ^^ )

Joergen Geerds

unread,
Dec 14, 2009, 5:06:26 PM12/14/09
to PTGui Support
I spoke with joost, and ptgui does indeed move much larger memory
chunks to the temp disk now, so if you like, try setting your RAID0
block settings to something larger, i.e. 1-4MB. maybe it gives you a
hair more speed.

joergen

Querdenker Ltd.

unread,
Dec 16, 2009, 7:17:38 AM12/16/09
to PTGui Support
made so many tests with different setup of my system

conclusion
- rectangular is way the fastest stitching possibility
- stitching in vista is much faster than in windows 7
- you cannot have enough ram ;) tomorrow I will have 72gb ^^
- raid systems are not as fast as hoped
- if you want to stitch very big gigapixel panoramas, you need at
least a workstation (4000€ and more), or you have month of time ;)

what I wish for:
- gpu support for way more performance
- a linux version ^^
- saving of intermediate steps, so that I don't have to start at the
beginning, when the system crashes - or when I only want to stitch
when I'm sleeping (doesn't matter if I have to stitch 2 more days
cause of that, but a little security in a one month stitch would be
great)

PTGui Support

unread,
Dec 16, 2009, 8:10:14 AM12/16/09
to pt...@googlegroups.com
On 16-Dec-09 13:17, Querdenker Ltd. wrote:
> made so many tests with different setup of my system
>
> conclusion
> - rectangular is way the fastest stitching possibility

Rectilinear, cylindrical or equirectangular in fact..

> - stitching in vista is much faster than in windows 7

This should make no difference at all for PTGui. If there's any
difference in speed this would be due to hardware or driver issues. I
suspect your particular RAID system is not working like it should in Win 7.

> - you cannot have enough ram ;) tomorrow I will have 72gb ^^
> - raid systems are not as fast as hoped
> - if you want to stitch very big gigapixel panoramas, you need at
> least a workstation (4000� and more), or you have month of time ;)

A simple gigapixel panorama should be doable in one to a few hours at
most one a modern computer (say, 64 bit & 8GB ram). But the following
things will cause considerable slow down:

- other projection than the above mentioned three
- HDR / exposure fusion
- Viewpoint correction
- using enblend/smartblend for very large panoramas
- using RAW source files. Better to batch convert to TIFF first

> what I wish for:
> - gpu support for way more performance

The benefits for gigapixel panoramas will be neglegible, really, since
memory and disk speed are the bottle necks.

> - a linux version ^^
> - saving of intermediate steps, so that I don't have to start at the
> beginning, when the system crashes - or when I only want to stitch
> when I'm sleeping (doesn't matter if I have to stitch 2 more days
> cause of that, but a little security in a one month stitch would be
> great)

This has been asked before but would be very complicated to implement.

Joost

Joergen Geerds

unread,
Dec 16, 2009, 11:12:04 AM12/16/09
to PTGui Support
On Dec 16, 8:10 am, PTGui Support <supp...@ptgui.com> wrote:
> > conclusion
> > - rectangular is way the fastest stitching possibility
> Rectilinear, cylindrical or equirectangular in fact...
> But the following things will cause considerable slow down:
>
> - other projection than the above mentioned three
> - HDR / exposure fusion
> - Viewpoint correction
> - using enblend/smartblend for very large panoramas
> - using RAW source files. Better to batch convert to TIFF first

how much less performance do we get from the code when we use
mercator, or from modified cylindrical projections? I know panini and
VP's aren't optimized at all and aren't using cropped tiles for temp,
but are all other cases also affected by that, or at least partially?
I also think that HDR projects are far too popular now to not to
benefit from the recent speed gains...

joergen

imag...@gmail.com

unread,
Dec 16, 2009, 12:04:50 PM12/16/09
to PTGui Support


> > - saving of intermediate steps, so that I don't have to start at the
> > beginning, when the system crashes - or when I only want to stitch
> > when I'm sleeping (doesn't matter if I have to stitch 2 more days
> > cause of that, but a little security in a one month stitch would be
> > great)
>

this can kind of be done already, while not an automated solution it
does work and is completely doable today.

In the Advanced section of Create Panorama tab you can choose which
source images to include in the stitch. If you process a sub selection
you can get a partial or imtermediate stitch section which can be
combined in PS later. This works and is really quite easy to do. Note,
it puts the subset in place so all you have to do is merge layers in
PS, the alignment is already done. If you could re-select the entire
set of images after selecting a subset it would be even easier.

Regards,

Robert

PTGui Support

unread,
Dec 17, 2009, 4:31:23 AM12/17/09
to pt...@googlegroups.com
On 16-Dec-09 17:12, Joergen Geerds wrote:
> On Dec 16, 8:10 am, PTGui Support<supp...@ptgui.com> wrote:
>>> conclusion
>>> - rectangular is way the fastest stitching possibility
>> Rectilinear, cylindrical or equirectangular in fact...
>> But the following things will cause considerable slow down:
>>
>> - other projection than the above mentioned three
>> - HDR / exposure fusion
>> - Viewpoint correction
>> - using enblend/smartblend for very large panoramas
>> - using RAW source files. Better to batch convert to TIFF first
>
> how much less performance do we get from the code when we use
> mercator, or from modified cylindrical projections?

That really depends on the project and the hardware..

> I know panini and
> VP's aren't optimized at all and aren't using cropped tiles for temp,
> but are all other cases also affected by that, or at least partially?
> I also think that HDR projects are far too popular now to not to
> benefit from the recent speed gains...

They definately do benefit from the recent improvements in virtual
memory handling.

Also the HDR/fusion algorithms are already optimized for speed in many
ways. But they're complex algorithms requiring a fair bit of processing.

Joost

Querdenker Ltd.

unread,
Dec 18, 2009, 3:45:53 AM12/18/09
to PTGui Support
On 16 Dez., 18:04, "image...@gmail.com" <image...@gmail.com> wrote:
> this can kind of be done already, while not an automated solution it
> does work and is completely doable today.
>
> In the Advanced section of Create Panorama tab you can choose which
> source images to include in the stitch. If you process a sub selection
> you can get a partial or imtermediate stitch section which can be
> combined in PS later. This works and is really quite easy to do. Note,
> it puts the subset in place so all you have to do is merge layers in
> PS, the alignment is already done. If you could re-select the entire
> set of images after selecting a subset it would be even easier.
>
> Regards,
>
> Robert

this helped very much :)

so I can partition my work, that ptgui only uses my ram :)
it is much faster :)

is there a possibility to uncheck more than one picture at the same
time?

PTGui Support

unread,
Dec 18, 2009, 4:00:10 AM12/18/09
to pt...@googlegroups.com
On 18-Dec-09 09:45, Querdenker Ltd. wrote:
> is there a possibility to uncheck more than one picture at the same
> time?

Yes, you can use shift+arrow keys, and then press the space bar.

Joost

Reply all
Reply to author
Forward
0 new messages