PTGUI stops responding and crash (trial versions 12 and 13)

[Michal Heppler]

Hi,

I found an issue with stitching pano, where the PTGui instantly crashes.

I have a pano (~130 36MPix RAW photos). The problem is that the photos was
captured in the middle of river and there is a problem with automated
control points (this is not problem, I understand why).

When I want to add some manual points, everything works fine, until I
need precise view. When I zoom on pictures to see some details in wider
context, the UI stops responding.

I can see high mem usage, I can see hugh CPU usega, but after few
moments cpu load drop. The application stays in unresposible state until
manually killed.

The same situation is with version 12.27 and 13.0, the same problem is
on PC with 40GB RAM...

Another problem is that the application is sometimes OOMKilled due
stitching. Memory usage is set to auto. Even if PTGui is almost only
running application.

I saw generic recommendation in the FAQ for RAM, but what is recommended
RAM for panos from 100 to 150 36MPix RAWs?


Thank you and have a nice day, Michal Heppler

--

_________________________________________
/ Přístup Zatraceného Hlupce Johnsona k \
| hudbě byl stejný jako jeho přístup k |
| jakémukoliv jinému vědeckému či |
| kulturnímu poli, která jím byla |
| ovlivněna jako brambory silnými |
| pozdními mrazy. Udělej to co |
| nejhlasitější, říkal. Co největší. |
| Všeobjímající. A proto byly obrovské |
| varhany Neviditelné univerzity jediné |
| varhany na světě, na něž se dala zahrát |
| celá symfonie napsaná pro bouři a |
| čvachtání rozšlápnutých žab. |
| |
\ -- T. Pratchett: Otec prasátek /
-----------------------------------------

[PTGui Support]

Hi Michal,

I think in either case it's caused by PTGui using more memory than
configured. It shouldn't do this but I've seen it happen too under
Linux. I will investigate further.

If you have some spare disk space, you could try adding it as swap. It's
difficult to estimate how much is needed, I'd guess somewhere between
100 and 200 GB.

sudo fallocate -l 200G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

If you still see it OOMkilled, try disabling it temporarily:
https://gist.github.com/t27/ad5219a7cdb7bcb977deccbc48a480d5

I will also try to reproduce the issue.

Kind regards,

Joost Nieuwenhuijse
www.ptgui.com

[Michal Heppler]

Hi,

thank you... I made some reorganization and I can see improvement.

I had 40GB RAM and 16GB swap, I increased swap to ~40GB, so I have
80GB of "virtual" RAM. I set RAM usage to 35GB and GPU RAM to 2GB
(it is shared, as it is on laptop). The RAM usage is around this limit,
but swap... The PTGui is killed in the moment, when consumes whole swap.
I think, disabling OOMKiller will not help, as the system is really out
of memory. But I will try.

This behavior seems OK to me, as the system itself stays responsible,
but still, this behavior is in conflict with expectation, but probably
in harm with https://gist.github.com/t27/ad5219a7cdb7bcb977deccbc48a480d5.

So, your recommendation for this size of pano is to have 200 to 250GB of RAM +
SWAP? I think, this is good to know and you can add it to FAQ chapter 3.11,
because, under Linux, 32GB is not definitely enough for pano of this
size.

(I have BTRFS on whole disk, so swapfile is not an option)

The improvements in version 13 are visible.

Have a nice day. Michal Heppler

V Thu, Feb 06, 2025 at 11:44:55AM +0100, 'PTGui Support' via PTGui Support napsal(a):

> --
> You received this message because you are subscribed to the Google Groups "PTGui Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/ptgui/42919483-c56d-489a-ab57-5019298dbb95%40ptgui.com.

--

_________________________________________
/ Bylo to zvláštní, když byla Sibyla v \
| Pseudopolském dvoře. Byl to jeden z |
| mnoha domů, který ještě před nedávnem |
| patřil Berankinově rodině, ale Sibyla |
| ho darovala Hlídce. Vyrůstala v něm |
| jako dívka. Byl to její domov. |
| |
| Jako kdyby do otrlých a okoralých duší |
| policistů prosáklo něco podobného |
| vzdálenému pochopení. Muži, kteří |
| nebyli právě proslulí svou elegancí a |
| uhlazenými způsoby, se přistihovali, |
| jak si čistí boty, když vstupují |
| dovnitř, a uctivě si sundávají přilby. |
| |
| Začali také poněkud jinak mluvit, |
| pomalu a váhavě, protože v duchu |
| pečlivě kontrolovali, co chtějí řici, |
| aby včas stihli vynechat hrubší výrazy. |
| Někdo dokonce našel koště a smetl smetí |
| do míst, kde nebylo tolik vidět. |
| |
\ -- T. Pratchett: Buch! /
-----------------------------------------

[PTGui Support]

In fact even 32G Ram and 32G swap should be sufficient. When PTGui
exceeds its configured memory limit, it will start writing to temp
files, and does not allocate any additional RAM.

This is how it should work, and it does so on Windows and Mac. But I've
also seen issues on Linux. I suspect it has something to do with the
temp files not getting flushed to disk immediately, so they take RAM
space in the VM cache until flushed. This results in the OOM.

The fact that we have several different file systems on linux doesn't
make this any easier; using btrfs might aggravate the problem, I'm not
sure. I need to investigate.

Kind regards,

Joost Nieuwenhuijse
www.ptgui.com

[Michal Heppler]

Joost, mystery solved!

It is easier then expected.

I started investigation based on described workflow...
And I found the source of troubles. The first question was where
are this temp files:

$ lsof -p <PTGui PID>
...
...
PTGui 92604 16u unix 0x00000000aa2c6941 0t0 1219050 /tmp/...(deleted) type=STREAM (LISTEN)
...

(this is for ilustration)

And here is it:

$ mount | grep '\/tmp'
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,nr_inodes=1048576,inode64)

OK, so simply changed folder for temporary files to not-ram-based FS
and this fixed the problem.

The hidden detail is this signature deleted. I was searching for
existing temp files, but they are deleted.

(For explanation: there is a trick, you can open a file, then you can
delete them, but you can write/read etc from/to this file until you
close them. So you do not see any existing file, but it consumes disk
space.)

Now, the PTGui respects set memory limit and I am able to stitch pano.
Another problem is that this stitching eats all available disk space,
but this is my problem, not in PTGui.

The problem with zoom in control points viewer still persists...


Have a nice day, Michal Heppler


V Thu, Feb 06, 2025 at 01:31:45PM +0100, 'PTGui Support' via PTGui Support napsal(a):

> To view this discussion visit https://groups.google.com/d/msgid/ptgui/81a89521-c151-43da-b8d0-b9f580271832%40ptgui.com.

--

_________________________________________
/ Organizovaně se zbavit zastaralých věcí \
| je jedinou bezpečnou metodou, kterou |
| může podnik zaměřit představivost a |
| energii svých pracovníků na inovace. |
| (str. 112) |
| |
\ -- Drucker /
-----------------------------------------

[PTGui Support]

Hi Michal,

Good find!

On 07-02-2025 09:34, 'Michal Heppler' via PTGui Support wrote:
> $ lsof -p <PTGui PID>
> ...
> ...
> PTGui 92604 16u unix 0x00000000aa2c6941 0t0 1219050 /tmp/...(deleted) type=STREAM (LISTEN)

But are you using PTGui 12 or 13?

Because in v13 PTGui should be using /var/tmp as the default temporary
folder, this was changed exactly for this reason. It will fall back to
/tmp only if /var/tmp does not exist.

> (For explanation: there is a trick, you can open a file, then you can
> delete them, but you can write/read etc from/to this file until you
> close them. So you do not see any existing file, but it consumes disk
> space.)

Yes this is what PTGui does. It is to ensure that no temporary files are
left behind if PTGui aborts anormally.

> The problem with zoom in control points viewer still persists...

Are you using v12 or v13? Running under wayland or x11? Please note that
v12 must be run with XDG_BACKEND=x11.

Joost

[Michal Heppler]

V Fri, Feb 07, 2025 at 11:07:21AM +0100, 'PTGui Support' via PTGui Support napsal(a):

> Hi Michal,
>
> Good find!
>
> On 07-02-2025 09:34, 'Michal Heppler' via PTGui Support wrote:
> > $ lsof -p <PTGui PID>
> > ...
> > ...
> > PTGui 92604 16u unix 0x00000000aa2c6941 0t0 1219050 /tmp/...(deleted) type=STREAM (LISTEN)
>
> But are you using PTGui 12 or 13?
>
> Because in v13 PTGui should be using /var/tmp as the default temporary
> folder, this was changed exactly for this reason. It will fall back to /tmp
> only if /var/tmp does not exist.

I have symlink - /var/tmp points to /tmp :(

I think, it will be nice to add even something like ~/.cache/PTGui,
because the /home is usually larges disk. And stitching pano can eat
hundreds Gigs of disk space. Or add a note to FAQ 3.11 about disk usage...

>
> > (For explanation: there is a trick, you can open a file, then you can
> > delete them, but you can write/read etc from/to this file until you
> > close them. So you do not see any existing file, but it consumes disk
> > space.)
>
> Yes this is what PTGui does. It is to ensure that no temporary files are
> left behind if PTGui aborts anormally.
>
> > The problem with zoom in control points viewer still persists...
>
> Are you using v12 or v13? Running under wayland or x11? Please note that v12
> must be run with XDG_BACKEND=x11.
>

This behavior is the same over v12 and v13. I have Wayland and with XDG_BACKEND=x11 on v12 nothing changed.
I will try it with X11 later, for sure...

> Joost

>
> --
> You received this message because you are subscribed to the Google Groups "PTGui Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/ptgui/4080ab61-2b26-4e28-8e0d-7bd2adf95b14%40ptgui.com.


--Michal Heppler

--

_________________________________________
/ Učenost na ženě jako chomout na Pegasu. \
| |
\ -- Procházka A. /
-----------------------------------------

[PTGui Support]

> I have symlink - /var/tmp points to /tmp

That's weird, which linux distribution is this? /var/tmp is supposed to
be persistent.

> This behavior is the same over v12 and v13. I have Wayland and with
XDG_BACKEND=x11 on v12 nothing changed.
> I will try it with X11 later, for sure...

I'll investigate, thanks.

Joost

[Michal Heppler]

V Fri, Feb 07, 2025 at 04:23:36PM +0100, 'PTGui Support' via PTGui Support napsal(a):

> > I have symlink - /var/tmp points to /tmp
>
> That's weird, which linux distribution is this? /var/tmp is supposed to be
> persistent.

This is not problem of spec. distro, but sin from the past. It is rolling
updates distro installed many years ago, which survived "few" laptops.

In the past, there was a problem with systemd, which had a problem
with /var/tmp/. I don't remember details yet, but workaround was
symlink to /tmp. And this is one of those things do-it-and-forget until
now...

>
> > This behavior is the same over v12 and v13. I have Wayland and with
> XDG_BACKEND=x11 on v12 nothing changed.
> > I will try it with X11 later, for sure...
>
> I'll investigate, thanks.

The PTGui works fine under X11 session. I was able to create manually
control points without any troubles and slowdown.


Btw. why the "Small planet" view consumes so much resources during
stitchig? The same pano with fisheye or any other view stitched in ~20
minutes, but small planet... After 12 hours only ~60% done 120GB
temp files on disk and 80GB RAM+SWAP... And 7TB written to disk (~60
times rewritten all temp files). I stopped it, because I like my SSD ;)
(But it was progressing now, not crashing - huge improvement!)

I am attaching small script for monitor resources usage.

--Michal Heppler

>
> Joost
>
> --
> You received this message because you are subscribed to the Google Groups "PTGui Support" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to ptgui+un...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/ptgui/76f05111-081e-453a-b511-4504fb0c2f0f%40ptgui.com.

--

_______________________________________
/ Musíš se mnoho učiti, abys poznal, že \
| málo víš. |
| |
\ -- Montaigne /
---------------------------------------

[PTGui Support]

On 2/9/25 13:40, 'Michal Heppler' via PTGui Support wrote:
>>> This behavior is the same over v12 and v13. I have Wayland and with
>> XDG_BACKEND=x11 on v12 nothing changed.
>>> I will try it with X11 later, for sure...
>>
>> I'll investigate, thanks.
>
> The PTGui works fine under X11 session. I was able to create manually
> control points without any troubles and slowdown.

Ok, good to know.

> Btw. why the "Small planet" view consumes so much resources during
> stitchig? The same pano with fisheye or any other view stitched in ~20
> minutes, but small planet... After 12 hours only ~60% done 120GB
> temp files on disk and 80GB RAM+SWAP... And 7TB written to disk (~60
> times rewritten all temp files). I stopped it, because I like my SSD ;)
> (But it was progressing now, not crashing - huge improvement!)

The pipeline is more optimized for equirectangular output. Those wide
circular projections are more difficult to do efficiently due to the
source images in the outer ring which stretch across the entire output,
so the source data is accessed in a very non-sequential way. If the
source images don't fit in ram then data is constantly swapped in and out.