Batch processing over 400000 PTO files.

[Mick Hellstrom]

Has anyone had any luck in getting ptbatchergui to process many many PTO files?
While I was building our house I built a pano rig of 5 camera, and took photos every minute.
I now have over 400,000 panos that I want to batch process.
I've built scripts to generate the PTO files, as all the cameras remained in the same position for months, with the occasional adjustment here and there.

Currently ptbatcher can only seem to handle only 200 PTOs at a time, and often crashes.
There seems to be no "exit upon completion" flag, and no way of detecting whether batch runs have finished from command line.
So, it means that it's very labour intensive.

Is there any way of adding in a simple option on the command line which returns the number of completed PTOs, so then I can build a script to
automatically pass new jobs to ptbatcher. Then I can run this on several PCs and get the job done in no time.

[Stefan Peter]

Dear Mike Hellstrom

Am 07.02.2015 um 00:26 schrieb Mick Hellstrom:
> Has anyone had any luck in getting ptbatchergui to process many many PTO
> files?

No, at least not me. I would recommend the use of pto2mk. This command
line utility will create a make file from the pto file which in turn can
be executed using make. In conjunction with a little scripting, the
whole process can be made running automatically.

> I now have over 400,000 panos that I want to batch process.

I'd hate to have to store and backup this number of images.

With kind regards

Stefan Peter

[T. Modes]

Am Samstag, 7. Februar 2015 00:26:19 UTC+1 schrieb Mick Hellstrom:

Has anyone had any luck in getting ptbatchergui to process many many PTO files?

PTBatcherGUI was not written for this purpose. So nobody tested this case. It works fine with a lower number of projects than your extreme case.
 

While I was building our house I built a pano rig of 5 camera, and took photos every minute.
I now have over 400,000 panos that I want to batch process.
I've built scripts to generate the PTO files, as all the cameras remained in the same position for months, with the occasional adjustment here and there.

For this use case you need only one project file and do it on the command line: Create one project file. Then call nona with the changed images and enblend.
see http://wiki.panotools.org/Panorama_scripting_in_a_nutshell#Simple_command-line_stitching for a description
 

There seems to be no "exit upon completion" flag, and no way of detecting whether batch runs have finished from command line.

The development version contains already this function.

[Gnome Nomad]

I have only ~48000 images in my collection, they only take up ~500GB. Also, from what Mike H said earlier, these were cameras mounted outside. Such cameras might not have very high resolution.

[Stefan Peter]

Dear Gnome Nomad

On 07.02.2015 10:27, Gnome Nomad wrote:
> I have only ~48000 images in my collection, they only take up ~500GB.

Way to go from 48'000 images to 400`000 panos x 5 images ;)

With kind regards

Stefan Peter

--
Any technology that does not appear magical is insufficiently advanced."
~ Gregory Benford

[Gnome Nomad]

Yah, a bit less than a factor of 10. What resolution are the source images?

With 6TB internal hard drives available, storage wouldn't be a problem.

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/54D639EA.3030908%40swissonline.ch.
For more options, visit https://groups.google.com/d/optout.

[Mick Hellstrom]

High everyone,
I've since started using pto2mk, (thanks Stefan), which was exactly what I was looking for. I've updated my scripts to use it, and it's beein chugging away for the last couple of days.
For those interested:
I built a pano rig of 5 cheap cameras, (10 years ago). So they have a res of 480x640, (flipped on the side).
I took a photo every minute from sunrise to sunset from Mar 2004 to Jan 2006.
There was a PC in a box stuck high up in a tree in the front yard.
I burnt around 20 DVDs for the raw images: around 100GB and over 400,000 images.

Each hour produces about an 8MB MP4, each day about 100MB MP4.

The raw images + half way through the conversion to MP4 consumes about 250GB.

The cameras stayed put the whole time, (which was the intent), with the occasional PC disk crash, etc, but overall quite successful.

I'll post it to youtube and post a link for anyone interested.

[Mick Hellstrom]

On Saturday, February 7, 2015 at 8:06:28 PM UTC+11, T. Modes wrote:

Am Samstag, 7. Februar 2015 00:26:19 UTC+1 schrieb Mick Hellstrom:

Has anyone had any luck in getting ptbatchergui to process many many PTO files?

PTBatcherGUI was not written for this purpose. So nobody tested this case. It works fine with a lower number of projects than your extreme case.
 

Yes, seems to be the case. It works very well otherwise.

 

While I was building our house I built a pano rig of 5 camera, and took photos every minute.
I now have over 400,000 panos that I want to batch process.
I've built scripts to generate the PTO files, as all the cameras remained in the same position for months, with the occasional adjustment here and there.

For this use case you need only one project file and do it on the command line: Create one project file. Then call nona with the changed images and enblend.
see http://wiki.panotools.org/Panorama_scripting_in_a_nutshell#Simple_command-line_stitching for a description

Hmmm. I might investigate. It should be much faster without the many forks used with the Makefile method.
 

 

There seems to be no "exit upon completion" flag, and no way of detecting whether batch runs have finished from command line.

The development version contains already this function.

 Thanks.

[Mick Hellstrom]

The attached image..... I know everyone will say it's not a good stitch, but I had to trade off a good front stitching vs bad distance stitching.
Parallalex errors are a bugger. Especially since the cameras were mounted "oddly".