Feature requests for Multiblend 2.0?

[Monkey]

It's been several years but I'm working on a new version of Multiblend that's a bit faster, produces much nicer results, and will blend much bigger mosaics.

Does anyone have any feature requests for a blender that I could consider incorporating?

Along the same lines, does anyone use Enblend's colour space features? Do they produce notably more "correct" results, or just different ones? I've added an approximately linear RGB mode to Multiblend, but it doesn't seem to produce great results so I'll only be leaving it there as a curiosity.

[Gunter Königsmann]

gimp-like memory management that allows to handle target images that are bigger than the RAM. don't know if it is already implemented. 

The gimp splits big images into tiles that small enough so a few of them fit into the RAM and then tries to work on these tiles one at a time. 

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/c0e1493b-014c-43b1-b897-39c1d7ac9d41n%40googlegroups.com.

[Monkey]

Output images that are bigger than RAM will be supported (on x64, at least; making the same available on x86 would be a lot more work). Input images that are bigger than RAM are not currently supported, but I'll see about adding that (if anyone's going to try blending gigapixel images together). It's done with plain memory mapped files rather than tiles.

[David W. Jones]

Hmm, I remember ages ago producing a 768MB panorama. Not a gigapixel
size image, of course. But I did it from the command line on a 32-bit
processor with 2GB RAM, starting with using cpfind on full-size 6mpx
images. Just running cpfind on a single image used 2GB RAM.

That just puts things in perspective. It took that little old laptop
about 8 hours to stitch the final panorama. That was using enblend back
then, of course. So apparently Linux was transparently managing memory
behind the scene.

I understand Photoshop does its own memory management, but it dates back
to the days when it ran under OSes that had no real memory management.

So why does an app like multiblend need to do its own memory management?

On 2/11/21 9:21 AM, Monkey wrote:
> Output images that are bigger than RAM will be supported (on x64, at

> least; making the same available on x86 would be a /lot /more work).

> Input images that are bigger than RAM are not currently supported, but
> I'll see about adding that (if anyone's going to try blending gigapixel
> images together). It's done with plain memory mapped files rather than
> tiles.
> On Thursday, 11 February 2021 at 18:39:34 UTC gunter.ko...@gmail.com wrote:
>
> gimp-like memory management that allows to handle target images that
> are bigger than the RAM. don't know if it is already implemented.
>
> The gimp splits big images into tiles that small enough so a few of
> them fit into the RAM and then tries to work on these tiles one at a
> time.
>
> On 11.02.21 18:33, Monkey wrote:
>> It's been several years but I'm working on a new version of
>> Multiblend that's a bit faster, produces much nicer results, and
>> will blend much bigger mosaics.
>>
>> Does anyone have any feature requests for a blender that I could
>> consider incorporating?
>>
>> Along the same lines, does anyone use Enblend's colour space
>> features? Do they produce notably more "correct" results, or just
>> different ones? I've added an approximately linear RGB mode to
>> Multiblend, but it doesn't seem to produce great results so I'll
>> only be leaving it there as a curiosity.


--

David W. Jones
gnome...@gmail.com
wandering the landscape of god
http://dancingtreefrog.com
My password is the last 8 digits of π.

[Monkey]

As I understand it, and I may not have this exactly right, an OS will manage memory up to the amount of physical memory. Cpfind was using 2Gb but this would have been swapped in and out of disk by the OS. Some versions of Enblend have their own caching.

Certainly, there is a harder limit defined by the size of the paging file. On Linux systems this might 1.5x - 2x the physical RAM. On my Windows computer, with 24Gb physical RAM, it's only currently 9Gb. I think it expands if required, but I did a quick test and I could not allocate 32Gb, despite having plenty of disk space. So I think each process is limited to 24Gb, which may be partially on disk so that other processes can continue to run. That way, every process can have its own 24Gb, but no more.

By bypassing that and generating memory mapped files (when allocating RAM files), Multiblend will be able to use as much "RAM" as there is disk space.

[Luís Henrique Camargo Quiroz]

    I have made some panoramas with circa 2.7 GB in a tif equirectangular file.  However I had to grew my linux swap partition more than one time until I managed to finish those panoramas, and my swap has 72 GB currently (and only 16 GB physical memory). 

  regards,

   Luís Henrique

--

A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/b152fafb-bf6f-4c1a-aa96-86c72e17bf90n%40googlegroups.com.

--

--
Luis Henrique Camargo Quiroz
http://luishcq.br.tripod.com - http://www.christusrex.org/www2/cantgreg
http://panoramaslh.net/

[Paul Womack]

True as far as it goes.

An OS will manage virtual memory, swapping data between RAM and Disc as needed, normally in pages of some fixed size.

However, if the application is "naive", some worst case behaviour can emerge. Imagine an application that routinely access the first byte of a page,

and then accessed the first byte of every other page before re-accessing the first byte of the first page again.

The application is not addressing many BYTES, but it will thrash the virtual memory badly. A typical image rotate will have this behaviour on either source of destination.

What you're aiming for is locality of accesses, where multiple accesses all fall within a page. For raster graphics this can be achieved by accessing data in a tiled way, so that adjacent pixels in both X and Y are "near". This can be done WITHOUT a tiled virtual memory system, you just need to access the data in a tiled order.

This can be done by finding the pseudo-tile T a (x,y) coord is in (simple divide X, Y by the the tile size), calculate the tile base address of the tile T (T * tile_bytes) and then work out the offset of the data from the tile base (x and y modulo tile side, then multiply the new Y by tile side).

I has a fancy 2D Object browser on SunOs that was thrashing, and by simply implementing "address_of_object()" as outlined above, the memory behaviour was immaculate.

   BugBear

[Monkey]

Anyone on Mac OS who can check if it compiles if I email a link to the source?

[Harry van der Wolf]

I still have my old Macbook with a hacked Catalina (as it officially doesn't support my old firmware) for my java testing.

I can compile it (or try to compile it) for you if there are not too many exotic dependencies. 

Harry

Op ma 15 feb. 2021 om 13:28 schreef Monkey <davidh...@gmail.com>:

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/78e53b5c-9b93-46e8-a098-07c83e610358n%40googlegroups.com.

[David W. Jones]

At which point perhaps, it seems to me, things become a maze of
OS-specific optimizations of how the app handles pseudo-tiles... but I
understand the problems of thrashing.

[David W. Jones]

I think the biggest I've made was 1.2GB as a TIFF. I have 64GB RAM and
64GB swap. The max memory use it hit during blending was 32GB, so it
never touched swap.

I'll have to see if I can produce bigger ones now since I've moved from
a 6mpix camera to a 20mpix one.

On 2/12/21 5:56 AM, Luís Henrique Camargo Quiroz wrote:

>     I have made some panoramas with circa 2.7 GB in a tif
> equirectangular file.  However I had to grew my linux swap partition
> more than one time until I managed to finish those panoramas, and my
> swap has 72 GB currently (and only 16 GB physical memory).
>
>
>   regards,
>
>    Luís Henrique
>

[Paul Womack]

It's hard to see how improving locality of access wouldn't help, regardless of OS details.

   BugBear

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/b94163fa-cdf6-05e1-4960-bf4a2be6da0e%40gmail.com.

[David W. Jones]

I'm sure it would work, probably speed things up, :) and introduce new ways to trigger bugs?

On February 19, 2021 10:55:07 PM HST, Paul Womack wrote:
> It's hard to see how improving locality of access wouldn't help,
> regardless of OS details.
>
> BugBear
>

Sent from my Android device with F/LOSS K-9 Mail.

[Bob Campbell]

Now that I’ve got a working 2020 build for mac, I can take a shot at building multiblend.  

--
A list of frequently asked questions is available at: http://wiki.panotools.org/Hugin_FAQ
---
You received this message because you are subscribed to the Google Groups "hugin and other free panoramic software" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hugin-ptx+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/78e53b5c-9b93-46e8-a098-07c83e610358n%40googlegroups.com.

[Harry van der Wolf]

Hi,

I already built it on MacOS Catalina and reported my findings to David.

I also created a portable version with all dylibs in one zip.

(I also built the MacOS bundle quite some years ago. 2007-2011)

Harry

Op ma 22 feb. 2021 om 05:34 schreef Bob Campbell <thebobc...@gmail.com>:

To view this discussion on the web visit https://groups.google.com/d/msgid/hugin-ptx/14155AFE-B804-42DB-9E8C-C919E67E433C%40gmail.com.