Giga-pixel image
Nils Haeck
cool to generate a giga-pixel image. I've generated a jpeg file that is
33.344 x 33.344 pixels (total just over 1 trillion pixels) with my NativeJpg
library.
http://www.simdesign.nl/download/mandelbrot_gigapixel.zip
Now.. which application can open them? The ones I tested all crashed,
because in order to hold this (24bit colour) image entirely in memory, you
would need 3 Gigabyte of ram approximately. But the NativeJpg library can
load the image just fine in tile-mode, and you can scroll through it
effortlessly. You can even view it at half, quarter and one-eight size. It
doesn't hold the bitmap in memory, just the (undecoded) jpeg file, then
loads part of this file as necessary when scrolling.
This zip file contains both the image and the TileDemo executable to open
it. The image is a computer-generated rendering of a part of the Mandelbrot
fractal. The jpeg file itself is "just" 54 Mb, quite reasonable.
If printed out at 150 dpi, this image would cover 5.6 by 5.6 meters (31.8
square meters). I didn't do this, but it would be cool wallpaper.
Note that the end is in sight with the Jpeg format, as it can only go up to
65535 x 65535 pixels :)
More info on the NativeJpg library here:
http://www.simdesign.nl/nativejpg.html
Nils Haeck
www.simdesign.nl
mhieta
Nice fractal. I'm glad that my product can open that easily :)
Btw. I always test with these images (Also there is png,tiff images):
http://visibleearth.nasa.gov/
I agree that most image viewers cannot open these kind of images at all.
- Marko
"Nils Haeck" <b...@bla.com> wrote in message
news:47517187$1...@newsgroups.borland.com...
Mat Ballard
> While everyone is talking about mega-pixel images, I thought it would be
> cool to generate a giga-pixel image. I've generated a jpeg file that is
> 33.344 x 33.344 pixels (total just over 1 trillion pixels) with my NativeJpg
> library.
Nice - but my A3 securitized TIF in four separations at 2400 dpi (39685 x 28063)
beats it by about 2 M pixels. 8^)
Now, Borland, where is that D64 I've been waiting for ? I'd like to do A1.
Nils Haeck
Well, this was more to prove that it is possible to "randomly" load a tile
from a Jpeg file, while still working with huge files. So even modest
systems, running Win98 + 128 Mb (as many people in the world still use) can
load these files.
I know that TIFF is usually already tile-based, so that might be easier.
Btw, I could easily beat you by generating a Jpeg of size 65535 x 65535, but
I don't see the sense in that.. and I wanted a sensible image. The
generation of the fractal already took a few hours, for that bigger size it
would probably run at least a day. Now the download size is also still
reasonable, because there are probably many readers downloading it from my
website. A bigger size would strain my website's bandwidth and download
quota.
If you're serious about 64bit though, you could try freepascal. One of my
customers actually bought e.g. my NativeXML component with the question if
it worked in freepascal (which it does). When I asked why, he told me that
he had a rather big xml file. When I asked *how* big, he told me it was 300
Gigabyte.. and the system he worked on had about 800 Gigabyte of memory.
Hmm, not an off-the-shelft consumer system obviously :) For these kinds of
systems, 64bit compilers are a must.
Nils
"Mat Ballard" <m...@chemwares.com> schreef in bericht
news:475326ef$1...@newsgroups.borland.com...
Mat Ballard
you've got me beat on Win98 - mine needs a *fast* 2G RAM system to do its work.
I was being a just a little bit facetious about "My one is bigger than your one
..." - but it's fun to work right up against the limits of the system.
Thanks for the tip on FreePascal - I have it installed already and have played
with it a bit. The big downer for me is that their implementation of TBitmap is
radically different from Borland's. I do a lot of low-level Scanline[] stuff
that does not translate well.
A 300 G XML file - while it's amazing that your NativeXML can actually handle
it, it makes me wonder if there might be a different programming paradigm that
might be more efficient for his problem. Still, these *big64* problems are
cropping up more and more frequently now.
cheers,
Mat
John Venture
it seems to be problematic to use it under Vista 64 :
Current Pos: 000000
<SOI marker>
Current Pos: 000002
<APP0 JFIF marker, length:16>
4A464946-00010201-02580258-00000000-...
JFIF
Current Pos: 000022
<DQT marker, length:130>
QTable=0 precision=8bit
6 4 4 6 10 16 20 24
5 5 6 8 10 23 24 22
6 5 6 10 16 23 28 22
6 7 9 12 20 35 32 25
7 9 15 22 27 44 41 31
10 14 22 26 32 42 45 37
20 26 31 35 41 48 48 40
29 37 38 39 45 40 41 40
QTable=1 precision=8bit
7 7 10 19 40 40 40 40
7 8 10 26 40 40 40 40
10 10 22 40 40 40 40 40
19 26 40 40 40 40 40 40
40 40 40 40 40 40 40 40
40 40 40 40 40 40 40 40
40 40 40 40 40 40 40 40
40 40 40 40 40 40 40 40
Current Pos: 000156
<SOF0 marker, length:15>
Coding method: Baseline DCT
Image 33344x33344, 3 frames, 8bit samples
Frame 0: 2x2 sampling ID=01 QTable=0
Frame 1: 1x1 sampling ID=02 QTable=1
Frame 2: 1x1 sampling ID=03 QTable=1
Exception during decode: Range check error
any plan to add zoom (with resampling) possibilities ?
same thing with pixel manipulation routines (I mean something like
canvas.pixels, bitblt, stretchdraw, etc...)
As you may know I'm very interested by LibVips that manage very large
picture without problem and has a lot of drawing routines, but
unfortunately there's no delphi headers.
Thanks
John
Alan Garny
news:4753c21f$1...@newsgroups.borland.com...
> Hi Nils,
>
> it seems to be problematic to use it under Vista 64 :
Same system here (Vista Ultimate x64) and no problem.
Alan.
John Venture
sometimes it works sometimes not...
Nils Haeck
> it seems to be problematic to use it under Vista 64 <snip>
You must turn off range checking under project options, then it will work
probably.
> any plan to add zoom (with resampling) possibilities ?
I have other code doing that, but I didn't want to complicate this example
with it. You can however load the image at 1/1, 1/2, 1/4 and 1/8 size
already.
> same thing with pixel manipulation routines (I mean something like
> canvas.pixels, bitblt, stretchdraw, etc...)
>
> As you may know I'm very interested by LibVips that manage very large
> picture without problem and has a lot of drawing routines, but
> unfortunately there's no delphi headers.
The example uses my TsdVirtualScrollbox component. However, virtual
scrollbox is not the same as a class that manages huge bitmaps (with the
methods you propose). If you want something like that, I could write it for
you (paid work), but then again, It should not be too hard to write
something like that yourself? Just hold a list with tiles (partial bitmaps)
in memory, and load them from a file whenever you need them. Dispose them
whenever they are no longer needed.
Nils
John Venture
>
>> it seems to be problematic to use it under Vista 64 <snip>
>>
>
> You must turn off range checking under project options, then it will work
> probably.
>
>
>
> The example uses my TsdVirtualScrollbox component. However, virtual
> scrollbox is not the same as a class that manages huge bitmaps (with the
> methods you propose). If you want something like that, I could write it for
> you (paid work), but then again, It should not be too hard to write
> something like that yourself? Just hold a list with tiles (partial bitmaps)
> in memory, and load them from a file whenever you need them. Dispose them
> whenever they are no longer needed.
>
>
will look at your comp before...
Alan Garny
>I have tryed with nils sample on a core 2 duo and 4Gb of Ram.
> sometimes it works sometimes not...
Interesting, same hardware and no problem (?!)...
Alan.
Alan Garny
> cool to generate a giga-pixel image. I've generated a jpeg file that is
> 33.344 x 33.344 pixels (total just over 1 trillion pixels) with my
> NativeJpg
> library.
> http://www.simdesign.nl/download/mandelbrot_gigapixel.zip
Interesting, I have recently done something similar for BMP files. Our
research group generates some pretty big BMP files (1.85GB is the biggest so
far), so I decide to write a small program to read them.
Source code: http://www.hellix.com/Inc/download.php?id=10
Executable: http://www.hellix.com/Inc/download.php?id=11
Alan.
Nils Haeck
Did you compile the tiledemo or the jpegtest? Since you're copying debug
information, I suspect that you actually compiled the jpegtest example. This
one does not use a virtual scrollbox, so it *will* allocate the complete
bitmap, and thus.. you might get range check errors because you go past
32-bit pointer boundaries.
Please compile the TileDemo.dpr example instead.
Nils
"John Venture" <nos...@nospam.com> schreef in bericht
news:4753c21f$1...@newsgroups.borland.com...
Nils Haeck
That's indeed the same principle, I think.
However, I wonder why you put all the image loading capability right in the
Main.pas unit (which is part of the GUI). From a software design
perspective, it would make sense to split this into a separate unit, so it
is easily re-useable.
Furthermore I see you use a paintbox for the display of the bitmap. With our
VirtualScrollbox this would probably be easier, and more efficient..
Example: you only have to draw parts that truly update. So if you scroll
horizontally, only the small strip left or right needs updating (and loading
from the underlying file). The rest of the screen is just copied.
Nils
"Alan Garny" <som...@somewhere.com> schreef in bericht
news:47542f46$1...@newsgroups.borland.com...
Alan Garny
> That's indeed the same principle, I think.
>
> However, I wonder why you put all the image loading capability right in
> the Main.pas unit (which is part of the GUI). From a software design
> perspective, it would make sense to split this into a separate unit, so it
> is easily re-useable.
Well, that's something that I wrote very quickly so we could view our big
BMP files. It was, in no way, intended to be a serious piece of software.
> Furthermore I see you use a paintbox for the display of the bitmap. With
> our VirtualScrollbox this would probably be easier, and more efficient..
> Example: you only have to draw parts that truly update. So if you scroll
> horizontally, only the small strip left or right needs updating (and
> loading from the underlying file). The rest of the screen is just copied.
Yes, that's in fact something I could have done too. No time for that
though... :)
Alan.
Alan Garny
> Hi Alan,
>
> That's indeed the same principle, I think.
>
> However, I wonder why you put all the image loading capability right in
> the Main.pas unit (which is part of the GUI). From a software design
> perspective, it would make sense to split this into a separate unit, so it
> is easily re-useable.
>
> Furthermore I see you use a paintbox for the display of the bitmap. With
> our VirtualScrollbox this would probably be easier, and more efficient..
> Example: you only have to draw parts that truly update. So if you scroll
> horizontally, only the small strip left or right needs updating (and
> loading from the underlying file). The rest of the screen is just copied.
By the way, before uploading my BMP viewer to my website, I decided to
remove the dependency to FastMM4/FastCode/FastMove and it's quite amazing to
see how much 'slower' my program became. So, yes, it's definitely worth
using those 'Fast' libraries!
Alan.
PS: the executable I uploaded does make use of those libraries.
Sanyin
> While everyone is talking about mega-pixel images, I thought it would be
> cool to generate a giga-pixel image. I've generated a jpeg file that is
> 33.344 x 33.344 pixels (total just over 1 trillion pixels) with my
> NativeJpg
> library.
> http://www.simdesign.nl/download/mandelbrot_gigapixel.zip
I would like to do that with ordinary bitmap, but its too slow,you must use
multithreading to improve performance.
Sanyin
"Alan Garny" <som...@somewhere.com> wrote in message
news:47542f46$1...@newsgroups.borland.com...
Well, you are using stretch procedure for streching...I think correct method
would be using memory mapped files,
and multithreading, line VIPS library (see some posts before).
John Venture
do you have some sample file ?
Thanks
John
Nils Haeck
The fact that this works on low-mem systems is because the jpeg is held
completely in memory, but it is compressed, so it doesn't take that much
room. If you want to keep a bitmap in memory, you are limited by memory size
*or* you must use a filestream, which is indeed slow.
Nils
"Sanyin" <dsf...@dfdasf.sff> schreef in bericht
news:4754...@newsgroups.borland.com...
Alan Garny
news:4755...@newsgroups.borland.com...
There is no doubt in my mind that my BMP viewer could be improved. Remember
that this was written 'very quickly' to answer an urgent need that we had.
At the end of the day, the program does what we want and is definitely fast
enough for what we need to do, so we are happy with it. :)
The way the program works is that if it can load the whole BMP file in
memory, then it will do so, otherwise it will just read the bit that needs
to be displayed from the file itself. Also, because the first thing I do is
to create a thumbnail, it means that (under Vista at least, because of the
way it handles memory) it will then be faster to read bits of the BMP file
from the file itself.
Regarding my use of StretchDraw, I believe I may not have much of a choice,
since I want to be able to create a thumbnail (and therefore needs to
stretch bits of the BMP file down), unless you know a better of doing that.
Otherwise, I am not sure how a memory mapped file could improve things here.
Could you expand on your idea?
Finally, I agree about multithreading (on multi-core processors that is).
That would make the loading of the BMP file and the creation of the
thumbnail much faster, no doubt.
Alan.
PS: I don't know anything about that "line VIPS library".
Alan Garny
news:4755...@newsgroups.borland.com...
> Alan,
>
> do you have some sample file ?
I do, but they are research files, so not for general consumption... :)
You may want to try http://www.efg2.com/Lab/Graphics/VeryLargeBitmap.htm.
You can, using that program, create a 768MB BMP file (just make sure that
*both* "Save to File" and "Count Colors" are checked).
Alan.
John Venture
http://www.vips.ecs.soton.ac.uk/index.php?title=Libvips
John Venture
case you can use high capacity (like 32 gb) memory card or USB
flashdisk for saving the stream,...
just a thought..
John Venture
to manage huge picture and huge vars,
what value to you use in stack and heap ? (linker parameter)
Thanks
John
GrandmasterB
> load the image just fine in tile-mode, and you can scroll through it
> effortlessly.
Thats awsome Nils! :-)
liran
file using memory-map-file without actually loading all the information
to memory, and so very large files (i tested with 1gb one) can be viewed
on a "normal" pc systems
you will need to build the tile-loading system yourself...
Liran