dealing with very big (upto 1Gb) svg file
meemsheen
Hi,
I have to visualize some values from a big text file with svg rect and with
different colors,
but the svg file I generate is so big in size that firefox/safari crashes to
open it.
Description about the input text file:
Its a comma separated csv text file which contains float values like:
0.70, 0.50,0.98,0.78,0.61,0.88.........
160 of the above values in one line and more than 100,000 lines.
I have a c script which parse the csv file and writes <rect> tags in svg and
with style=fill: rgb( , , ) values filled from the input csv file.
If I just parse some 1000 lines from the input csv file the generated svg is
about 50-100MB and I can
view it as I wanted, but when I parse more and more, the generated svg
becomes very big and
safari/firefox hangs to open this big svg file.
I have 4GB RAM in my machine.
Any help to deal with this problem or do it in another better way will be
much appreciated.
Thanks,
Best regards,
--
meemsheen
--
View this message in context: http://old.nabble.com/dealing-with-very-big-%28upto-1Gb%29-svg-file-tp27574066p27574066.html
Sent from the Mozilla - SVG mailing list archive at Nabble.com.
Stanimir Stamenkov
> (...)
> If I just parse some 1000 lines from the input csv file the generated svg is
> about 50-100MB and I can
> view it as I wanted, but when I parse more and more, the generated svg
> becomes very big and
> safari/firefox hangs to open this big svg file.
> I have 4GB RAM in my machine.
>
> Any help to deal with this problem or do it in another better way will be
> much appreciated.
You may ensure you don't use formatting white space in the generated
SVG text which will make the built DOM much smaller, I guess.
If the data is really that big, may be generate and serve a bitmap
image, instead? Or if it happens on the client side - one could
draw directly on a canvas:
https://developer.mozilla.org/en/HTML/Canvas
--
Stanimir
codedread
> Hi,
>
> I have to visualize some values from a big text file with svg rect and with
> different colors,
> but the svg file I generate is so big in size that firefox/safari crashes to
> open it.
>
> Description about the input text file:
>
> Its a comma separated csv text file which contains float values like:
> 0.70, 0.50,0.98,0.78,0.61,0.88.........
> 160 of the above values in one line and more than 100,000 lines.
>
> I have a c script which parse the csv file and writes <rect> tags in svg and
> with style=fill: rgb( , , ) values filled from the input csv file.
>
> If I just parse some 1000 lines from the input csv file the generated svg is
> about 50-100MB and I can
> view it as I wanted, but when I parse more and more, the generated svg
> becomes very big and
> safari/firefox hangs to open this big svg file.
> I have 4GB RAM in my machine.
>
> Any help to deal with this problem or do it in another better way will be
> much appreciated.
>
Is it required to turn everything into rectangles and each one having
a different color? The browsers may be choking on the number of
elements (rects) since each element adds DOM overhead (memory usage).
Have you looked at parsing the data into SVG path commands? Using
just one path (with subpaths) for each color would drastically cut
down on memory usage in the browser.
<path fill="blue" d="M100,100 L200,200 z"/>
This would be a path that moves the pen to (100,100) then draws a line
to (200,200), then closes the path.
Jeff
meemsheen
No, its not required to turn everything into rects, but the different colors
are important,
its just similar to visualize such a data with a "heatmap"...but a java
program which I've for that
also gives me 'out of mem error', and I also can't use utilities like
matrix2png...
Your suggestion of using Path and subpaths looks interesting, that it will
cut down the load,
but can I use different fill colors for subpaths? (i.e red color to values
from 0.50 to green color for values upto 0.99)
--
meemsheen
--
View this message in context: http://old.nabble.com/dealing-with-very-big-%28upto-1Gb%29-svg-file-tp27574066p27586725.html
codedread
>
> Your suggestion of using Path and subpaths looks interesting, that it will
> cut down the load,
> but can I use different fill colors for subpaths? (i.e red color to values
> from 0.50 to green color for values upto 0.99)
>
No, subpaths are all painted with the same paint of the path element.
However, if you take some time to sort things into the same colors,
then you could group all same-colors into one path.
Jeff
HelderMagalhaes
> > but can I use different fill colors for subpaths? (i.e red color to values
> > from 0.50 to green color for values upto 0.99)
>
> No, subpaths are all painted with the same paint of the path element.
> However, if you take some time to sort things into the same colors,
> then you could group all same-colors into one path.
Along with the whitespace optimizations already described, I'd also
give emphasis towards property sharing. This will reduce memory and
speed up loading. So, instead of:
<rect fill="red" stroke="none" x="10" width="10" height="10"/>
<rect fill="green" stroke="none" x="20" width="10" height="10"/>
<rect fill="blue" stroke="none" x="30" width="10" height="10"/>
One could simply write:
<g fill="red" stroke="none">
<rect stroke="none" x="10" width="10" height="10"/>
<rect fill="green" x="20" width="10" height="10" />
<rect fill="blue" x="30" width="10" height="10"/>
</g>
This works pretty well if you are able to identify a set of shared
property values. The gains become more obvious as the number of child
elements increase.
Also, you may find CSS interesting for avoiding repetitive style-
related attributes: just create a class for them! :-)
Finally, you may also use XML entities for repetitive text, if you can
identify such patterns. ;-)
Hope this helps,
Helder