CyArks new online 3D viewer - a case study on way too many digits

[Martin Isenburg]

Hello,

Congrats to CyArk for creating this wonderful online experience that
allows me to inspect amazing cultural treasures all across the world
in interactive 3D as lidarnews reports:

http://www.lidarnews.com/content/view/9138/

As I am very involved with point cloud formats I was instantly curious
about how CyArk's 3D Viewer was storing these point clouds - after all
- users will have to download them. Turns out that each model is
stored as several gzipped ASCII files that are around 2 MB in size and
contain a JSON string that describes the point cloud. The "Tikal
Temple I" model, for example, is stored in 5 files that total 10.8 MB
(see links at the very end).

On further inspection I saw that each file contained a list of points
as [x,y,z,"RGBintensity", "RGBcolor"]. Here are the first six points
from the first file of the "Tikal Temple I" model:

[28.286682500009,-17.910553000169,-22.492218,"00ff13","473618"]
[-29.050201500009,19.081512999721,-20.314026,"f56400","958b4e"]
[-29.855194500007,19.4510499998,-20.298309,"f56700","938749"]
[-29.754943500011,19.124969999772,-20.32785,"f98100","9c9255"
[-29.297363499994,19.565826999722,-20.301452,"e91700","9a914f"]
[-29.503723500005,19.244445999851,-20.305786,"fcb500","958b4b"]

I suggested to the CyArk group to be a little less generous with the
decimal digits and store something more compact instead by rounding to
millimeters:

[28.287,-17.911,-22.492,"00ff13","473618"]
[-29.05,19.082,-20.314,"f56400","958b4e"]
[-29.855,19.451,-20.298,"f56700","938749"]
[-29.755,19.125,-20.328,"f98100","9c9255"]
[-29.297,19.565,-20.301,"e91700","9a914f"]
[-29.504,19.244,-20.306,"fcb500","958b4b"]

Millimeter accuracy should be more than sufficient for a laser scanned
temple. The answer I got was that they had considered that shaving a
few digits would give some savings but that they "[...] generally
don't round anything. Any loss of quality, no matter how academic, is
usually frowned upon around here."

I would like to take this as an educational opportunity on scanner
precision. (-: Folks who have seen the first 5 minutes of my LASzip
LiDAR compression video on youtube (http://www.youtube.com/watch?
v=A0s0fVktj6U) know that I am on "a mission" to educate folks about
not storing excessive precision in their scanned data. (-:

Why excessive? The majority of the decimal digits that CyArk stores in
those files are not actual data. The binary IEEE format that is used
to represents a double-precision floating-point number often makes it
seem as if there are many non-zero decimal digits ... but most of them
are completely meaningless.

How meaningless? CyArk stored the x and y coordinate for the points of
the "Tikal Temple I" model with picometer (!) precision (see
illustration below) and let me quote wikipedia [1]: "The picometre's
length is of an order such that its application is almost entirely
confined to particle physics and quantum physics. Atoms are between 62
and 520 pm in diameter." So clearly a bit of an overkill ... (-:

28.29 = centimeter
28.287 = millimeter
28.2867 = 0.1 millimeter
28.28668 = 0.01 millimeter
28.286682 = micrometer
28.2866825 = 0.1 micrometer
28.28668250 = 0.01 micrometer
28.286682500 = nanometer
28.2866825000 = 0.1 nanometer
28.28668250001 = 0.01 nanometer
28.286682500009 = picometer

Here is a link to my version of the "Tikal I Temple" model in good old
VRML. It is displayed in the ancient Java Engine from Shout3D so it
may not show on your Mac (but on your Solaris Sparc) but you can still
download the model.

http://www.cs.unc.edu/~isenburg/pmc/tikal_temple_i.html
http://www.cs.unc.edu/~isenburg/pmc/media_TIK_20071003_151724.wrl.gz

Part of my "campaign" (*) was the release of pointzip [2] for
compressing terrestrial scanned data. It features a precision selector
very prominent in the GUI that "forces" the user to make a choice and
reflect about the precision in the data.

Regards,

Martin @rapidlasso

(*) This message was brought to you by the people for better 3D
compression through storing less decimal places by knowing your
precision.

[1] http://en.wikipedia.org/wiki/Picometre
[2] http://pointzip.org

The 5 JSON files storing the "Tikal I Temple" model:

http://archive.cyark.org/projects/TIK/area_07-10-02-18-28-52__2335//media_TIK_20071003_151724_ori-0.js.gz
http://archive.cyark.org/projects/TIK/area_07-10-02-18-28-52__2335//media_TIK_20071003_151724_ori-1.js.gz
http://archive.cyark.org/projects/TIK/area_07-10-02-18-28-52__2335//media_TIK_20071003_151724_ori-2.js.gz
http://archive.cyark.org/projects/TIK/area_07-10-02-18-28-52__2335/media_TIK_20071003_151724_ori-3.js.gz
http://archive.cyark.org/projects/TIK/area_07-10-02-18-28-52__2335/media_TIK_20071003_151724_ori-4.js.gz