Precision in LandMapR
31 views
Skip to first unread message
Sarah Lepp
Jun 6, 2012, 3:54:11 PM6/6/12
to open-geomorph...@googlegroups.com
How precise should the the data handling be in LandMapR?
Currently, the data variable type is a double which means inputs and outputs allow for 15 digit precision. If we switch to float type that means the modules will be much more efficient, but will be reduced to 7 digit precision. Can we switch from double to float? Would that cause problems for anyone?
docduncan
Jun 7, 2012, 10:31:59 AM6/7/12
to Open-geomorphometry-project
One of the reasons that CAD systems use double precision floats is for
positioning accuracy. For example, given a CAD model of the Earth and
the Moon there is sufficient precision in the system to allow the
accurate positioning of models of plaques at both Tranquility Base and
the Johnson Spaceflight Centre. In a system like 3DStudio Max, which
is the same 3D engine but only allows float precision large models
suffer some pretty severe positioning problems at the edges - in fact,
models are pretty much limited to city size. Given that we want to
cover an entire province (or continent...) with one model, I think its
possible we may need even more precision, or at least a way to deal
with large expanses.
positioning accuracy. For example, given a CAD model of the Earth and
the Moon there is sufficient precision in the system to allow the
accurate positioning of models of plaques at both Tranquility Base and
the Johnson Spaceflight Centre. In a system like 3DStudio Max, which
is the same 3D engine but only allows float precision large models
suffer some pretty severe positioning problems at the edges - in fact,
models are pretty much limited to city size. Given that we want to
cover an entire province (or continent...) with one model, I think its
possible we may need even more precision, or at least a way to deal
with large expanses.
bobmacm
Jun 8, 2012, 2:58:17 PM6/8/12
to open-geomorph...@googlegroups.com
For most uses single is sufficient.
But I did encounter some problems related to rounding when I dealt with very large numbers.
I leave it up to others more knowledgeable than me to make a final decision.
jeandaniel.sylvain
Jun 16, 2012, 7:56:02 PM6/16/12
to open-geomorph...@googlegroups.com
I think we can go over the digit concept by using a factor of 10, 100 or 1000 if digits are necessary but it could also represent a problem if user is not well advise.
For my part, one to three digits could be enough for typical work with landmapr (flat terrain), but maybe in the future we will need more digits for new covariables or parameters that could be derive with LandmapR 2.0. We should not limit ourself to single digit if it's not a physical limitation.
I also think that the treatment of larger area with double type could represent a big problem with the actual technology (RAM). In my work, I had some problem in SAGA with big extent because SAGA always use double precision. Maybe we should let the option to the user?
For my part, one to three digits could be enough for typical work with landmapr (flat terrain), but maybe in the future we will need more digits for new covariables or parameters that could be derive with LandmapR 2.0. We should not limit ourself to single digit if it's not a physical limitation.
I also think that the treatment of larger area with double type could represent a big problem with the actual technology (RAM). In my work, I had some problem in SAGA with big extent because SAGA always use double precision. Maybe we should let the option to the user?
Reply all
Reply to author
Forward
0 new messages