extracting coordinates to 9 decimal places.

1,043 views
Skip to first unread message

Cummings, Mike

unread,
Jun 4, 2009, 6:46:03 PM6/4/09
to mapi...@googlegroups.com

Without getting into all the details; and yes, I know that the extra 3 decimal places are not significant; but, I have been using them for Identification.  

I would like to extract the coordinates of selected points to 9 decimal places.  Is this possible?

Thanks all,

mike

Bill Thoen

unread,
Jun 4, 2009, 8:44:59 PM6/4/09
to mapi...@googlegroups.com
You could either move them into a decimal (14,9) column or export them
into a fixed precision with Format$(x, "0.##########"). Otherwise you'll
find that floats get truncated to about 3 or 4 decimal places!

- Bill Thoen

Cummings, Mike

unread,
Jun 5, 2009, 1:23:41 PM6/5/09
to mapi...@googlegroups.com
Bill and all,
I tried that and it gives me a number with three trailing zeros. I'm
really looking for a result that looks like a real number. Maybe I
should to explain the problem.
At the time (~1997); I really should have listened to the little voice.
But they told me that we were never going to change projections from
NAD27 to NAD83. We had started to use a Trimble GPS to verify well
locations. Instead of adding a column to our database to indicate the
source of location; I started exporting the data from the Trimble
software with 9 decimal places and storing the location in an access
database. Everything is based on this access table. I based queries on
the length of the latitude & longitude fields.
They have now decided to change projections and I really would like to
make as few changes to all of our forms and code as possible. If I
could convert the data to NAD83 & get the 9 decimal places, put the data
in the Latitude & longitude fields of the access table and replace some
MI tables; I would be done.

I have thought about just converting the data and adding .00000001; but,
I just am not happy with doing it- it would have to be the last resort.


Anyway, I have been trying to convert and get the 9 decimal places in MI
and haven't had any luck.

I'm not sure that I can get what I want done with MI.
Now that I have explained this; does anyone have a suggestion?

Rob Schneider - Aurchem Exploration

unread,
Jun 5, 2009, 1:33:19 PM6/5/09
to mapi...@googlegroups.com
Just wondering - is the NAD27 -> NAD83 conversion algorithm even accurate to 9 decimal places? Or are you actually adding error?




.

  

David R Sherrod

unread,
Jun 5, 2009, 2:11:35 PM6/5/09
to mapi...@googlegroups.com, mapi...@googlegroups.com

Rob's comment is the one I would have offered earlier in the week (I couldn't hit 9 decimal places if I had a total-station tacheometer and shot lengths of a few meters), but you stated your preference to maintain or even crank up the precision, regardless.

The inner workings of the MapInfo conversion are vague to me, so I tried experimenting. (Most of my MapInfo solutions are empirical.)
I set some coordinates to 9 decimal places in an excel spreadsheet, imported those to MapInfo and NAD27, and then reestablished the table for new columns that I could update in a table projected as NAD83.  First I tried it with State-scale latitude and longitude limits (Oregon, in my case), and ran into your problem, which is the MapInfo default of "millionths of a degree for latitude-longitude."

I then tried setting the bounds for the table to an area only 0.2 degrees by 0.2 degrees (my coord examples were within that).  I was able to get Column Update values sporadically to the ninth decimal place (besides the "000" fillings), but the values were either "999" or "001," which sounds something like your efforts.  Mike, if you're going to be doing the conversion with State-scale databases, the great latitudinal extent of California may squeeze you out at 6 decimal places for conversions within MapInfo.

Unless you use Scott Daugherty's ENZYTE version.

Dave S.
-----------------------------------------



From:
Rob Schneider - Aurchem Exploration <r...@aurchem.ca>
To: mapi...@googlegroups.com
Date: 06/05/2009 10:34 AM
Subject:
[MI-L] Re: extracting coordinates to 9 decimal places.
Sent by: mapi...@googlegroups.com


Cummings, Mike

unread,
Jun 5, 2009, 2:11:34 PM6/5/09
to mapi...@googlegroups.com

I imagine any time you convert; you can add error.

 

I’m not concerned about the accuracy beyond 6 decimal places; I need the 9 decimal places to identify that the well location was verified with the Trimble GPS.   

Bill Thoen

unread,
Jun 5, 2009, 3:02:24 PM6/5/09
to mapi...@googlegroups.com
Well, those three zeros at the ends of your coordinates might really be
what the values are. After all, MapInfo stores their coordinates as
32-bit integers, and that says to me that the maximum number of
significant digits you can have is about 9 or 10. So when you expand a
coordinate to 14 significant digits the last 4 or 5 are totally
meaningless.

However, you can extract the coordinates to a float field and then
you'll have 52 bits to play with, which will give you log(2) * 52 = 15.7
significant digits that a float data type can preserve. Then you can
stuff whatever you want in the digits beyond 10 and not affect the
precision of the coordinate as far as MapInfo is concerned. And if you
dump it to a character field, you can invent as much precision as you like.

Since you're using this extra "precision" only as an identifier, there's
nothing wrong with stuffing whatever digits you please in there beyond
the seventh decimal. With a coordinate of six decimal places we're only
talking about 1/10 m between longitude 0.000000 and 0.00001 at the
equator, or about a centimeter if you go to 7 decimal places. So my
advice is to just go ahead and store your coordinates in a system that
can preserve the detail, and add whatever numbers you need to uniquely
identify that location. You won't be hurting anything location-wise,
because the precision beyond 7 decimals was not there to begin with.

Cummings, Mike

unread,
Jun 5, 2009, 3:35:03 PM6/5/09
to mapi...@googlegroups.com
Bill,
I understand and agree. I knew nothing past the sixth decimal was
significant when I started storing the data. I saved the 9 decimal
places to keep from creating a source field in the table and having to
worry about getting populated correctly.

My concern is: these numbers will be on our website. I can image folks
looking at these numbers, contacting us about the format and asking if
they are real. I'm wishing I had written a bunch of code differently.

I feel like I may just add to the end and move on.

Thanks,
mike
Reply all
Reply to author
Forward
0 new messages