Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

edm ppm correction formula Leica TCA2003

678 views
Skip to first unread message

Steve

unread,
May 23, 2004, 3:00:33 AM5/23/04
to
I'll probably be able to track this down from Leica eventually but I
am wondering if anyone has this formula available..


I want ppm expressed in terms of :
Temperature ( centigrade)
Pressure (mb) and
Relative humidity (%)

The documtation thjat came with the TCA2003 does not define this
relationship.


Thanks in advance


Steve

André

unread,
May 29, 2004, 8:44:25 AM5/29/04
to
I am afraid you're out of luck.
There are no such formula.

In the manual for the 2003 there is a table showing the relationship over a
limited range. (normal working range)
The relationships are not linear and do not follow any known math. function
as far as I know.
Over a limited range one might be able to say ppm changes 1 for each n
degrees temp; same for pressure.
You can take the values by reading the graph in the manual. These values are
only valid within the indicated range.
Ignore humidity. it's the smallest factor by far.

André

"Steve" <srawli...@hotmail.com> schreef in bericht
news:40b04af1...@news-server.bigpond.net.au...

bob

unread,
May 30, 2004, 10:09:19 PM5/30/04
to
One place to find the formulae needed is in NOAA Technical Memorandum
NOS NGS-10, Use of Calibration Baselines, from the follwing link:

http://www.ngs.noaa.gov/PUBS_LIB/TMNOSNGS10.pdf

Pages 6-9 give you the formulas and the method.

Another source is Surveying, by Moffitt and Bouchard (1987). I have the
eight edition and the formula start on page 49.

None of these give the ppm directly or in the units you have asked, but
it wouldn't take much effort to derive them....

--Bob

Steve

unread,
Jun 5, 2004, 7:31:30 PM6/5/04
to
Thanks for the ngs URL. There is some good stuff in that pdf

Leica support came to my aid quite quickly in pointing out that the
TPS1100 Professional Series User manual lists the ppm formulae for not
only the infra red EMD (which is also used in the TC2003) but also
for the visible laser EDM as well.

The pdf file of this manual is called "System_UserManual_en.pdf"
and the formulae are on page 152

The file can be downloaded from
http://downloads.leica-geosystems.com/downloads/areas/zz/tps/tps1100/manuals/index.htm

On that page select System user manual download. The current version
of this manual is 2.21.

Those two formulae are considerably easier to interpret than the one
in the ngs document but still are quite complicated.

I rewrote the formula for infrared in xls format as follows

=283.04-(0.29195*P/(1+0.003660858*T)-(4.126*H/(10000*(1+0.003660858*T)))*POWER(10,((7.5*T)/(273.3+T)+0.7857)))

P is pressure in millibars
H is humidity in %
T is temperature in celsius

I tested it against the LEICA GSI word index 59 and it gave agreement
to 0.1ppm though the manual states it is vaild of 1013.25mb 12
degrees c and 60 % humidity

The other interesting thing I was told by Leica support was that the
index n for infrared of 1.0002830 is corrected in the TC2003 for
temperature change, (presumably the temp in the instrument) This can
be seen by calling up the readout of the instananeous refractive index
from the TCA2003 onboard meu edm test options. The readout of
refractive index is constantly updated and actually changes slightly
as you watch it.

Quite a bit of documentation on this technology is available from
LEICA


Steve

On Mon, 31 May 2004 02:09:19 GMT, bob <rmfo...@worldnet.att.net>
wrote:

Steve

unread,
Jun 5, 2004, 10:49:35 PM6/5/04
to
On Sat, 05 Jun 2004 23:31:30 GMT, srawli...@hotmail.com (Steve)
wrote:

>
>The other interesting thing I was told by Leica support was that the
>index n for infrared of 1.0002830 is corrected in the TC2003 for
>temperature change, (presumably the temp in the instrument) This can
>be seen by calling up the readout of the instananeous refractive index
>from the TCA2003 onboard meu edm test options. The readout of
>refractive index is constantly updated and actually changes slightly
>as you watch it.
>
>Quite a bit of documentation on this technology is available from
>LEICA
>


Now that I'm at my other box I can reference the email from Leica
support staff whch went as follows

"This is known as a TCXO - Temperature Corrected Crystal Oscillator.

For further info see this web sites:

http://www.wenzel.com/documents/tcxo.html

http://osc.conwin.com/pages/tcxo.html

http://www.ctscorp.com/components/tcxo.htm

Do you remember the old EDM's that used an oven to warm the crystal up
to a
certain temperature? This is now obsolete because of TCXO's"


Steve

0 new messages