clarification on the GPS time stored in LAS files and the gps_time range reported by lasinfo

[Martin Isenburg]

Hello,

I just got the following note:

"While testing lasinfo, I noticed two bugs in the gps_time range it reported. 

lasinfo reported:

gps_time 174893536.000000 174895488.000000

Which translates to these timestamps:

Jul 22, 1985 05:32:12 UTC

Jul 22, 1985 06:04:44 UTC

 

However, the correct GPS time range is:

gps_time 1174893536.000000 1174895488.000000

Which translates to these correct timestamps:

Mar 30, 2017 07:51:10 UTC

Mar 30, 2017 07:18:38 UTC

Notice that the leading “1” is missing in the lasinfo reported gps_time range."

-----------------------

What's going on? The LAS format stores in the "gps time" field the moment that the laser pulse was fired for all the returns it generates. The GPS time can be stored in one of two ways:

(1) GPS time of the week is a number between 0 and 3600*24*7 that expresses the number of seconds that have passed since the last Sunday night of the week in the moment that the pulse was shot.

(2) Adjusted Standard GPS time is a number that continuously increases that expresses the number of seconds that have passed since the beginning of GPS time (some date in 1971) in the moment that the pulse was shot *MINUS* one bilion seconds (i.e. the 'adjusted' part).

lasinfo simply reports the actual values stored. It does not map the range to the Standard GPS time. That would be trivial for the second representation (add one billion before printing the value but - a little warning - you will loose time stamp resolution if you simply add one billion to the 64 bit floating point Adjusted Standard GPS time stamps). But that would be impossible for the first representation as the GPS week is not stored with each point.

Regards.

Martin @rapidlasso

 

[Jordan, Tom]

Martin,

Thank  you for your prompt reply clarifying gps_time in LiDAR files. Also, I just saw your article “Please use Adjusted Standard GPS time stamps” and agree with you that “GPS seconds of the week” makes no sense if the date of the referenced Sunday is not also stored.

 

As I understand it, there is no way to store the referenced Sunday, so it is never known without some other metadata file, correct?

 

Additionally, we can assert the following:

·         The units of gps_time must be “Adjusted Standard GPS time” (not “GPS time of the week”), if gps_time > 604,800

·         gps_time > 604,800 for all acquisition times after Sep 21, 2011 01:46:25 UTC (GPS Standard time = 1,000,604,800)

 

It’s unfortunate that “gps_time” can have two different units, and that such an algorithm is required to infer which of the two it is, and that algorithm doesn’t always work (for gps_time <= 604800).

 

However, I believe there are two ways that lasinfo can help disambiguate this:

1.       Provide a GPS standard time when it can be calculated (and omit it when it can’t be determined). For instance,

            gps_time: 174915136 174917088

gps standard time: 2017-03-30Z13:18:38  2017-03-30Z13:51:10

 

2.       Provide the “gps time units” with a value of either “adjusted standard GPS time”, or “ambiguous”. For example:

gps_time: 174915136 174917088

gps time units: adjusted standard GPS time

 

Tom Jordan, MSCS
Software Engineer

Geospatial data processing and visualization

Space and intelligence systems / HARRIS CORPORATION

Melbourne, FL / HTC-C3014

[Terje Mathisen]

Martin, you are of course (as usual :-) ) correct when you state that
all time stamps should use the Adjusted Standard GPS time instead of
time of week, but I still think lasinfo has a bug when it reports that
number as if the "Adjusted" part could be omitted!

I.e. lasinfo should definitely add 1e9 to the number of seconds before
converting to YMD HMS representation.

Re. your claim about losing resolution:

When a timestamp is stored as a FP number, the specification is
seriously messed up IMHO. :-(

I've been working in the NTP Hackers team for many years and that
protocol always uses fixed-point format for all timestamps specifically
to avoid steps in the resolution as the integer field gets larger. As
you note a double with 53-bit mantissa and an integer (seconds) part
between 1 and 2G would require 31 bits for the seconds leaving just 22
fractional bits. The latter would correspond to a resolution of ~250 ns
which would be marginal for pulse timestamps so the "Adjusted" bias is
required.

For Adj GPS timestamps the easiest way to decode it would be something
like this:

int64_t seconds = (int64_t) agpstime;
agpstime -= (double) seconds;
int32_t nanoseconds = (int32_t) (agpstime * 1.0e9);
seconds += ADJGPSBIAS; // 1E9
seconds += TAI_to_UTC_offset(seconds);

Terje

> --
> Download LAStools at
> http://lastools.org
> http://rapidlasso.com
> Be social with LAStools at
> http://facebook.com/LAStools
> http://twitter.com/LAStools
> http://linkedin.com/groups/LAStools-4408378
> Manage your settings at
> http://groups.google.com/group/lastools/subscribe


--
- <Terje.M...@tmsw.no>
"almost all programming can be viewed as an exercise in caching"

[Martin Isenburg]

Hello,

Tom, which of the two formats for specifying a GPS time stamp is used does in general not need to be "guessed" by a formula but is specified by a bit in the "global encoding" bit-field in the LAS header. However, a more robust software implementation may use a formula to "guess" that the time stamp is in “Adjusted Standard GPS time” by simply checking whether they fall outside of the GPS week range of [0, 604800) and that would be correct for all but that one week in September 2011 when then  “Adjusted Standard GPS time” had its zero crossing.

Terje, I would not say that lasinfo has a bug. It simply reports the min / max values stored in the different attribute fields of all point records without interpreting them. Note also that the extended scan angle is not scaled by 0.006. And also that the reported X, Y, and Z integer coordinates are not scaled and offset using the information from the header. It simply reports what values are stored. Also lasinfo does not report the GPS time range converted to a YMD HMS representation (maybe it should in a future version), so also there is no bug.

[...]

reporting minimum and maximum for all LAS point record entries ...

  X           -49999500   49999499

  Y           -49999500   49999500

  Z            -1534499    1534500

  intensity           0      65535

  return_number       1          1

  number_of_returns   1          1

  edge_of_flight_line 0          0

  scan_direction_flag 0          0

  classification      0          7

  scan_angle_rank    15         15

  user_data           0          0

  point_source_ID     0          0

  gps_time 174893536.000000 174895488.000000

  extended_return_number          1      1

  extended_number_of_returns      1      1

  extended_classification         1     81

  extended_scan_angle          2500   2500

  extended_scanner_channel        0      0

[...]

Regards,

Martin @rapidlasso

[Terje Mathisen]

Martin Isenburg wrote:
> Terje, I would not say that lasinfo has a bug. It simply reports the
> min / max values stored in the different attribute fields of all point
> records without interpreting them. Note also that the extended scan
> angle is not scaled by 0.006. And also that the reported X, Y, and Z
> integer coordinates are not scaled and offset using the information
> from the header. It simply reports what values are stored. Also
> lasinfo does not report the GPS time range converted to a YMD HMS
> representation (maybe it should in a future version), so also there is
> no bug.

Oops, me and my big mouth. Mea Culpa!

I should have remembered that, I just saw the message with translated
(typically using the localtime()/gmtime() call?) timestamp and forgot
that lasinfo doesn't do that. That does make your code correct of course.

OTOH I do think it would be a good idea to also translate any agpstime
to ascii since it would provide another often needed sanity check. I
know that I personally would love such a feature when I'm looking a new
lidar scans here in Norway where a single small area can get laz blocks
from up to 5 or 6 different projects.

Terje

[Jordan, Tom]

I agree with Terje and suggest an enhancement to lasinfo so that it reports an additional field "gps standard time" when there's sufficient information to determine it (such as when it can be asserted that gps_time represents "Adjusted Standard GPS time"), otherwise it's omitted. For example:
gps_time: 174915136.000000 174917088.000000

gps standard time: 2017-03-30Z13:18:38 2017-03-30Z13:51:10

This would be very helpful as the acquisition start and end times could then be directly understood without requiring additional software or manual steps to convert to gps_time to a human readable form.

Note that the format of the timestamps in the example above are designed to be both human readable and easily parsed by software code.

Tom Jordan, MSCS
Software Engineer
Geospatial data processing and visualization

SPACE AND INTELLIGENCE SYSTEMS / HARRIS CORPORATION
Melbourne, FL / HTC-C3014

[Jean-François Bourdon]

I would add my voice to those of Terje and Tom. I often have to check acquisition dates inside a tile and had to write some addition code to convert gps_time to gps standard time. However, if something like that is implemented into lasinfo, I must warn that using the GPS Time Type bit to evaluate if the time is an GPS seconds-of-week or not can be misleading. It is probably rare, but I encountered somes LAZ files where the GPS Time Type bit was set to 1, yet the timestamp were in GPS seconds-of-week (I knew the real date via other metadata). The point cloud had been classified with TerraScan, so maybe it has something to do with this software.

All this to say, +1 for human readable time info in lasinfo, but I think that warnings should be cast when the timestamp is the range 0-604800 even if the GPS Time Type bit is set to 1.