PulseWaves VLR information, can it be retrieved?

[Joe McGlinchy]

Hello,

I am working with a puslewaves file from a RIEGL scanner, which records the received waveform with multiple channels. There are gains and other parameters associated with each channel which are described in the output of running the PulseTools pulseinfo.exe tool, but don't seem to be in any of the data when using PyLidar getWaveformInfo(), getReceived(), getTransmitted(), or the header object. Is that information somewhere else? Also, I understand that each waveform segment has a duration associated with it. I am not seeing that information anywhere, either. If it is being returned by PyLidar, but I am not seeing it, can you please advise as to where that information may be located or able to be retrieved??

Thanks!

Joe

[gilling...@gmail.com]

Joe,

This information is definitely available when reading a Riegl .rxp file I haven't got a pulsewaves file generated by Riegl to test what you are saying. Again John Armston should be able to comment on this....

Sam.

[Joe McGlinchy]

Hi Sam,

Still stumped on this. I can share the file with you if you like. I have resampled the pulsewaves files so that there are approximately 5k pulses/waveforms so it loads quickly.

-Joe

[gilling...@gmail.com]

Yes, please post a link to the file and I'll have a look at it when I get time.

Sam.

[Joe McGlinchy]

Sam, here is a link to the .pls and .wvs files. Please let me know if you cannot access it.

https://drive.google.com/drive/folders/1ltAnJvYjA0W3JtzgczBXQ_0jDMwCDOG5?usp=sharing

-Joe

[gilling...@gmail.com]

Joe,

Have a look at the CHANNEL, RANGE_TO_WAVEFORM_START, RECEIVE_WAVE_GAIN, RECEIVE_WAVE_OFFSET, TRANS_WAVE_GAIN and TRANS_WAVE_OFFSET fields of the array returned by getWaveformInfo(). Is this what you are after?

This first axis of the array will be the waveform number, second axis is the pulse. I can't see how to obtain the waveform duration from pulsewaves, but RANGE_TO_WAVEFORM_START contains the duration to the start of it.

Hope this helps.
Sam.

[Joe McGlinchy]

Sam,

Thanks for looking at the data. I had a look at one of the waveforms which consisted of a return waveform with 3 segments to try and get some ideas on how to move forward. Here is some information on this particular waveform id# 12515 (0-based index). There are 3 return waveform segments, all of which only have 80 valid data values and are masked out to 240 digitizations per segment

The printout for the pulse taken from the output of the getPulses() method, and the waveform information from the getWaveformInfo() method. It looks like the offsets and gains for each segment are 0.0 and 1.0, respectively. The RANGE_TO_WAVEFORM_START is something I am having a bit of trouble interpreting. A lot of the field names for the outputs of getWaveformInfo() and getPulses() structured arrays don't seem to match up 1:1 with what is in the PulseWaves specification. I may just be having trouble interpreting the outputs relative to the spec, but if I can back out the duration of each segment along with the scaling factor and offset for those segment durations using the RANGE_TO_WAVEFORM_START values, then I will do that. Do you have any thoughts on how best to do that with PulseWaves formatted data?? 

X_LAST: -6849.52348
X_FIRST: -6837.211071
Y_LAST: -3205.60924
Y_FIRST: -3206.235323
Z_LAST: 2328.1286
Z_FIRST: 2349.90707
Segments: 4.      (if 2, one Tx, one Rx)
seg 1 len: 80 (masked)
seg 1 len: 240 (unmasked)
seg 2 len: 80 (masked)
seg 2 len: 240 (unmasked)
seg 3 len: 80 (masked)
seg 3 len: 240 (unmasked)

info for pulse 12515 in batch 0
TIME: 404883410321
OFFSET: 1668286
X_ORIGIN: -6463.194
Y_ORIGIN: -3225.254
Z_ORIGIN: 3011.477
X_TARGET: -6536.921
Y_TARGET: -3221.505
Z_TARGET: 2881.067
FIRST_RETURNING_SAMPLE: 5073
LAST_RETURNING_SAMPLE: 5240
DESCRIPTOR_INDEX: 5
INTENSITY: 0
WFM_START_IDX: 26021
NUMBER_OF_WAVEFORM_SAMPLES: 4
NUMBER_OF_RETURNS: 1
PTS_START_IDX: 12515

info for waveform 12515 in batch 0
NUMBER_OF_WAVEFORM_RECEIVED_BINS: [0L 80L 80L 80L]
RECEIVED_START_IDX: [0L 1161680L 1161760L 1161840L]
NUMBER_OF_WAVEFORM_TRANSMITTED_BINS: [28L 0L 0L 0L]
TRANSMITTED_START_IDX: [350420L 0L 0L 0L]
RANGE_TO_WAVEFORM_START: [-11.998270034790039 5073.11279296875 5161.111328125 5074.8408203125]
CHANNEL: [3 1 1 0]
RECEIVE_WAVE_GAIN: [1.0 1.0 1.0 1.0]
RECEIVE_WAVE_OFFSET: [0.0 0.0 0.0 0.0]
TRANS_WAVE_GAIN: [1.0 1.0 1.0 1.0]
TRANS_WAVE_OFFSET: [0.0 0.0 0.0 0.0]

If I look at the pulseinfo.exe output for this same file, I can located the Pulse Desciptor index for PULSEdescriptor 5, which i think is correct for this pulse given DESCRIPTOR_INDEX: 5 in the printout above:

variable length header record 10 of 17:

  reserved             0

  user ID              'PulseWaves_Spec'

  record ID            200005

  length after header  404

  description          'PulseWaves 0.3 r11 (140921) by rapidlasso'

    PULSEdescriptor 5

      optical_center_to_anchor_point:  0x0 ('coincide')

      number_of_extra_waves_bytes:     0

      number_of_samplings:             3

      sample_units:                    1 [nanoseconds]

      scanner_index:                   1

      compression:                     0

      description:                     '1 x RP, 2 x LP, 1 x HP'

        sampling[0]

          type:        1 ('outgoing')

          channel:     3

          bits_for_duration_from_anchor:            32

          scale_for_duration_from_anchor:            0.00667312

          offset_for_duration_from_anchor:           0

          bits_for_number_of_segments:               0

          bits_for_number_of_samples:               16

          number_of_segments:   1

          number_of_samples:    0

          bits_per_sample:      8

          lookup_table_index:   1

          sample_units:       1.0 [nanoseconds]

          compression:          0

          description:        '1 reference segment(s)'

        sampling[1]

          type:        2 ('returning')

          channel:     1

          bits_for_duration_from_anchor:            32

          scale_for_duration_from_anchor:            0.00667312

          offset_for_duration_from_anchor:           0

          bits_for_number_of_segments:               0

          bits_for_number_of_samples:               16

          number_of_segments:   2

          number_of_samples:    0

          bits_per_sample:      8

          lookup_table_index:   1

          sample_units:       1.0 [nanoseconds]

          compression:          0

          description:        '2 lowpower segment(s)'

        sampling[2]

          type:        2 ('returning')

          channel:     0

          bits_for_duration_from_anchor:            32

          scale_for_duration_from_anchor:            0.00667312

          offset_for_duration_from_anchor:           0

          bits_for_number_of_segments:               0

          bits_for_number_of_samples:               16

          number_of_segments:   1

          number_of_samples:    0

          bits_per_sample:      8

          lookup_table_index:   2

          sample_units:       1.0 [nanoseconds]

          compression:          0

          description:        '1 highpower segment(s)'

Do you think it is necessary to write a parser to produce some kind of VLR dictionary to look up some of this additional information? This is all in the header information, so while it tells me how many bits are used to store the duration_from_anchor value, it doesn't actually give me the value, but it also gives me the scale and offset for the duration_from_anchor.

Thanks!

Joe

[gilling...@gmail.com]

Joe,

I think you might be best linking to the PulseWaves library directly if you want to access all the data in the way the specification describes. You will need to use C++. Pylidar attempts to present the information in a similar manner to our SPD V4 format. Hence the names and structure will be different.

Yes I checked and the offsets and gains are set to 0 and 1 as this information does not appear to be made available by the PulseWaves library. I put this in to match the SPD V4 spec.

Sam.