access waveform data in PulseWaves file

[Joe McGlinchy]

Hello,

I am working with some PulseWaves format data and trying to access the waveform data using PyLidar. I hav elooked over the examples on the documentation site but don't see any example on accessing the waveform data. Is there a simple way to do that with PyLidar? I have tried something like

from pylidar.lidarformats import pulsewaves
 from pylidar.lidarformats import generic
    header = pw.getHeader()
    tot_pulses = pw.getTotalNumberPulses()
with pulsewaves.PulseWavesFile(inFile, generic.READ, 'test','test' ) as pw:

but this gives me an error saying I don't have enough inputs. 

Joe 

[gilling...@gmail.com]

Hi Joe,

You need to be using the lidarprocessor.doProcessing function to loop over the data as shown in the examples.

I've just added a short example do the documentation (http://pylidar.org/en/latest/processorexamples.html#reading-waveform-data) to show how to read waveform data.

Let me know if you need more information.

Sam.

[Joe McGlinchy]

Thanks for providing this example, Sam. I tried it with a PLS file which I obtained after decompressing from a PLZ file with PulseTools. I edited the example to return only the waveformInfo piece, but that appears to be a None object. Is that expected? I have attached a screenshot of my code and outputs.

[Joe McGlinchy]

UPDATE:

I was able to get this to work with the following. I am still getting used to how to correctly use the lidarprocessor class! Here is the code I used to get the waveforminfo, received, and transmitted pulses:

def readFunc(data, otherArgs):

    # returns 2d masked structured array with info about waveforms

    # first axis is waveform number, second is pulse

    otherArgs.waveinfo = data.input1.getWaveformInfo()

    

    # returns masked 3d radiance array

    # first axis is waveform bin, second is waveform number, third is pulse

    otherArgs.recv = data.input1.getReceived()

    otherArgs.trans = data.input1.getTransmitted()

    

inFile = r"<path to .pls file>"

dataFiles = lidarprocessor.DataFiles()

dataFiles.input1 = lidarprocessor.LidarFile(inFile, lidarprocessor.READ)

otherArgs = lidarprocessor.OtherArgs()

otherArgs.waveinfo = 0

otherArgs.recv = 1

otherArgs.trans = 2

lidarprocessor.doProcessing(readFunc, dataFiles, otherArgs=otherArgs)

Then I am able to check the shapes of the otherArgs objects and peak into them, as shown in the attached picture.

-Joe

[gilling...@gmail.com]

Just be aware that readFunc gets called repeatedly with each 'chunk' of data read from the file. So your code will only be saving the last chunk.....

[Joe McGlinchy]

Thanks Sam, good call. I had noticed the array sizes were a little small.. That is fine for now. I think i can take note of the processChunk example to extract the full file contents when I get there. Thanks for your help on it.

[Joe McGlinchy]

Hello,

I wanted to revisit this topic to hopefully understand where some additional pulse metadata may be located. For a given pulse I am looking at the data returned by the getWaveformInfo() method, and it returns a structured array. For example (id is a number, 0 is the batch of waveforms):

>>>otherArgs.allWaveInfo[0][:,id] 

masked_array(data = [(0L, 0L, 48L, 84016L, 0.0, 0, 1.0, 0.0, 1.0, 0.0)

(72L, 94504L, 0L, 0L, 7253.0, 0, 1.0, 0.0, 1.0, 0.0)

(--, --, --, --, --, --, --, --, --, --)],

             mask = [(False, False, False, False, False, False, False, False, False, False)

(False, False, False, False, False, False, False, False, False, False)

(True, True, True, True, True, True, True, True, True, True)],

       fill_value = (999999L, 999999L, 999999L, 999999L, 1e+20, 63, 1.0000000200408773e+20, 1.0000000200408773e+20, 1.0000000200408773e+20, 1.0000000200408773e+20),

            dtype = [('NUMBER_OF_WAVEFORM_RECEIVED_BINS', '<u4'), ('RECEIVED_START_IDX', '<u8'), ('NUMBER_OF_WAVEFORM_TRANSMITTED_BINS', '<u4'), ('TRANSMITTED_START_IDX', '<u8'), ('RANGE_TO_WAVEFORM_START', '<f8'), ('CHANNEL', 'u1'), ('RECEIVE_WAVE_GAIN', '<f4'), ('RECEIVE_WAVE_OFFSET', '<f4'), ('TRANS_WAVE_GAIN', '<f4'), ('TRANS_WAVE_OFFSET', '<f4')])

I understand from the PulseWaves format there should be an anchor point and direction vector associated with the pulse, as well as a duration. Is that returned with the getWaveformInfo() method when reading the data or is it somewhere else? I haven't been able to locate it while searching through the source code.

Thanks!

-Joe

[gilling...@gmail.com]

I think you are after the x_origin, y_origin, z_origin, x_target, y_target and z_target fields of the pulse info (see data.input1.getPulses).

[Joe McGlinchy]

that was it. Thank you again!