need help reading trajectory data from .pbstream file in Python

[Akshay Bharadwaj]

Hey,

I am trying to read the trajectory data in *.pbstream file in python. I am able to to read the identifier number and the size of the data but i'm stuck parsing the uncompressed data to meaning full pose information. My python code so far looks like

#!/usr/bin/env python

import os
import sys
import gzip
import struct
import numpy as np
import zlib

def pbstreamreader(posefile):
    print "Reading .pbstream data"
    idnumber = '0x7b1d1f7b5bf501db'
    filehandle = open(posefile, 'rb')
    # make sure the data
    size = readthefile(filehandle)
    if idnumber!= size:
        print "unknown string"
        return        
       
    #get the total size of the data
    size = readthefile(filehandle)
    print int(size,0)
   
    compressed_trajectory_data = readdatafromfile(filehandle,int(size,0))
   
    trajectory_data = zlib.decompress(compressed_trajectory_data,16+zlib.MAX_WBITS)
   
    # read the entire data of the file
    data = struct.unpack_from('b',trajectory_data)
    print data
   
   
   
    filehandle.close()

def readthefile(fileobj):
    size =0
    byte=None
    word = []
    for i in range(0,8):
        byte = fileobj.read(1).encode("hex")
        word.append(byte)
    word = word[::-1]          
    word = '0x'+''.join(word)
    return word

def readdatafromfile(fileobj, size):
    return fileobj.read(size)

   
   
def main():
    if len(sys.argv)!=2:
        print "Incorrect usage. generateassetsfrombag.py <path/to/the/*.pbstream>"
        return
    pbstreamreader(sys.argv[1])
     
   
if __name__=='__main__':
    main()

from the proro_stream.h, I believe that the data is  being parsed from string to a buffer with

proto->ParseFromString(decompressed_data)

where `ParseFromString`  seems to be a protobuf api. So, do I need to use protobuf Api to read the pose? My goal is to created an vector of all Poses of the robot.

From the github link they suggest to use gzip and struct to achieve the same. the o/p of the struct.upack doesn't seem to be correct.

I'm not sure if I am on the right track and how to proceed further. I am kinda confused.

Any help is greatly appreciated.

Thanks
Akshay

[Mac Mason]

Yes, you will need to use Python protobufs; writing your own protobuf parser isn't a good use of your time.

If you're in a real hurry, I'd suggest using C++ to parse the protos (as in the link you provided) and then have the C++ code output some easily-parsed format (JSON?) to a text file, which you then read from Python.

--
You received this message because you are subscribed to the Google Groups "google-cartographer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-cartogra...@googlegroups.com.
To post to this group, send email to google-ca...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cartographer/e0d57e09-3102-4275-a805-be17633ec474%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Julian (Mac) Mason

Toyota Research Institute

[Michael Carroll]

Here's a gist for what I've been doing to manipulate the protobuf files from Python

https://gist.github.com/mjcarroll/f620ef8d88efc3a03711259d6fc9a578

Generating the python deserialization is pretty straightforward on Ubuntu, 

# sudo apt-get install protobuf-compiler

# roscd cartographer && mkdir py_export

# protoc --python_out ./py_export -I . `find . -iname '*.proto'`

Hope this helps.

To unsubscribe from this group and stop receiving emails from it, send an email to google-cartographer+unsub...@googlegroups.com.

To post to this group, send email to google-ca...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-cartographer/e0d57e09-3102-4275-a805-be17633ec474%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Akshay Bharadwaj]

Hey,

Thanks a lot for your replies. Thanks Micael for the code snippet. Finally got it working some time back.

This is what I have to read the trajectories. https://gist.github.com/anonymous/2dea6dd8c98b1aebdffd798f5af9827a

Thanks
Akshay