RPC, problems with the response

[Robert Gregat]

Hello everyone,

I use Crossbar.io together with Autobahn|Python Twisted on my Raspberry Pi 2 (Ubuntu Mate).

For some testing i query a sqlite/spatialite-db from an angularJS-App. For the communication i use angular-wamp.

So I have a procedure in Python which do all the heavy work (prepare the query, execute it and produce a geojson-string from the result). 

My idea behind this is to have something similar like a vector-tile query based on slippy-map Tiles (zoom/xtile/ytile). 

Everything works fine, dispite of one thing. If i execute the call 2-3 or more times i get my response after all calls are finished.

On my Pi 2 i did some console.log() to see what happened inside of the procedure. The procedure works normal and execute one by one, but on the client side i get every result after every call is processed at once.

My question is, when my procedure execute one by one, why did i get the response only after no more execution are going on?

Stripped down python code:

class AppSessionSpatialite(ApplicationSession):

    @inlineCallbacks

    def onJoin(self, details):

        def get_data_db(queryData):

            ....

            return result

        yield self.register(get_data_db, 'de.map.data.request')

[Johnny W. Santos]

I think you have to make get_data_db as a coroutine as well, and also your db connection may be blocking the request. Are you using an async driver for the db?

--
You received this message because you are subscribed to the Google Groups "Crossbar" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crossbario+...@googlegroups.com.
To post to this group, send email to cross...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/crossbario/b89d8d7c-ff1d-4612-a1ad-d07c4a794985%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Johnny W. dos Santos

[Tobias Oberstein]

We need to see what you do in the get_data_db procedure. You must not block the reactor .. eg need to use appropriate db code as db drivers are blocking most often

Sent from Mobile (Google Nexus 5)

[Robert Gregat]

That could be one problem, yes.

I'm using pysqlite.

To open and close the spatiaLite-db i have two separate functions which have their own subscription. The problem with this approach is, that i can't

execute a query in a different thread. A workaround could be to open the db connection inside of the get_data_db function.

def open_db(status):
 global conn # Verbindungsobjekt
 global dbOpen
 dbOpen = False
 try:
 print("open_db: {}".format(1))
 conn = db.connect('/home/robert/crossbar_setting/Autobahn_Test/node2/python/sqlite/berlin_building.sqlite') # Oeffnen einer Datenbankverbindung
 conn.enable_load_extension(True) # Das einladen von extension aktivieren
 cur = conn.cursor() # Erzeugen eines cursors mit dem SQL-Statements ausgefuerht werdne koennen
 cur.execute("select load_extension('libspatialite.so')") # Einladen der libspatialite.so
 conn.enable_load_extension(False) # Funktionalitaet zum Einladen von extensions wieder deaktivieren
 conn.row_factory = dict_factory # Festlegen wie ein Ergebnis formatiert werden soll
 cur.close() # Cursor wieder schliessen
 dbOpen = True
 except:
 dbOpen = False
 self.publish(u'de.db.isopen', dbOpen)


 sub = yield self.subscribe(open_db, 'de.open.db')
 print("subscribed to topic 'open_db': {}".format(sub))

def close_db(status):
 print("close_db: {}".format(1))
 dbClosed = False
 try:
 conn.close()
 dbClosed = True
 dbOpen = False
 except:
 dbClosed = False
 self.publish(u'de.db.isclosed', dbClosed)
 
 sub = yield self.subscribe(close_db, 'de.close.db')
 print("subscribed to topic 'close_db': {}".format(sub))

def get_data_db(queryData):
 print("get_data_db: {}".format(queryData))
 
 """
 Idee:
 Eine Liste mit allen Informationen wird zusammengestellt.
 die letztem drei Eintraege beinhaltet die tilenummer, vor der tilenummer kommt der tabellenname.
 Alle weiteren Eintraege stellen Spaltennamen dar. Der erste Eintrag beinhaltet den Spaltennamen
 id, wenn in der Tabelle vorhanden.
 ["id", "column2", "column3", "columnx", "tabellenname", "zoom", "xtile", "ytile"]
 Aus diesen Informationen wird das Select-Statement konstruiert.
 """
 
 # Den String splitten
 splitList = queryData.split("/")
 
 # Anzahl an Elementen in der Liste ermitteln
 listCount = len(splitList)
 
 # Tilenummer extrahieren und neu zusammensetzen
 tileNumber = splitList[listCount-3] + "/" + splitList[listCount-2] + "/" + splitList[listCount-1]
 
 print(tileNumber)
 
 # BBOX berechnen
 maputil = MapUtil()
 bbox = maputil.num2deg(str(tileNumber)) # Umwandeln der TileNummer zu einer BoundingBox
 
 # Tabellennamen aus der Liste extrahieren
 tabellenname = splitList[listCount-4]
 
 # Alle Spalten aus der Liste extrahieren und in einer neuen Liste abspeichern
 spalten = splitList[0:listCount-4]
 
 # Anzahl an Spalten ermitteln
 listCount = len(spalten)
 
 spaltenSQL = ""
 forCounter = 0
 
 # Zusammensetzen eines String mit allen Spalten
 for spalte in spalten:
 spaltenSQL += spalte
 if(forCounter < listCount-1):
 spaltenSQL += ","
 forCounter += 1
 
 # Zusammensetzen aller Informationen zu einem SQL-Befehl
 cmd = """
 SELECT asgeojson(CastToPolygon(st_intersection(BuildMBR({2}, {3}, {4}, {5}), Geometry))) as geojson, {0}
 FROM {1}
 WHERE MBRIntersects(BuildMBR({2}, {3}, {4}, {5}), Geometry);""".format(spaltenSQL, tabellenname, bbox[0], bbox[1], bbox[2], bbox[3])
 
 cur = conn.cursor()
 cur.execute(cmd)
 
 result = cur.fetchall()
 cur.close()
 
 featureCollection = list()
 
 for row in result:
 geom = geojson.loads(row['geojson'])
 row.pop('geojson')
 #idBuilding = geojson.loads(row['id'])
 #row.pop('id')
 feature = geojson.Feature(geometry = geom, properties = row)


 featureCollection.append(feature)
 
 finalFeatureCollection = geojson.FeatureCollection(featureCollection)
 geoJsonDump = geojson.dumps(finalFeatureCollection, sort_keys=True)
 
 print(geoJsonDump)
 
 return geoJsonDump
 #reactor.callLater(0, dbThread, geoJsonDump)
 
 
 reg = yield self.register(get_data_db, 'de.map.data.request')
 print("Funktion registriert: get_data_db")