Pub Sub to Big Table using Cloud functions

76 views
Skip to first unread message

Ravi Teja Chava

unread,
Sep 3, 2021, 1:49:07 AM9/3/21
to Google Cloud Bigtable Discuss
Hi, 

I was trying to achieve writing data to Big Table from Pub sub using cloud function

But I want some help to achieve the below-specified points:

  1. How can we compare the existing value in the Big table for a specific row and update the record(Eg: version from pub-sub could be v1, v2 or 1, 2, 3... and version is application-specific and not big table specfic)
  2. If the version of data from Pub Sub is less than the Big Table record, the update has to be skipped
  3. If the row key doesn't exist, we have to insert the record

Below is the code snippet I tried using for my testing

def writeToBigTable(project_id, instance_id, table_id, data):
client = bigtable.Client(project=project_id, admin=True)
instance = client.instance(instance_id)
table = instance.table(table_id)
#timestamp = datetime.datetime.utcnow()
# Timestamp is set zero since we are maintaining only latest record
timestamp = datetime.datetime.utcnow()
data = json.loads(data)
row_key = data['sys_id']
row = table.row(row_key)
try:
     for col in data.keys():
             value = data[col]
              row.set_cell('stats',col,value.encode('UTF8'),timestamp=timestamp)
 except Exception as e:
      print('RowKey is {}'.format(row_key))
      print("Excepttion is {}".format(e))
       raise e
table.mutate_rows([row])
return data

Thanks,
-Ravi   
Reply all
Reply to author
Forward
0 new messages