Hi,
I was trying to achieve writing data to Big Table from Pub sub using cloud function
But I want some help to achieve the below-specified points:
- How can we compare the existing value in the Big table for a specific row and update the record(Eg: version from pub-sub could be v1, v2 or 1, 2, 3... and version is application-specific and not big table specfic)
- If the version of data from Pub Sub is less than the Big Table record, the update has to be skipped
- If the row key doesn't exist, we have to insert the record
Below is the code snippet I tried using for my testing
def writeToBigTable(project_id, instance_id, table_id, data):
client = bigtable.Client(project=project_id, admin=True)
instance = client.instance(instance_id)
table = instance.table(table_id)
#timestamp = datetime.datetime.utcnow()
# Timestamp is set zero since we are maintaining only latest record
timestamp = datetime.datetime.utcnow()
data = json.loads(data)
row_key = data['sys_id']
row = table.row(row_key)
try:
for col in data.keys():
value = data[col]
row.set_cell('stats',col,value.encode('UTF8'),timestamp=timestamp)
except Exception as e:
print('RowKey is {}'.format(row_key))
print("Excepttion is {}".format(e))
raise e
table.mutate_rows([row])
return data
Thanks,
-Ravi