Firebase Storing Vectors only working half the time

[Arjun Mulchandani]

Hey everyone! I am using firebase in python right now and need to store vectors (1024 dimensions) as a part of a project to search documents. When I run the code below which essentially adds the vector to the document dicts then updates the record, the code errors with the message (500 Internal Server Error) About half the time. If I cut out the vectors from the record it works every single time. I have found this behavior quite random and could not find a common pattern between the data for which the code errored. I know it is not a request size issue due to one the of the records that worked containing 15 vectors and tons of text while one that didn't work only contained one vector and 2 tiny strings. I am incredibly sorry for the long description, but I have been searching for hours and could not find anyone having the same error. Thank you in advance!

# Process each record and update Firestore

for record in records:

UID, data = record

modified = False

for i in range(len(data["Documents"])):

doc = data["Documents"][i]

if doc["Status"] in ["Awaiting Upload"]:

if "TA.CT-ContextAware" not in doc:

continue

TACT = doc["TA.CT-ContextAware"]

TACT_string = TACT_stringifier(TACT)

TACT["Alignment"] = Vector(model.encode(TACT_string))

doc["TA.CT.A"] = TACT

doc["STRCF.RE"] = doc["STRCF.RE-Justified"]

modified = True

del doc["TA.CT-ContextAware"]

del doc["STRCF.RE-Justified"]

if modified:

FIREDB.collection("CENTRAL").document(UID+"-Aligned").set(data)