Running out of memory whilst precomputing representations

[Arjun Singh]

Hi,

I have been trying to generate the precomputed representations for the training data using the BERT model checkpoint provided.

I wanted to see if it worked since my plan is to fine-tune the BERT model you have used and then generate new precomputed representations.

Unfortunately when I try to do so for the training data, I am running out of memory (24GB) on the GPU that I have access to.

I have been trying to look at "extract_features.py" in "data/get_bert_embeddings" to see where the issue lies but I am not able to find it.

Is there a way to reduce the memory burden on the GPUs, maybe by batching updates to the output file or clearing up unused objects?

Any suggestions will be really appreciated.

Thank you for taking the time to answer this.

Kind regards,

Arjun