Thanks for your replies Zhongliang and Joar!
I ended up using the albumentations library and it works quite well in our pipeline. It has a lot of supporting documentation and a lot of functions for image transformations (along with transformations of the actual annotations - bounding boxes, segmentation, keypoints...).
Xabier Lekunberri also pointed this tutorial out from OpenCV, which he used to change contrast and brightness, and might be useful to other people in the group:
Cheers,
Catarina