First double check if the Pi is not throttling due to overheating or lack of USB power. This may cause the slowdown.
Usually 30/50 px of text height is fine. IF the problem is tesseract, try to use the fast model (or "normal" if using best). I assume you are using the 4.x release.
Try tesseract -v to see if you are using all the available CPU optimizations.
Try to move the SetSourceResolution outside the loop and see if it changes something (MAYBE it may invalidate some caches or something).
The time you are referring to is one single api.GetUTF8Text() call, correct?
Lorenzo