3-4 seconds for a single page is probably not that slow depending on the page content and layout.
We have a huge OCR project with approximately 16 million images to process. Our configuration has 4 virtual machines each with 8 cores and 16GB memory. They work against a single input queue and process 8 pages at a time per VM. The CPU utilization on each VM is kept close to 100% by design.
We use a producer/consumer model with multiple threads and can process in excess of 10,000 pages per hour (we even do multiple OCR passes on certain pages where the orientation is uncertain). This set up did require fair amount of development to create but is working fine and has processed more than 3 million pages already. Tesseract and Leptonica engines have performed flawlessly in this rather demanding setup.
The point is that a single page on a single thread may feel slow, but a batch process with a large degree of parallelism performs rather well.