BERT-based Models for Next Version?

[Brad Windsor]

Hi folks!

First off, great library, super useful. Wanted to ask if there had been any discussions on the next NN architecture for tesseract 5:

1. Will we move the LSTM-based code to a transformer-based architecture? [0] 

2. Going further, we know that large pre-trained Transformer models have good next-word prediction properties which can correct text filled with OCR errors[1]. Have we tried fine-tuning such models for the OCR task itself? If not, would a small demo be welcome? 

[0] https://arxiv.org/pdf/1706.03762.pdf

[1] https://dl.acm.org/doi/abs/10.1145/3383583.3398605

Thanks,

Brad