I find the translateLocally models quite amazing (a 14MB CPU-decoded model giving nynorsk→english translation of very readable quality). I have an anaphora resolution task, I just want to figure out if a certain word should be he/she/it, but relevant context can be quite large, so transformers seems like a good fit, and I'd love to be able to use something lightweight like this. Before I delve into the code, can I use marian for this kind of classification task? I see
https://github.com/marian-nmt/marian-dev/issues/885 which looks promising (I could mask the word I want to guess), but I haven't seen any end-to-end examples of using it as a language model in that way rather than translation. Is this feasible with marian, or should I be looking somewhere else for lightweight (decoding on CPU) transformers?