Transformer AutoEncoder

[Cédric Colas]

Hi,

I was quite excited by the paper on Transformer AutoEncoders and wanted to play with it. Is there any pre-trained model available? From the Github, it seems I can easily access the MusicTransformer but I don't see checkpoints for the transformer autoencoder anywhere.

If it's not online, is there an expected date of release?

If not, can we access the midi dataset converted from a youtube dataset anywhere? I'm talking about the one described here and used to train the MusicTransformer available from the Colab.

Is there any training script for the transformer autoencoder (in addition to the implementation of the model itself)?

If it's not there, I'd like to train it myself, but without access to the dataset or training script it seems like a near impossible task. I could use the MAESTRO dataset but I'd like to train it on a more diverse one if possible.

Would appreciate any link / hint that could make the task easier!

Thanks a lot for the work, and your time!

Best,
Cédric