Hello,
there is no such thing as a (single) dataset for "the" model. madmom contains several models trained on various datasets, some of them are publicly available, some are not. But none of the dataset has anything to do with emotion recognition.
If you clarify your intent on what to port to a NN framework, we could probably help you. Besides that, if you only want to port over the inference part, this is (relatively) easy to accomplish, since all weights/parameters are stored in the models anyways.
HTH