In my tests, most of the custom automatic formatting rules (Options > Automatic Formatting > Automatic Formatting Rules) I created in my v4 database weren't retained when opening that database in v5. I tried doing an export/import, but again, most of the custom rules weren't recreated.
That's as far as I got in my testing. There could be other issues but I haven't had time to dig in further.
Did you try importing auto-formatting rules to MLO 5 as described here: https://www.mylifeorganized.net/announcements/windows-mlo-5-upgrade.shtml#mlo5_installation ? This should help you restore your customizations.