Excuse the late reply, I just got back from my vacation :).
It's quite hard to exactly figure out why it's so slow on your site without code. However, are you rendering all those categories in a navbar/mega menu maybe? And if yes, how are you doing this?
You could always cache those mega menu's to prevent slow websites with the built in django cache templatetag.
Let me know!
Joey