I have built a system using django admin for the main interface. It's doable - however - I wouldn't do it again. The main reason is that getting things working quickly is easy. You get a simple interface that can do CRUD. You then add grappelli (or some other system) to get a nicer interface. And then you start nesting things.
Now your problems start. The admin interface gives you a lot of functionality out of the box, BUT the problem is that you need to follow the exact guidelines for this to work. You start working with formsets and specialcases and it takes a long time to write the code. So the initial fast setup makes it hard do develop features in the long run.
At least that's my experience.