We just did raw mutipart data. So, if you wanted to send something without a file, it would accept JSON/XML - but if you wanted to send a file, then multipart with the extra "file" field.
One thing that we did was add our own custom Deserializer that accepts arrays, so we still use the JMS Serializer for decoding even raw POST data (which excludes the file, of course). So our logic doesn't care what the input format is - it just checks the request to see if it included any files.
The inconsistency bothers me too, still - but I couldn't think of another really practical way.