Unfortunately, you're right. For inventory to work, you need a product per item/size/color/style which really multiplies the workload. And, then product selection for the customer is not friendly, you don't pick an item (jacket) and then select properties (color/size/etc)
I have seen some customers do some trick things to make it work better for fashon items, but they are customizations that are not provided by XMPie.