This is more a limitation of how Enaml deals with constraints, not the constraint solver itself.
Enaml assumes that you are building “desktop-style” applications, where each “widget” has its own preferred intrinsic size. This means that parent containers expand to fit the size of their children. The layout is done “inside-out”. It sounds like you want an “outside-in” layout, where the children are sized to fit within their parent.
You may be able to get this to work by abusing the “hug-width” and “hug-height” attributes, but it might be a bit like ice skating uphill.
If it become a royal PITA, you may need to resort to RawWidget and your own custom QLayout.