Models should be crafted to support commands and protect invariants while remaining expressive and true to the ubiquitous language. That's how you define the associations of your model.
If the only invariant that constrains the Product-Category relationship is that a Product has to be categorized, then Product could hold the category value (or the identity of the category if it's an AR).
Another fictive invariant could be that a Category may only be associated with a maximum of N products. In that case, a Category may hold a collection of ProductId values or perhaps ProductCategory values because you need a boundary that includes all the associations in order to strongly enforce the invariant.
There are times where strongly enforcing an invariant may require a very large consistency boundary (a huge aggregate), one that may not be scalable. At that point, you will have to turn to business experts in order to see if the rule is a true invariant and understand the business impact of violating that rule. You may found out that eventual consistency is acceptable, either through an automated process or a manual one (e.g identity categories that are busting the rule so they can be addressed).
I do not have a lot of experience yet, but from what I could understand so far, invariants are key for modeling aggregates...