As far as I'm aware the reason for Chromium having this permission is so that it can look up whether URLs that get loaded in the browser should be turned into intents to launch other apps (e.g. navigating to
youtube.com launching the Youtube app instead of loading it in Chromium) - since any URL could be potentially matched by any app, Chromium needs the QUERY_ALL_PACKAGES permission to get correct results.
This is why the policy lists "browsers" as one of the acceptable use cases for this permission, because any browser that wants to implement this standard Android logic will need the permission.
So, if your app behaves like a browser and allows users to follow links to arbitrary sites, and you want this standard behaviour where they will be redirected to external apps when appropriate, then you will need this permission for your app, and will have to justify this as your app being browser-like.
If your app doesn't do that, then you may be able to just remove the permission, but you may also need to make changes to the Chromium code that's responsible for intent dispatching as it may not work as expected without the permission. I'm not aware of an existing configuration option to control this.