[Server] Port Policy Engine to Native Go: Implementation & Alignment Report

[Guan-Ming Chiu]

Hi everyone,

  The native Go policy engine replacing OPA/Rego for relationship evaluation is ready for review.

  PR: https://github.com/meshery/meshery/pull/17330
  Design doc: https://docs.google.com/document/d/1_z_KRZw2RRQrMGTXgrwFUoavEIN6S0hKJRf7IQc8jCU
  Slides: https://docs.google.com/presentation/d/1gwg115bq-sSSqUzdhsGxvCKm1dH27IzQ23RuN1lqKSY/

  Why migrate?

  OPA/Rego adds significant overhead. Across 382 real catalog designs, 161 (42%) time out under OPA. All remaining divergences trace to 6 documented OPA bugs. Zero cases
  where OPA produces correct results that Go does not.

  What's implemented:

  - Full 3-phase evaluation pipeline in native Go: Validate, Identify, Actions
  - All 5 relationship policies via RelationshipPolicy interface: MatchLabels, Alias, EdgeNonBinding, EdgeBinding, HierarchicalParentChild
  - Evaluation trace properly populated (componentsAdded/Removed/Updated, relationshipsAdded/Removed/Updated)
  - Feature-flagged via USE_GO_POLICY_ENGINE. OPA remains the default for gradual rollout

  Testing across 382 catalog designs:

  49 designs produce content-level identical results between engines. The remaining divergences break down into OPA timeouts (161 designs hanging >10s), the legacy prepass
   deleting all relationships before the pipeline runs (51 designs), set conversion dropping matchlabel relationships (10 designs), Go finding more relationships that OPA
  misses (40 designs), and OPA retaining deleted relationships that Go correctly removes (1 design).

  Performance:

  - 7.7x overall speedup (17.2s vs 132.2s across 312 designs)
  - 16.9x full pipeline speedup on micro-benchmarks
  - 6x less memory per evaluation (72 KB vs 438 KB)

  Go engine improvements over OPA:

  - Self-referencing prevention (blocks components from relating to themselves)
  - MatchLabels cap (20 groups) to prevent combinatorial explosion
  - Per-design timeout protection
  - Proper deleted relationship cleanup
  - Inventory additions handled via standard RelationshipPolicy interface (no legacy prepass)

  Looking forward to your review and feedback.

Best  
Guan-Ming