MLflow 3.17.0

[Haoji Tang]

MLflow 3.17.0 has been released!

MLflow 3.17.0 brings TypeSafe’s Jev models to evaluation, tracing, and the AI Gateway, along with finer access controls and improvements to trace analytics and evaluation results. Here are the highlights of the release:

⚡ Faster LLM-as-a-Judge with Jev: Use TypeSafe’s Jev models for Boolean and categorical checks in your existing mlflow.genai.evaluate() workflow. Review structured decisions and probability metadata to identify close calls.

🔭 Observability for System One models: Trace synchronous and asynchronous Jev calls with mlflow.typesafe.autolog(). Inspect each structured answer and its probability details alongside the question and state that produced it.

🚪 Jev in MLflow AI Gateway: Route Jev calls through managed endpoints with server-side API keys, usage tracking, and budgets. Change the configured model without updating every application that calls it.

🔐 Fine-grained permissions for shared resources: Set permissions for runs, traces, and model versions using wildcard grants and explicit deny rules.

📊 Faster trace analytics: Opt into daily rollups to reduce SQL analytics query overhead for large trace datasets.

🔗 Dataset-linked evaluation metrics: Link aggregate evaluation metrics to the managed dataset that produced them, so you can see which dataset each result comes from.

Database upgrade: SQL-backed servers require a coordinated database upgrade, even with daily rollups disabled. Stop all writers, take a restorable backup, run mlflow db upgrade DATABASE_URL, and restart every replica on the new MLflow version. Mixed-version rolling upgrades are not supported. Upgrade guide.

For visual demos of the above features, check out the release post on our website! For a complete list of updates and bug fixes, check out the full changelog.

Share Your Feedback:
We’d love to hear about your experience with these new features:

• GitHub Issues - Report bugs or request features
• ⭐ Star us on GitHub - Show your support for the project