MLflow 2.17.0rc0

[Daniel Lok]

We are excited to announce MLflow 2.17.0rc0, the release candidate for MLflow 2.17.0. This release includes several enhancements to extend the functionality of MLflow's ChatModel interface to further extend its versatility for handling custom GenAI application use cases. We're also starting the work on improving both the utility and the versatility of MLflow's evaluate functionality for GenAI, initially with support for callable GenAI evaluation metrics.

To install, run the following command:

pip install mlflow==2.17.0rc0

Please try it out and report any issues on the issue tracker.

Major Features and notifications

• ChatModel enhancements - As the GenAI-focused 'cousin' of PythonModel, ChatModel is getting some sizable functionality extensions. From native support for tool calling (a requirement for creating a custom agent), simpler conversions to the internal dataclass constructs needed to interface with ChatModel via the introduction of from_dict methods to all data structures, the addition of a metadata field to allow for full input payload customization, handling of the new refusal response type, to the inclusion of the interface type to the response structure to allow for greater integration compatibility. (#13191, #13180, #13143, @daniellok-db, #13102, #13071, @BenWilson2)

• Callable GenAI Evaluation Metrics - As the intial step in a much broader expansion of the functionalities of mlflow.evaluate for GenAI use cases, we've converted the GenAI evaluation metrics to be callable. This allows you to use them directly in packages that support callable GenAI evaluation metrics, as well as making it simpler to debug individual responses when prototyping solutions. (#13144, @serena-ruan)

• Audio file support in the MLflow UI - You can now directly 'view' audio files that have been logged and listen to them from within the MLflow UI's artifact viewer pane. (#13017, @sydneyw-spotify)

• MLflow AI Gateway is no longer deprecated - We've decided to revert our deprecation for the AI Gateway feature. We had renamed it to the MLflow Deployments Server, but have reconsidered and reverted the naming and namespace back to the original configuration.

Features:

• [Models] Add tool calling support for ChatModel (#13191, @daniellok-db)
• [Models] Add from_dict() function to ChatModel dataclasses (#13180, @daniellok-db)
• [Models] Add metadata field for ChatModel (#13143, @daniellok-db)
• [Models] Update ChatCompletionResponse to populate object type (#13102, @BenWilson2)
• [Models] Add support for LLM response refusal (#13071, @BenWilson2)
• [Eval] Make Evaluation metrics callable (#13144, @serena-ruan)
• [Databricks] Add support for route_optimized parameter in databricks deployment client (#13222, @prabhatkgupta)

Bug fixes:

• [Tracking] Fix tracing for LangGraph (#13215, @B-Step62)
• [Model Registry] Fix retry and credential refresh issues with artifact downloads from the model registry (#12935, @rohitarun-db)
• [Tracking] Fix LangChain autologging so that langchain-community is not required for partner packages (#13172, @B-Step62)
• [Artifacts] Fix issues with file removal for the local artifact repository (#13005, @rzalawad)

Documentation updates:

• [Docs] Add guide for building custom GenAI apps with ChatModel (#13207, @BenWilson2)
• [Docs] Add updates to the MLflow AI Gateway documentation (#13217, @daniellok-db)
• [Docs] Remove MLflow AI Gateway deprecation status (#13153, @BenWilson2)
  

For a comprehensive list of changes, see the release change log, and check out the latest documentation on mlflow.org.