mlflow.entities
The mlflow.entities
module defines entities returned by the MLflow
REST API.
- class mlflow.entities.Assessment(name: str, source: AssessmentSource, trace_id: Optional[str] = None, rationale: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None, assessment_id: Optional[str] = None, error: Optional[AssessmentError] = None, expectation: Optional[mlflow.entities.assessment.ExpectationValue] = None, feedback: Optional[mlflow.entities.assessment.FeedbackValue] = None, overrides: Optional[str] = None, valid: Optional[bool] = None)[source]
Note
Experimental: This class may change or be removed in a future release without warning.
Base class for assessments that can be attached to a trace. An Assessment should be one of the following types:
- Expectations: A label that represents the expected value for a particular operation.
For example, an expected answer for a user question from a chatbot.
- Feedback: A label that represents the feedback on the quality of the operation.
Feedback can come from different sources, such as human judges, heuristic scorers, or LLM-as-a-Judge.
- error: Optional[AssessmentError] = None
- classmethod from_dictionary(d: dict[str, typing.Any]) Assessment [source]
- classmethod from_proto(proto)[source]
- source: AssessmentSource
- to_dictionary()[source]
- to_proto()[source]
- class mlflow.entities.AssessmentError(error_code: str, error_message: Optional[str] = None, stack_trace: Optional[str] = None)[source]
Note
Experimental: This class may change or be removed in a future release without warning.
Error object representing any issues during generating the assessment.
For example, if the LLM-as-a-Judge fails to generate an feedback, you can log an error with the error code and message as shown below:
from mlflow.entities import AssessmentError error = AssessmentError( error_code="RATE_LIMIT_EXCEEDED", error_message="Rate limit for the judge exceeded.", stack_trace="...", ) mlflow.log_feedback( trace_id="1234", name="faithfulness", source=AssessmentSourceType.LLM_JUDGE, error=error, # Skip setting value when an error is present )
- Parameters
error_code – The error code.
error_message – The detailed error message. Optional.
stack_trace – The stack trace of the error. Truncated to 1000 characters before being logged to MLflow. Optional.
- classmethod from_dictionary(error_dict)[source]
- classmethod from_proto(proto)[source]
- to_dictionary()[source]
- to_proto()[source]
- class mlflow.entities.AssessmentSource(source_type: str, source_id: str = 'default')[source]
Note
Experimental: This class may change or be removed in a future release without warning.
Source of an assessment (human, LLM as a judge with GPT-4, etc).
When recording an assessment, MLflow mandates providing a source information to keep track of how the assessment is conducted.
- Parameters
source_type – The type of the assessment source. Must be one of the values in the AssessmentSourceType enum.
source_id – An identifier for the source, e.g. user ID or LLM judge ID. If not provided, the default value “default” is used.
Example:
Human annotation can be represented with a source type of “HUMAN”:
import mlflow from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType source = AssessmentSource( source_type=AssessmentSourceType.HUMAN, source_id="bob@example.com", )
LLM-as-a-judge can be represented with a source type of “LLM_JUDGE”:
import mlflow from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType source = AssessmentSource( source_type=AssessmentSourceType.LLM_JUDGE, source_id="gpt-4o-mini", )
Heuristic evaluation can be represented with a source type of “CODE”:
import mlflow from mlflow.entities.assessment import AssessmentSource, AssessmentSourceType source = AssessmentSource( source_type=AssessmentSourceType.CODE, source_id="repo/evaluation_script.py", )
To record more context about the assessment, you can use the metadata field of the assessment logging APIs as well.
- classmethod from_dictionary(source_dict: dict[str, typing.Any]) AssessmentSource [source]
- classmethod from_proto(proto)[source]
- to_dictionary() dict[str, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.AssessmentSourceType(source_type: str)[source]
Note
Experimental: This class may change or be removed in a future release without warning.
- classmethod from_proto(proto_source_type) str [source]
- class mlflow.entities.Dataset(name: str, digest: str, source_type: str, source: str, schema: Optional[str] = None, profile: Optional[str] = None)[source]
Dataset object associated with an experiment.
- classmethod from_proto(proto)[source]
- to_dictionary()[source]
- to_proto()[source]
- class mlflow.entities.DatasetInput(dataset: Dataset, tags: Optional[list[InputTag]] = None)[source]
DatasetInput object associated with an experiment.
- property dataset: Dataset
Dataset.
- classmethod from_proto(proto)[source]
- property tags: list[InputTag]
Array of input tags.
- to_dictionary()[source]
- to_proto()[source]
- class mlflow.entities.Document(page_content: str, metadata: dict[str, typing.Any] = <factory>, id: Optional[str] = None)[source]
An entity used in MLflow Tracing to represent retrieved documents in a RETRIEVER span.
- Parameters
page_content – The content of the document.
metadata – A dictionary of metadata associated with the document.
id – The ID of the document.
- classmethod from_langchain_document(document)[source]
- classmethod from_llama_index_node_with_score(node_with_score)[source]
- to_dict()[source]
- class mlflow.entities.Expectation(name: str, value: Any, source: Optional[AssessmentSource] = None, trace_id: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None)[source]
Note
Experimental: This class may change or be removed in a future release without warning.
Represents an expectation about the output of an operation, such as the expected response that a generative AI application should provide to a particular user query.
- Parameters
name – The name of the assessment.
value – The expected value of the operation. This can be any JSON-serializable value.
source – The source of the assessment. If not provided, the default source is HUMAN.
trace_id – The ID of the trace associated with the assessment. If unset, the assessment is not associated with any trace yet. should be specified.
metadata – The metadata associated with the assessment.
span_id – The ID of the span associated with the assessment, if the assessment should be associated with a particular span in the trace.
create_time_ms – The creation time of the assessment in milliseconds. If unset, the current time is used.
last_update_time_ms – The last update time of the assessment in milliseconds. If unset, the current time is used.
Example
from mlflow.entities import AssessmentSource, Expectation expectation = Expectation( name="expected_response", value="The capital of France is Paris.", source=AssessmentSource( source_type=AssessmentSourceType.HUMAN, source_id="john@example.com", ), metadata={"project": "my-project"}, )
- classmethod from_dictionary(d: dict[str, typing.Any]) Expectation [source]
- classmethod from_proto(proto) Expectation [source]
- class mlflow.entities.Experiment(experiment_id, name, artifact_location, lifecycle_stage, tags=None, creation_time=None, last_update_time=None)[source]
Experiment object.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.ExperimentTag(key, value)[source]
Tag object associated with an experiment.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.Feedback(name: str = 'feedback', value: Optional[Union[float, int, str, bool, dict[str, typing.Union[float, int, str, bool]], list[typing.Union[float, int, str, bool]]]] = None, error: Optional[Union[Exception, AssessmentError]] = None, source: Optional[AssessmentSource] = None, trace_id: Optional[str] = None, metadata: Optional[dict[str, str]] = None, span_id: Optional[str] = None, create_time_ms: Optional[int] = None, last_update_time_ms: Optional[int] = None, rationale: Optional[str] = None, overrides: Optional[str] = None, valid: bool = True)[source]
Note
Experimental: This class may change or be removed in a future release without warning.
Represents feedback about the output of an operation. For example, if the response from a generative AI application to a particular user query is correct, then a human or LLM judge may provide feedback with the value
"correct"
.- Parameters
name – The name of the assessment. If not provided, the default name “feedback” is used.
value – The feedback value. This can be one of the following types: - float - int - str - bool - list of values of the same types as above - dict with string keys and values of the same types as above
error – An optional error associated with the feedback. This is used to indicate that the feedback is not valid or cannot be processed. Accepts an exception object, or an
Expectation
object.rationale – The rationale / justification for the feedback.
source – The source of the assessment. If not provided, the default source is CODE.
trace_id – The ID of the trace associated with the assessment. If unset, the assessment is not associated with any trace yet. should be specified.
metadata – The metadata associated with the assessment.
span_id – The ID of the span associated with the assessment, if the assessment should be associated with a particular span in the trace.
create_time_ms – The creation time of the assessment in milliseconds. If unset, the current time is used.
last_update_time_ms – The last update time of the assessment in milliseconds. If unset, the current time is used.
Example
from mlflow.entities import AssessmentSource, Feedback feedback = Feedback( name="correctness", value=True, rationale="The response is correct.", source=AssessmentSource( source_type="HUMAN", source_id="john@example.com", ), metadata={"project": "my-project"}, )
- property error_code: Optional[str]
The error code of the error that occurred when the feedback was created.
- property error_message: Optional[str]
The error message of the error that occurred when the feedback was created.
- classmethod from_proto(proto)[source]
- source: AssessmentSource
- class mlflow.entities.FileInfo(path, is_dir, file_size)[source]
Metadata about a file or directory.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.InferenceTableLocation(full_table_name: str)[source]
Represents the location of a Databricks inference table.
- Parameters
full_table_name – The fully qualified name of the inference table where the trace is stored, in the format of <catalog>.<schema>.<table>.
- classmethod from_dict(d: dict[str, typing.Any]) InferenceTableLocation [source]
- classmethod from_proto(proto) InferenceTableLocation [source]
- to_dict() dict[str, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.InputTag(key: str, value: str)[source]
Input tag object associated with a dataset.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.LifecycleStage[source]
-
- classmethod is_valid(lifecycle_stage)[source]
- classmethod matches_view_type(view_type, lifecycle_stage)[source]
- classmethod view_type_to_stages(view_type=3)[source]
- class mlflow.entities.LiveSpan(otel_span: opentelemetry.trace.span.Span, trace_id: str, span_type: str = 'UNKNOWN')[source]
A “live” version of the
Span
class.The live spans are those being created and updated during the application runtime. When users start a new span using the tracing APIs within their code, this live span object is returned to get and set the span attributes, status, events, and etc.
- add_event(event: SpanEvent)[source]
Add an event to the span.
- Parameters
event – The event to add to the span. This should be a
SpanEvent
object.
- from_dict(data: dict[str, typing.Any]) Span [source]
Create a Span object from the given dictionary.
- record_exception(exception: Union[str, Exception])[source]
Record an exception on the span, adding an exception event and setting span status to ERROR.
- Parameters
exception – The exception to record. Can be an Exception instance or a string describing the exception.
- set_attribute(key: str, value: Any)[source]
Set a single attribute to the span.
- set_attributes(attributes: dict[str, typing.Any])[source]
Set the attributes to the span. The attributes must be a dictionary of key-value pairs. This method is additive, i.e. it will add new attributes to the existing ones. If an attribute with the same key already exists, it will be overwritten.
- set_inputs(inputs: Any)[source]
Set the input values to the span.
- set_outputs(outputs: Any)[source]
Set the output values to the span.
- set_span_type(span_type: str)[source]
Set the type of the span.
- set_status(status: Union[SpanStatusCode, str])[source]
Set the status of the span.
- Parameters
status – The status of the span. This can be a
SpanStatus
object or a string representing of the status code defined inSpanStatusCode
e.g."OK"
,"ERROR"
.
- class mlflow.entities.LoggedModel(experiment_id: str, model_id: str, name: str, artifact_location: str, creation_timestamp: int, last_updated_timestamp: int, model_type: Optional[str] = None, source_run_id: Optional[str] = None, status: Union[LoggedModelStatus, int] = LoggedModelStatus.READY, status_message: Optional[str] = None, tags: Optional[Union[list[LoggedModelTag], dict[str, str]]] = None, params: Optional[Union[list[LoggedModelParameter], dict[str, str]]] = None, metrics: Optional[list[Metric]] = None)[source]
MLflow entity representing a Model logged to an MLflow Experiment.
- property creation_timestamp: int
Integer. Model creation timestamp (milliseconds since the Unix epoch).
- classmethod from_proto(proto)[source]
- property last_updated_timestamp: int
Integer. Timestamp of last update for this Model (milliseconds since the Unix epoch).
- property metrics: Optional[list[Metric]]
List of metrics associated with this Model.
- property status: LoggedModelStatus
String. Current status of this Model.
- to_dictionary() dict[str, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.LoggedModelInput(model_id: str)[source]
ModelInput object associated with a Run.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.LoggedModelOutput(model_id: str, step: int)[source]
ModelOutput object associated with a Run.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.LoggedModelParameter(key, value)[source]
MLflow entity representing a parameter of a Model.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.LoggedModelStatus(value)[source]
Enum for status of an
mlflow.entities.LoggedModel
.- classmethod from_int(status_int: int) LoggedModelStatus [source]
- classmethod from_proto(proto)[source]
- static is_finalized(status) bool [source]
Determines whether or not a LoggedModelStatus is a finalized status. A finalized status indicates that no further status updates will occur.
- to_int() int [source]
- to_proto()[source]
- class mlflow.entities.LoggedModelTag(key, value)[source]
Tag object associated with a Model.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.Metric(key, value, timestamp, step, model_id: Optional[str] = None, dataset_name: Optional[str] = None, dataset_digest: Optional[str] = None, run_id: Optional[str] = None)[source]
Metric object.
- classmethod from_dictionary(metric_dict)[source]
Create a Metric object from a dictionary.
- Parameters
metric_dict (dict) – Dictionary containing metric information.
- Returns
The Metric object created from the dictionary.
- Return type
- classmethod from_proto(proto)[source]
- to_dictionary()[source]
Convert the Metric object to a dictionary.
- Returns
The Metric object represented as a dictionary.
- Return type
dict
- to_proto()[source]
- class mlflow.entities.MlflowExperimentLocation(experiment_id: str)[source]
Represents the location of an MLflow experiment.
- Parameters
experiment_id – The ID of the MLflow experiment where the trace is stored.
- classmethod from_dict(d: dict[str, typing.Any]) MlflowExperimentLocation [source]
- classmethod from_proto(proto) MlflowExperimentLocation [source]
- to_dict() dict[str, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.NoOpSpan[source]
No-op implementation of the Span interface.
This instance should be returned from the mlflow.start_span context manager when span creation fails. This class should have exactly the same interface as the Span so that user’s setter calls do not raise runtime errors.
E.g.
with mlflow.start_span("span_name") as span: # Even if the span creation fails, the following calls should pass. span.set_inputs({"x": 1}) # Do something
- end(outputs: Optional[Any] = None, attributes: Optional[dict[str, typing.Any]] = None, status: Optional[Union[SpanStatus, str]] = None, end_time_ns: Optional[int] = None)[source]
- record_exception(exception: Union[str, Exception])[source]
- set_attribute(key: str, value: Any)[source]
- set_attributes(attributes: dict[str, typing.Any])[source]
- set_inputs(inputs: dict[str, typing.Any])[source]
- set_outputs(outputs: dict[str, typing.Any])[source]
- set_status(status: SpanStatus)[source]
- class mlflow.entities.Param(key, value)[source]
Parameter object.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.Prompt(name: str, version: int, template: str, commit_message: Optional[str] = None, creation_timestamp: Optional[int] = None, version_metadata: Optional[dict[str, str]] = None, prompt_tags: Optional[dict[str, str]] = None, aliases: Optional[list[str]] = None)[source]
An entity representing a prompt (template) for GenAI applications.
- Parameters
name – The name of the prompt.
version – The version number of the prompt.
template – The template text of the prompt. It can contain variables enclosed in double curly braces, e.g. {{variable}}, which will be replaced with actual values by the format method. MLflow use the same variable naming rules same as Jinja2 https://jinja.palletsprojects.com/en/stable/api/#notes-on-identifiers
commit_message – The commit message for the prompt version. Optional.
creation_timestamp – Timestamp of the prompt creation. Optional.
version_metadata – A dictionary of metadata associated with the prompt version. This is useful for storing version-specific information, such as the author of the changes. Optional.
prompt_tags – A dictionary of tags associated with the entire prompt. This is different from the version_metadata as it is not tied to a specific version of the prompt.
- format(allow_partial: bool = False, **kwargs) Union[Prompt, str] [source]
Format the template text with the given keyword arguments. By default, it raises an error if there are missing variables. To format the prompt text partially, set allow_partial=True.
Example:
prompt = Prompt("my-prompt", 1, "Hello, {{title}} {{name}}!") formatted = prompt.format(title="Ms", name="Alice") print(formatted) # Output: "Hello, Ms Alice!" # Partial formatting formatted = prompt.format(title="Ms", allow_partial=True) print(formatted) # Output: Prompt(name=my-prompt, version=1, template="Hello, Ms {{name}}!")
- Parameters
allow_partial – If True, allow partial formatting of the prompt text. If False, raise an error if there are missing variables.
kwargs – Keyword arguments to replace the variables in the template.
- classmethod from_model_version(model_version: ModelVersion, prompt_tags: Optional[dict[str, str]] = None) Prompt [source]
Create a Prompt object from a ModelVersion object.
- Parameters
model_version – The ModelVersion object to convert to a Prompt.
prompt_tags – The prompt-level tags (from RegisteredModel). Optional.
- to_single_brace_format() str [source]
Convert the template text to single brace format. This is useful for integrating with other systems that use single curly braces for variable replacement, such as LangChain’s prompt template. Default is False.
- class mlflow.entities.Run(run_info: RunInfo, run_data: RunData, run_inputs: Optional[RunInputs] = None, run_outputs: Optional[RunOutputs] = None)[source]
Run object.
- property data: RunData
The run data, including metrics, parameters, and tags.
- Return type
- classmethod from_proto(proto)[source]
- property info: RunInfo
The run metadata, such as the run id, start time, and status.
- Return type
- property inputs: RunInputs
The run inputs, including dataset inputs.
- Return type
- property outputs: RunOutputs
The run outputs, including model outputs.
- Return type
- to_dictionary() dict[typing.Any, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.RunData(metrics=None, params=None, tags=None)[source]
Run data (metrics and parameters).
- classmethod from_proto(proto)[source]
- property metrics
Dictionary of string key -> metric value for the current run. For each metric key, the metric value with the latest timestamp is returned. In case there are multiple values with the same latest timestamp, the maximum of these values is returned.
- to_dictionary()[source]
- to_proto()[source]
- class mlflow.entities.RunInfo(run_id, experiment_id, user_id, status, start_time, end_time, lifecycle_stage, artifact_uri=None, run_name=None)[source]
Metadata about a run.
- property artifact_uri[source]
String root artifact URI of the run.
- property end_time[source]
End time of the run, in number of milliseconds since the UNIX epoch.
- classmethod from_proto(proto)[source]
- classmethod get_orderable_attributes()[source]
- classmethod get_searchable_attributes()[source]
- property lifecycle_stage
One of the values in
LifecycleStage
describing the lifecycle stage of the run.
- property run_id[source]
String containing run id.
- property run_name[source]
String containing run name.
- property start_time[source]
Start time of the run, in number of milliseconds since the UNIX epoch.
- property status[source]
One of the values in
mlflow.entities.RunStatus
describing the status of the run.
- to_proto()[source]
- property user_id[source]
String ID of the user who initiated this run.
- class mlflow.entities.RunInputs(dataset_inputs: list[DatasetInput], model_inputs: Optional[list[LoggedModelInput]] = None)[source]
RunInputs object.
- property dataset_inputs: list[DatasetInput]
Array of dataset inputs.
- classmethod from_proto(proto)[source]
- property model_inputs: list[LoggedModelInput]
Array of model inputs.
- to_dictionary() dict[str, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.RunOutputs(model_outputs: list[LoggedModelOutput])[source]
RunOutputs object.
- classmethod from_proto(proto)[source]
- property model_outputs: list[LoggedModelOutput]
Array of model outputs.
- to_dictionary() dict[typing.Any, typing.Any] [source]
- to_proto()[source]
- class mlflow.entities.RunStatus[source]
Enum for status of an
mlflow.entities.Run
.- static all_status()[source]
- static from_string(status_str)[source]
- static is_terminated(status)[source]
- static to_string(status)[source]
- class mlflow.entities.RunTag(key, value)[source]
Tag object associated with a run.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.SourceType[source]
Enum for originating source of a
mlflow.entities.Run
.- static from_string(status_str)[source]
- static to_string(status)[source]
- class mlflow.entities.Span(otel_span: opentelemetry.sdk.trace.ReadableSpan)[source]
A span object. A span represents a unit of work or operation and is the building block of Traces.
This Span class represents immutable span data that is already finished and persisted. The “live” span that is being created and updated during the application runtime is represented by the
LiveSpan
subclass.- property attributes: dict[str, typing.Any]
Get all attributes of the span.
- Returns
A dictionary of all attributes of the span.
- property events: list[SpanEvent]
Get all events of the span.
- Returns
A list of all events of the span.
- classmethod from_dict(data: dict[str, typing.Any]) Span [source]
Create a Span object from the given dictionary.
- classmethod from_dict_v2(data: dict[str, typing.Any]) Span [source]
Create a Span object from the given dictionary in v2 schema.
- get_attribute(key: str) Optional[Any] [source]
Get a single attribute value from the span.
- Parameters
key – The key of the attribute to get.
- Returns
The value of the attribute if it exists, otherwise None.
- property status: SpanStatus
The status of the span.
- to_dict() dict[str, typing.Any] [source]
- to_proto()[source]
Convert into OTLP compatible proto object to sent to the Databricks Trace Server.
- class mlflow.entities.SpanEvent(name: str, timestamp: int = <factory>, attributes: dict[str, typing.Union[str, bool, int, float, typing.Sequence[str], typing.Sequence[bool], typing.Sequence[int], typing.Sequence[float]]] = <factory>)[source]
An event that records a specific occurrences or moments in time during a span, such as an exception being thrown. Compatible with OpenTelemetry.
- Parameters
name – Name of the event.
timestamp – The exact time the event occurred, measured in nanoseconds. If not provided, the current time will be used.
attributes – A collection of key-value pairs representing detailed attributes of the event, such as the exception stack trace. Attributes value must be one of
[str, int, float, bool, bytes]
or a sequence of these types.
- attributes: dict[str, typing.Union[str, bool, int, float, typing.Sequence[str], typing.Sequence[bool], typing.Sequence[int], typing.Sequence[float]]]
- classmethod from_exception(exception: Exception)[source]
Create a span event from an exception.
- json()[source]
- to_proto()[source]
Convert into OTLP compatible proto object to sent to the Databricks Trace Server.
- class mlflow.entities.SpanStatus(status_code: SpanStatusCode, description: str = '')[source]
Status of the span or the trace.
- Parameters
status_code – The status code of the span or the trace. This must be one of the values of the
mlflow.entities.SpanStatusCode
enum or a string representation of it like “OK”, “ERROR”.description – Description of the status. This should be only set when the status is ERROR, otherwise it will be ignored.
- status_code: SpanStatusCode
- class mlflow.entities.SpanStatusCode(value)[source]
Enum for status code of a span
- static from_proto_status_code(status_code: str) SpanStatusCode [source]
Convert a string status code to the corresponding SpanStatusCode enum value.
- class mlflow.entities.SpanType[source]
Predefined set of span types.
- class mlflow.entities.Trace(info: TraceInfo, data: TraceData)[source]
A trace object.
- Parameters
info – A lightweight object that contains the metadata of a trace.
data – A container object that holds the spans data of a trace.
- data: TraceData
- info: TraceInfo
- static pandas_dataframe_columns() list[str] [source]
- search_assessments(name: Optional[str] = None, *, span_id: Optional[str] = None, all: bool = False, type: Optional[Literal['expectation', 'feedback']] = None) list['Assessment'] [source]
Get assessments for a given name / span ID. By default, this only returns assessments that are valid (i.e. have not been overridden by another assessment). To return all assessments, specify all=True.
- Parameters
name – The name of the assessment to get. If not provided, this will match all assessment names.
span_id – The span ID to get assessments for. If not provided, this will match all spans.
all – If True, return all assessments regardless of validity.
type – The type of assessment to get (one of “feedback” or “expectation”). If not provided, this will match all assessment types.
- Returns
A list of assessments that meet the given conditions.
- search_spans(span_type: Optional[SpanType] = None, name: Optional[Union[str, re.Pattern]] = None, span_id: Optional[str] = None) list[Span] [source]
Search for spans that match the given criteria within the trace.
- Parameters
span_type – The type of the span to search for.
name – The name of the span to search for. This can be a string or a regular expression.
span_id – The ID of the span to search for.
- Returns
A list of spans that match the given criteria. If there is no match, an empty list is returned.
import mlflow import re from mlflow.entities import SpanType @mlflow.trace(span_type=SpanType.CHAIN) def run(x: int) -> int: x = add_one(x) x = add_two(x) x = multiply_by_two(x) return x @mlflow.trace(span_type=SpanType.TOOL) def add_one(x: int) -> int: return x + 1 @mlflow.trace(span_type=SpanType.TOOL) def add_two(x: int) -> int: return x + 2 @mlflow.trace(span_type=SpanType.TOOL) def multiply_by_two(x: int) -> int: return x * 2 # Run the function and get the trace y = run(2) trace_id = mlflow.get_last_active_trace_id() trace = mlflow.get_trace(trace_id) # 1. Search spans by name (exact match) spans = trace.search_spans(name="add_one") print(spans) # Output: [Span(name='add_one', ...)] # 2. Search spans by name (regular expression) pattern = re.compile(r"add.*") spans = trace.search_spans(name=pattern) print(spans) # Output: [Span(name='add_one', ...), Span(name='add_two', ...)] # 3. Search spans by type spans = trace.search_spans(span_type=SpanType.LLM) print(spans) # Output: [Span(name='run', ...)] # 4. Search spans by name and type spans = trace.search_spans(name="add_one", span_type=SpanType.TOOL) print(spans) # Output: [Span(name='add_one', ...)]
- to_dict() dict[str, typing.Any] [source]
- to_json(pretty=False) str [source]
- to_pandas_dataframe_row() dict[str, typing.Any] [source]
- to_proto()[source]
Convert into a proto object to sent to the MLflow backend.
- NB: The Trace definition in MLflow backend doesn’t include the data field,
but rather only contains TraceInfoV3.
- class mlflow.entities.TraceData(spans: Optional[list[Span]] = None, **kwargs)[source]
A container object that holds the spans data of a trace.
- Parameters
spans – List of spans that are part of the trace.
- classmethod from_dict(d)[source]
- property intermediate_outputs: Optional[dict[str, typing.Any]]
Returns intermediate outputs produced by the model or agent while handling the request. There are mainly two flows to return intermediate outputs: 1. When a trace is generate by the mlflow.log_trace API, return intermediate_outputs attribute of the span. 2. When a trace is created normally with a tree of spans, aggregate the outputs of non-root spans.
- spans: list[Span]
- to_dict() dict[str, typing.Any] [source]
- class mlflow.entities.TraceInfo(trace_id: str, trace_location: mlflow.entities.trace_location.TraceLocation, request_time: int, state: mlflow.entities.trace_state.TraceState, request_preview: Optional[str] = None, response_preview: Optional[str] = None, client_request_id: Optional[str] = None, execution_duration: Optional[int] = None, trace_metadata: dict[str, str] = <factory>, tags: dict[str, str] = <factory>, assessments: list[mlflow.entities.assessment.Assessment] = <factory>)[source]
Metadata about a trace, such as its ID, location, timestamp, etc.
- Parameters
trace_id – The primary identifier for the trace.
trace_location – The location where the trace is stored, represented as a
TraceLocation
object. MLflow currently support MLflow Experiment or Databricks Inference Table as a trace location.request_time – Start time of the trace, in milliseconds.
state – State of the trace, represented as a
TraceState
enum. Can be one of [OK, ERROR, IN_PROGRESS, STATE_UNSPECIFIED].request_preview – Request to the model/agent, equivalent to the input of the root, span but JSON-encoded and can be truncated.
response_preview – Response from the model/agent, equivalent to the output of the root span but JSON-encoded and can be truncated.
client_request_id – Client supplied request ID associated with the trace. This could be used to identify the trace/request from an external system that produced the trace, e.g., a session ID in a web application.
execution_duration – Duration of the trace, in milliseconds.
trace_metadata – Key-value pairs associated with the trace. They are designed for immutable values like run ID associated with the trace.
tags – Tags associated with the trace. They are designed for mutable values, that can be updated after the trace is created via MLflow UI or API.
assessments – List of assessments associated with the trace.
- assessments: list[Assessment]
- property experiment_id: Optional[str]
An MLflow experiment ID associated with the trace, if the trace is stored in MLflow tracking server. Otherwise, None.
- classmethod from_dict(d: dict[str, typing.Any]) TraceInfo [source]
Create a TraceInfoV3 object from a dictionary.
- state: TraceState
- to_dict() dict[str, typing.Any] [source]
Convert the TraceInfoV3 object to a dictionary.
- to_proto()[source]
- property token_usage: Optional[dict[str, int]]
Returns the aggregated token usage for the trace.
- Returns
A dictionary containing the aggregated LLM token usage for the trace. - “input_tokens”: The total number of input tokens. - “output_tokens”: The total number of output tokens. - “total_tokens”: Sum of input and output tokens.
Note
The token usage tracking is not supported for all LLM providers. Refer to the MLflow Tracing documentation for which providers support token usage tracking.
- trace_location: TraceLocation
- class mlflow.entities.TraceInfoV2(request_id: str, experiment_id: str, timestamp_ms: int, execution_time_ms: Optional[int], status: mlflow.entities.trace_status.TraceStatus, request_metadata: dict[str, str] = <factory>, tags: dict[str, str] = <factory>, assessments: list[mlflow.entities.assessment.Assessment] = <factory>)[source]
Metadata about a trace.
- Parameters
request_id – id of the trace.
experiment_id – id of the experiment.
timestamp_ms – start time of the trace, in milliseconds.
execution_time_ms – duration of the trace, in milliseconds.
status – status of the trace.
request_metadata – Key-value pairs associated with the trace. Request metadata are designed for immutable values like run ID associated with the trace.
tags – Tags associated with the trace. Tags are designed for mutable values like trace name, that can be updated by the users after the trace is created, unlike request_metadata.
- assessments: list[Assessment]
- classmethod from_dict(trace_info_dict)[source]
Convert trace info dictionary to TraceInfo object.
- classmethod from_proto(proto, assessments=None)[source]
- to_dict()[source]
Convert trace info to a dictionary for persistence. Update status field to the string value for serialization.
- to_proto()[source]
- class mlflow.entities.TraceLocation(type: TraceLocationType, mlflow_experiment: Optional[MlflowExperimentLocation] = None, inference_table: Optional[InferenceTableLocation] = None)[source]
Represents the location where the trace is stored.
Currently, MLflow supports two types of trace locations:
MLflow experiment: The trace is stored in an MLflow experiment.
Inference table: The trace is stored in a Databricks inference table.
- Parameters
type – The type of the trace location, should be one of the
TraceLocationType
enum values.mlflow_experiment – The MLflow experiment location. Set this when the location type is MLflow experiment.
inference_table – The inference table location. Set this when the location type is Databricks Inference table.
- classmethod from_dict(d: dict[str, typing.Any]) TraceLocation [source]
- classmethod from_experiment_id(experiment_id: str) TraceLocation [source]
- classmethod from_proto(proto) TraceLocation [source]
- inference_table: Optional[InferenceTableLocation] = None
- mlflow_experiment: Optional[MlflowExperimentLocation] = None
- to_dict() dict[str, typing.Any] [source]
- to_proto()[source]
- type: TraceLocationType
- class mlflow.entities.TraceLocationType(value)[source]
An enumeration.
- classmethod from_dict(d: dict[str, typing.Any]) TraceLocationType [source]
- classmethod from_proto(proto: int) TraceLocationType [source]
- to_proto()[source]
- class mlflow.entities.TraceState(value)[source]
Enum representing the state of a trace.
STATE_UNSPECIFIED
: Unspecified trace state.OK
: Trace successfully completed.ERROR
: Trace encountered an error.IN_PROGRESS
: Trace is currently in progress.
- static from_otel_status(otel_status: opentelemetry.trace.status.Status)[source]
Convert OpenTelemetry status code to MLflow TraceState.
- classmethod from_proto(proto: int) TraceState [source]
- to_proto()[source]
- class mlflow.entities.ViewType[source]
Enum to filter requested experiment types.
- classmethod from_proto(proto_view_type)[source]
- classmethod from_string(view_str)[source]
- classmethod to_proto(view_type)[source]
- classmethod to_string(view_type)[source]
- class mlflow.entities.model_registry.ModelVersion(name: str, version: str, creation_timestamp: int, last_updated_timestamp: Optional[int] = None, description: Optional[str] = None, user_id: Optional[str] = None, current_stage: Optional[str] = None, source: Optional[str] = None, run_id: Optional[str] = None, status: str = 'READY', status_message: Optional[str] = None, tags: Optional[list[ModelVersionTag]] = None, run_link: Optional[str] = None, aliases: Optional[list[str]] = None, model_id: Optional[str] = None, params: Optional[list[LoggedModelParameter]] = None, metrics: Optional[list[Metric]] = None, deployment_job_state: Optional[ModelVersionDeploymentJobState] = None)[source]
MLflow entity for Model Version.
- property creation_timestamp: int
Integer. Model version creation timestamp (milliseconds since the Unix epoch).
- property deployment_job_state: Optional[ModelVersionDeploymentJobState]
Deployment job state for the current model version.
- classmethod from_proto(proto) ModelVersion [source]
- property last_updated_timestamp: Optional[int]
Integer. Timestamp of last update for this model version (milliseconds since the Unix epoch).
- property metrics: Optional[list[Metric]]
List of metrics associated with this model version.
- property params: Optional[list[LoggedModelParameter]]
List of parameters associated with this model version.
- property run_link: Optional[str]
String. MLflow run link referring to the exact run that generated this model version.
- property tags: dict[str, str]
Dictionary of tag key (string) -> tag value for the current model version.
- to_proto()[source]
- class mlflow.entities.model_registry.ModelVersionDeploymentJobState(job_id, run_id, job_state, run_state, current_task_name)[source]
Deployment Job state object associated with a model version.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.model_registry.ModelVersionSearch(*args, **kwargs)[source]
- aliases()[source]
List of aliases (string) for the current model version.
- tags()[source]
Dictionary of tag key (string) -> tag value for the current model version.
- class mlflow.entities.model_registry.ModelVersionTag(key, value)[source]
Tag object associated with a model version.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.model_registry.RegisteredModel(name, creation_timestamp=None, last_updated_timestamp=None, description=None, latest_versions=None, tags=None, aliases=None, deployment_job_id=None, deployment_job_state=None)[source]
MLflow entity for Registered Model.
- property creation_timestamp
Integer. Model version creation timestamp (milliseconds since the Unix epoch).
- classmethod from_proto(proto)[source]
- property last_updated_timestamp
Integer. Timestamp of last update for this model version (milliseconds since the Unix epoch).
- property latest_versions
List of the latest
mlflow.entities.model_registry.ModelVersion
instances for each stage.
- to_proto()[source]
- class mlflow.entities.model_registry.RegisteredModelAlias(alias, version)[source]
Alias object associated with a registered model.
- classmethod from_proto(proto)[source]
- to_proto()[source]
- class mlflow.entities.model_registry.RegisteredModelDeploymentJobState[source]
Enum for registered model deployment state of an
mlflow.entities.model_registry.RegisteredModel
.- static all_states()[source]
- static from_string(state_str)[source]
- static to_string(state)[source]
- class mlflow.entities.model_registry.RegisteredModelSearch(*args, **kwargs)[source]
- aliases()[source]
Dictionary of aliases (string) -> version for the current registered model.
- tags()[source]
Dictionary of tag key (string) -> tag value for the current registered model.