mlflow.gateway
- mlflow.gateway.get_gateway_uri() str [source]
Warning
MLflow AI gateway is deprecated and has been replaced by the deployments API for generative AI. See https://mlflow.org/docs/latest/llms/gateway/migration.html for migration.
Returns the currently set MLflow AI Gateway server uri iff set. If the Gateway uri has not been set by using
set_gateway_uri
, anMlflowException
is raised.
- mlflow.gateway.set_gateway_uri(gateway_uri: str)[source]
Warning
MLflow AI gateway is deprecated and has been replaced by the deployments API for generative AI. See https://mlflow.org/docs/latest/llms/gateway/migration.html for migration.
- Sets the uri of a configured and running MLflow AI Gateway server in a global context.
Providing a valid uri and calling this function is required in order to use the MLflow AI Gateway fluent APIs.
- Args:
- gateway_uri: The full uri of a running MLflow AI Gateway server or, if running on
Databricks, “databricks”.
- class mlflow.gateway.base_models.ConfigModel[source]
A pydantic model representing Gateway configuration data, such as an OpenAI completions route definition including route name, model name, API keys, etc.
- class mlflow.gateway.config.AI21LabsConfig(*, ai21labs_api_key: str)[source]
-
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_ai21labs_api_key(value)[source]
- class mlflow.gateway.config.AWSBaseConfig(*, aws_region: Optional[str] = None)[source]
- class mlflow.gateway.config.AWSIdAndKey(*, aws_region: Optional[str] = None, aws_access_key_id: str, aws_secret_access_key: str, aws_session_token: Optional[str] = None)[source]
- class mlflow.gateway.config.AWSRole(*, aws_region: Optional[str] = None, aws_role_arn: str, session_length_seconds: int = 900)[source]
- class mlflow.gateway.config.AliasedConfigModel[source]
Enables use of field aliases in a configuration model for backwards compatibility
- class mlflow.gateway.config.AmazonBedrockConfig(*, aws_config: Union[mlflow.gateway.config.AWSRole, mlflow.gateway.config.AWSIdAndKey, mlflow.gateway.config.AWSBaseConfig])[source]
- class mlflow.gateway.config.AnthropicConfig(*, anthropic_api_key: str, anthropic_version: str = '2023-06-01')[source]
-
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_anthropic_api_key(value)[source]
- class mlflow.gateway.config.CohereConfig(*, cohere_api_key: str)[source]
-
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_cohere_api_key(value)[source]
- class mlflow.gateway.config.GatewayConfig(*, endpoints: list[mlflow.gateway.config.RouteConfig])[source]
- endpoints: list[mlflow.gateway.config.RouteConfig]
- class mlflow.gateway.config.GeminiConfig(*, gemini_api_key: str)[source]
-
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_gemini_api_key(value)[source]
- class mlflow.gateway.config.HuggingFaceTextGenerationInferenceConfig(*, hf_server_url: str)[source]
- class mlflow.gateway.config.Limit(*, calls: int, key: Optional[str] = None, renewal_period: str)[source]
- class mlflow.gateway.config.LimitsConfig(*, limits: Optional[list[mlflow.gateway.config.Limit]] = [])[source]
- limits: Optional[list[mlflow.gateway.config.Limit]]
- class mlflow.gateway.config.MistralConfig(*, mistral_api_key: str)[source]
-
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_mistral_api_key(value)[source]
- class mlflow.gateway.config.MlflowModelServingConfig(*, model_server_url: str)[source]
- class mlflow.gateway.config.Model(*, name: Optional[str] = None, provider: Union[str, mlflow.gateway.config.Provider], config: Optional[mlflow.gateway.base_models.ConfigModel] = None)[source]
- config: Optional[mlflow.gateway.base_models.ConfigModel]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- provider: Union[str, mlflow.gateway.config.Provider]
- classmethod validate_config(info, values)[source]
- classmethod validate_provider(value)[source]
- class mlflow.gateway.config.ModelInfo(*, name: Optional[str] = None, provider: mlflow.gateway.config.Provider)[source]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- provider: mlflow.gateway.config.Provider
- class mlflow.gateway.config.MosaicMLConfig(*, mosaicml_api_key: str, mosaicml_api_base: Optional[str] = None)[source]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_mosaicml_api_key(value)[source]
- class mlflow.gateway.config.OpenAIAPIType(value)[source]
An enumeration.
- class mlflow.gateway.config.OpenAIConfig(*, openai_api_key: str, openai_api_type: mlflow.gateway.config.OpenAIAPIType = OpenAIAPIType.OPENAI, openai_api_base: Optional[str] = None, openai_api_version: Optional[str] = None, openai_deployment_name: Optional[str] = None, openai_organization: Optional[str] = None)[source]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- openai_api_type: mlflow.gateway.config.OpenAIAPIType
- classmethod validate_field_compatibility(info: dict[str, typing.Any])[source]
- classmethod validate_openai_api_key(value)[source]
- class mlflow.gateway.config.PaLMConfig(*, palm_api_key: str)[source]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_palm_api_key(value)[source]
- class mlflow.gateway.config.Route(*, name: str, route_type: str, model: mlflow.gateway.config.RouteModelInfo, route_url: str, limit: Optional[mlflow.gateway.config.Limit] = None)[source]
- class Config[source]
- limit: Optional[mlflow.gateway.config.Limit]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'json_schema_extra': {'example': {'model': {'name': 'gpt-4o-mini', 'provider': 'openai'}, 'name': 'openai-completions', 'route_type': 'llm/v1/completions', 'route_url': '/gateway/routes/completions/invocations'}}}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- to_endpoint()[source]
- class mlflow.gateway.config.RouteConfig(*, name: str, endpoint_type: mlflow.gateway.config.RouteType, model: mlflow.gateway.config.Model, limit: Optional[mlflow.gateway.config.Limit] = None)[source]
- endpoint_type: mlflow.gateway.config.RouteType
- limit: Optional[mlflow.gateway.config.Limit]
- model: mlflow.gateway.config.Model
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore', 'populate_by_name': True, 'validate_by_alias': True, 'validate_by_name': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- to_route() mlflow.gateway.config.Route [source]
- classmethod validate_endpoint_name(route_name)[source]
- classmethod validate_limit(value)[source]
- classmethod validate_model(model)[source]
- classmethod validate_route_type(value)[source]
- classmethod validate_route_type_and_model_name(values)[source]
- class mlflow.gateway.config.RouteModelInfo(*, name: Optional[str] = None, provider: str)[source]
- class mlflow.gateway.config.RouteType(value)[source]
An enumeration.
- class mlflow.gateway.config.TogetherAIConfig(*, togetherai_api_key: str)[source]
- model_config: ClassVar[ConfigDict] = {'extra': 'ignore'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- classmethod validate_togetherai_api_key(value)[source]