Package org.mlflow.tracking

Class MlflowClient

java.lang.Object
org.mlflow.tracking.MlflowClient
All Implemented Interfaces:
Closeable, Serializable, AutoCloseable
public class MlflowClient extends Object implements Serializable, Closeable
Client to an MLflow Tracking Sever.
See Also:

  • Field Details

  • Constructor Details

    • MlflowClient

      public MlflowClient()
      Return a default client based on the MLFLOW_TRACKING_URI environment variable.

    • MlflowClient

      public MlflowClient(String trackingUri)
      Instantiate a new client using the provided tracking uri.

    • MlflowClient

      public MlflowClient(MlflowHostCredsProvider hostCredsProvider)
      Create a new MlflowClient; users should prefer constructing ApiClients via MlflowClient() or MlflowClient(String) if possible.

  • Method Details

    • getRun

      public Service.Run getRun(String runId)
      Get metadata, params, tags, and metrics for a run. A single value is returned for each metric key: the most recently logged metric value at the largest step.
      Returns:
      Run associated with the ID.

    • getMetricHistory

      public List<Service.Metric> getMetricHistory(String runId, String key)

    • createRun

      public Service.RunInfo createRun()
      Create a new run under the default experiment with no application name.
      Returns:
      RunInfo created by the server.

    • createRun

      public Service.RunInfo createRun(String experimentId)
      Create a new run under the given experiment.
      Returns:
      RunInfo created by the server.

    • createRun

      public Service.RunInfo createRun(Service.CreateRun request)
      Create a new run. This method allows providing all possible fields of CreateRun, and can be invoked as follows: 
         import org.mlflow.api.proto.Service.CreateRun;
   CreateRun.Builder request = CreateRun.newBuilder();
   request.setExperimentId(experimentId);
   request.setSourceVersion("my-version");
   createRun(request.build());
      Returns:
      RunInfo created by the server.

    • listRunInfos

      public List<Service.RunInfo> listRunInfos(String experimentId)
      Returns:
      A list of all RunInfos associated with the given experiment.

    • searchRuns

      public List<Service.RunInfo> searchRuns(List<String> experimentIds, String searchFilter)
      Deprecated.
      As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
      Return RunInfos from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc = 0.9" If null, the result will be equivalent to having an empty search filter.
      Returns:
      A list of all RunInfos that satisfy search filter.

    • searchRuns

      public List<Service.RunInfo> searchRuns(List<String> experimentIds, String searchFilter, Service.ViewType runViewType)
      Deprecated.
      As of 1.1.0 - please use searchRuns(List, String, ViewType, int) or similar that returns a page of Run results.
      Return RunInfos from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      Returns:
      A list of all RunInfos that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns(List<String> experimentIds, String searchFilter, Service.ViewType runViewType, int maxResults)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      Returns:
      A list of all Runs that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns(List<String> experimentIds, String searchFilter, Service.ViewType runViewType, int maxResults, List<String> orderBy)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      Returns:
      A list of all Runs that satisfy search filter.

    • searchRuns

      public RunsPage searchRuns(List<String> experimentIds, String searchFilter, Service.ViewType runViewType, int maxResults, List<String> orderBy, String pageToken)
      Return runs from provided list of experiments that satisfy the search query.
      Parameters:
      experimentIds - List of experiment IDs.
      searchFilter - SQL compatible search query string. Format of this query string is similar to that specified on MLflow UI. Example : "params.model = 'LogisticRegression' and metrics.acc != 0.9" If null, the result will be equivalent to having an empty search filter.
      runViewType - ViewType for expected runs. One of (ACTIVE_ONLY, DELETED_ONLY, ALL) If null, only runs with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of runs desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      pageToken - String token specifying the next page of results. It should be obtained from a call to searchRuns(List, String).
      Returns:
      A page of Runs that satisfy the search filter.

    • searchExperiments

      public ExperimentsPage searchExperiments(String searchFilter, Service.ViewType experimentViewType, int maxResults, List<String> orderBy)
      Return experiments that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "attribute.name = 'MyExperiment'" - "tags.problem_type = 'iris_regression'" If null, the result will be equivalent to having an empty search filter.
      experimentViewType - ViewType for expected experiments. One of (ACTIVE_ONLY, DELETED_ONLY, ALL). If null, only experiments with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of experiments desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      Returns:
      A page of experiments that satisfy the search filter.

    • searchExperiments

      public ExperimentsPage searchExperiments()
      Return up to 1000 active experiments.
      Returns:
      A page of active experiments with up to 1000 items.

    • searchExperiments

      public ExperimentsPage searchExperiments(String searchFilter)
      Return up to the first 1000 active experiments that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "attribute.name = 'MyExperiment'" - "tags.problem_type = 'iris_regression'" If null, the result will be equivalent to having an empty search filter.
      Returns:
      A page of up to active 1000 experiments that satisfy the search filter.

    • searchExperiments

      public ExperimentsPage searchExperiments(String searchFilter, Service.ViewType experimentViewType, int maxResults, List<String> orderBy, String pageToken)
      Return experiments that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "attribute.name = 'MyExperiment'" - "tags.problem_type = 'iris_regression'" If null, the result will be equivalent to having an empty search filter.
      experimentViewType - ViewType for expected experiments. One of (ACTIVE_ONLY, DELETED_ONLY, ALL). If null, only experiments with viewtype ACTIVE_ONLY will be searched.
      maxResults - Maximum number of experiments desired in one page.
      orderBy - List of properties to order by. Example: "metrics.acc DESC".
      pageToken - String token specifying the next page of results. It should be obtained from a call to searchExperiments(String).
      Returns:
      A page of experiments that satisfy the search filter.

    • getExperiment

      public Service.Experiment getExperiment(String experimentId)
      Returns:
      An experiment with the given ID.

    • getExperimentByName

      public Optional<Service.Experiment> getExperimentByName(String experimentName)
      Returns:
      The experiment associated with the given name or Optional.empty if none exists.

    • createExperiment

      public String createExperiment(String experimentName)
      Create a new experiment using the default artifact location provided by the server.
      Parameters:
      experimentName - Name of the experiment. This must be unique across all experiments.
      Returns:
      Experiment ID of the newly created experiment.

    • createExperiment

      public String createExperiment(Service.CreateExperiment request)
      Create a new experiment. This method allows providing all possible fields of CreateExperiment, and can be invoked as follows: 
         import org.mlflow.api.proto.Service.CreateExperiment;
   CreateExperiment.Builder request = CreateExperiment.newBuilder();
   request.setName(name);
   request.setArtifactLocation(artifactLocation);
   request.addTags(experimentTag);
   createExperiment(request.build());
      Returns:
      ID of the experiment created by the server.

    • deleteExperiment

      public void deleteExperiment(String experimentId)
      Mark an experiment and associated runs, params, metrics, etc. for deletion.

    • restoreExperiment

      public void restoreExperiment(String experimentId)
      Restore an experiment marked for deletion.

    • renameExperiment

      public void renameExperiment(String experimentId, String newName)
      Update an experiment's name. The new name must be unique.

    • deleteRun

      public void deleteRun(String runId)
      Delete a run with the given ID.

    • restoreRun

      public void restoreRun(String runId)
      Restore a deleted run with the given ID.

    • logParam

      public void logParam(String runId, String key, String value)
      Log a parameter against the given run, as a key-value pair. This cannot be called against the same parameter key more than once.

    • logMetric

      public void logMetric(String runId, String key, double value)
      Log a new metric against the given run, as a key-value pair. Metrics are recorded against two axes: timestamp and step. This method uses the number of milliseconds since the Unix epoch for the timestamp, and it uses the default step of zero.
      Parameters:
      runId - The ID of the run in which to record the metric.
      key - The key identifying the metric for which to record the specified value.
      value - The value of the metric.

    • logMetric

      public void logMetric(String runId, String key, double value, long timestamp, long step)
      Log a new metric against the given run, as a key-value pair. Metrics are recorded against two axes: timestamp and step.
      Parameters:
      runId - The ID of the run in which to record the metric.
      key - The key identifying the metric for which to record the specified value.
      value - The value of the metric.
      timestamp - The timestamp at which to record the metric value.
      step - The step at which to record the metric value.

    • setExperimentTag

      public void setExperimentTag(String experimentId, String key, String value)
      Log a new tag against the given experiment as a key-value pair.
      Parameters:
      experimentId - The ID of the experiment on which to set the tag
      key - The key used to identify the tag.
      value - The value of the tag.

    • setTag

      public void setTag(String runId, String key, String value)
      Log a new tag against the given run, as a key-value pair.
      Parameters:
      runId - The ID of the run on which to set the tag
      key - The key used to identify the tag.
      value - The value of the tag.

    • deleteTag

      public void deleteTag(String runId, String key)
      Delete a tag on the run ID with a specific key. This is irreversible.
      Parameters:
      runId - String ID of the run
      key - Name of the tag

    • logBatch

      public void logBatch(String runId, Iterable<Service.Metric> metrics, Iterable<Service.Param> params, Iterable<Service.RunTag> tags)
      Log multiple metrics, params, and/or tags against a given run (argument runId). Argument metrics, params, and tag iterables can be nulls.

    • setTerminated

      public void setTerminated(String runId)
      Set the status of a run to be FINISHED at the current time.

    • setTerminated

      public void setTerminated(String runId, Service.RunStatus status)
      Set the status of a run to be completed at the current time.

    • setTerminated

      public void setTerminated(String runId, Service.RunStatus status, long endTime)
      Set the status of a run to be completed at the given endTime.

    • sendGet

      public String sendGet(String path)
      Send a GET to the following path, including query parameters. This is mostly an internal API, but allows making lower-level or unsupported requests.
      Returns:
      JSON response from the server.

    • sendPost

      public String sendPost(String path, String json)
      Send a POST to the following path, with a String-encoded JSON body. This is mostly an internal API, but allows making lower-level or unsupported requests.
      Returns:
      JSON response from the server.

    • sendPatch

      public String sendPatch(String path, String json)

    • logArtifact

      public void logArtifact(String runId, File localFile)
      Upload the given local file or directory to the run's root artifact directory. For example, 
         logArtifact(runId, "/my/localModel")
   listArtifacts(runId) // returns "localModel"
      Parameters:
      runId - Run ID of an existing MLflow run.
      localFile - File or directory to upload. Must exist.

    • logArtifact

      public void logArtifact(String runId, File localFile, String artifactPath)
      Upload the given local file or directory to an artifactPath within the run's root directory. For example, 
         logArtifact(runId, "/my/localModel", "model")
   listArtifacts(runId, "model") // returns "model/localModel"
      (i.e., the localModel file is now available in model/localModel). If logging a directory, the directory is renamed to artifactPath.
      Parameters:
      runId - Run ID of an existing MLflow run.
      localFile - File or directory to upload. Must exist.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • logArtifacts

      public void logArtifacts(String runId, File localDir)
      Upload all files within the given local directory the run's root artifact directory. For example, if /my/local/dir/ contains two files "file1" and "file2", then 
         logArtifacts(runId, "/my/local/dir")
   listArtifacts(runId) // returns "file1" and "file2"
      Parameters:
      runId - Run ID of an existing MLflow run.
      localDir - Directory to upload. Must exist, and must be a directory (not a simple file).

    • logArtifacts

      public void logArtifacts(String runId, File localDir, String artifactPath)
      Upload all files within the given local director an artifactPath within the run's root artifact directory. For example, if /my/local/dir/ contains two files "file1" and "file2", then 
         logArtifacts(runId, "/my/local/dir", "model")
   listArtifacts(runId, "model") // returns "model/file1" and "model/file2"
      (i.e., the contents of the local directory are now available in model/).
      Parameters:
      runId - Run ID of an existing MLflow run.
      localDir - Directory to upload. Must exist, and must be a directory (not a simple file).
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • listArtifacts

      public List<Service.FileInfo> listArtifacts(String runId)
      List the artifacts immediately under the run's root artifact directory. This does not recursively list; instead, it will return FileInfos with isDir=true where further listing may be done.
      Parameters:
      runId - Run ID of an existing MLflow run.

    • listArtifacts

      public List<Service.FileInfo> listArtifacts(String runId, String artifactPath)
      List the artifacts immediately under the given artifactPath within the run's root artifact directory. This does not recursively list; instead, it will return FileInfos with isDir=true where further listing may be done.
      Parameters:
      runId - Run ID of an existing MLflow run.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • downloadArtifacts

      public File downloadArtifacts(String runId)
      Return a local directory containing *all* artifacts within the run's artifact directory. Note that this will download the entire directory path, and so may be expensive if the directory has a lot of data.
      Parameters:
      runId - Run ID of an existing MLflow run.

    • downloadArtifacts

      public File downloadArtifacts(String runId, String artifactPath)
      Return a local file or directory containing all artifacts within the given artifactPath within the run's root artifactDirectory. For example, if "model/file1" and "model/file2" exist within the artifact directory, then 
         downloadArtifacts(runId, "model") // returns a local directory containing "file1" and "file2"
   downloadArtifacts(runId, "model/file1") // returns a local *file* with the contents of file1.
      Note that this will download the entire subdirectory path, and so may be expensive if the subdirectory has a lot of data.
      Parameters:
      runId - Run ID of an existing MLflow run.
      artifactPath - Artifact path relative to the run's root directory. Should NOT start with a /.

    • getLatestVersions

      public List<ModelRegistry.ModelVersion> getLatestVersions(String modelName)
      Return the latest model version for each stage. The current available stages are: [None, Staging, Production, Archived]. 
              import org.mlflow.api.proto.ModelRegistry.ModelVersion;
        List<ModelVersion> detailsList = getLatestVersions("model");

        for (ModelVersion details : detailsList) {
            System.out.println("Model Name: " + details.getModelVersion()
                                                       .getRegisteredModel()
                                                       .getName());
            System.out.println("Model Version: " + details.getModelVersion().getVersion());
            System.out.println("Current Stage: " + details.getCurrentStage());
        }
      Parameters:
      modelName - The name of the model
      Returns:
      A collection of ModelRegistry.ModelVersion

    • getLatestVersions

      public List<ModelRegistry.ModelVersion> getLatestVersions(String modelName, Iterable<String> stages)
      Return the latest model version for each stage requested. The current available stages are: [None, Staging, Production, Archived]. 
              import org.mlflow.api.proto.ModelRegistry.ModelVersion;
        List<ModelVersion> detailsList =
          getLatestVersions("model", Lists.newArrayList<String>("Staging"));

        for (ModelVersion details : detailsList) {
            System.out.println("Model Name: " + details.getModelVersion()
                                                       .getRegisteredModel()
                                                       .getName());
            System.out.println("Model Version: " + details.getModelVersion().getVersion());
            System.out.println("Current Stage: " + details.getCurrentStage());
        }
      Parameters:
      modelName - The name of the model
      stages - A list of stages
      Returns:
      The latest model version ModelRegistry.ModelVersion

    • getModelVersion

      public ModelRegistry.ModelVersion getModelVersion(String modelName, String version)
             import org.mlflow.api.proto.ModelRegistry.ModelVersion;
       ModelVersion modelVersion = getModelVersion("model", "version");
      Parameters:
      modelName - Name of the containing registered model. *
      version - Version number as a string of the model version.
      Returns:
      a single model version ModelRegistry.ModelVersion

    • getRegisteredModel

      public ModelRegistry.RegisteredModel getRegisteredModel(String modelName)
      Returns a RegisteredModel from the model registry for the given model name. 
             import org.mlflow.api.proto.ModelRegistry.RegisteredModel;
       RegisteredModel registeredModel = getRegisteredModel("model");
      Parameters:
      modelName - Name of the containing registered model. *
      Returns:
      a registered model ModelRegistry.RegisteredModel

    • getModelVersionDownloadUri

      public String getModelVersionDownloadUri(String modelName, String version)
      Return the model URI containing for the given model version. The model URI can be used to download the model version artifacts. 
              String modelUri = getModelVersionDownloadUri("model", 0);
      Parameters:
      modelName - The name of the model
      version - The version number of the model
      Returns:
      The specified model version's URI.

    • downloadModelVersion

      public File downloadModelVersion(String modelName, String version)
      Returns a directory containing all artifacts within the given registered model version. The method will download the model version artifacts to the local file system. Note that this method will not work if the `download_uri` refers to a single file (and not a directory) due to the way many ArtifactRepository's `download_artifacts` handle empty subpaths. 
              File modelVersionDir = downloadModelVersion("model", 0);
      Parameters:
      modelName - The name of the model
      version - The version number of the model
      Returns:
      A directory (File) containing model artifacts

    • downloadLatestModelVersion

      public File downloadLatestModelVersion(String modelName, String stage)
      Returns a directory containing all artifacts within the latest registered model version in the given stage. The method will download the model version artifacts to the local file system. 
              File modelVersionDir = downloadLatestModelVersion("model", "Staging");
      (i.e., the contents of the local directory are now available).
      Parameters:
      modelName - The name of the model
      stage - The name of the stage
      Returns:
      A directory (File) containing model artifacts

    • searchModelVersions

      public ModelVersionsPage searchModelVersions(String searchFilter, int maxResults, List<String> orderBy)
      Return model versions that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "name = 'model_name'" - "run_id = '...'" If null, the result will be equivalent to having an empty search filter.
      maxResults - Maximum number of model versions desired in one page.
      orderBy - List of properties to order by. Example: "name DESC".
      Returns:
      A page of model versions that satisfy the search filter.

    • searchModelVersions

      public ModelVersionsPage searchModelVersions()
      Return up to 1000 model versions.
      Returns:
      A page of model versions with up to 1000 items.

    • searchModelVersions

      public ModelVersionsPage searchModelVersions(String searchFilter)
      Return up to 1000 model versions that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "name = 'model_name'" - "run_id = '...'" If null, the result will be equivalent to having an empty search filter.
      Returns:
      A page of model versions with up to 1000 items.

    • searchModelVersions

      public ModelVersionsPage searchModelVersions(String searchFilter, int maxResults, List<String> orderBy, String pageToken)
      Return model versions that satisfy the search query.
      Parameters:
      searchFilter - SQL compatible search query string. Examples: - "name = 'model_name'" - "run_id = '...'" If null, the result will be equivalent to having an empty search filter.
      maxResults - Maximum number of model versions desired in one page.
      orderBy - List of properties to order by. Example: "name DESC".
      pageToken - String token specifying the next page of results. It should be obtained from a call to searchModelVersions(String).
      Returns:
      A page of model versions that satisfy the search filter.

    • close

      public void close()
      Closes the MlflowClient and releases any associated resources.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable