• GenAI applications and agents: LLM-powered apps, chatbots, RAG pipelines, tool-calling agents. Key MLflow features include tracing for observability, evaluation with LLM judges, and prompt management, among others.
• Traditional ML / deep learning models: scikit-learn, PyTorch, TensorFlow, XGBoost, etc. Key MLflow features include experiment tracking (parameters, metrics, artifacts), model logging, and model deployment, among others.

• Imports from LLM client libraries:

  openai

  ,

  anthropic

  ,

  google.generativeai

  ,

  langchain

  ,

  langchain_openai

  ,

  langgraph

  ,

  llamaindex

  ,

  litellm

  ,

  autogen

  ,

  crewai

  ,

  dspy

• Imports from MLflow GenAI modules:

  mlflow.genai

  ,

  mlflow.tracing

  ,

  mlflow.openai

  ,

  mlflow.langchain

• Usage of chat completions, embeddings, or agent frameworks
• Prompt templates or prompt engineering code

• Imports from ML frameworks:

  sklearn

  ,

  torch

  ,

  tensorflow

  ,

  keras

  ,

  xgboost

  ,

  lightgbm

  ,

  catboost

  ,

  statsmodels

  ,

  scipy

• Imports from MLflow ML modules:

  mlflow.sklearn

  ,

  mlflow.pytorch

  ,

  mlflow.tensorflow

• Model training loops,

  .fit()

  calls, hyperparameter tuning code
• Dataset loading with tabular/image/time-series data

undefined

mlflow.experimentKind

mlflow experiments get --experiment-id <EXPERIMENT_ID> --output json > /tmp/exp_detail.json
jq -r '.tags["mlflow.experimentKind"] // "not set"' /tmp/exp_detail.json

• genai_development

  → GenAI use case

• custom_model_development

  → Traditional ML use case
• Not set → Proceed to Signal 3

Are you building a GenAI application (e.g., an LLM-powered chatbot, RAG pipeline, or tool-calling agent) or a traditional ML/deep learning model (e.g., training a classifier, regression model, or neural network)?

• Tracing Quickstart (https://mlflow.org/docs/latest/genai/tracing/quickstart/) — Enabling automatic tracing for LLM calls. Covers starting an MLflow server, creating an experiment, enabling autologging, and viewing traces in the UI.
  • Python + OpenAI variant: https://mlflow.org/docs/latest/genai/tracing/quickstart/python-openai/
  • TypeScript + OpenAI variant: https://mlflow.org/docs/latest/genai/tracing/quickstart/typescript-openai
  • OpenTelemetry (language-agnostic) variant: also linked from the quickstart page
• Evaluation Quickstart (https://mlflow.org/docs/latest/genai/eval-monitor/quickstart/) — Evaluating GenAI application quality using LLM judges (scorers). Covers defining datasets, prediction functions, and built-in + custom scorers.
• Version Tracking Quickstart (https://mlflow.org/docs/latest/genai/version-tracking/quickstart/) — Prompt management, application versioning, and connecting tracing to versioned prompts.

• Tracking Quickstart (https://mlflow.org/docs/latest/ml/tracking/quickstart/) — Experiment tracking with scikit-learn: autologging, manual parameter/metric/model logging, and exploring results in the MLflow UI.
• Deep Learning Tutorial (https://mlflow.org/docs/latest/ml/getting-started/deep-learning/) — Training a PyTorch model with MLflow logging: parameters, metrics, checkpoints, and system metrics (GPU utilization, memory).
• Hyperparameter Tuning Tutorial (https://mlflow.org/docs/latest/ml/getting-started/hyperparameter-tuning/) — Running hyperparameter searches with Optuna + MLflow, comparing results, and selecting the best model.

@mlflow.trace

mlflow.start_span()

import mlflow

mlflow.set_experiment("example-genai-app")

@mlflow.trace
def mock_chat(query: str) -> str:
    with mlflow.start_span(name="retrieve_context") as span:
        context = "Mock retrieved context for: " + query
        span.set_inputs({"query": query})
        span.set_outputs({"context": context})
    with mlflow.start_span(name="generate_response") as span:
        response = "Mock response based on: " + context
        span.set_inputs({"context": context, "query": query})
        span.set_outputs({"response": response})
    return response

mock_chat("What is MLflow?")

1. Autologging — If the user's code uses a supported framework, a single line automatically traces all calls to their LLM provider. See https://mlflow.org/docs/latest/genai/tracing/ for the full list of supported providers. If the provider is supported:
   python

   import mlflow
   
   # Pick the one that matches the user's LLM provider:
   mlflow.openai.autolog()       # OpenAI SDK
   mlflow.anthropic.autolog()    # Anthropic SDK
   mlflow.langchain.autolog()    # LangChain / LangGraph
   mlflow.litellm.autolog()      # LiteLLM

   Add this call once at application startup (e.g., top of

   main.py

   ,

   app.py

   , or the entry point module). It must execute before any LLM calls are made.
   If the provider is not supported by autologging, skip to step 3 (Custom tracing) and use

   @mlflow.trace

   to manually instrument the relevant functions.
2. Experiment configuration — Set the experiment so traces are organized:
   python

   mlflow.set_experiment("my-genai-app")

   Or via environment variable:

   export MLFLOW_EXPERIMENT_NAME="my-genai-app"

3. Custom tracing (optional) — For functions that aren't automatically traced (custom tools, business logic), use the

   @mlflow.trace

   decorator:
   python

   @mlflow.trace
   def my_custom_tool(query: str) -> str:
       # ... tool logic ...
       return result

openai.OpenAI()

ChatOpenAI()

1. Autologging — If the user's code uses a supported framework, a single line automatically logs parameters, metrics, and models during training. See https://mlflow.org/docs/latest/ml/ for the full list of supported frameworks. If the framework is supported:
   python

   import mlflow
   
   # Pick the one that matches the user's ML framework:
   mlflow.sklearn.autolog()      # scikit-learn
   mlflow.pytorch.autolog()      # PyTorch / PyTorch Lightning
   mlflow.tensorflow.autolog()   # TensorFlow / Keras
   mlflow.xgboost.autolog()      # XGBoost
   mlflow.lightgbm.autolog()     # LightGBM

   Add this call once before training starts. It automatically captures

   model.fit()

   calls, logged metrics, and model artifacts.
   If the framework is not supported by autologging, skip to step 3 (Manual logging) and use

   mlflow.log_param()

   ,

   mlflow.log_metric()

   , and

   mlflow.log_artifact()

   to log data explicitly.
2. Experiment configuration — Set the experiment so runs are organized:
   python

   mlflow.set_experiment("my-ml-experiment")

   Or via environment variable:

   export MLFLOW_EXPERIMENT_NAME="my-ml-experiment"

3. Manual logging (optional) — For metrics or parameters not captured by autologging:
   python

   with mlflow.start_run():
       mlflow.log_param("custom_param", value)
       mlflow.log_metric("custom_metric", value)

model.fit()

1. Run the application and trigger at least one LLM call
2. Check for traces:
   bash

   mlflow traces search \
     --experiment-id <EXPERIMENT_ID> \
     --max-results 5 \
     --extract-fields 'info.trace_id,info.state,info.request_time' \
     --output json > /tmp/verify_traces.json
   jq '.traces | length' /tmp/verify_traces.json

3. If traces appear, open the MLflow UI to inspect them visually

1. Run the training script
2. Check for runs:
   bash

   mlflow runs search \
     --experiment-id <EXPERIMENT_ID> \
     --max-results 5 \
     --output json > /tmp/verify_runs.json
   jq '.runs | length' /tmp/verify_runs.json

3. If runs appear, open the MLflow UI to inspect logged parameters, metrics, and artifacts