MLflow Tracking & Experiments
Databricks · Machine Learning & AI
Theory
MLflow is the open-source ML lifecycle tool built into Databricks; current workspaces use MLflow 3. Tracking records each training run: parameters (such as max_depth), metrics (such as accuracy or RMSE), tags and artifacts like plots, plus the model itself as a logged model. Runs are grouped into experiments, which you browse under Experiments in the sidebar to sort, filter and compare runs side by side. mlflow.autolog() captures most of this automatically for libraries like scikit-learn, XGBoost and Spark ML. MLflow 3 also adds GenAI features: tracing of LLM and agent calls and evaluation with judges and scorers.
Example: Asha trains three churn models with different max_depth values on main.sales data. In the experiment view she sorts by val_auc, sees max_depth=6 scored 0.87, and picks that run to register.
Why we use it
ML work involves many trial runs; without tracking, nobody remembers which data, code and settings produced the best model. MLflow keeps a searchable, reproducible record and connects runs to registered models and serving.
When to use it
Use MLflow tracking for every model training, from notebooks or jobs. Use autolog for quick coverage and explicit log_param/log_metric for custom values. For GenAI apps and agents, use MLflow Tracing and evaluation rather than classic metrics alone.
How it works
- mlflow.set_experiment(...) chooses where runs go
- Start a run and train the model
- Log params, metrics and the model
- More settings to try? Yes: Start another run with new params. No: Compare runs in the Experiments UI.
- Pick the best run and register its model in Unity Catalog
Syntax
import mlflow
mlflow.set_experiment("/Users/<you>/<experiment>")
mlflow.autolog()
with mlflow.start_run(run_name="..."):
mlflow.log_param("k", v); mlflow.log_metric("m", x)
mlflow.sklearn.log_model(model, name="model")
mlflow.search_runs(order_by=["metrics.val_auc DESC"])
UI: Experiments → <experiment> → Runs table → select runs → CompareExample
import mlflow
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score
mlflow.set_experiment("/Users/asha@techtown.in/churn")
df = spark.table("main.sales.customer_features").toPandas() # small feature table
X, y = df.drop(columns=["customer_id", "churned"]), df["churned"]
for depth in [4, 6, 8]:
with mlflow.start_run(run_name=f"rf_depth_{depth}"):
model = RandomForestClassifier(max_depth=depth).fit(X, y)
auc = roc_auc_score(y, model.predict_proba(X)[:, 1])
mlflow.log_param("max_depth", depth)
mlflow.log_metric("val_auc", auc)
mlflow.sklearn.log_model(model, name="model", input_example=X.head(5))
best = mlflow.search_runs(order_by=["metrics.val_auc DESC"]).iloc[0]Key points
- An experiment groups runs; each run logs params, metrics, tags and artifacts
- mlflow.autolog() logs most of this automatically for popular libraries
- MLflow 3 uses name= instead of artifact_path= when logging a model
- Compare runs in the Experiments UI or with mlflow.search_runs()
- MLflow Tracing records each step of LLM and agent calls
- Use a proper validation split; the example scores training data only for brevity