Building and Optimizing Intelligent Machine Learning Pipelines with TPOT for Complete Automation and Performance Enhancement
We start this tutorial to show how you can harness TPOT to automate and optimize machine studying pipelines virtually. By working straight in Google Colab, we make sure the setup is light-weight, reproducible, and accessible. We stroll via loading knowledge, defining a customized scorer, tailoring the search area with superior fashions like XGBoost, and organising a cross-validation technique. As we proceed, we discover how evolutionary algorithms in TPOT seek for high-performing pipelines, offering us transparency via Pareto fronts and checkpoints. Take a look at the FULL CODES here.
!pip -q set up tpot==0.12.2 xgboost==2.0.3 scikit-learn==1.4.2 graphviz==0.20.3
import os, json, math, time, random, numpy as np, pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import make_scorer, f1_score, classification_report, confusion_matrix
from sklearn.pipeline import Pipeline
from tpot import TPOTClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
SEED = 7
random.seed(SEED); np.random.seed(SEED); os.environ["PYTHONHASHSEED"]=str(SEED)We start by putting in the libraries and importing all of the important modules that assist knowledge dealing with, mannequin constructing, and pipeline optimization. We set a set random seed to make sure our outcomes stay reproducible each time we run the pocket book. Take a look at the FULL CODES here.
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=SEED)
scaler = StandardScaler().match(X_tr)
X_tr_s, X_te_s = scaler.rework(X_tr), scaler.rework(X_te)
def f1_cost_sensitive(y_true, y_pred):
return f1_score(y_true, y_pred, common='binary', pos_label=1)
cost_f1 = make_scorer(f1_cost_sensitive, greater_is_better=True)Right here, we load the breast most cancers dataset and cut up it into coaching and testing units whereas preserving class stability. We standardize the options for stability after which outline a customized F1-based scorer, permitting us to judge pipelines with a give attention to successfully capturing optimistic circumstances. Take a look at the FULL CODES here.
tpot_config = {
'sklearn.linear_model.LogisticRegression': {
'C': [0.01, 0.1, 1.0, 10.0],
'penalty': ['l2'], 'solver': ['lbfgs'], 'max_iter': [200]
},
'sklearn.naive_bayes.GaussianNB': {},
'sklearn.tree.DecisionTreeClassifier': {
'criterion': ['gini','entropy'], 'max_depth': [3,5,8,None],
'min_samples_split':[2,5,10], 'min_samples_leaf':[1,2,4]
},
'sklearn.ensemble.RandomForestClassifier': {
'n_estimators':[100,300], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.ExtraTreesClassifier': {
'n_estimators':[200], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.GradientBoostingClassifier': {
'n_estimators':[100,200], 'learning_rate':[0.03,0.1],
'max_depth':[2,3], 'subsample':[0.8,1.0]
},
'xgboost.XGBClassifier': {
'n_estimators':[200,400], 'max_depth':[3,5], 'learning_rate':[0.05,0.1],
'subsample':[0.8,1.0], 'colsample_bytree':[0.8,1.0],
'reg_lambda':[1.0,2.0], 'min_child_weight':[1,3],
'n_jobs':[0], 'tree_method':['hist'], 'eval_metric':['logloss'],
'gamma':[0,1]
}
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)We outline a customized TPOT configuration that mixes linear fashions, tree-based learners, ensembles, and XGBoost, using rigorously chosen hyperparameters. We additionally established a stratified 5-fold cross-validation technique, making certain that each candidate pipeline is examined pretty throughout balanced splits of the dataset. Take a look at the FULL CODES here.
t0 = time.time()
tpot = TPOTClassifier(
generations=5,
population_size=40,
offspring_size=40,
scoring=cost_f1,
cv=cv,
subsample=0.8,
n_jobs=-1,
config_dict=tpot_config,
verbosity=2,
random_state=SEED,
max_time_mins=10,
early_stop=3,
periodic_checkpoint_folder="tpot_ckpt",
warm_start=False
)
tpot.match(X_tr_s, y_tr)
print(f"n
First search took {time.time()-t0:.1f}s")
def pareto_table(tpot_obj, ok=5):
rows=[]
for ind, meta in tpot_obj.pareto_front_fitted_pipelines_.gadgets():
rows.append({
"pipeline": ind, "cv_score": meta['internal_cv_score'],
"dimension": len(str(meta['pipeline'])),
})
df = pd.DataFrame(rows).sort_values("cv_score", ascending=False).head(ok)
return df.reset_index(drop=True)
pareto_df = pareto_table(tpot, ok=5)
print("nTop Pareto pipelines (cv):n", pareto_df)
def eval_pipeline(pipeline, X_te, y_te, title):
y_hat = pipeline.predict(X_te)
f1 = f1_score(y_te, y_hat)
print(f"n[{name}] F1(take a look at) = {f1:.4f}")
print(classification_report(y_te, y_hat, digits=3))
print("nEvaluating high pipelines on take a look at:")
for i, (ind, meta) in enumerate(sorted(
tpot.pareto_front_fitted_pipelines_.gadgets(),
key=lambda kv: kv[1]['internal_cv_score'], reverse=True)[:3], 1):
eval_pipeline(meta['pipeline'], X_te_s, y_te, title=f"Pareto#{i}")We launch an evolutionary search with TPOT, cap the runtime for practicality, and checkpoint progress, permitting us to reproducibly hunt for sturdy pipelines. We then examine the Pareto entrance to determine the highest trade-offs, convert it right into a compact desk, and choose leaders primarily based on the cross-validation rating. Lastly, we consider one of the best candidates on the held-out take a look at set to verify real-world efficiency with F1 and a full classification report. Take a look at the FULL CODES here.
print("n
Heat-start for additional refinement...")
t1 = time.time()
tpot2 = TPOTClassifier(
generations=3, population_size=40, offspring_size=40,
scoring=cost_f1, cv=cv, subsample=0.8, n_jobs=-1,
config_dict=tpot_config, verbosity=2, random_state=SEED,
warm_start=True, periodic_checkpoint_folder="tpot_ckpt"
)
strive:
tpot2._population = tpot._population
tpot2._pareto_front = tpot._pareto_front
besides Exception:
move
tpot2.match(X_tr_s, y_tr)
print(f" Heat-start additional search took {time.time()-t1:.1f}s")
best_model = tpot2.fitted_pipeline_ if hasattr(tpot2, "fitted_pipeline_") else tpot.fitted_pipeline_
eval_pipeline(best_model, X_te_s, y_te, title="BestAfterWarmStart")
export_path = "tpot_best_pipeline.py"
(tpot2 if hasattr(tpot2, "fitted_pipeline_") else tpot).export(export_path)
print(f"n
Exported finest pipeline to: {export_path}")
from importlib import util as _util
spec = _util.spec_from_file_location("tpot_best", export_path)
tbest = _util.module_from_spec(spec); spec.loader.exec_module(tbest)
reloaded_clf = tbest.exported_pipeline_
pipe = Pipeline([("scaler", scaler), ("model", reloaded_clf)])
pipe.match(X_tr, y_tr)
eval_pipeline(pipe, X_te, y_te, title="ReloadedExportedPipeline")
report = {
"dataset": "sklearn breast_cancer",
"train_size": int(X_tr.form[0]), "test_size": int(X_te.form[0]),
"cv": "StratifiedKFold(5)",
"scorer": "customized F1 (binary)",
"search": {"gen_1": 5, "gen_2_warm": 3, "pop": 40, "subsample": 0.8},
"exported_pipeline_first_120_chars": str(reloaded_clf)[:120]+"...",
}
print("n
Mannequin Card:n", json.dumps(report, indent=2))We proceed the search with a heat begin, reusing the realized heat begin to refine candidates and choose one of the best performer on our take a look at set. We export the profitable pipeline, reload it alongside our scaler to imitate deployment, and confirm its outcomes. Lastly, we generate a compact mannequin card to doc the dataset, search settings, and the abstract of the exported pipeline for reproducibility.
In conclusion, we see how TPOT permits us to maneuver past trial-and-error mannequin choice and as an alternative depend on automated, reproducible, and explainable optimization. We export one of the best pipeline, validate it on unseen knowledge, and even reload it for deployment-style use, confirming that the workflow is not only experimental however production-ready. By combining reproducibility, flexibility, and interpretability, we finish with a strong framework that we will confidently apply to extra advanced datasets and real-world issues.
Take a look at the FULL CODES here. Be at liberty to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be happy to comply with us on Twitter and don’t overlook to hitch our 100k+ ML SubReddit and Subscribe to our Newsletter.
The submit Building and Optimizing Intelligent Machine Learning Pipelines with TPOT for Complete Automation and Performance Enhancement appeared first on MarkTechPost.
