Advanced Machine Learning: Complete Visual Guide

💻 3.9 Hands-On: GridSearchCV in Python

Pass your model, the dictionary of hyperparameters to test, and let scikit-learn do the heavy lifting of testing every combination.

from sklearn.model_selection import GridSearchCV
from sklearn.ensemble import RandomForestClassifier

# 1. Define the baseline model
rf_model = RandomForestClassifier(random_state=1)

# 2. Define the Hyperparameter Grid (the combinations to try)
param_grid = {
    'n_estimators': [50, 100, 150],    # Number of trees
    'max_depth': [3, 5, 7]             # Depth of trees
} # 3 options × 3 options = 9 total combinations to test

# 3. Setup GridSearchCV using 5-fold CV internally
grid_search = GridSearchCV(estimator=rf_model, param_grid=param_grid, cv=5, scoring='accuracy')

# 4. Fit it to the data (this runs 9 combinations × 5 folds = 45 models!)
grid_search.fit(X_train, y_train)

# 5. Extract the winner
print("Best Parameters Selected:", grid_search.best_params_)
print("Best CV Accuracy Score:", grid_search.best_score_)

# The grid_search object can now naturally predict using the best model!
y_pred = grid_search.predict(X_test)

4. The Bias-Variance Tradeoff (The Dartboard)

Tuning hyperparameters is simply a balancing act between Bias (Underfitting) and Variance (Overfitting).

3. The "Last Mile": API Deployment Framework

A Web Application (like Netflix or a Banking App) cannot execute code inside your Jupyter Notebook. You must package your model and put it behind a lightweight web server (FastAPI) so the App can request predictions over the internet.

# --- STEP 1: In your Jupyter Notebook (Save the Model) ---
import joblib
# Save your finished, trained Scikit-Learn pipeline to a physical file
joblib.dump(secure_model_pipeline, 'fraud_model.pkl')


# --- STEP 2: In a new file called app.py (The Web Server) ---
from fastapi import FastAPI
import joblib

app = FastAPI()

# Load the serialized model back into the server's memory
model = joblib.load('fraud_model.pkl')

@app.post("/predict")
def predict_fraud(transaction_amount: float, user_age: int):
    # 1. Take incoming JSON data from the mobile/web app
    incoming_data = [[transaction_amount, user_age]]
    
    # 2. Run it through the loaded model pipeline
    prediction = model.predict(incoming_data)
    
    # 3. Send the verdict back to the web application across the internet!
    return {"is_fraud": bool(prediction[0])}

import optuna
from imblearn.pipeline import Pipeline # Using imblearn.pipeline ensures SMOTE runs ONLY on training folds
from imblearn.over_sampling import SMOTE
from xgboost import XGBClassifier
from sklearn.model_selection import cross_val_score, StratifiedKFold

def objective(trial):
    # 1. Optuna intelligently queries the hyperparameter space
    params = {
        'n_estimators': trial.suggest_int('n_estimators', 100, 1000),
        'learning_rate': trial.suggest_float('learning_rate', 1e-3, 0.3, log=True),
        'max_depth': trial.suggest_int('max_depth', 3, 9)
    }
    
    # 2. Build the Leak-Proof Architecture using an Imblearn Pipeline
    # The pipeline guarantees SMOTE is NEVER applied to validation sets during cross-validation!
    pipeline = Pipeline([
        ('balancer', SMOTE(random_state=42)),
        ('classifier', XGBClassifier(**params, eval_metric="logloss"))
    ])
    
    # 3. Use Stratified 5-Fold Cross Validation to protect imbalanced class distributions
    cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
    
    # 4. Evaluate using Recall or F1 (Never just Accuracy for Imbalanced Data!)
    score = cross_val_score(pipeline, X_train, y_train, cv=cv, scoring='f1').mean()
    return score

# 5. Let Optuna hunt down the global maximum metric across 50 intelligent trials
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=50)
print("Absolute Best Production Parameters:", study.best_params)