Scikit-learn

Building a Custom Estimator with Scikit-learn

Scikit-learn allows users to define their own custom estimators by creating Python classes that implement the standard interface. This is especially helpful when you want to encapsulate custom preprocessing, transformation, or model behavior and integrate it into Scikit-learn’s Pipeline and model selection tools.

Key Characteristics

  • Fully compatible with Pipeline, GridSearchCV, and cross-validation tools
  • Requires implementation of fit() and optionally transform() or predict()
  • Useful for custom preprocessing or model behavior
  • Can also support get_params() and set_params() for hyperparameter tuning

Basic Rules

  • Must inherit from BaseEstimator and TransformerMixin (or define similar interface)
  • Always implement fit() method
  • Implement transform() if used in data preprocessing
  • Implement predict() if building a custom classifier or regressor
  • Define class-level parameters using __init__

Syntax Table

SL NO Technique Syntax Example Description
1 Import Base Classes from sklearn.base import BaseEstimator, TransformerMixin Required for defining custom estimators
2 Create Class class MyTransformer(BaseEstimator, TransformerMixin) Start of custom class definition
3 Constructor def __init__(self, param=default): Defines parameters with defaults
4 Fit Method def fit(self, X, y=None): return self Learns internal structure, returns self
5 Transform or Predict def transform(self, X): or def predict(self, X): Converts or classifies data

Syntax Explanation

1. Import Base Classes

What is it?
Imports Scikit-learn’s base classes that define standard estimator interfaces.

Syntax:

from sklearn.base import BaseEstimator, TransformerMixin

Explanation:

  • BaseEstimator gives you access to Scikit-learn features like parameter inspection and cloning.
  • TransformerMixin provides the .fit_transform() utility based on your fit() and transform() methods.
  • These base classes ensure full compatibility with Scikit-learn pipelines and utilities.

2. Create Class

What is it?
Defines the custom estimator or transformer by extending Scikit-learn base classes.

Syntax:

class MyTransformer(BaseEstimator, TransformerMixin):
    pass

Explanation:

  • Class should inherit from both BaseEstimator and TransformerMixin to behave like native transformers.
  • Enables easy integration with Pipeline, GridSearchCV, and cloning.
  • Avoids boilerplate by inheriting useful methods like get_params() and set_params().

3. Constructor (__init__)

What is it?
Initializes class with configurable hyperparameters.

Syntax:

def __init__(self, multiplier=1):
    self.multiplier = multiplier

Explanation:

  • All parameters must be explicitly listed in __init__() without logic.
  • Enables Scikit-learn to inspect and tune parameters via get_params().
  • Store parameters as instance attributes to use in later methods.
  • Avoid performing computation or validations in __init__().

4. Fit Method

What is it?
Trains the estimator or prepares it by learning internal statistics.

Syntax:

def fit(self, X, y=None):
    return self

Explanation:

  • Must accept X and optionally y. Always return self.
  • This method can learn statistics (mean, std, min, max, etc.) needed for later transformation or prediction.
  • No transformation is applied here—just model fitting.
  • Required for both estimators and transformers in pipelines.

5. Transform or Predict

What is it?
Executes the main functionality—either transforming input data or predicting outcomes.

Syntax (transform):

def transform(self, X):
    return X * self.multiplier

Syntax (predict):

def predict(self, X):
    return X > 0.5

Explanation:

  • transform() is for feature engineering, data scaling, encoding, etc.
  • predict() is used in classifiers or regressors to return predictions.
  • Output must match input dimensions (for transform) or be label-compatible (for predict).
  • Can include logic based on hyperparameters passed during __init__().
  • Should handle both NumPy arrays and Pandas DataFrames if possible.

Real-Life Project: Custom Feature Multiplier Transformer

Project Overview

Multiply all features by a given constant using a reusable transformer class.

Code Example

import numpy as np
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

# Custom Transformer
class FeatureMultiplier(BaseEstimator, TransformerMixin):
    def __init__(self, multiplier=1):
        self.multiplier = multiplier

    def fit(self, X, y=None):
        return self

    def transform(self, X):
        return X * self.multiplier

# Sample usage
X = np.array([[1, 2], [3, 4]])
y = [0, 1]
pipeline = Pipeline([
    ('multiply', FeatureMultiplier(multiplier=10)),
    ('clf', LogisticRegression())
])
pipeline.fit(X, y)

Expected Output

  • Pipeline multiplies features by 10 before classification
  • Logistic regression learns on modified features

Common Mistakes to Avoid

  • ❌ Forgetting to return self in fit()
  • ❌ Not listing parameters explicitly in __init__()
  • ❌ Performing logic or validations in __init__()
  • ❌ Returning transformed data in fit() instead of transform()

Further Reading Recommendation

📘 Hands-On Python and Scikit-Learn: A Practical Guide to Machine Learning by Sarful Hassan

🔗 Available on Amazon