gators.feature_selection package#

Submodules#

gators.feature_selection.information_value module#

gators.feature_selection.information_value.compute_iv(X, y, regularization=0.01)[source]#

Compute the Information Value (IV) for each categorical feature in the dataset.

To convert continuous features to categorical, consider using the binning module to create bins before computing IV.

Parameters:

  • X (pl.DataFrame) – The input features.

  • y (pl.Series) – The target variable (binary).

  • regularization (float, default=0.01) – Regularization parameter to avoid division by zero in WOE/IV calculation.

Returns:

A DataFrame containing the IV values for each feature.

Return type:

pl.DataFrame

Examples

>>> import polars as pl
>>> from gators.feature_selection import compute_iv

>>> X = pl.DataFrame({
...     "feature1": ["a", "a", "b", "c"],
...     "feature2": ["x", "x", "x", "y"],
...     "target": [1, 0, 1, 0]
... })
>>> iv = compute_iv(X.drop("target"), X["target"])
>>> print(iv)
shape: (2, 2)
┌──────────┬────────────┐
│ feature  │ iv         │
│ ---      │ ---        │
│ str      │ f64        │
╞══════════╪════════════╡
│ feature1 │ 0.693147   │
│ feature2 │ 0.287682   │
└──────────┴────────────┘

gators.feature_selection.feature_stability_index module#

gators.feature_selection.feature_stability_index.feature_stability_index(estimator, skf, X: polars.DataFrame, y: polars.Series, importance_threshold: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])] = 0.0, k: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1)])] = 5)[source]#

Compute Feature Stability Index (FSI) using repeated estimator feature importance.

Measures how consistently a feature is selected across different training folds. Higher FSI indicates more stable/reliable feature importance.

Parameters:

  • estimator (estimator object) – Any estimator with a feature_importances_ attribute (e.g., XGBoost, RandomForest).

  • skf (sklearn fold splitter object) – Any sklearn fold splitter object (e.g., StratifiedKFold, KFold) for splitting the data.

  • X (pl.DataFrame) – Feature DataFrame with shape (n_samples, n_features).

  • y (pl.Series) – Target series for training.

  • importance_threshold (Annotated[float, Field(ge=0.0, le=1.0)], default=0.0) – Minimum importance value for a feature to be considered “selected” in a run. Must be between 0.0 and 1.0.

  • k (Annotated[int, Field(ge=1)], default=100) – Number of selected features. Must be at least 1.

Returns:

DataFrame with columns:

  • feature: Feature name

  • fsi: Feature Stability Index (0 to 1, higher is more stable)

  • importance: Average importance across all runs

Sorted by FSI and importance in descending order, filtered to fsi > 0.

Return type:

pl.DataFrame

Module contents#

gators.feature_selection.compute_iv(X, y, regularization=0.01)[source]

Compute the Information Value (IV) for each categorical feature in the dataset.

To convert continuous features to categorical, consider using the binning module to create bins before computing IV.

Parameters:

  • X (pl.DataFrame) – The input features.

  • y (pl.Series) – The target variable (binary).

  • regularization (float, default=0.01) – Regularization parameter to avoid division by zero in WOE/IV calculation.

Returns:

A DataFrame containing the IV values for each feature.

Return type:

pl.DataFrame

Examples

>>> import polars as pl
>>> from gators.feature_selection import compute_iv

>>> X = pl.DataFrame({
...     "feature1": ["a", "a", "b", "c"],
...     "feature2": ["x", "x", "x", "y"],
...     "target": [1, 0, 1, 0]
... })
>>> iv = compute_iv(X.drop("target"), X["target"])
>>> print(iv)
shape: (2, 2)
┌──────────┬────────────┐
│ feature  │ iv         │
│ ---      │ ---        │
│ str      │ f64        │
╞══════════╪════════════╡
│ feature1 │ 0.693147   │
│ feature2 │ 0.287682   │
└──────────┴────────────┘
class gators.feature_selection.CorrelationSelector[source]

Bases: gators.feature_selection._base_selector._BaseSelector

Drop redundant features by removing highly correlated ones, keeping the most important.

For every pair of numeric features (restricted to those present in importance) whose Pearson correlation exceeds max_corr, the feature with the lower importance score is discarded. The greedy pass iterates over pairs in column order: once a feature is marked for removal it is skipped for all subsequent comparisons.

Only numeric columns (non-String, non-Boolean, non-Categorical, non-Enum) that appear in importance are candidates for removal. All other columns are kept untouched.

Parameters:

  • importance (dict[str, float]) – Mapping of feature name to importance score. Only columns listed here are considered for correlation filtering; all other columns are passed through unchanged.

  • max_corr (float, default=0.95) – Correlation threshold above which a pair is considered redundant. Must be in the range (0, 1].

  • use_abs (bool, default=True) – When True the absolute value of the correlation is compared against max_corr, so strong negative correlations (e.g. −0.97) are treated the same as strong positive ones. When False only correlations that are strictly greater than max_corr (positive) trigger removal.

selected_features_

Column names that survive the filter (set after fit).

Type:

list[str]

columns_to_drop_

Column names removed because a more important correlated feature exists (set after fit).

Type:

list[str]

Examples

>>> import polars as pl
>>> from gators.feature_selection import CorrelationSelector

>>> X = pl.DataFrame({
...     "a": [1.0, 2.0, 3.0, 4.0, 5.0],
...     "b": [1.1, 2.1, 3.1, 4.1, 5.1],   # nearly identical to "a"
...     "c": [5.0, 3.0, 1.0, 4.0, 2.0],   # independent
... })
>>> importance = {"a": 0.9, "b": 0.4, "c": 0.7}

Example 1: Default threshold (0.95)

>>> selector = CorrelationSelector(importance=importance)
>>> selector.fit(X)
CorrelationSelector(importance={'a': 0.9, 'b': 0.4, 'c': 0.7}, max_corr=0.95, use_abs=True)
>>> selector.columns_to_drop_
['b']
>>> selector.transform(X).columns
['a', 'c']

Example 2: Stricter threshold keeps both correlated features

>>> selector2 = CorrelationSelector(importance=importance, max_corr=0.999)
>>> selector2.fit(X)
CorrelationSelector(importance={'a': 0.9, 'b': 0.4, 'c': 0.7}, max_corr=0.999, use_abs=True)
>>> selector2.columns_to_drop_
[]

Example 3: use_abs=False ignores negative correlations

>>> X_neg = pl.DataFrame({
...     "a": [1.0, 2.0, 3.0, 4.0, 5.0],
...     "b": [-1.0, -2.0, -3.0, -4.0, -5.0],  # perfectly negatively correlated
...     "c": [5.0, 3.0, 1.0, 4.0, 2.0],
... })
>>> importance_neg = {"a": 0.9, "b": 0.4, "c": 0.7}
>>> selector3 = CorrelationSelector(importance=importance_neg, max_corr=0.95, use_abs=False)
>>> selector3.fit(X_neg)
CorrelationSelector(importance={'a': 0.9, 'b': 0.4, 'c': 0.7}, max_corr=0.95, use_abs=False)
>>> selector3.columns_to_drop_   # negative corr not filtered when use_abs=False
[]
fit(X: polars.DataFrame, y: polars.Series | None = None) → gators.feature_selection.correlation_selector.CorrelationSelector[source]

Compute pairwise Pearson correlations and record which columns to drop.

Parameters:

  • X (pl.DataFrame) – Input DataFrame.

  • y (pl.Series, default=None) – Not used; present for sklearn API compatibility.

Returns:

The fitted transformer instance.

Return type:

CorrelationSelector

gators.feature_selection.feature_stability_index(estimator, skf, X: polars.DataFrame, y: polars.Series, importance_threshold: Annotated[float, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=0.0), Le(le=1.0)])] = 0.0, k: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1)])] = 5)[source]

Compute Feature Stability Index (FSI) using repeated estimator feature importance.

Measures how consistently a feature is selected across different training folds. Higher FSI indicates more stable/reliable feature importance.

Parameters:

  • estimator (estimator object) – Any estimator with a feature_importances_ attribute (e.g., XGBoost, RandomForest).

  • skf (sklearn fold splitter object) – Any sklearn fold splitter object (e.g., StratifiedKFold, KFold) for splitting the data.

  • X (pl.DataFrame) – Feature DataFrame with shape (n_samples, n_features).

  • y (pl.Series) – Target series for training.

  • importance_threshold (Annotated[float, Field(ge=0.0, le=1.0)], default=0.0) – Minimum importance value for a feature to be considered “selected” in a run. Must be between 0.0 and 1.0.

  • k (Annotated[int, Field(ge=1)], default=100) – Number of selected features. Must be at least 1.

Returns:

DataFrame with columns:

  • feature: Feature name

  • fsi: Feature Stability Index (0 to 1, higher is more stable)

  • importance: Average importance across all runs

Sorted by FSI and importance in descending order, filtered to fsi > 0.

Return type:

pl.DataFrame

gators.feature_selection.select_k_best_stable_features(estimator, skf, X: polars.DataFrame, y: polars.Series, k: Annotated[int, FieldInfo(annotation=NoneType, required=True, metadata=[Ge(ge=1)])] = 200)[source]

Select features that consistently appear in the top-k across all folds.

For each fold, fits the estimator on the training split and selects the top-k features by importance. Returns the intersection — features that ranked in the top-k in every fold.

Parameters:

  • estimator (estimator object) – Any estimator with a feature_importances_ attribute (e.g., XGBoost, RandomForest).

  • skf (sklearn fold splitter object) – Any sklearn fold splitter object (e.g., StratifiedKFold, KFold) for splitting the data.

  • X (pl.DataFrame) – Feature DataFrame with shape (n_samples, n_features).

  • y (pl.Series) – Target series for training.

  • k (Annotated[int, Field(ge=1)], default=200) – Number of top features to consider per fold. Must be at least 1.

Returns:

DataFrame with columns:

  • feature: Feature name

  • importance: Average feature importance across all folds

Contains only features that appeared in the top-k in every fold, sorted by importance in descending order.

Return type:

pl.DataFrame

class gators.feature_selection.FeatureStabilitySelector[source]

Bases: gators.feature_selection._base_selector._BaseSelector

Drop columns whose Feature Stability Index falls below a threshold.

Wraps feature_stability_index() into a fit/transform interface. The FSI measures how consistently a feature is selected across cross-validation folds; features with low FSI are considered unstable and are dropped.

Note

Sklearn estimators require NumPy arrays. The fit method calls .to_numpy() internally — this is unavoidable and correct.

Parameters:

  • estimator (estimator object) – Any fitted estimator with a feature_importances_ attribute (e.g., RandomForestClassifier, XGBClassifier).

  • skf (sklearn splitter object) – Any sklearn cross-validation splitter (e.g., StratifiedKFold).

  • threshold (float, default=0.5) – Minimum FSI required to keep a column. Columns with FSI strictly below this value are dropped.

  • importance_threshold (float, default=0.0) – Minimum per-fold importance for a feature to count as “selected” in that fold.

selected_features_

Column names that survive the FSI threshold.

Type:

list[str]

columns_to_drop_

Column names dropped due to low FSI.

Type:

list[str]

fsi_scores_

Full FSI DataFrame (feature, fsi, importance) computed during fit.

Type:

pl.DataFrame

Examples

>>> import polars as pl
>>> from sklearn.ensemble import RandomForestClassifier
>>> from sklearn.model_selection import StratifiedKFold
>>> from gators.feature_selection import FeatureStabilitySelector

>>> X = pl.DataFrame({
...     "stable":   [i % 2 for i in range(100)],
...     "unstable": [i % 7 for i in range(100)],
... })
>>> y = pl.Series("target", [i % 2 for i in range(100)])
>>> estimator = RandomForestClassifier(n_estimators=10, random_state=0)
>>> skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
>>> selector = FeatureStabilitySelector(estimator=estimator, skf=skf, threshold=0.5)
>>> selector.fit(X, y)
>>> X_transformed = selector.transform(X)
fit(X: polars.DataFrame, y: polars.Series | None = None) → gators.feature_selection.feature_stability_selector.FeatureStabilitySelector[source]

Compute FSI for each column and record which to drop.

Parameters:

  • X (pl.DataFrame) – Input DataFrame.

  • y (pl.Series) – Target series for training the estimator.

Returns:

The fitted transformer instance.

Return type:

FeatureStabilitySelector

class gators.feature_selection.InformationValueSelector[source]

Bases: gators.feature_selection._base_selector._BaseSelector

Drop columns whose Information Value falls below a threshold.

Computes the IV for each categorical (String/Categorical/Enum) column and drops those whose IV is below threshold. When a discretizer is provided, numeric columns are first binned by the discretizer before IV computation; the discretized values are used only for scoring and are not applied during transform — numeric columns that survive the threshold remain numeric in the output. Without a discretizer, numeric columns are excluded from IV computation and are always kept.

Parameters:

  • threshold (float, default=0.02) – Minimum IV required to keep a column. Columns with IV strictly below this value are dropped.

  • regularization (float, default=0.01) – Regularization applied to WOE/IV calculation to avoid division by zero.

  • discretizer (_BaseDiscretizer or None, default=None) – Optional discretizer used to bin numeric columns before computing IV. When None, numeric columns are excluded from IV computation and always kept (backward-compatible behaviour).

selected_features_

All column names that survive the threshold (set after fit).

Type:

list[str]

columns_to_drop_

Column names dropped because their IV was too low (set after fit).

Type:

list[str]

iv_values_

Mapping of feature name to its computed IV value (set after fit).

Type:

dict[str, float]

Examples

>>> import polars as pl
>>> from gators.feature_selection import InformationValueSelector

>>> X = pl.DataFrame({
...     "cat_strong": ["a", "b", "a", "b", "a", "b"],
...     "cat_weak":   ["x", "x", "x", "x", "y", "y"],
...     "numeric":    [1.0, 2.0, 3.0, 4.0, 5.0, 6.0],
... })
>>> y = pl.Series("target", [1, 0, 1, 0, 1, 0])
>>> selector = InformationValueSelector(threshold=0.02)
>>> selector.fit(X, y)
>>> X_transformed = selector.transform(X)

With a discretizer to score numeric columns:

>>> from gators.discretizers import EqualSizeDiscretizer
>>> selector = InformationValueSelector(
...     threshold=0.02,
...     discretizer=EqualSizeDiscretizer(num_bins=7),
... )
>>> selector.fit(X, y)
>>> X_transformed = selector.transform(X)  # numeric columns remain numeric
fit(X: polars.DataFrame, y: polars.Series | None = None) → gators.feature_selection.information_value_selector.InformationValueSelector[source]

Compute IV for each applicable column and record which to drop.

Categorical columns are always scored. Numeric columns are scored only when a discretizer is provided; the discretizer is fitted and applied to X internally and its output is passed to compute_iv.

Parameters:

  • X (pl.DataFrame) – Input DataFrame.

  • y (pl.Series) – Binary target series.

Returns:

The fitted transformer instance.

Return type:

InformationValueSelector

class gators.feature_selection.PermutationImportanceSelector[source]

Bases: gators.feature_selection._base_selector._BaseSelector

Drop columns whose permutation importance falls below a threshold.

Fits estimator on the training data, then measures how much the model score degrades when each feature is randomly shuffled (permuted). Features whose mean importance across n_repeats permutations is below threshold are dropped.

Note

Sklearn estimators require NumPy arrays. The fit method calls .to_numpy() internally — this is unavoidable and correct.

Parameters:

  • estimator (estimator object) – A fitted or unfitted sklearn-compatible estimator. Must implement fit and score.

  • n_repeats (int, default=5) – Number of times to permute each feature.

  • threshold (float, default=0.0) – Minimum mean importance drop required to keep a feature. Features with mean permutation importance strictly below this value are dropped. A value of 0.0 keeps all features that contribute at least marginally.

selected_features_

Column names that survive the importance threshold.

Type:

list[str]

columns_to_drop_

Column names dropped due to low permutation importance.

Type:

list[str]

importances_

Mean permutation importance for each input feature.

Type:

dict[str, float]

Examples

>>> import polars as pl
>>> from sklearn.ensemble import RandomForestClassifier
>>> from gators.feature_selection import PermutationImportanceSelector

>>> X = pl.DataFrame({
...     "informative": [i % 2 for i in range(100)],
...     "noise":       [0] * 100,
... })
>>> y = pl.Series("target", [i % 2 for i in range(100)])
>>> estimator = RandomForestClassifier(n_estimators=10, random_state=0)
>>> selector = PermutationImportanceSelector(
...     estimator=estimator, n_repeats=5, threshold=0.0
... )
>>> selector.fit(X, y)
>>> X_transformed = selector.transform(X)
fit(X: polars.DataFrame, y: polars.Series | None = None) → gators.feature_selection.permutation_importance_selector.PermutationImportanceSelector[source]

Fit estimator and compute permutation importances.

Parameters:

  • X (pl.DataFrame) – Input DataFrame.

  • y (pl.Series) – Target series.

Returns:

The fitted transformer instance.

Return type:

PermutationImportanceSelector

class gators.feature_selection.PSIFilter[source]

Bases: gators.transformer._base_transformer._BaseTransformer

Drop columns whose Population Stability Index exceeds a threshold.

PSI quantifies how much a feature’s distribution has shifted between a reference dataset (typically training data) and the current dataset. High PSI signals distributional drift; such features are unreliable at inference time and are dropped.

PSI interpretation:

  • PSI < 0.10 — stable, no significant change

  • 0.10 ≤ PSI < 0.25 — moderate shift, investigate

  • PSI ≥ 0.25 — significant shift, feature is unstable

Only numeric (Float64, Float32, Int64, Int32) columns are evaluated for PSI. Non-numeric columns are always kept.

Parameters:

  • reference_df (pl.DataFrame) – Reference DataFrame whose distributions define the baseline.

  • threshold (float, default=0.2) – Maximum PSI allowed. Columns with PSI strictly above this value are dropped.

  • n_bins (int, default=10) – Number of quantile-based bins used when computing PSI.

  • subset (list[str] or None, default=None) – Numeric columns to evaluate. If None, all numeric columns shared between reference_df and the DataFrame passed to fit are used.

psi_scores_

PSI score for each evaluated column (set after fit).

Type:

dict[str, float]

columns_to_drop_

Columns dropped because their PSI exceeded the threshold.

Type:

list[str]

selected_features_

Columns kept after filtering.

Type:

list[str]

Examples

>>> import polars as pl
>>> from gators.feature_selection import PSIFilter

>>> reference = pl.DataFrame({
...     "stable": [float(i % 10) for i in range(100)],
...     "drifted": [float(i) for i in range(100)],
... })
>>> current = pl.DataFrame({
...     "stable":  [float(i % 10) for i in range(100)],
...     "drifted": [float(i + 200) for i in range(100)],
... })
>>> selector = PSIFilter(reference_df=reference, threshold=0.2)
>>> selector.fit(current)
>>> X_transformed = selector.transform(current)
fit(X: polars.DataFrame, y: polars.Series | None = None) → gators.feature_selection.psi_filter.PSIFilter[source]

Compute PSI for each numeric column against the reference DataFrame.

Parameters:

  • X (pl.DataFrame) – Current DataFrame to compare against reference_df.

  • y (pl.Series, default=None) – Not used; present for sklearn compatibility.

Returns:

The fitted transformer instance.

Return type:

PSIFilter

transform(X: polars.DataFrame) → polars.DataFrame[source]

Drop high-PSI columns from the DataFrame.

Parameters:

X (pl.DataFrame) – Input DataFrame to transform.

Returns:

DataFrame with high-PSI columns removed.

Return type:

pl.DataFrame