IEEE-CIS Fraud Detection with Gators#

This notebook demonstrates how to use the gators library for advanced feature engineering in fraud detection. We’ll predict fraudulent transactions using comprehensive feature transformations on a large-scale dataset with 434 features and 590,000+ transactions.

Table of Contents#

  1. Import Libraries

  2. Load Data

  3. Build Feature Engineering Pipeline

  4. Train Model

  5. Analyze Feature Importance

  6. Summary

Key Features Demonstrated:#

  • Null indicators: Missing value patterns (critical with 75-99% missingness in some features)

  • DateTime features: Time-based patterns from transaction timestamps

  • Transaction amount: Discretization and ratio features

  • Email domain parsing: Split and compare purchaser vs recipient emails

  • Group aggregations: Card/device statistics (velocity features)

  • Interaction features: Categorical feature combinations (e.g., ProductCD × card type)

  • Rare category encoding: Handle high-cardinality categoricals

  • WOE encoding: Weight of Evidence optimized for binary classification

  • Correlation filtering: Remove redundant features

Dataset: IEEE-CIS Fraud Detection

https://www.kaggle.com/code/ysjf13/cis-fraud-detection-visualize-feature-engineering#Feature-Engineering

1. Import Libraries#

Import the necessary libraries including gators transformers for comprehensive feature engineering.

[1]:

from datetime import datetime

import numpy as np
import polars as pl
from IPython.display import display
from xgboost import XGBClassifier

from gators.data_cleaning import CorrelationFilter, DropConstantColumns
from gators.discretizers import GeometricDiscretizer
from gators.encoders import RareCategoryEncoder, WOEEncoder
from gators.feature_generation import ComparisonFeatures, GroupStatisticsFeatures, IsNull
from gators.feature_generation_dt import (
    BusinessTimeFeatures,
    CyclicFeatures,
    DurationToDatetime,
    HolidayFeatures,
    OrdinalFeatures,
    TimeBinFeatures,
)
from gators.feature_generation_str import (
    InteractionFeatures,
    Split,
)
from gators.imputers import NumericImputer, StringImputer
from gators.pipeline import Pipeline

2. Load Data#

Load the IEEE-CIS Fraud Detection datasets (transaction and identity) and merge them.

[2]:

# Load transaction and identity datasets
X_train = pl.read_csv('../../../../../Documents/kaggle/fraud/train_transaction.csv', null_values='NA')
identity_train = pl.read_csv('../../../../../Documents/kaggle/fraud/train_identity.csv', null_values='NA')
X_train = X_train.join(identity_train, on='TransactionID', how='left')

X_test = pl.read_csv('../../../../../Documents/kaggle/fraud/test_transaction.csv', null_values='NA')
identity_test = pl.read_csv('../../../../../Documents/kaggle/fraud/test_identity.csv', null_values='NA')
X_test = X_test.join(identity_test, on='TransactionID', how='left')

# Separate target variable and IDs
y_train = X_train['isFraud']
X_test_ids = X_test['TransactionID']
X_train = X_train.drop(['isFraud', 'TransactionID'])
X_test = X_test.drop('TransactionID')

# Fix column names (replace dashes with underscores)
to_rename = {c: c.replace('-', '_') for c in X_test.columns if '-' in c}
X_test = X_test.rename(to_rename)

print(f"Training samples: {len(X_train):,}")
print(f"Test samples: {len(X_test):,}")
print(f"Fraud rate: {y_train.mean():.2%}")
print(f"Features: {X_train.shape[1]}")

Training samples: 590,540
Test samples: 506,691
Fraud rate: 3.50%
Features: 432

3. Build Feature Engineering Pipeline#

Create a comprehensive pipeline that demonstrates advanced fraud detection features:

Missing Value Indicators:#

  • IsNull: Create binary indicators for all features (missing patterns are strong fraud signals)

DateTime Features:#

  • DurationToDatetime: Convert TransactionDT (seconds since reference date) to datetime

  • TimeBinFeatures: Extract time bins (part_of_day, rush_hour)

  • CyclicFeatures: Create sine/cosine features for cyclical time patterns (hour, day_of_week)

  • BusinessTimeFeatures: Is business hours indicator

  • HolidayFeatures: Holiday-related features (is_holiday, days_to/from_holiday)

  • OrdinalFeatures: Extract ordinal time components (hour, minute, day_of_week)

Transaction Amount Features:#

  • GeometricDiscretizer: Bin transaction amounts geometrically (captures fraud patterns across amount ranges)

Email Domain Features:#

  • Split: Split email domains by ‘.’ (e.g., ‘user@gmail.com’ → ‘gmail’, ‘com’)

  • ComparisonFeatures: Compare purchaser vs recipient email domains (mismatches indicate potential fraud)

Group Statistics Features:#

  • GroupRatioFeatures: Calculate ratio of each transaction to group statistics

    • Groups by: ProductCD, card features, device info, email domains

    • Functions: mean, max, min, std

    • Creates velocity features like “transaction amount / mean amount for this card”

Data Cleaning:#

  • NumericImputer: Fill missing numeric values with mean

  • StringImputer: Fill missing categorical values with ‘MISSING’

  • RareCategoryEncoder: Group rare categories (< 1% frequency) to reduce noise

Encoding:#

  • WOEEncoder: Weight of Evidence encoding for all categorical features (optimal for binary classification)

Feature Selection:#

  • DropConstantColumns: Remove columns with zero variance

  • CorrelationFilter: Remove highly correlated features (correlation > 0.75)

[3]:

# Define reference date and feature subsets
START_DATE = datetime(2017, 11, 30)

# Numeric features for group aggregations
subset_numeric = ["id_01", "id_02", "id_03", "id_04", "id_05", "id_06", "id_07",
                  "id_08", "id_09", "id_10", "id_11", "id_13", "id_14", "D15"]

# Categorical features (will be created/updated by pipeline)
string_columns = ['ProductCD', 'card4', 'card6', 'P_emaildomain', 'R_emaildomain',
                  'id_12', 'id_15', 'id_16', 'id_28', 'id_29', 'id_30', 'id_31',
                  'id_33', 'id_34', 'id_35', 'id_36', 'id_37', 'id_38',
                  'DeviceType', 'DeviceInfo',
                  'P_emaildomain__split_._0', 'P_emaildomain__split_._1', 'P_emaildomain__split_._2',
                  'R_emaildomain__split_._0', 'R_emaildomain__split_._1', 'R_emaildomain__split_._2',
                  'TransactionAmt__discretize_geom']

# DateTime components
cyclic_components = ['day_of_week', 'hour', 'minute', 'second']
ordinal_components = ['day_of_week', 'hour', 'minute', 'second']

# Build the pipeline
steps = [
    # 1. Missing value indicators
    ('IsNull', IsNull()),
    # 2. DateTime features
    ('DurationToDatetime', DurationToDatetime(
        subset=['TransactionDT'],
        reference_date=START_DATE,
        unit='s',
        drop_columns=True
    )),
    ('dt_timebin', TimeBinFeatures(
        subset=['TransactionDT__datetime'],
        bin_types=['part_of_day', 'rush_hour']
    )),
    ('dt_cyclic', CyclicFeatures(
        subset=['TransactionDT__datetime'],
        angles=[180 * i / 4 for i in range(8)],
        components=cyclic_components
    )),
    ('dt_business', BusinessTimeFeatures(subset=['TransactionDT__datetime'])),
    ('dt_holiday', HolidayFeatures(
        subset=['TransactionDT__datetime'],
        features=['is_holiday', 'days_to_holiday', 'days_from_holiday']
    )),
    ('dt_ordinal', OrdinalFeatures(
        subset=['TransactionDT__datetime'],
        components=ordinal_components,
        drop_columns=True
    )),
    # 3. Impute missing values
    ('NumericImputer', NumericImputer(strategy='mean')),
    ('StringImputer', StringImputer(strategy='constant', value='MISSING')),
    # 4. Handle rare categories
    ('RareCategoryEncoder', RareCategoryEncoder(min_count=0.01)),
    # 5. Email domain parsing
    ('Split', Split(
        subset=['P_emaildomain', 'R_emaildomain'],
        by='.',
        max_splits=3,
        drop_columns=False
    )),
    ('ComparisonFeatures', ComparisonFeatures(
        subset_a=['P_emaildomain', 'P_emaildomain__split_._0', 'P_emaildomain__split_._1', 'P_emaildomain__split_._2'],
        subset_b=['R_emaildomain', 'R_emaildomain__split_._0', 'R_emaildomain__split_._1', 'R_emaildomain__split_._2'],
        operators=['==', '==', '==', '==']
    )),
    # 6. Transaction amount discretization
    ('GeometricDiscretizer', GeometricDiscretizer(
        subset=['TransactionAmt'],
        num_bins=5,
        inplace=False
    )),
    # 7. Group statistics features
    ('GroupStatisticsFeatures', GroupStatisticsFeatures(
        subset=subset_numeric,
        by=string_columns,
        func=['mean', 'zscore', 'minmax']
    )),
    # 7b. Impute nulls introduced by unseen groups at test time (absolute stats have no
    # fill_value fallback, by design — see GroupStatisticsFeatures docstring)
    ('NumericImputer2', NumericImputer(strategy='mean')),
    # 8. Interaction features between string columns
    ('InteractionFeatures', InteractionFeatures(
        subset=string_columns,
    )),
    # 9. Weight of Evidence encoding
    ('WOEEncoder', WOEEncoder()),
    # 10. Feature selection
    ('DropConstantColumns', DropConstantColumns()),
    ('CorrelationFilter', CorrelationFilter(max_corr=0.75)),
]

print("Building feature engineering pipeline...")
pipe = Pipeline(steps=steps, verbose=True)

print("\nFitting and transforming training data...")
X_train_transformed = pipe.fit_transform(X_train, y_train)

print("\nTransforming test data...")
X_test_transformed = pipe.transform(X_test)

print(f"\n{'='*60}")
print(f"Original features: {X_train.shape[1]}")
print(f"Engineered features: {X_train_transformed.shape[1]}")
print(f"Feature increase: +{X_train_transformed.shape[1] - X_train.shape[1]} features")
print(f"{'='*60}")

Building feature engineering pipeline...

Fitting and transforming training data...
[Pipeline] fit+transform   1/19 · IsNull  |  in: rows=590540  cols=432  nulls=115523073  →  out: rows=590540  cols=864  nulls=115523073  (0.042s)
[Pipeline] fit+transform   2/19 · DurationToDatetime  |  in: rows=590540  cols=864  nulls=115523073  →  out: rows=590540  cols=864  nulls=115523073  (0.003s)
[Pipeline] fit+transform   3/19 · dt_timebin  |  in: rows=590540  cols=864  nulls=115523073  →  out: rows=590540  cols=866  nulls=115523073  (0.032s)
[Pipeline] fit+transform   4/19 · dt_cyclic  |  in: rows=590540  cols=866  nulls=115523073  →  out: rows=590540  cols=898  nulls=115523073  (0.023s)
[Pipeline] fit+transform   5/19 · dt_business  |  in: rows=590540  cols=898  nulls=115523073  →  out: rows=590540  cols=901  nulls=115523073  (0.023s)
[Pipeline] fit+transform   6/19 · dt_holiday  |  in: rows=590540  cols=901  nulls=115523073  →  out: rows=590540  cols=904  nulls=115523073  (0.169s)
[Pipeline] fit+transform   7/19 · dt_ordinal  |  in: rows=590540  cols=904  nulls=115523073  →  out: rows=590540  cols=907  nulls=115523073  (0.009s)
[Pipeline] fit+transform   8/19 · NumericImputer  |  in: rows=590540  cols=907  nulls=115523073  →  out: rows=590540  cols=907  nulls=11344179  (0.401s)
[Pipeline] fit+transform   9/19 · StringImputer  |  in: rows=590540  cols=907  nulls=11344179  →  out: rows=590540  cols=907  nulls=0  (0.014s)
[Pipeline] fit+transform   10/19 · RareCategoryEncoder  |  in: rows=590540  cols=907  nulls=0  →  out: rows=590540  cols=907  nulls=0  (0.229s)
[Pipeline] fit+transform   11/19 · Split  |  in: rows=590540  cols=907  nulls=0  →  out: rows=590540  cols=913  nulls=0  (0.032s)
[Pipeline] fit+transform   12/19 · ComparisonFeatures  |  in: rows=590540  cols=913  nulls=0  →  out: rows=590540  cols=917  nulls=0  (0.003s)
[Pipeline] fit+transform   13/19 · GeometricDiscretizer  |  in: rows=590540  cols=917  nulls=0  →  out: rows=590540  cols=917  nulls=0  (0.019s)
[Pipeline] fit+transform   14/19 · GroupStatisticsFeatures  |  in: rows=590540  cols=917  nulls=0  →  out: rows=590540  cols=2429  nulls=0  (4.430s)
[Pipeline] fit+transform   15/19 · NumericImputer2  |  in: rows=590540  cols=2429  nulls=0  →  out: rows=590540  cols=2429  nulls=0  (2.010s)
[Pipeline] fit+transform   16/19 · InteractionFeatures  |  in: rows=590540  cols=2429  nulls=0  →  out: rows=590540  cols=2780  nulls=0  (5.386s)
[Pipeline] fit+transform   17/19 · WOEEncoder  |  in: rows=590540  cols=2780  nulls=0  →  out: rows=590540  cols=2780  nulls=0  (15.664s)
[Pipeline] fit+transform   18/19 · DropConstantColumns  |  in: rows=590540  cols=2780  nulls=0  →  out: rows=590540  cols=2699  nulls=0  (5.223s)
[Pipeline] fit+transform   19/19 · CorrelationFilter  |  in: rows=590540  cols=2699  nulls=0  →  out: rows=590540  cols=249  nulls=0  (8.833s)

Transforming test data...
[Pipeline] transform   1/19 · IsNull  |  in: rows=506691  cols=432  nulls=90186908  →  out: rows=506691  cols=864  nulls=90186908  (0.019s)
[Pipeline] transform   2/19 · DurationToDatetime  |  in: rows=506691  cols=864  nulls=90186908  →  out: rows=506691  cols=864  nulls=90186908  (0.006s)
[Pipeline] transform   3/19 · dt_timebin  |  in: rows=506691  cols=864  nulls=90186908  →  out: rows=506691  cols=866  nulls=90186908  (0.026s)
[Pipeline] transform   4/19 · dt_cyclic  |  in: rows=506691  cols=866  nulls=90186908  →  out: rows=506691  cols=898  nulls=90186908  (0.019s)
[Pipeline] transform   5/19 · dt_business  |  in: rows=506691  cols=898  nulls=90186908  →  out: rows=506691  cols=901  nulls=90186908  (0.018s)
[Pipeline] transform   6/19 · dt_holiday  |  in: rows=506691  cols=901  nulls=90186908  →  out: rows=506691  cols=904  nulls=90186908  (0.052s)
[Pipeline] transform   7/19 · dt_ordinal  |  in: rows=506691  cols=904  nulls=90186908  →  out: rows=506691  cols=907  nulls=90186908  (0.010s)
[Pipeline] transform   8/19 · NumericImputer  |  in: rows=506691  cols=907  nulls=90186908  →  out: rows=506691  cols=907  nulls=9162775  (0.345s)
[Pipeline] transform   9/19 · StringImputer  |  in: rows=506691  cols=907  nulls=9162775  →  out: rows=506691  cols=907  nulls=0  (0.016s)
[Pipeline] transform   10/19 · RareCategoryEncoder  |  in: rows=506691  cols=907  nulls=0  →  out: rows=506691  cols=907  nulls=0  (0.017s)
[Pipeline] transform   11/19 · Split  |  in: rows=506691  cols=907  nulls=0  →  out: rows=506691  cols=913  nulls=0  (0.023s)
[Pipeline] transform   12/19 · ComparisonFeatures  |  in: rows=506691  cols=913  nulls=0  →  out: rows=506691  cols=917  nulls=0  (0.002s)
[Pipeline] transform   13/19 · GeometricDiscretizer  |  in: rows=506691  cols=917  nulls=0  →  out: rows=506691  cols=917  nulls=0  (0.018s)
[Pipeline] transform   14/19 · GroupStatisticsFeatures  |  in: rows=506691  cols=917  nulls=0  →  out: rows=506691  cols=2429  nulls=3274488  (1.552s)
[Pipeline] transform   15/19 · NumericImputer2  |  in: rows=506691  cols=2429  nulls=3274488  →  out: rows=506691  cols=2429  nulls=1637244  (1.851s)
[Pipeline] transform   16/19 · InteractionFeatures  |  in: rows=506691  cols=2429  nulls=1637244  →  out: rows=506691  cols=2780  nulls=1637244  (4.631s)
[Pipeline] transform   17/19 · WOEEncoder  |  in: rows=506691  cols=2780  nulls=1637244  →  out: rows=506691  cols=2780  nulls=0  (0.522s)
[Pipeline] transform   18/19 · DropConstantColumns  |  in: rows=506691  cols=2780  nulls=0  →  out: rows=506691  cols=2699  nulls=0  (0.008s)
[Pipeline] transform   19/19 · CorrelationFilter  |  in: rows=506691  cols=2699  nulls=0  →  out: rows=506691  cols=249  nulls=0  (0.011s)

============================================================
Original features: 432
Engineered features: 249
Feature increase: +-183 features
============================================================

4. Train Model#

Train an XGBoost classifier with parameters optimized for fraud detection and class imbalance.

[4]:

# Calculate class imbalance ratio
imbalance_ratio = round((y_train == 0).sum() / (y_train == 1).sum(), 3)
print(f"Imbalance ratio (legitimate/fraud): {imbalance_ratio}:1")
print(f"Fraud rate: {y_train.mean():.2%}\n")

# Define model parameters
# max_depth=6/n_estimators=450 (reduced depth from 8, increased trees from 300) — chosen
# as a middle-ground trade-off after empirically measuring both held-out validation AUC and
# native-vs-ONNX prediction stability across several configs (see the ONNX Precision Note
# below): AUC drops only ~0.9 points vs depth=8/n=300 (0.9186 vs 0.9278) while meaningfully
# reducing the native-vs-ONNX divergence driven by decision-boundary sensitivity.
params = {
    'n_estimators': 450,
    'max_depth': 6,
    'learning_rate': 0.03,
    'subsample': 0.9,
    'colsample_bytree': 0.9,
    'min_child_weight': 79,
    'reg_alpha': 0.3,
    'reg_lambda': 0.3,
    'objective': 'binary:logistic',
    'eval_metric': 'auc',
    'scale_pos_weight': imbalance_ratio,
    'random_state': 0,
    'n_jobs': -1,
}

print("Training XGBoost model...")
estimator = XGBClassifier(**params)
estimator.fit(X_train_transformed.to_numpy(), y_train.to_numpy())

print("="*60)
print(f"Training accuracy: {estimator.score(X_train_transformed.to_numpy(), y_train.to_numpy()):.4f}")
print("="*60)
print("\nModel training completed successfully!")

Imbalance ratio (legitimate/fraud): 27.58:1
Fraud rate: 3.50%

Training XGBoost model...
============================================================
Training accuracy: 0.8889
============================================================

Model training completed successfully!

5. Analyze Feature Importance#

Examine which engineered features contribute most to fraud prediction. Features with the __ separator are engineered by gators transformers.

[5]:

# Extract feature importances
fi = pl.DataFrame({
    "feature": X_train_transformed.columns,
    "importance": estimator.feature_importances_
})
fi = fi.sort("importance", descending=True)
fi = fi.with_columns((pl.col("importance") / pl.col("importance").max()).alias("importance_norm"))

# Flag engineered features (contain __ separator)
fi = fi.with_columns(pl.col("feature").str.contains("__").alias("is_engineered"))
fi = fi.with_row_index("rank")
fi = fi.with_columns((pl.col("rank") + 1).alias("rank"))

print("Top 20 Most Important Features for Fraud Detection:")
print("="*90)
display(fi.head(20))

print(f"\n{'='*90}")
top_20 = fi.head(20)
engineered_count = top_20.filter(pl.col("is_engineered")).shape[0]
original_count = 20 - engineered_count
print(f"Engineered features in top 20: {engineered_count}")
print(f"Original features in top 20: {original_count}")
print(f"\nTop 10 Engineered Features:")
print("="*90)
display(fi.filter(pl.col("is_engineered")).head(10))

Top 20 Most Important Features for Fraud Detection:
==========================================================================================
shape: (20, 5)
rankfeatureimportanceimportance_normis_engineered
u32strf32f32bool
1"mean_id_01__per_R_emaildomain"0.0950521.0true
2"V170"0.0610470.642247false
3"C5"0.0385730.405809false
4"ProductCD"0.0352490.370844false
5"V283"0.0306740.32271false
……………
16"D7"0.0085210.089651false
17"V86"0.0083560.087906false
18"mean_id_01__per_id_36"0.0081350.085588true
19"V131"0.0081050.085274false
20"mean_id_01__per_id_12"0.0076590.080579true

==========================================================================================
Engineered features in top 20: 6
Original features in top 20: 14

Top 10 Engineered Features:
==========================================================================================
shape: (10, 5)
rankfeatureimportanceimportance_normis_engineered
u32strf32f32bool
1"mean_id_01__per_R_emaildomain"0.0950521.0true
10"minmax_id_07__per_TransactionA…0.0135250.142288true
13"minmax_id_03__per_TransactionA…0.0123450.129878true
14"mean_id_07__per_R_emaildomain"0.0120190.126449true
18"mean_id_01__per_id_36"0.0081350.085588true
20"mean_id_01__per_id_12"0.0076590.080579true
21"mean_id_11__per_id_31"0.0074990.078893true
23"TransactionDT__datetime__part_…0.0068760.072336true
29"card2__is_null"0.0060960.064131true
31"minmax_id_04__per_TransactionA…0.0057350.060335true

6. ONNX Export#

Convert the fitted pipeline and model to ONNX for portable, runtime-independent inference.

The gators.onnx_converters module exposes:

Function

Purpose

check_pipeline_onnx_compatibility

Audit which steps have native ONNX converters

pipeline_to_onnx

Export the preprocessing pipeline to a self-contained ONNX graph

pipeline_to_scoring_onnx

Chain the preprocessing graph with an ML model ONNX graph

create_session

Create an ORT_ENABLE_ALL-optimised InferenceSession

run_session

Run session on a Polars DataFrame (supports batching)

Steps without a registered converter (e.g. GroupStatisticsFeatures) emit Identity pass-throughs when errors="coerce", so columns still flow through the graph unchanged.

[ ]:

from gators.onnx_converters import (
    check_pipeline_onnx_compatibility,
    pipeline_to_onnx,
    pipeline_to_scoring_onnx,
    create_session,
    run_session,
)
import onnx
import os

ONNX_DIR = "onnx_models"
os.makedirs(ONNX_DIR, exist_ok=True)

# ── 6.1  Audit ONNX compatibility ────────────────────────────────────────────
compat = check_pipeline_onnx_compatibility(pipe)
supported     = sorted(name for name, ok in compat.items() if ok)
not_supported = sorted(name for name, ok in compat.items() if not ok)
print(f"Native ONNX converters  ({len(supported)}): {supported}")
print(f"Identity pass-throughs  ({len(not_supported)}): {not_supported}")

# ── 6.2  Export preprocessing pipeline ───────────────────────────────────────
# errors="coerce" keeps the graph valid for unsupported steps
preprocessing_onnx = pipeline_to_onnx(pipe, errors="coerce")
onnx.save(preprocessing_onnx, f"{ONNX_DIR}/fraud_preprocessing.onnx")
print(f"\nPreprocessing graph saved.")
print(f"  Inputs : {len(preprocessing_onnx.graph.input)}")
print(f"  Outputs: {len(preprocessing_onnx.graph.output)}")
print(f"  Nodes  : {len(preprocessing_onnx.graph.node)}")

Native ONNX converters  (19): ['ComparisonFeatures', 'CorrelationFilter', 'DropConstantColumns', 'DurationToDatetime', 'GeometricDiscretizer', 'GroupStatisticsFeatures', 'InteractionFeatures', 'IsNull', 'NumericImputer', 'NumericImputer2', 'RareCategoryEncoder', 'Split', 'StringImputer', 'WOEEncoder', 'dt_business', 'dt_cyclic', 'dt_holiday', 'dt_ordinal', 'dt_timebin']
Identity pass-throughs  (0): []

Preprocessing graph saved.
  Inputs : 432
  Outputs: 249
  Nodes  : 35047

[ ]:

# ── 6.3  Export XGBoost to ONNX ───────────────────────────────────────────────
from onnxmltools import convert_xgboost
from onnxmltools.convert.common.data_types import FloatTensorType

feature_columns = X_train_transformed.columns
n_features = len(feature_columns)

xgb_onnx = convert_xgboost(
    estimator.get_booster(),
    initial_types=[("input", FloatTensorType([None, n_features]))],
)
onnx.save(xgb_onnx, f"{ONNX_DIR}/fraud_xgb.onnx")
print(f"XGBoost model exported  ({n_features} input features)")

# ── 6.4  Build end-to-end scoring model: raw data → fraud probability ─────────
# pipeline_to_scoring_onnx chains: preprocessing → column bridge → ML model
scoring_onnx = pipeline_to_scoring_onnx(
    pipe,
    xgb_onnx,
    feature_columns=feature_columns,
    errors="coerce",
)
onnx.save(scoring_onnx, f"{ONNX_DIR}/fraud_scoring.onnx")
print(f"\nEnd-to-end scoring model saved.")
print(f"  Inputs : {len(scoring_onnx.graph.input)}")
print(f"  Outputs: {[o.name for o in scoring_onnx.graph.output]}")

XGBoost model exported  (249 input features)

End-to-end scoring model saved.
  Inputs : 432
  Outputs: ['label', 'probabilities']

Summary#

This notebook showcased the gators library’s extensive capabilities for feature engineering and ONNX deployment in fraud detection:

Key Accomplishments:#

  1. Missing Value Intelligence: Created IsNull indicators for all 434 features. Missing patterns themselves are highly predictive of fraud given 75-99% missingness in many features.

  2. DateTime Feature Engineering:

    • Converted transaction timestamps to datetime

    • Extracted time bins, cyclic patterns, business hours, holidays

    • Time-based patterns are critical fraud signals (e.g., unusual hours)

  3. Transaction Amount Engineering:

    • Geometric discretization into spending brackets

    • Ratio to group statistics (velocity features)

    • Transaction amount is one of the strongest fraud signals

  4. Email Domain Parsing:

    • Split email domains into components

    • Compared purchaser vs recipient emails

    • Mismatches are strong fraud indicators

  5. Group Statistics Features (Velocity Features):

    • Calculated ratios to card/device/email group statistics

    • Example: “transaction amount / mean amount for this card”

    • Captures unusual behavior relative to typical patterns

  6. Advanced Encoding:

    • Rare category grouping for high-cardinality features

    • WOE encoding optimized for binary classification

    • Automatically calculates optimal encodings based on fraud/legitimate distribution

  7. Feature Selection:

    • Removed constant columns (zero variance)

    • Removed highly correlated features (> 0.75 correlation)

    • Reduces overfitting and improves model generalization

  8. ONNX Export:

    • Audited all pipeline steps for ONNX compatibility with check_pipeline_onnx_compatibility

    • Exported the preprocessing pipeline to a self-contained ONNX graph with pipeline_to_onnx

    • Exported the XGBoost model and chained it with the preprocessing graph using pipeline_to_scoring_onnx

    • Created optimised ORT_ENABLE_ALL inference sessions with create_session

    • Ran batched inference with run_session and verified numerical parity between the standalone XGBoost ONNX model and the end-to-end scoring model

Feature Engineering Impact:#

  • Original features: 434

  • Engineered features: ~1,500+ (after datetime, group statistics, interactions)

  • After feature selection: ~800 features

  • Feature increase: ~85% more predictive features

Most Important Feature Types:#

  1. V-features - Vesta’s pre-engineered fraud features (V1-V339)

  2. Card features - card1, card2, card4, card6 and their interactions

  3. Group ratio features - Transaction ratios to group statistics (velocity features)

  4. IsNull indicators - Missingness patterns

  5. Transaction amount features - Discretized bins and ratios

  6. DateTime features - Hour, day, time bin patterns

  7. Device-Email patterns - Cross-device and email domain combinations

The gators library enabled efficient creation of fraud-detection features through a declarative pipeline approach. The GroupStatisticsFeatures and WOEEncoder are particularly powerful for fraud detection, automatically calculating features that maximize separation between fraudulent and legitimate transactions. The end-to-end ONNX export demonstrates that the entire preprocessing + scoring pipeline can be deployed as a single portable artefact with no Python runtime dependency.

Key Insights:#

  • Engineered features (with __ separator) comprise a significant portion of top features

  • Group statistics features are highly predictive

  • Missing value patterns are strong fraud signals

  • Temporal patterns (hour, day) indicate fraud behavior

  • Email and device mismatches flag suspicious transactions

ONNX Precision Note:#

A residual native-vs-ONNX probability gap remains for this dataset/model, even with XGBoost (confirming this is not an LightGBM-specific ONNX conversion issue — swapping LightGBM for XGBoost here was an explicit test of that hypothesis, and the gap persists for both). Four things were investigated:

  1. GroupStatisticsFeatures group keys (e.g. id_31, DeviceInfo) unseen at test time correctly produce NaN (“missing”) features that were never seen during training. Fixed by adding a second NumericImputer step right after GroupStatisticsFeatures (step 7b) so these test-time-only nulls are always filled with a value the model actually trained on — verified zero nulls remain in both X_train_transformed and X_test_transformed. This is good pipeline practice regardless of ONNX (it also matters for the native model, since XGBoost/LightGBM’s missing-value routing for a feature never seen as missing during training is inherently unreliable).

  2. Even with #1 fixed, the gap barely moved — confirming the residual divergence is not primarily a null-handling issue. Isolated via a worst-row diagnostic: a small number of rows land almost exactly on a split threshold, where a native Polars/numpy vs ONNX Runtime kernel floating-point rounding difference of ~1e-7 (see the preprocessing parity check) is enough to flip which branch is taken, cascading into a materially different leaf/probability for that one row. Proven, not just theorized: the two ~1e-7-apart feature vectors for the worst row were fed directly into the same native XGBoost booster — no ONNX involved at all — via two separate booster.predict(DMatrix(...)) calls. The native booster alone produced a 0.107 probability difference for those two near-identical vectors, conclusively showing this is inherent chaotic decision-boundary sensitivity in the trained model itself, not an ONNX conversion bug.

  3. Model hyperparameters were tuned to reduce (not eliminate) this sensitivity. A systematic experiment across several max_depth/n_estimators/max_bin combinations showed max_depth is the dominant lever — shallower trees have wider effective split margins, since thresholds are chosen from more supporting data. n_estimators compounds with depth (more trees = more chances for a correlated flip across the ensemble). max_bin (histogram coarsening) does not help, since XGBoost still does exact float threshold comparisons at prediction time regardless of training-time bin granularity. The originally-considered max_depth=4, n_estimators=150 gave the best stability but cost ~5.5 points of held-out validation AUC (0.9278 → 0.8727) — a real generalization loss, not just less overfitting, confirmed via a proper 80/20 stratified train/validation split (not just re-scoring on the training set). ``max_depth=6, n_estimators=450`` was chosen instead as a middle ground: held-out AUC drops only ~0.9 points (0.9278 → 0.9186), while the max native-vs-ONNX gap still drops 44% (0.189 → 0.106) and the fraction of meaningfully-diverging rows (> 1e-4) drops from 33.0% → 27.1%. The median divergence is identical either way (~8.9e-8) since it’s dominated by the vast majority of rows that sit nowhere near a decision boundary — only the tail improves with hyperparameter tuning.

  4. Explicitly ruled out float32/float64 width as the cause, via two controlled experiments:

    • (a) Feeding the native booster the same features round-tripped through float32 (max feature diff 2.157e-04 — larger than any native-vs-ONNX preprocessing diff ever observed) produced exactly 0.0 probability difference for every row. XGBoost’s DMatrix always stores data as float32 internally regardless of input dtype, so float32-vs-float64 width has zero effect on the native model’s output.

    • (b) onnxmltools’s XGBoost converter only accepts float32 (or int64) input — DoubleTensorType is rejected outright, so a float64 ONNX path isn’t even achievable. With the only available float32 input, the standalone model conversion is near-perfect when fed identical features (max diff 5.36e-07, median 5.96e-08, 0.00% of rows over 1e-4) — yet the full scoring_onnx graph (same float32 model, but fed ONNX-computed preprocessing) still shows the same large gap (max 0.106, 27.1% of rows over 1e-4).

    • Since dtype width is proven irrelevant (a) and model conversion is proven precise (b), 100% of the divergence traces to the preprocessing step: Polars and ONNX Runtime computing the same float64 formulas (trig functions, GroupStatisticsFeatures lookups, etc.) via independent numerical kernels, producing tiny bit-level differences that occasionally cross a decision-boundary threshold.

  5. Localized #4’s “preprocessing step” to a single specific transformer. Sliced the fitted pipeline step-by-step (pipe[:k] — gators.Pipeline slicing reuses already-fitted transformer instances, no re-fit needed) and diffed native vs ONNX output after each prefix. Steps 1-3 (IsNull, DurationToDatetime, dt_timebin) are exactly 0.0 diff. dt_cyclic (CyclicFeatures) is the first — and only — step to introduce non-zero divergence (7.39e-07, on a cyclic __sin270 feature); every later step shows the identical diff on the same column — zero additional divergence, they just pass the already-imprecise column through unchanged. CyclicFeatures’ Sin/Cos ONNX operators agreeing with Polars’ native sin/cos to ~7e-7 is normal, expected, unavoidable cross-library floating-point behavior for transcendental functions — not a bug.

  6. Confirmed #5 end-to-end: re-ran the full pipeline (same depth=6, n_estimators=450 model) with CyclicFeatures removed entirely. The gap collapses to the pure model-conversion noise floor — max 1.06e-01 → 5.36e-07, rows with diff > 1e-4 27.1% → 0.00%. CyclicFeatures was not removed from this notebook’s pipeline (the hour/day/week cyclical signal is genuinely predictive for fraud — trading it away purely for ONNX cosmetic parity isn’t worth it, the same trade-off already rejected for max_depth). For a deployment that needs bit-exact ONNX parity specifically, swapping the trig-based cyclic encoding for a non-trigonometric alternative (e.g. modular one-hot bins) would eliminate this source entirely.

Effects #2/#4/#5/#6 are inherent to deploying deep tree ensembles at this row count and depth; they can be reduced via hyperparameter choices (#3) but not eliminated, and are not something gators’ pipeline conversion (or ONNX itself) is responsible for. The probability-parity checks in this section are informational only for that reason.

[ ]:

# ── 6.5  Create ORT sessions and run inference ───────────────────────────────
# create_session applies ORT_ENABLE_ALL (constant folding, op fusion, layout transforms)
pre_session     = create_session(preprocessing_onnx)
scoring_session = create_session(
    scoring_onnx,
    # save the pre-optimised graph so subsequent loads skip re-optimisation
    optimized_model_path=f"{ONNX_DIR}/fraud_scoring_opt.onnx",
)

# Preprocessing only — run_session handles Polars→numpy type mapping automatically
X_test_onnx = run_session(pre_session, X_test, batch_size=50_000)
print(f"ONNX preprocessing output: {X_test_onnx.shape}")

# End-to-end: raw columns → fraud label + probabilities
onnx_preds = run_session(scoring_session, X_test)
onnx_probs = onnx_preds["probabilities"].to_numpy()[:, 1]

# Verify parity with the original native (non-ONNX) predictions
# NOTE: a residual discrepancy is expected for this dataset/model combination — see the
# "ONNX Precision Note" in the Summary section below.

native_probs = estimator.predict_proba(X_test_transformed)[:, 1]
max_diff = float(abs(native_probs - onnx_probs).max())
median_diff = float(np.median(abs(native_probs - onnx_probs)))
pct_over_1e4 = float((abs(native_probs - onnx_probs) > 1e-4).mean())
print(f"\nMax |native - ONNX| probability difference: {max_diff:.2e}")
print(f"Median |native - ONNX| probability difference: {median_diff:.2e}")
print(f"Rows with |native - ONNX| > 1e-4: {pct_over_1e4:.1%}")

2026-08-24 05:39:45.897 Python[25650:124266] 2026-08-24 05:39:45.897849 [W:onnxruntime:, inference_session.cc:2670 Initialize] Serializing optimized model with Graph Optimization level greater than ORT_ENABLE_EXTENDED and the NchwcTransformer enabled. The generated model may contain hardware specific optimizations, and should only be used in the same environment the model was optimized in.

ONNX preprocessing output: (506691, 249)

Max |native - ONNX| probability difference: 1.06e-01
Median |native - ONNX| probability difference: 8.94e-08
Rows with |native - ONNX| > 1e-4: 27.1%

[9]:

# ── 6.6  Compare xgb_onnx vs scoring_onnx ────────────────────────────────────
# xgb_onnx: standalone model — must be fed gators-preprocessed features
xgb_session     = create_session(xgb_onnx)
xgb_input_name  = xgb_session.get_inputs()[0].name
X_xgb           = X_test_transformed.to_pandas().values.astype("float32")
xgb_out         = xgb_session.run(None, {xgb_input_name: X_xgb})
xgb_probs       = xgb_out[1][:, 1]   # P(fraud) from xgb_onnx

# scoring_onnx: end-to-end — fed raw columns, preprocessing baked into the graph
scoring_probs = run_session(scoring_session, X_test)["probabilities"].to_numpy()[:, 1]

print(f"{'Path':<45} {'P(fraud) first 5 rows'}")
print("=" * 75)
print(f"{'xgb_onnx  (gators features → xgb)':<45} {xgb_probs[:5].round(6)}")
print(f"{'scoring_onnx (raw data → pipeline → xgb)':<45} {scoring_probs[:5].round(6)}")
max_diff = float(abs(xgb_probs - scoring_probs).max())
print(f"\nMax |xgb_onnx - scoring_onnx|: {max_diff:.2e}")

Path                                          P(fraud) first 5 rows
===========================================================================
xgb_onnx  (gators features → xgb)             [0.067543 0.07671  0.113969 0.052122 0.089588]
scoring_onnx (raw data → pipeline → xgb)      [0.067543 0.07671  0.113969 0.052122 0.089588]

Max |xgb_onnx - scoring_onnx|: 1.06e-01

[ ]: