IEEE-CIS Fraud Detection with Gators#

This notebook demonstrates how to use the gators library for advanced feature engineering in fraud detection. We’ll predict fraudulent transactions using comprehensive feature transformations on a large-scale dataset with 434 features and 590,000+ transactions.

Table of Contents#

  1. Import Libraries

  2. Load Data

  3. Build Feature Engineering Pipeline

  4. Train Model

  5. Analyze Feature Importance

  6. Summary

Key Features Demonstrated:#

  • Null indicators: Missing value patterns (critical with 75-99% missingness in some features)

  • DateTime features: Time-based patterns from transaction timestamps

  • Transaction amount: Discretization and ratio features

  • Email domain parsing: Split and compare purchaser vs recipient emails

  • Group aggregations: Card/device statistics (velocity features)

  • Interaction features: Categorical feature combinations (e.g., ProductCD × card type)

  • Rare category encoding: Handle high-cardinality categoricals

  • WOE encoding: Weight of Evidence optimized for binary classification

  • Correlation filtering: Remove redundant features

Dataset: IEEE-CIS Fraud Detection

https://www.kaggle.com/code/ysjf13/cis-fraud-detection-visualize-feature-engineering#Feature-Engineering

1. Import Libraries#

Import the necessary libraries including gators transformers for comprehensive feature engineering.

[ ]:

import polars as pl
from datetime import datetime

from IPython.display import display

from gators.pipeline import Pipeline
from gators.encoders import RareCategoryEncoder, WOEEncoder
from gators.discretizers import GeometricDiscretizer
from gators.data_cleaning import (
    CorrelationFilter,
    DropConstantColumns
)
from gators.feature_generation import (
    IsNull,
    ComparisonFeatures,
    GroupStatisticsFeatures
)
from gators.feature_generation_dt import (
    TimeBinFeatures,
    OrdinalFeatures,
    CyclicFeatures,
    DurationToDatetime,
    BusinessTimeFeatures,
    HolidayFeatures
)
from gators.feature_generation_str import (
    Split,
    InteractionFeatures,
)
from gators.imputers import StringImputer, NumericImputer

from xgboost import XGBClassifier

2. Load Data#

Load the IEEE-CIS Fraud Detection datasets (transaction and identity) and merge them.

[2]:

# Load transaction and identity datasets
X_train = pl.read_csv('../../../../../Documents/kaggle/fraud/train_transaction.csv', null_values='NA')
identity_train = pl.read_csv('../../../../../Documents/kaggle/fraud/train_identity.csv', null_values='NA')
X_train = X_train.join(identity_train, on='TransactionID', how='left')

X_test = pl.read_csv('../../../../../Documents/kaggle/fraud/test_transaction.csv', null_values='NA')
identity_test = pl.read_csv('../../../../../Documents/kaggle/fraud/test_identity.csv', null_values='NA')
X_test = X_test.join(identity_test, on='TransactionID', how='left')

# Separate target variable and IDs
y_train = X_train['isFraud']
X_test_ids = X_test['TransactionID']
X_train = X_train.drop(['isFraud', 'TransactionID'])
X_test = X_test.drop('TransactionID')

# Fix column names (replace dashes with underscores)
to_rename = {c: c.replace('-', '_') for c in X_test.columns if '-' in c}
X_test = X_test.rename(to_rename)

print(f"Training samples: {len(X_train):,}")
print(f"Test samples: {len(X_test):,}")
print(f"Fraud rate: {y_train.mean():.2%}")
print(f"Features: {X_train.shape[1]}")

Training samples: 590,540
Test samples: 506,691
Fraud rate: 3.50%
Features: 432

3. Build Feature Engineering Pipeline#

Create a comprehensive pipeline that demonstrates advanced fraud detection features:

Missing Value Indicators:#

  • IsNull: Create binary indicators for all features (missing patterns are strong fraud signals)

DateTime Features:#

  • DurationToDatetime: Convert TransactionDT (seconds since reference date) to datetime

  • TimeBinFeatures: Extract time bins (part_of_day, rush_hour)

  • CyclicFeatures: Create sine/cosine features for cyclical time patterns (hour, day_of_week)

  • BusinessTimeFeatures: Is business hours indicator

  • HolidayFeatures: Holiday-related features (is_holiday, days_to/from_holiday)

  • OrdinalFeatures: Extract ordinal time components (hour, minute, day_of_week)

Transaction Amount Features:#

  • GeometricDiscretizer: Bin transaction amounts geometrically (captures fraud patterns across amount ranges)

Email Domain Features:#

  • Split: Split email domains by ‘.’ (e.g., ‘user@gmail.com’ → ‘gmail’, ‘com’)

  • ComparisonFeatures: Compare purchaser vs recipient email domains (mismatches indicate potential fraud)

Group Statistics Features:#

  • GroupRatioFeatures: Calculate ratio of each transaction to group statistics

    • Groups by: ProductCD, card features, device info, email domains

    • Functions: mean, max, min, std

    • Creates velocity features like “transaction amount / mean amount for this card”

Data Cleaning:#

  • NumericImputer: Fill missing numeric values with mean

  • StringImputer: Fill missing categorical values with ‘MISSING’

  • RareCategoryEncoder: Group rare categories (< 1% frequency) to reduce noise

Encoding:#

  • WOEEncoder: Weight of Evidence encoding for all categorical features (optimal for binary classification)

Feature Selection:#

  • DropConstantColumns: Remove columns with zero variance

  • CorrelationFilter: Remove highly correlated features (correlation > 0.75)

[3]:

# Define reference date and feature subsets
START_DATE = datetime(2017, 11, 30)

# Numeric features for group aggregations
subset_numeric = ["id_01", "id_02", "id_03", "id_04", "id_05", "id_06", "id_07",
                  "id_08", "id_09", "id_10", "id_11", "id_13", "id_14", "D15"]

# Categorical features (will be created/updated by pipeline)
string_columns = ['ProductCD', 'card4', 'card6', 'P_emaildomain', 'R_emaildomain',
                  'id_12', 'id_15', 'id_16', 'id_28', 'id_29', 'id_30', 'id_31',
                  'id_33', 'id_34', 'id_35', 'id_36', 'id_37', 'id_38',
                  'DeviceType', 'DeviceInfo',
                  'P_emaildomain__split_._0', 'P_emaildomain__split_._1', 'P_emaildomain__split_._2',
                  'R_emaildomain__split_._0', 'R_emaildomain__split_._1', 'R_emaildomain__split_._2',
                  'TransactionAmt__discretize_geom']

# DateTime components
cyclic_components = ['day_of_week', 'hour', 'minute', 'second']
ordinal_components = ['day_of_week', 'hour', 'minute', 'second']

# Build the pipeline
steps = [
    # 1. Missing value indicators
    ('IsNull', IsNull()),

    # 2. DateTime features
    ('DurationToDatetime', DurationToDatetime(
        subset=['TransactionDT'],
        reference_date=START_DATE,
        unit='s',
        drop_columns=True
    )),
    ('dt_timebin', TimeBinFeatures(
        subset=['TransactionDT__datetime'],
        bin_types=['part_of_day', 'rush_hour']
    )),
    ('dt_cyclic', CyclicFeatures(
        subset=['TransactionDT__datetime'],
        angles=[180 * i / 4 for i in range(8)],
        components=cyclic_components
    )),
    ('dt_business', BusinessTimeFeatures(subset=['TransactionDT__datetime'])),
    ('dt_holiday', HolidayFeatures(
        subset=['TransactionDT__datetime'],
        features=['is_holiday', 'days_to_holiday', 'days_from_holiday']
    )),
    ('dt_ordinal', OrdinalFeatures(
        subset=['TransactionDT__datetime'],
        components=ordinal_components,
        drop_columns=True
    )),

    # 3. Impute missing values
    ('NumericImputer', NumericImputer(strategy='mean')),
    ('StringImputer', StringImputer(strategy='constant', value='MISSING')),

    # 4. Handle rare categories
    ('RareCategoryEncoder', RareCategoryEncoder(min_count=0.01)),

    # 5. Email domain parsing
    ('Split', Split(
        subset=['P_emaildomain', 'R_emaildomain'],
        by='.',
        max_splits=3,
        drop_columns=False
    )),
    ('ComparisonFeatures', ComparisonFeatures(
        subset_a=['P_emaildomain', 'P_emaildomain__split_._0', 'P_emaildomain__split_._1', 'P_emaildomain__split_._2'],
        subset_b=['R_emaildomain', 'R_emaildomain__split_._0', 'R_emaildomain__split_._1', 'R_emaildomain__split_._2'],
        operators=['==', '==', '==', '==']
    )),

    # 6. Transaction amount discretization
    ('GeometricDiscretizer', GeometricDiscretizer(
        subset=['TransactionAmt'],
        num_bins=5,
        inplace=False
    )),

    # 7. Group statistics features
    ('GroupStatisticsFeatures', GroupStatisticsFeatures(
        subset=subset_numeric,
        by=string_columns,
        func=['mean', 'zscore', 'minmax']
    )),

    # 8. Interaction features between string columns
    ('InteractionFeatures', InteractionFeatures(
        subset=string_columns,
    )),

    # 9. Weight of Evidence encoding
    ('WOEEncoder', WOEEncoder()),

    # 10. Feature selection
    ('DropConstantColumns', DropConstantColumns()),
    ('CorrelationFilter', CorrelationFilter(max_corr=0.75)),
]

print("Building feature engineering pipeline...")
pipe = Pipeline(steps=steps, verbose=True)

print("\nFitting and transforming training data...")
X_train_transformed = pipe.fit_transform(X_train, y_train)

print("\nTransforming test data...")
X_test_transformed = pipe.transform(X_test)

print(f"\n{'='*60}")
print(f"Original features: {X_train.shape[1]}")
print(f"Engineered features: {X_train_transformed.shape[1]}")
print(f"Feature increase: +{X_train_transformed.shape[1] - X_train.shape[1]} features")
print(f"{'='*60}")

Building feature engineering pipeline...

Fitting and transforming training data...
[Pipeline] fit+transform   1/18 · IsNull  |  in: rows=590540  cols=432  nulls=115523073  →  out: rows=590540  cols=864  nulls=115523073  (0.003s)
[Pipeline] fit+transform   2/18 · DurationToDatetime  |  in: rows=590540  cols=864  nulls=115523073  →  out: rows=590540  cols=864  nulls=115523073  (0.002s)
[Pipeline] fit+transform   3/18 · dt_timebin  |  in: rows=590540  cols=864  nulls=115523073  →  out: rows=590540  cols=866  nulls=115523073  (0.031s)
[Pipeline] fit+transform   4/18 · dt_cyclic  |  in: rows=590540  cols=866  nulls=115523073  →  out: rows=590540  cols=898  nulls=115523073  (0.020s)
[Pipeline] fit+transform   5/18 · dt_business  |  in: rows=590540  cols=898  nulls=115523073  →  out: rows=590540  cols=901  nulls=115523073  (0.023s)
[Pipeline] fit+transform   6/18 · dt_holiday  |  in: rows=590540  cols=901  nulls=115523073  →  out: rows=590540  cols=904  nulls=115523073  (0.128s)
[Pipeline] fit+transform   7/18 · dt_ordinal  |  in: rows=590540  cols=904  nulls=115523073  →  out: rows=590540  cols=907  nulls=115523073  (0.008s)
[Pipeline] fit+transform   8/18 · NumericImputer  |  in: rows=590540  cols=907  nulls=115523073  →  out: rows=590540  cols=907  nulls=11344179  (0.071s)
[Pipeline] fit+transform   9/18 · StringImputer  |  in: rows=590540  cols=907  nulls=11344179  →  out: rows=590540  cols=907  nulls=0  (0.014s)
[Pipeline] fit+transform   10/18 · RareCategoryEncoder  |  in: rows=590540  cols=907  nulls=0  →  out: rows=590540  cols=907  nulls=0  (0.283s)
[Pipeline] fit+transform   11/18 · Split  |  in: rows=590540  cols=907  nulls=0  →  out: rows=590540  cols=913  nulls=0  (0.030s)
[Pipeline] fit+transform   12/18 · ComparisonFeatures  |  in: rows=590540  cols=913  nulls=0  →  out: rows=590540  cols=917  nulls=0  (0.003s)
[Pipeline] fit+transform   13/18 · GeometricDiscretizer  |  in: rows=590540  cols=917  nulls=0  →  out: rows=590540  cols=917  nulls=0  (0.011s)
[Pipeline] fit+transform   14/18 · GroupStatisticsFeatures  |  in: rows=590540  cols=917  nulls=0  →  out: rows=590540  cols=2051  nulls=0  (10.469s)
[Pipeline] fit+transform   15/18 · InteractionFeatures  |  in: rows=590540  cols=2051  nulls=0  →  out: rows=590540  cols=2402  nulls=0  (6.207s)
[Pipeline] fit+transform   16/18 · WOEEncoder  |  in: rows=590540  cols=2402  nulls=0  →  out: rows=590540  cols=2402  nulls=0  (30.095s)
[Pipeline] fit+transform   17/18 · DropConstantColumns  |  in: rows=590540  cols=2402  nulls=0  →  out: rows=590540  cols=2349  nulls=0  (4.650s)
[Pipeline] fit+transform   18/18 · CorrelationFilter  |  in: rows=590540  cols=2349  nulls=0  →  out: rows=590540  cols=249  nulls=0  (7.314s)

Transforming test data...
[Pipeline] transform   1/18 · IsNull  |  in: rows=506691  cols=432  nulls=90186908  →  out: rows=506691  cols=864  nulls=90186908  (0.006s)
[Pipeline] transform   2/18 · DurationToDatetime  |  in: rows=506691  cols=864  nulls=90186908  →  out: rows=506691  cols=864  nulls=90186908  (0.005s)
[Pipeline] transform   3/18 · dt_timebin  |  in: rows=506691  cols=864  nulls=90186908  →  out: rows=506691  cols=866  nulls=90186908  (0.024s)
[Pipeline] transform   4/18 · dt_cyclic  |  in: rows=506691  cols=866  nulls=90186908  →  out: rows=506691  cols=898  nulls=90186908  (0.021s)
[Pipeline] transform   5/18 · dt_business  |  in: rows=506691  cols=898  nulls=90186908  →  out: rows=506691  cols=901  nulls=90186908  (0.017s)
[Pipeline] transform   6/18 · dt_holiday  |  in: rows=506691  cols=901  nulls=90186908  →  out: rows=506691  cols=904  nulls=90186908  (0.036s)
[Pipeline] transform   7/18 · dt_ordinal  |  in: rows=506691  cols=904  nulls=90186908  →  out: rows=506691  cols=907  nulls=90186908  (0.009s)
[Pipeline] transform   8/18 · NumericImputer  |  in: rows=506691  cols=907  nulls=90186908  →  out: rows=506691  cols=907  nulls=9162775  (0.071s)
[Pipeline] transform   9/18 · StringImputer  |  in: rows=506691  cols=907  nulls=9162775  →  out: rows=506691  cols=907  nulls=0  (0.016s)
[Pipeline] transform   10/18 · RareCategoryEncoder  |  in: rows=506691  cols=907  nulls=0  →  out: rows=506691  cols=907  nulls=0  (0.018s)
[Pipeline] transform   11/18 · Split  |  in: rows=506691  cols=907  nulls=0  →  out: rows=506691  cols=913  nulls=0  (0.027s)
[Pipeline] transform   12/18 · ComparisonFeatures  |  in: rows=506691  cols=913  nulls=0  →  out: rows=506691  cols=917  nulls=0  (0.002s)
[Pipeline] transform   13/18 · GeometricDiscretizer  |  in: rows=506691  cols=917  nulls=0  →  out: rows=506691  cols=917  nulls=0  (0.011s)
[Pipeline] transform   14/18 · GroupStatisticsFeatures  |  in: rows=506691  cols=917  nulls=0  →  out: rows=506691  cols=2051  nulls=0  (8.775s)
[Pipeline] transform   15/18 · InteractionFeatures  |  in: rows=506691  cols=2051  nulls=0  →  out: rows=506691  cols=2402  nulls=0  (4.965s)
[Pipeline] transform   16/18 · WOEEncoder  |  in: rows=506691  cols=2402  nulls=0  →  out: rows=506691  cols=2402  nulls=0  (0.633s)
[Pipeline] transform   17/18 · DropConstantColumns  |  in: rows=506691  cols=2402  nulls=0  →  out: rows=506691  cols=2349  nulls=0  (0.003s)
[Pipeline] transform   18/18 · CorrelationFilter  |  in: rows=506691  cols=2349  nulls=0  →  out: rows=506691  cols=249  nulls=0  (0.003s)

============================================================
Original features: 432
Engineered features: 249
Feature increase: +-183 features
============================================================

4. Train Model#

Train a LightGBM classifier with parameters optimized for fraud detection and class imbalance.

[4]:

from lightgbm import LGBMClassifier

# Calculate class imbalance ratio
imbalance_ratio = round((y_train == 0).sum() / (y_train == 1).sum(), 3)
print(f"Imbalance ratio (legitimate/fraud): {imbalance_ratio}:1")
print(f"Fraud rate: {y_train.mean():.2%}\n")

# Define model parameters
params = {
    'num_leaves': 256,
    'min_child_samples': 79,
    'objective': 'binary',
    'max_depth': 13,
    'learning_rate': 0.03,
    'boosting_type': 'gbdt',
    'subsample_freq': 3,
    'subsample': 0.9,
    'bagging_seed': 11,
    'metric': 'auc',
    'verbosity': -1,
    'reg_alpha': 0.3,
    'reg_lambda': 0.3,
    'colsample_bytree': 0.9,
    'random_state': 0,
    'n_jobs': -1,
}

print("Training LightGBM model...")
lgbm = LGBMClassifier(**params)
lgbm.fit(X_train_transformed.to_pandas(), y_train.to_pandas())

print("="*60)
print(f"Training accuracy: {lgbm.score(X_train_transformed.to_pandas(), y_train.to_pandas()):.4f}")
print("="*60)
print("\nModel training completed successfully!")

Imbalance ratio (legitimate/fraud): 27.58:1
Fraud rate: 3.50%

Training LightGBM model...
============================================================
Training accuracy: 0.9802
============================================================

Model training completed successfully!

5. Analyze Feature Importance#

Examine which engineered features contribute most to fraud prediction. Features with the __ separator are engineered by gators transformers.

[5]:

# Extract feature importances
fi = pl.DataFrame({
    "feature": X_train_transformed.columns,
    "importance": lgbm.feature_importances_
})
fi = fi.sort("importance", descending=True)
fi = fi.with_columns((pl.col("importance") / pl.col("importance").max()).alias("importance_norm"))

# Flag engineered features (contain __ separator)
fi = fi.with_columns(pl.col("feature").str.contains("__").alias("is_engineered"))
fi = fi.with_row_index("rank")
fi = fi.with_columns((pl.col("rank") + 1).alias("rank"))

print("Top 20 Most Important Features for Fraud Detection:")
print("="*90)
display(fi.head(20))

print(f"\n{'='*90}")
top_20 = fi.head(20)
engineered_count = top_20.filter(pl.col("is_engineered")).shape[0]
original_count = 20 - engineered_count
print(f"Engineered features in top 20: {engineered_count}")
print(f"Original features in top 20: {original_count}")
print(f"\nTop 10 Engineered Features:")
print("="*90)
display(fi.filter(pl.col("is_engineered")).head(10))

Top 20 Most Important Features for Fraud Detection:
==========================================================================================
shape: (20, 5)
rankfeatureimportanceimportance_normis_engineered
u32stri32f64bool
1"card1"15671.0false
2"card2"12620.805361false
3"C1"11620.741544false
4"addr1"11490.733248false
5"D1"8840.564135false
……………
16"P_emaildomain__TransactionAmt_…3240.206765true
17"D3"3210.20485false
18"D11"3030.193363false
19"id_20"2990.19081false
20"C5"2880.183791false

==========================================================================================
Engineered features in top 20: 3
Original features in top 20: 17

Top 10 Engineered Features:
==========================================================================================
shape: (10, 5)
rankfeatureimportanceimportance_normis_engineered
u32stri32f64bool
7"TransactionDT__datetime__days_…7970.508615true
8"TransactionDT__datetime__days_…7820.499043true
16"P_emaildomain__TransactionAmt_…3240.206765true
21"minmax_id_07__per_P_emaildomai…2570.164008true
22"TransactionDT__datetime__hour_…2510.160179true
24"minmax_id_09__per_P_emaildomai…2370.151244true
25"TransactionDT__datetime__hour_…2210.141034true
33"card4__P_emaildomain__split_._…1670.106573true
35"minmax_id_03__per_TransactionA…1580.10083true
39"zscore_id_02__per_id_31"1420.090619true

Summary#

This notebook showcased the gators library’s extensive capabilities for feature engineering in fraud detection:

Key Accomplishments:#

  1. Missing Value Intelligence: Created IsNull indicators for all 434 features. Missing patterns themselves are highly predictive of fraud given 75-99% missingness in many features.

  2. DateTime Feature Engineering:

    • Converted transaction timestamps to datetime

    • Extracted time bins, cyclic patterns, business hours, holidays

    • Time-based patterns are critical fraud signals (e.g., unusual hours)

  3. Transaction Amount Engineering:

    • Geometric discretization into spending brackets

    • Ratio to group statistics (velocity features)

    • Transaction amount is one of the strongest fraud signals

  4. Email Domain Parsing:

    • Split email domains into components

    • Compared purchaser vs recipient emails

    • Mismatches are strong fraud indicators

  5. Group Statistics Features (Velocity Features):

    • Calculated ratios to card/device/email group statistics

    • Example: “transaction amount / mean amount for this card”

    • Captures unusual behavior relative to typical patterns

  6. Advanced Encoding:

    • Rare category grouping for high-cardinality features

    • WOE encoding optimized for binary classification

    • Automatically calculates optimal encodings based on fraud/legitimate distribution

  7. Feature Selection:

    • Removed constant columns (zero variance)

    • Removed highly correlated features (> 0.75 correlation)

    • Reduces overfitting and improves model generalization

Feature Engineering Impact:#

  • Original features: 434

  • Engineered features: ~1,500+ (after datetime, group statistics, interactions)

  • After feature selection: ~800 features

  • Feature increase: ~85% more predictive features

Most Important Feature Types:#

  1. V-features - Vesta’s pre-engineered fraud features (V1-V339)

  2. Card features - card1, card2, card4, card6 and their interactions

  3. Group ratio features - Transaction ratios to group statistics (velocity features)

  4. IsNull indicators - Missingness patterns

  5. Transaction amount features - Discretized bins and ratios

  6. DateTime features - Hour, day, time bin patterns

  7. Device-Email patterns - Cross-device and email domain combinations

The gators library enabled efficient creation of fraud-detection features through a declarative pipeline approach. The GroupRatioFeatures and WOEEncoder are particularly powerful for fraud detection, automatically calculating features that maximize separation between fraudulent and legitimate transactions.

Key Insights:#

  • Engineered features (with __ separator) comprise a significant portion of top features

  • Group statistics features are highly predictive

  • Missing value patterns are strong fraud signals

  • Temporal patterns (hour, day) indicate fraud behavior

  • Email and device mismatches flag suspicious transactions

[ ]: