San Francisco Crime Classification with Gators#

This notebook demonstrates how to use the gators library for feature engineering in a multiclass classification problem. We’ll predict crime categories from the San Francisco Crime dataset using various gators transformers.

Key Features Demonstrated:#

  • Data preprocessing and cleaning

  • DateTime feature engineering (cyclic, ordinal, business time, holidays)

  • String feature engineering (pattern detection)

  • Spatial feature engineering (coordinate rotation)

  • Group-based imputation

  • Feature encoding and interaction

1. Import Libraries#

Import the necessary libraries including gators transformers for feature engineering.

[6]:

import polars as pl
import pandas as pd
from IPython.display import display

from gators.pipeline import Pipeline
from gators.encoders import OneHotEncoder, CountEncoder
from gators.data_cleaning import DropColumns, CastColumns
from gators.feature_generation import PlanRotationFeatures
from gators.imputers import GroupByImputer
from gators.feature_generation_dt import (
    CyclicFeatures,
    OrdinalFeatures,
    BusinessTimeFeatures,
    TimeBinFeatures,
    HolidayFeatures,
)
from gators.feature_generation_str import (
    Contains,
    InteractionFeatures,
)

from xgboost import XGBClassifier

2. Load and Preprocess Data#

Load the San Francisco Crime dataset and perform initial preprocessing:

  • Remove redundant columns

  • Handle outlier coordinate values (replace invalid coordinates with null)

  • Create a distance-to-center feature using San Francisco’s geographic center

[7]:

# Load data
X_train = pl.read_parquet('../../../../../Documents/kaggle/sf/train.parquet')
X_test = pl.read_parquet('../../../../../Documents/kaggle/sf/test.parquet')

# Drop unnecessary columns
X_train = X_train.drop(["Descript", "Resolution", "DayOfWeek"])
X_test = X_test.drop(["DayOfWeek"])

# Extract target variable
target = 'Category'
y_train = X_train[target]
X_train = X_train.drop(target)

# Replace invalid coordinate values with null
X_train = X_train.with_columns([
    pl.when(pl.col('X') == -120.5).then(None).otherwise(pl.col('X')).alias('X'),
    pl.when(pl.col('Y') == 90.0).then(None).otherwise(pl.col('Y')).alias('Y')
])

X_test = X_test.with_columns([
    pl.when(pl.col('X') == -120.5).then(None).otherwise(pl.col('X')).alias('X'),
    pl.when(pl.col('Y') == 90.0).then(None).otherwise(pl.col('Y')).alias('Y')
])

# Create distance to San Francisco center feature
sf_center_x, sf_center_y = -122.4194, 37.7749
X_train = X_train.with_columns([
    (((pl.col('X') - sf_center_x)**2 + (pl.col('Y') - sf_center_y)**2)**0.5).alias('distance_to_center')
])
X_test = X_test.with_columns([
    (((pl.col('X') - sf_center_x)**2 + (pl.col('Y') - sf_center_y)**2)**0.5).alias('distance_to_center')
])

print(f"Training data shape: {X_train.shape}")
print(f"Test data shape: {X_test.shape}")

Training data shape: (878049, 6)
Test data shape: (884262, 7)

3. Prepare Target Variable#

Convert crime category labels to numeric format for model training.

[8]:

# Define all crime categories
crime_categories = [
    "ARSON", "ASSAULT", "BAD CHECKS", "BRIBERY", "BURGLARY",
    "DISORDERLY CONDUCT", "DRIVING UNDER THE INFLUENCE", "DRUG/NARCOTIC",
    "DRUNKENNESS", "EMBEZZLEMENT", "EXTORTION", "FAMILY OFFENSES",
    "FORGERY/COUNTERFEITING", "FRAUD", "GAMBLING", "KIDNAPPING",
    "LARCENY/THEFT", "LIQUOR LAWS", "LOITERING", "MISSING PERSON",
    "NON-CRIMINAL", "OTHER OFFENSES", "PORNOGRAPHY/OBSCENE MAT",
    "PROSTITUTION", "RECOVERED VEHICLE", "ROBBERY", "RUNAWAY",
    "SECONDARY CODES", "SEX OFFENSES FORCIBLE", "SEX OFFENSES NON FORCIBLE",
    "STOLEN PROPERTY", "SUICIDE", "SUSPICIOUS OCC", "TREA", "TRESPASS",
    "VANDALISM", "VEHICLE THEFT", "WARRANTS", "WEAPON LAWS"
]

# Convert target to numeric labels
mapping = {cat: str(i) for i, cat in enumerate(crime_categories)}
y_train = y_train.replace(mapping).cast(pl.Int64)

print(f"Number of classes: {len(crime_categories)}")
print(f"Target shape: {y_train.shape}")

Number of classes: 39
Target shape: (878049,)

4. Build Feature Engineering Pipeline#

The gators Pipeline orchestrates multiple feature transformers:

Data Cleaning & Casting:#

  • GroupByImputer: Fill missing X/Y coordinates using mean values grouped by police district

  • CastColumns: Convert date strings to datetime type

Datetime Features:#

  • TimeBinFeatures: Create time-of-day bins (morning, afternoon, evening, night)

  • DatetimeCyclicFeatures: Generate cyclic features for temporal patterns (sin/cos transformations)

  • BusinessTimeFeatures: Extract business-related time features (weekday vs weekend)

  • HolidayFeatures: Identify whether crimes occurred on holidays

  • DatetimeOrdinalFeatures: Create ordinal features (month, day, hour, etc.)

String Features:#

  • Contains: Detect patterns in address strings (e.g., contains ‘Block’, ‘AV’, ‘ST’)

  • InteractionFeatures: Create interactions between categorical features (e.g., day_of_week × part_of_day)

Spatial Features:#

  • PlanRotationFeatures: Rotate X/Y coordinates at multiple angles to capture spatial patterns

Encoding:#

  • OneHotEncoder: Encode categorical variables as binary features

  • CountEncoder: Encode categories by their frequency

[9]:

# Define datetime components to extract
ordinal_components = [
    "month", "week", "day_of_week", "day_of_month",
    "day_of_year", "hour", "minute", "weekend"
]

cyclic_components = [
    "month", "week", "day_of_week", "day_of_month",
    "day_of_year", "hour", "minute"
]

# Build the pipeline with gators transformers
steps = [
    # Impute missing coordinates using group averages
    ("group_imputer", GroupByImputer(
        group_by_column='PdDistrict',
        strategy='mean',
        subset=['X', 'Y']
    )),

    # Convert dates to datetime type
    ("cast", CastColumns(subset=['Dates'], dtype=pl.Datetime, inplace=True)),

    # Extract time-of-day bins
    ("dt_timebin", TimeBinFeatures(subset=['Dates'])),

    # Create cyclic datetime features (captures cyclical nature of time)
    ("dt_cyclic", CyclicFeatures(
        subset=['Dates'],
        angles=[180 * i / 4 for i in range(8)],
        components=cyclic_components
    )),

    # Extract business time features
    ("dt_business", BusinessTimeFeatures(subset=['Dates'])),

    # Add holiday indicator
    ("dt_holiday", HolidayFeatures(subset=['Dates'], features=['is_holiday'])),

    # Extract ordinal datetime components
    ("dt_ordinal", OrdinalFeatures(subset=['Dates'], components=ordinal_components)),

    # Extract patterns from address strings
    ("contains", Contains(contains_dict={'Address': ['/', 'Block', 'AV', 'ST']})),

    # Drop original columns after feature extraction
    ("drop", DropColumns(subset=["Address", "Dates"])),

    # Create interaction features
    ("interaction_time", InteractionFeatures(subset=['Dates__day_of_week', 'Dates__part_of_day'])),
    ("interaction_district", InteractionFeatures(subset=['PdDistrict', 'Dates__part_of_day'])),

    # Rotate coordinates to capture spatial patterns
    ("plan_rotation", PlanRotationFeatures(
        columns=[['X', 'Y']],
        angles=[180 * i / 4 for i in range(8)]
    )),

    # Encode categorical variables
    ("onehot_encoder", OneHotEncoder()),
    ("count_encoder", CountEncoder()),
]

# Create and fit the pipeline
pipeline = Pipeline(steps=steps, verbose=True)
X_train_transformed = pipeline.fit_transform(X_train, y_train)
X_test_transformed = pipeline.transform(X_test)

print(f"\nOriginal features: {X_train.shape[1]}")
print(f"Engineered features: {X_train_transformed.shape[1]}")

[Pipeline] fit+transform   1/14 · group_imputer  |  in: rows=878049  cols=6  nulls=201  →  out: rows=878049  cols=6  nulls=67  (0.018s)
[Pipeline] fit+transform   2/14 · cast  |  in: rows=878049  cols=6  nulls=67  →  out: rows=878049  cols=6  nulls=67  (0.032s)
[Pipeline] fit+transform   3/14 · dt_timebin  |  in: rows=878049  cols=6  nulls=67  →  out: rows=878049  cols=11  nulls=67  (0.044s)
[Pipeline] fit+transform   4/14 · dt_cyclic  |  in: rows=878049  cols=11  nulls=67  →  out: rows=878049  cols=67  nulls=67  (0.089s)
[Pipeline] fit+transform   5/14 · dt_business  |  in: rows=878049  cols=67  nulls=67  →  out: rows=878049  cols=70  nulls=67  (0.034s)
[Pipeline] fit+transform   6/14 · dt_holiday  |  in: rows=878049  cols=70  nulls=67  →  out: rows=878049  cols=71  nulls=67  (0.113s)
[Pipeline] fit+transform   7/14 · dt_ordinal  |  in: rows=878049  cols=71  nulls=67  →  out: rows=878049  cols=79  nulls=67  (0.012s)
[Pipeline] fit+transform   8/14 · contains  |  in: rows=878049  cols=79  nulls=67  →  out: rows=878049  cols=83  nulls=67  (0.013s)
[Pipeline] fit+transform   9/14 · drop  |  in: rows=878049  cols=83  nulls=67  →  out: rows=878049  cols=81  nulls=67  (0.001s)
[Pipeline] fit+transform   10/14 · interaction_time  |  in: rows=878049  cols=81  nulls=67  →  out: rows=878049  cols=82  nulls=67  (0.023s)
[Pipeline] fit+transform   11/14 · interaction_district  |  in: rows=878049  cols=82  nulls=67  →  out: rows=878049  cols=83  nulls=67  (0.029s)
[Pipeline] fit+transform   12/14 · plan_rotation  |  in: rows=878049  cols=83  nulls=67  →  out: rows=878049  cols=99  nulls=67  (0.012s)
[Pipeline] fit+transform   13/14 · onehot_encoder  |  in: rows=878049  cols=99  nulls=67  →  out: rows=878049  cols=189  nulls=67  (0.133s)
[Pipeline] fit+transform   14/14 · count_encoder  |  in: rows=878049  cols=189  nulls=67  →  out: rows=878049  cols=189  nulls=67  (0.042s)
[Pipeline] transform   1/14 · group_imputer  |  in: rows=884262  cols=7  nulls=228  →  out: rows=884262  cols=7  nulls=76  (0.014s)
[Pipeline] transform   2/14 · cast  |  in: rows=884262  cols=7  nulls=76  →  out: rows=884262  cols=7  nulls=76  (0.033s)
[Pipeline] transform   3/14 · dt_timebin  |  in: rows=884262  cols=7  nulls=76  →  out: rows=884262  cols=12  nulls=76  (0.048s)
[Pipeline] transform   4/14 · dt_cyclic  |  in: rows=884262  cols=12  nulls=76  →  out: rows=884262  cols=68  nulls=76  (0.094s)
[Pipeline] transform   5/14 · dt_business  |  in: rows=884262  cols=68  nulls=76  →  out: rows=884262  cols=71  nulls=76  (0.035s)
[Pipeline] transform   6/14 · dt_holiday  |  in: rows=884262  cols=71  nulls=76  →  out: rows=884262  cols=72  nulls=76  (0.004s)
[Pipeline] transform   7/14 · dt_ordinal  |  in: rows=884262  cols=72  nulls=76  →  out: rows=884262  cols=80  nulls=76  (0.012s)
[Pipeline] transform   8/14 · contains  |  in: rows=884262  cols=80  nulls=76  →  out: rows=884262  cols=84  nulls=76  (0.012s)
[Pipeline] transform   9/14 · drop  |  in: rows=884262  cols=84  nulls=76  →  out: rows=884262  cols=82  nulls=76  (0.001s)
[Pipeline] transform   10/14 · interaction_time  |  in: rows=884262  cols=82  nulls=76  →  out: rows=884262  cols=83  nulls=76  (0.025s)
[Pipeline] transform   11/14 · interaction_district  |  in: rows=884262  cols=83  nulls=76  →  out: rows=884262  cols=84  nulls=76  (0.031s)
[Pipeline] transform   12/14 · plan_rotation  |  in: rows=884262  cols=84  nulls=76  →  out: rows=884262  cols=100  nulls=76  (0.005s)
[Pipeline] transform   13/14 · onehot_encoder  |  in: rows=884262  cols=100  nulls=76  →  out: rows=884262  cols=190  nulls=76  (0.072s)
[Pipeline] transform   14/14 · count_encoder  |  in: rows=884262  cols=190  nulls=76  →  out: rows=884262  cols=190  nulls=76  (0.023s)

Original features: 6
Engineered features: 189

5. Train Model and Generate Predictions#

Train an XGBoost classifier with the engineered features and generate predictions.

[10]:

[c for c in X_test_transformed.columns if c not in X_train_transformed.columns]

[10]:

['Id']

[11]:

# Define model parameters
xgb_params = {
    'max_depth': 6,
    'learning_rate': 0.1,
    'n_estimators': 100,
    'min_child_weight': 5,
    'subsample': 0.85,
    'colsample_bytree': 0.85,
    'max_delta_step': 2,
    'tree_method': 'hist',
    'eval_metric': 'mlogloss',
    'random_state': 0,
    'n_jobs': -1
}

# Train the model
estimator = XGBClassifier(**xgb_params)
estimator.fit(X_train_transformed, y_train)

print("Model training completed!")

# Generate predictions
submission = pd.DataFrame(
    estimator.predict_proba(X_test_transformed.drop("Id")),
    index=X_test["Id"].to_pandas(),
    columns=crime_categories
)
submission.to_csv("submission.csv.zip", compression="zip")
print("Submission file created successfully!")

Model training completed!
Submission file created successfully!

6. Feature Importance Analysis#

Analyze which features contribute most to the model’s predictions.

[12]:

# Extract feature importances
feat_imp = pl.DataFrame({
    "feature": estimator.feature_names_in_,
    "importance": estimator.feature_importances_
})
feat_imp = feat_imp.sort("importance", descending=True)

# Display top 20 most important features
print("Top 20 Most Important Features:")
display(feat_imp.head(20))

Top 20 Most Important Features:
shape: (20, 2)
featureimportance
strf32
"Address__contains_/"0.044727
"Address__contains_Block"0.040278
"Dates__minute__sin90"0.036214
"Dates__minute__sin270"0.035519
"Dates__minute"0.033772
……
"Dates__rush_hour__morning_rush"0.012069
"PdDistrict__Dates__part_of_day…0.011952
"PdDistrict__Dates__part_of_day…0.011936
"PdDistrict__Dates__part_of_day…0.011771
"Dates__hour__sin315"0.01068

Summary#

This notebook demonstrates the power of the gators library for feature engineering in machine learning pipelines:

Key Takeaways:#

  1. Modular Pipeline: Easily compose complex feature engineering workflows using gators transformers

  2. Datetime Engineering: Rich set of temporal features (cyclic, ordinal, business time, holidays)

  3. Spatial Features: Coordinate rotation and distance-based features for geographic data

  4. String Processing: Pattern extraction from text fields

  5. Smart Imputation: Group-based imputation leveraging categorical relationships

  6. Seamless Integration: Works with Polars DataFrames and scikit-learn/XGBoost models

The gators library significantly reduced feature engineering complexity while creating a comprehensive feature set for multiclass classification.