feature_engineering

predspot.feature_engineering

Feature Engineering Module

Turns the spatio-temporal series produced by a mapping (see crime_mapping) into lagged features, one row per (t, places). Each feature class applies a time series transformation to the history of every place and then builds lags lagged columns from it:

• AR — the raw series (autoregressive features);
• Diff — first differences;
• Seasonality — the seasonal component of an STL decomposition;
• Trend — the trend component of an STL decomposition.

The output always contains one extra row for the period right after the last observed one, so that the fitted model can forecast the next period.

TimeSeriesFeatures

TimeSeriesFeatures(lags, tfreq=None)

Bases: BaseEstimator, TransformerMixin

Base class for lagged time series features.

Parameters:

Name	Type	Description	Default
lags	int	Number of lagged columns to create (> 1).	required
tfreq	str	Time frequency of the series ('M', 'W' or 'D'). If omitted it is inferred from the series index.	None

Source code in src/predspot/feature_engineering.py

def __init__(self, lags, tfreq=None):
    if not isinstance(lags, int) or lags < 2:
        raise ValueError("`lags` must be an integer greater than 1.")
    self.lags = lags
    self.tfreq = tfreq
    self._offset = tfreq_offset(tfreq) if tfreq is not None else None

label property

label

str: Prefix of the feature columns (override in subclasses).

apply_ts_decomposition abstractmethod

apply_ts_decomposition(ts)

Transform the time series of one place before lagging it.

Parameters:

Name	Type	Description	Default
ts	Series	Series of one place indexed by time.	required

Returns:

Type	Description
Series	Transformed series.

Source code in src/predspot/feature_engineering.py

@abstractmethod
def apply_ts_decomposition(self, ts):
    """
    Transform the time series of one place before lagging it.

    Args:
        ts (pandas.Series): Series of one place indexed by time.

    Returns:
        pandas.Series: Transformed series.
    """

fit

fit(x=None, y=None)

No-op; present for scikit-learn compatibility.

Source code in src/predspot/feature_engineering.py

def fit(self, x=None, y=None):
    """No-op; present for scikit-learn compatibility."""
    return self

make_lag_df

make_lag_df(ts)

Build the lagged feature columns of one time series.

Parameters:

Name	Type	Description	Default
ts	Series	Series indexed by time.	required

Returns:

Type	Description
tuple	(lag_df, ts_aligned) — the lagged features and the original series restricted to the same index.

Source code in src/predspot/feature_engineering.py

def make_lag_df(self, ts):
    """
    Build the lagged feature columns of one time series.

    Args:
        ts (pandas.Series): Series indexed by time.

    Returns:
        tuple: ``(lag_df, ts_aligned)`` — the lagged features and the
        original series restricted to the same index.
    """
    if len(ts) <= self.lags:
        raise ValueError("`lags` is higher than the number of time periods.")
    lag_df = pd.concat([ts.shift(lag) for lag in range(1, self.lags + 1)], axis=1)
    lag_df = lag_df.iloc[self.lags :]
    lag_df.columns = [f"{self.label}_{i}" for i in range(1, self.lags + 1)]
    return lag_df, ts.loc[lag_df.index]

transform

transform(stseries)

Compute lagged features for every place.

Parameters:

Name	Type	Description	Default
stseries	Series	Series indexed by (t, places).	required

Returns:

Type	Description
DataFrame	Features indexed by (t, places), including one row for the period after the last observed one.

Source code in src/predspot/feature_engineering.py

def transform(self, stseries):
    """
    Compute lagged features for every place.

    Args:
        stseries (pandas.Series): Series indexed by ``(t, places)``.

    Returns:
        pandas.DataFrame: Features indexed by ``(t, places)``, including
        one row for the period after the last observed one.
    """
    times = stseries.index.get_level_values("t")
    offset = self._offset if self._offset is not None else infer_offset(times)
    places = stseries.index.get_level_values("places").unique()
    logger.debug(
        "%s: computing %d lags for %d places", type(self).__name__, self.lags, len(places)
    )
    frames = []
    for place in places:
        ts = stseries.xs(place, level="places").sort_index()
        ts = self.apply_ts_decomposition(ts)
        ts.loc[ts.index[-1] + offset] = None  # next period, to be forecast
        f, _ = self.make_lag_df(ts)
        f["places"] = place
        frames.append(f.set_index("places", append=True))
    X = pd.concat(frames)
    X.index.names = ["t", "places"]
    return X.sort_index()

AR

AR(lags, tfreq=None)

Bases: TimeSeriesFeatures

Autoregressive features: lags of the raw series.

Source code in src/predspot/feature_engineering.py

def __init__(self, lags, tfreq=None):
    if not isinstance(lags, int) or lags < 2:
        raise ValueError("`lags` must be an integer greater than 1.")
    self.lags = lags
    self.tfreq = tfreq
    self._offset = tfreq_offset(tfreq) if tfreq is not None else None

Diff

Diff(lags, tfreq=None)

Bases: TimeSeriesFeatures

Lags of the first difference of the series.

Source code in src/predspot/feature_engineering.py

def __init__(self, lags, tfreq=None):
    if not isinstance(lags, int) or lags < 2:
        raise ValueError("`lags` must be an integer greater than 1.")
    self.lags = lags
    self.tfreq = tfreq
    self._offset = tfreq_offset(tfreq) if tfreq is not None else None

Seasonality

Seasonality(lags, tfreq=None)

Bases: _STLFeatures

Lags of the seasonal component of an STL decomposition (period = lags).

Source code in src/predspot/feature_engineering.py

def __init__(self, lags, tfreq=None):
    if not isinstance(lags, int) or lags < 2:
        raise ValueError("`lags` must be an integer greater than 1.")
    self.lags = lags
    self.tfreq = tfreq
    self._offset = tfreq_offset(tfreq) if tfreq is not None else None

Trend

Trend(lags, tfreq=None)

Bases: _STLFeatures

Lags of the trend component of an STL decomposition (period = lags).

Source code in src/predspot/feature_engineering.py

def __init__(self, lags, tfreq=None):
    if not isinstance(lags, int) or lags < 2:
        raise ValueError("`lags` must be an integer greater than 1.")
    self.lags = lags
    self.tfreq = tfreq
    self._offset = tfreq_offset(tfreq) if tfreq is not None else None

FeatureScaling

FeatureScaling(estimator)

Bases: TransformerMixin, BaseEstimator

Wrap a scikit-learn scaler so that it returns DataFrames.

Parameters:

Name	Type	Description	Default
estimator	TransformerMixin	Any scikit-learn transformer (e.g. QuantileTransformer).	required

Source code in src/predspot/feature_engineering.py

def __init__(self, estimator):
    self.estimator = estimator

infer_offset

infer_offset(time_index)

Infer the pandas.DateOffset between consecutive periods.

Parameters:

Name	Type	Description	Default
time_index	DatetimeIndex	Unique, sorted period labels.	required

Returns:

Type	Description
DateOffset	The offset separating consecutive periods.

Raises:

Type	Description
ValueError	If the frequency cannot be inferred (e.g. fewer than 3 periods); pass tfreq explicitly in that case.

Source code in src/predspot/feature_engineering.py

def infer_offset(time_index):
    """
    Infer the ``pandas.DateOffset`` between consecutive periods.

    Args:
        time_index (pandas.DatetimeIndex): Unique, sorted period labels.

    Returns:
        pandas.DateOffset: The offset separating consecutive periods.

    Raises:
        ValueError: If the frequency cannot be inferred (e.g. fewer than 3
            periods); pass ``tfreq`` explicitly in that case.
    """
    time_index = pd.DatetimeIndex(time_index).unique().sort_values()
    freq = pd.infer_freq(time_index) if len(time_index) >= 3 else None
    if freq is None:
        raise ValueError(
            "Could not infer the time frequency of the series; pass `tfreq` explicitly."
        )
    return pd.tseries.frequencies.to_offset(freq)