laurel.pipelines.describe_dwells package

Submodules

laurel.pipelines.describe_dwells.nodes module

Kedro pipeline nodes for the describe_dwells pipeline (Model Module 2 — Augment dwell data).

Transforms the raw telematics trip records into a clean, shift-annotated DwellSet that is ready for the electrification simulation. This pipeline implements the second part of Model Module 2 (Augment Dwell Data): coalescing brief return trips that break a dwell without meaningfully changing the vehicle’s location, and marking the start of each new FMCSA-compliant driver shift.

Pipeline overview

1. format_trips_columns — Parses timestamps, converts H3 hex strings to integers, recodes the vehicle-ID column as a compact integer category, and optionally persists the Dask DataFrame in memory.

2. calc_derived_trip_cols — Computes trip duration in hours from start and end timestamps.

3. create_dwells — Converts the trip-oriented DataFrame into a DwellSet (one row per dwell event) using DwellSet.from_trips.

4. coalesce_interrupted_dwells — Merges dwells that are separated by a short “circle trip” (same origin and destination, below distance and duration thresholds) into a single, longer dwell.

5. mark_vehicle_shifts — Marks dwell events that start a new driver shift (dwell duration >= min_refresh_hrs, per FMCSA HOS regulations) and assigns a monotonically increasing shift ID within each vehicle.

6. calc_rolling_dwell_ratios — Computes a rolling time-window dwell ratio for each (vehicle, location) pair as the fraction of the vehicle’s total dwell time spent at that location.

7. map_location_groups — Joins freight-activity-class labels from the describe_locations pipeline onto each dwell row.

Key design decisions

• Coalescing via ``accum_masked``: Rather than a forward-fill or group-join, coalescing is implemented by propagating the latest end-time of the interrupted dwell sequence backwards through the masked rows and then dropping the non-masked rows. This avoids a sort-dependent join and is compatible with Dask partitioned DataFrames.

• Reset column neutralisation: The DwellSet.reset column has special semantics inside accum_masked. Setting it to False before calling accum_masked and restoring it afterwards prevents the coalescing logic from treating existing shift boundaries as accumulation barriers.

• FMCSA 6.9-hour threshold: The min_refresh_hrs parameter encodes the FMCSA 8-hour off-duty rest requirement. A slightly lower threshold of 6.9 hours is used to accommodate GPS rounding in the telematics data.

• Deprecated rolling-ratio functions: calc_inter_visit_stats, calc_inter_visit_times, describe_veh_loc_pairs, filter_substantial_dwells, and cluster_veh_loc_pairs are preserved for potential future use but are not connected to the active pipeline as of 2025-10-20.

References

Passow, F., & Rajagopal, R. (2026). Identifying indicators to inform proactive substation upgrades for charging electric heavy-duty trucks. Applied Energy.

Federal Motor Carrier Safety Administration. Hours of Service Regulations, 49 CFR Part 395.

laurel.pipelines.describe_dwells.nodes.calc_derived_trip_cols(trips, params)

Compute trip duration in hours from start and end timestamp columns.

Parameters:

• trips (DataFrame) – Trips DataFrame with UTC-aware timestamp columns.

• params (dict) –

  Pipeline parameters dict with keys:

  • time_cols (dict): sub-keys trip_start and trip_end naming the timestamp columns.

  • persist (bool): if True, trigger Dask computation and pin the result in distributed memory.

Return type:

DataFrame

Returns:

The trips DataFrame with a new trip_hrs column (float).

laurel.pipelines.describe_dwells.nodes.calc_inter_visit_stats(dw)

Compute inter-visit time and mileage for each (vehicle, location) pair. :rtype: DwellSet

Deprecated since version Not: connected to the active pipeline as of 2025-10-20. Preserved for potential future use.

laurel.pipelines.describe_dwells.nodes.calc_inter_visit_times(grp, hex_col, end_col, start_col)

Calculate inter-visit times for a single vehicle’s dwells, sorted by time.

Return type:

DataFrame

laurel.pipelines.describe_dwells.nodes.calc_rolling_dwell_ratios(dw, params)

Compute each dwell’s maximum rolling location-specific dwell ratio.

For each dwell, the rolling ratio is the fraction of the vehicle’s total dwell time (within a rolling time window) that was spent at the same hexagon. The maximum of this ratio across all windows in the observation period is recorded for each (vehicle, location) pair and then mapped back onto individual dwell rows.

Dispatches to _calc_rolling_dwell_ratios_part via map_partitions for Dask-backed DwellSet instances, or calls it directly for pandas.

Parameters:

• dw (DwellSet) – Shift-annotated DwellSet (output of mark_vehicle_shifts).

• params (dict) –

  Pipeline parameters dict with keys:

  • output_ratio_col (str): name of the output rolling-ratio column.

  • rolling_kwargs (dict): keyword arguments forwarded to pd.Series.rolling (e.g., window, min_periods).

Return type:

DwellSet

Returns:

The DwellSet with params["output_ratio_col"] added to dw.data.

laurel.pipelines.describe_dwells.nodes.cluster_veh_loc_pairs(veh_locs, params)

Cluster (vehicle, location) pairs using HDBSCAN on log-scaled, standardised features.

Deprecated since version Not: connected to the active pipeline as of 2025-10-20. Preserved for potential future use.

Selects params["feature_cols"], optionally subsamples, applies log10 transformation (skipping ratio columns), standardises with StandardScaler, and fits HDBSCAN with a minimum cluster size of n_obs / min_cluster_size_denom.

Parameters:

• veh_locs (DataFrame) – (vehicle, location) summary DataFrame (output of describe_veh_loc_pairs).

• params (dict) –

  Pipeline parameters dict with keys:

  • feature_cols (list[str]): feature columns for clustering.

  • sample (dict): active (bool), n (int), seed (int) for optional subsampling.

  • min_cluster_size_denom (int): denominator for deriving HDBSCAN’s min_cluster_size from n_obs.

  • cluster_col (str): output cluster-label column name.

Return type:

DataFrame

Returns:

The input DataFrame with a categorical cluster_col column added.

laurel.pipelines.describe_dwells.nodes.coalesce_interrupted_dwells(dw, params)

Merge dwells that are split by short circle trips into single, continuous dwells.

A “circle trip” is a trip whose origin and destination hexagon are identical and whose distance and duration both fall below configurable thresholds. Such trips most likely represent GPS noise or brief vehicle movements within a depot and should not break the surrounding dwell into two separate events.

The algorithm proceeds as follows:

1. Mark non-circle trips with is_not_short_circle = True.

2. Temporarily set dw.reset to False for all rows to prevent shift boundaries from interrupting the accumulation.

3. Use DwellSet.accum_masked with CumAggFunc.MAX to propagate the latest end time backwards through each circle-trip gap (reverse=True), so that the dwell preceding the circle trip absorbs the end time of the dwell following it.

4. Drop rows marked as circle trips.

5. Restore dw.reset from the saved copy and rename accumulated columns back to their original names.

The distances and durations of the dropped circle trips are not accumulated into the surviving dwell (they are treated as negligible).

Parameters:

• dw (DwellSet) – Input DwellSet freshly created from trips.

• params (dict) –

  Pipeline parameters dict with keys:

  • max_short_dist_miles (float): maximum trip distance (miles) for a trip to qualify as a circle trip.

  • max_short_dur_hrs (float): maximum trip duration (hours) for a trip to qualify as a circle trip.

Return type:

DwellSet

Returns:

The DwellSet with circle trips coalesced; dw.data has the same schema as the input but with fewer rows.

laurel.pipelines.describe_dwells.nodes.create_dwells(trips, params)

Convert a trip-oriented DataFrame into a DwellSet of dwell events.

A dwell event represents the period during which a vehicle is stationary at a location between two consecutive trips. DwellSet.from_trips infers each dwell’s start time, end time, and hexagon from the surrounding trip records.

An optional debug subsample limits computation to the first n rows of the Dask DataFrame for rapid iteration. If load_into_memory is set, the entire Dask DataFrame is materialised before conversion, which can be faster when working on smaller datasets.

Parameters:

• trips (DataFrame) – Trips Dask DataFrame (output of calc_derived_trip_cols).

• params (dict) –

  Pipeline parameters dict with keys:

  • debug_subsample (dict): active (bool) and n (int) to limit the input to the first n rows.

  • load_into_memory (bool): if True, call .compute() before conversion.

  • drop_cols (list[str]): trip columns to remove before conversion.

  • col_renamer (dict[str, str]): additional column renames to apply before conversion.

  • from_trips_cols (dict): column-name keyword arguments forwarded to DwellSet.from_trips (e.g., veh, hex, start, end).

  • verify_sorting (bool): if True, DwellSet.from_trips verifies that trips are sorted by (vehicle, time).

  • set_index_kwargs (dict): additional keyword arguments for setting the index in DwellSet.from_trips.

Return type:

DwellSet

Returns:

A DwellSet with one dwell record per stationary event, sorted by (vehicle, start time).

laurel.pipelines.describe_dwells.nodes.describe_veh_loc_pairs(dw)

Summarise each (vehicle, location) pair with visit counts, dwell hours, and inter-visit statistics. :rtype: DataFrame

Deprecated since version Not: connected to the active pipeline as of 2025-10-20. Preserved for potential future use.

laurel.pipelines.describe_dwells.nodes.filter_substantial_dwells(dw, params)

Drop dwells shorter than a duration threshold, accumulating trip stats across gaps. :rtype: DwellSet

Deprecated since version Not: connected to the active pipeline as of 2025-10-20. Preserved for potential future use.

laurel.pipelines.describe_dwells.nodes.format_trips_columns(trips, params)

Parse timestamps, encode H3 hex strings, and recode vehicle IDs in the raw trips DataFrame.

Categorising the vehicle-ID column before converting timestamp columns prevents a known Dask issue in which dd.to_datetime silently converts integer category codes to float64. H3 hexagon strings are converted to uint64 integers for memory efficiency and join performance.

Parameters:

• trips (DataFrame) – Raw trips Dask DataFrame as loaded from the 01_raw catalog.

• params (dict) –

  Pipeline parameters dict with keys:

  • category_columns (list[str]): columns to categorise before timestamp conversion.

  • time_columns (list[str]): columns to convert to UTC-aware datetime64.

  • h3_columns (list[str]): H3 hexagon string columns to convert to uint64.

  • col_renamer (dict[str, str]): mapping from raw to internal column names (applied after all other transformations).

  • veh_id_col (str): vehicle identifier column; category codes are cast to int64 after renaming.

  • persist (bool): if True, trigger Dask computation and pin the result in distributed memory.

Return type:

DataFrame

Returns:

A Dask DataFrame with parsed timestamps, integer hex IDs, compact integer vehicle IDs, and columns renamed to internal names.

laurel.pipelines.describe_dwells.nodes.map_location_groups(dw, hex_corresp, params)

Join freight-activity-class labels from the hex correspondence table onto dwells.

Maps the cluster/group label for each hexagon (computed by the describe_locations pipeline) onto the DwellSet rows by hexagon ID, via merge_dataframes_node. That helper preserves dw.data’s index (veh_id) across the merge and, for Dask-backed DwellSet instances, performs a partition-wise broadcast merge rather than Series.map(), which would otherwise embed the correspondence table as an unmanaged, unspillable graph literal and can stall workers under memory pressure.

Parameters:

• dw (DwellSet) – DwellSet with a hex-ID column.

• hex_corresp (DataFrame) – DataFrame indexed by hexagon ID with a location-group column.

• params (dict) –

  Pipeline parameters dict with keys:

  • location_group_col (str): name of the group column in hex_corresp and the output column added to dw.data.

  • missing_values (dict): sub-keys:

    • fill_missing (bool): whether to fill NaN group labels.

    • fill_value: value used to fill unmapped hexagons (e.g., "undeveloped").

Return type:

DwellSet

Returns:

The DwellSet with a new location-group column added to dw.data.

laurel.pipelines.describe_dwells.nodes.mark_vehicle_shifts(dw, params)

Identify driver shift boundaries and assign shift IDs.

A dwell whose duration meets or exceeds min_refresh_hrs is classified as a shift-refresh dwell (the driver’s mandatory off-duty rest period under FMCSA 49 CFR Part 395). The shift ID for the following dwell is incremented, so that the refresh dwell itself is the last event of the preceding shift. Shift IDs are vehicle-local (i.e., they restart at 0 for each vehicle).

Parameters:

• dw (DwellSet) – Coalesced DwellSet (output of coalesce_interrupted_dwells).

• params (dict) –

  Pipeline parameters dict with keys:

  • columns (dict): sub-keys:

    • dur (str): temporary column name for dwell duration.

    • refresh (str): boolean column name marking shift-refresh dwells.

    • shift_id (str): integer shift-ID column name.

  • min_refresh_hrs (float): minimum dwell duration in hours to classify as a shift refresh (typically 6.9 hours).

Return type:

DwellSet

Returns:

The DwellSet with refresh and shift_id columns added to dw.data.

laurel.pipelines.describe_dwells.pipeline module

Kedro pipeline definition for the describe_dwells pipeline.

Wires the nodes from laurel.pipelines.describe_dwells.nodes into a single Pipeline object. For full documentation of each node’s inputs, outputs, and algorithm, see laurel.pipelines.describe_dwells.nodes.

Sub-pipelines / tags

• format_trips — parses timestamps, converts H3 hex strings, and computes derived trip columns from the raw telematics trips.

• create_dwells / create_dwells_optional_stops — converts the trip DataFrame to a DwellSet; coalesces circle-trip interruptions; marks FMCSA driver-shift boundaries; computes rolling dwell ratios; and joins freight-activity-class labels from describe_locations.

To visualise the node graph interactively, run:

kedro viz run

then open http://localhost:4141 in a browser and select describe_dwells from the pipeline dropdown.

laurel.pipelines.describe_dwells.pipeline.create_pipeline(**kwargs)

Return type:

Pipeline

Module contents

This is a boilerplate pipeline ‘describe_dwells’ generated using Kedro 1.0.0

laurel.pipelines.describe_dwells.create_pipeline(**kwargs)

Return type:

Pipeline