InterHub Dataset

Overview

Driving interaction is one of the most complex and critical areas of autonomous-driving research. Dense interaction events are scarce and inconsistently defined in existing naturalistic-driving datasets, limiting exploration of complex scenarios. InterHub addresses this issue by combining Waymo, Lyft, nuPlan, and INTERACTION and mining their interaction events to create a dense driving-interaction dataset. Unified data interfaces and formal methods enable dense coverage and quantitative analysis of interaction scenarios. Experiments show that InterHub provides more challenging tests for trajectory prediction and planning, revealing limitations of existing algorithms in highly interactive scenarios. A user-friendly toolkit also extends InterHub to other public or private data. As the first comprehensive dataset for dense driving interaction, InterHub provides strong tools and data support for advancing autonomous-driving interaction research.

ArXiv：https://arxiv.org/abs/2411.18302

Project Link:https://github.com/zxc-tju/InterHub

Motivation and Challenges

Driving interaction is one of the most complex and critical areas of autonomous-driving research. Dense interaction events are scarce and inconsistently defined in existing naturalistic-driving datasets, limiting exploration of complex scenarios. InterHub addresses this issue by combining Waymo, Lyft, nuPlan, and INTERACTION and mining their interaction events to create a dense driving-interaction dataset. Unified data interfaces and formal methods enable dense coverage and quantitative analysis of interaction scenarios. Experiments show that InterHub provides more challenging tests for trajectory prediction and planning, revealing limitations of existing algorithms in highly interactive scenarios. A user-friendly toolkit also extends InterHub to other public or private data. As the first comprehensive dataset for dense driving interaction, InterHub provides strong tools and data support for advancing autonomous-driving interaction research.

Breakthrough: InterHub

To address these issues, this work develops the InterHub dataset, which integrates multiple public naturalistic-driving datasets and extracts interaction events to provide a unified, comprehensive data foundation. The main contributions are:

  1. A unified dense driving-interaction dataset:Dense driving-interaction events are extracted from widely used public naturalistic-driving datasets, unified, analyzed, and categorized to provide InterHub, a user-friendly data foundation for driving-interaction research.
  2. A quantitative, easy-to-understand definition of driving interaction:Formal methods clearly and quantitatively define driving-interaction events, while an open-source toolkit supports interdisciplinary interaction descriptions and user-side extraction of multi-agent interaction events.
  3. Validation of high-challenge interaction scenarios:Experiments demonstrate the challenge of high-interaction-intensity scenarios in InterHub, which are important for validating performance on key autonomous-driving tasks such as trajectory prediction and motion planning.

Figure 1 presents the overall research. Based on the NVIDIA-developed trajdata toolkit,multi-source driving records are standardized into a unified format. Formal methods extract multi-agent interaction events and build a dense interaction dataset. Dataset interaction characteristics are analyzed, semantic labels for scenario interaction features are provided, and the data are applied to key autonomous-driving tasks.

Technical Highlights：Formalized Interaction-Event Identification

This method addresses diverse, sparsely distributed driving-interaction events by describing and quantifying them through formal methods, improving identification accuracy and consistency.

Driving interaction is defined asa situation in which at least two road users affect one another because of potential spatial and temporal conflict, with three elements:

  • Number of interacting entities: involves at least two participants.
  • Spatiotemporal conflict: entities may occupy the same spatial region in the future.
  • Mutual influence: participants take actions to avoid collisions, altering each other’s behavior.

This semantic description creates a readable interaction scenario, conveying that interaction centers on the effort required to resolve potential conflict. To balance readability and mathematical precision, it is converted into a quantitative expression. Figure 2 illustrates the main idea. Interaction intensity is measured by the minimum sum of absolute accelerations or decelerations (MSAA) required to resolve potential conflict; higher MSAA indicates a stronger impact on participants.

The above quantitative method is used to formally define driving-interaction scenarios (Figure 3). See the original paper for details.

Data Structure

The figure below shows the overall InterHub information flow. The dataset provides three types of information:

  • Index: scenario ID, participant ID, and interaction start and end times;
  • Interaction metrics: interaction intensity (MSAA), post-encroachment time (PET), and more;
  • Scenario features: path relationships (such as parallel or crossing) and priority information.

Key information can index and locate interaction-event segments in raw scenarios, yielding standardized trajectory and map information.

Related Example: Interaction Scenarios

Figure 5 shows key frames of a driving-interaction event in InterHub and changes in interaction intensity.

Related Example: Data Statistics and Interaction Analysis

A total of interaction events were extracted from the Waymo, Lyft, nuPlan, and INTERACTION datasets 65,399 interaction events, substantially more than in any single dataset. Key statistics are:

  • Interaction intensity: mean intensity is 1.76 m/s²;
  • Interaction duration: most interactions last 0.25–3.5 seconds;
  • Participants: multi-agent interaction events involve 3.2 vehicles on average, with up to 7 vehicles.

Analysis of driving-interaction characteristics across datasets yields the following findings:

  • Sparse interaction scenarios: the proportion of valid interaction events is low—about 3% for two-agent scenarios and only 1% for multi-agent scenarios—and this sparsity is consistent across datasets.
  • Interaction participants: nuPlan and INTERACTION have more complex multi-agent interactions, while Waymo and Lyft are less complex and involve at most five vehicles.
  • Interaction Duration: most valid interactions last 0.25–3.5 seconds, with an average duration of about 0.6 seconds. nuPlan interactions are shorter, and INTERACTION events are shorter still because of clip-length constraints.
  • Interaction intensity: most interaction intensities are below 3 m/s², while Waymo contains more high-intensity interaction scenarios.
  • Safety evaluation: post-encroachment time (PET) evaluates interaction safety. Even without additional safety filtering, the data capture hazardous non-interaction scenarios, showing that InterHub can provide critical support for autonomous-driving validation.

Related Example: Key Autonomous-Driving Tasks

Applications to trajectory prediction and motion planning demonstrate the dense interaction dataset’s potential to advance autonomous-driving evaluation and development. Experiments show that existing prediction and planning models perform substantially worse in high-density interaction scenarios:

  • Trajectory prediction: for example, the minimum average displacement error (minADE) of UQnet and MTR on InterHub increases by 34.65% and 22.67%, respectively, relative to the original datasets.
  • Motion planning:Based on theOnSite Autonomous Driving Algorithm Challenge, held from 2023 to 2024, we evaluated the average performance scores of the top ten planning algorithms across more than 700 interaction scenarios. As interaction intensity increases, average performance scores decline significantly, validating InterHub’s ability to provide more challenging test scenarios.

Summary and Outlook

Naturalistic-driving datasets are essential resources for empirical research on autonomous vehicles and play a key role in recent advances. This work focuses on a specific but highly relevant subset: interactive driving scenarios.InterHub frameworkprovides a formal method for identifying and quantifying driving interactions and produces a dense driving-interaction event dataset. It also offers toolkits for data extraction, retrieval, visualization, and cross-dataset application, advancing holistic interaction analysis.

The first key step in solving any problem is accurate identification. This dataset addresses existing gaps in autonomous-driving research through:

  • Easy cross-dataset use: the InterHub framework provides standardized tools that improve data reproducibility, facilitate comparative analysis across datasets, and increase the generality and reliability of autonomous-driving interaction research.
  • Interaction challenges in autonomous-driving tasks: high-risk driving scenarios are rare, yet discovering them is vital for testing autonomous-driving systems. Validation under highly interactive conditions helps address rare but critical long-tail challenges.
  • Potential dataset misuse: the INTERACTION dataset is limited in scale, while larger datasets such as WOMD provide more sparse interaction data. The InterHub toolkit helps researchers combine dense and sparse data for deeper analysis.
  • Recommendations for future dataset development: autonomous-driving-perspective data are valuable for studying interactions between autonomous and human-driven vehicles, but a lack of purely human-interaction scenarios limits research. Inconsistent map formats hinder cross-dataset research; unified map formats can improve efficiency.
  • From driving automation to future transportation systems: InterHub creates new opportunities for traffic management, supporting study of AV integration with mixed traffic systems and its effects. Understanding AV-specific interactions is vital for the future of transportation ecosystems.