Availability of Neural NDE and NADE (D2RL) implementations

[lee-scieninc]

Hello TeraSim Team,

I am writing to you from a Tier 4 partner organization. We have been actively adopting TeraSim for our internal development and simulation needs. First of all, I would like to express our appreciation for this excellent research and for open-sourcing this platform. It has been very insightful for our team.

I am currently attempting to reproduce the results presented in the TeraSim paper (arXiv:2503.03629v1). While exploring the codebase to understand the generative capabilities highlighted in the paper, I noticed some discrepancies between the methodology described and the current implementation in the main branch.

Specifically, I am looking for the implementation of Neural NDE and NADE with D2RL, but they appear to be missing or replaced by rule-based placeholders.

1. Neural NDE Implementation
   Paper Description: Section III.B.2 mentions that "NeuralNDE [2] proposed a generative framework..." and implies a model calibrated from large-scale naturalistic driving data.

Code Observation: In packages/terasim-nde-nade/terasim_nde_nade/vehicle/nde_decision_model.py, the NDEDecisionModel seems to inherit from IDMModel.

Lines 83-89 essentially return CommandType.DEFAULT, delegating control to SUMO's default behavior or the rule-based IDM, rather than a learned neural network model.

2. NADE with D2RL (Deep RL)
   Paper Description: Section III.B.4 states that "NADE leverages dense deep reinforcement learning (D2RL)" to dynamically regulate adversity probabilities based on the evolving traffic state.

Code Observation: In packages/terasim-nde-nade/terasim_nde_nade/envs/nade.py, I could not find an RL training loop or a policy network definition.

Instead, get_IS_prob (Lines 605-639) appears to implement Importance Sampling (IS) using fixed constants (e.g., IS_MAGNITUDE_DEFAULT = 20 at Line 30) rather than a learned policy.

Questions
To help us better understand the platform and reproduce the study's results, could you please clarify the following:

Code Availability: Are the Neural NDE model weights and the D2RL inference code available in a separate branch or a different repository that we might have missed?

Release Plan: If they are not currently public, is there a plan to release the training/inference code for NADE (D2RL) and Neural NDE in the near future?

Documentation: If these components are intended to remain proprietary or internal-only, would it be possible to update the README to clarify that the public repository currently supports the rule-based (IDM/SUMO) fallback? This would greatly assist researchers in understanding the scope of the open-source release.

Thank you again for your hard work on TeraSim. We look forward to your response.

Best regards,

[miracelplus]

Thank you very much for your detailed message and for the careful reading of the TeraSim paper and codebase. We truly appreciate your team’s interest in TeraSim and your active adoption of the platform in your internal development and simulation workflows.

We are happy to confirm that your understanding and observations are correct, and we would like to clarify the current status and our roadmap in more detail below.

---

1. Current Focus of Our Development

At present, our work on TeraSim is mainly concentrated on the following directions:

(1) Strengthening TeraSim–Autoware integration
We are actively enhancing the native integration between TeraSim and Autoware, including standardized testing environments, testing APIs, and closed-loop evaluation workflows to better support practical AV development and validation.

(2) Finalizing the NeuralNDE interface design
We do have an internally running version of NeuralNDE, calibrated from large-scale naturalistic driving data. However, before releasing it publicly, we are working on:

• Completing and cleaning up the training pipeline
• Unifying the model interface and configuration
• Ensuring the released version is reproducible and well-documented

Our intention is to open-source not only the inference code, but also the full training workflow, which requires additional time to properly prepare.

---

2. NADE and D2RL in the Open-Source Release

Regarding NADE with D2RL, your observation is also accurate.

In the current open-source version, we intentionally retain a rule-based / fixed-parameter implementation for stress testing. The main reason is the intrinsic nature of D2RL:

• The optimal D2RL parameters are task-dependent, varying across:

  • Different cities
  • Different traffic distributions
  • Different test horizons and evaluation objectives

• A single trained D2RL policy does not generalize reliably across all scenarios out of the box.

To ensure that TeraSim is immediately usable across diverse tasks, we therefore provide a default, fixed-parameter formulation (e.g., constant IS magnitude) that can run robustly without retraining.

That said, the training and inference code for D2RL itself is fully available and can be found here:

https://github.com/michigan-traffic-lab/Dense-Deep-Reinforcement-Learning

This repository uses the same D2RL formulation as described in the TeraSim paper. We are actively exploring better abstractions and interfaces to make D2RL more generalizable and plug-and-play within TeraSim in future releases.

---

3. Documentation and Transparency

Your suggestion regarding documentation is very helpful. We agree that it would be beneficial to explicitly clarify in the README that:

• The current public release supports rule-based (IDM / SUMO) fallback behavior
• NeuralNDE and fully trained D2RL-based NADE are not yet included in the main branch

We will incorporate this clarification to improve transparency and usability for the research community.

---

Once again, thank you for your thoughtful questions and for your interest in TeraSim. We truly value feedback from active users like your team, and we look forward to continued discussions and collaboration.

Please feel free to reach out if you have further questions.

Best regards,
TeraSim Team