Thank you for your careful reading of Appendix D.2 and for framing the evaluation-cell filtering boundary as a clear, constructive question. We address each part of your inquiry below.
What exactly causes rejection before an affordance evaluation cell is created? ... Please distinguish invalid initializations, rejected action candidates within a task, and entire valid initializations for which the dock permits no feasible action.
To directly answer your question: only physically invalid object initializations (Category 1) can be rejected before cell creation; a valid object initialization is never omitted even if the current dock admits no feasible manipulation target (Category 3).
Specifically, the three categories you highlighted are distinguished and handled as follows:
If the last category can be excluded, how is it accounted for when comparing candidate docks? The frequency of such exclusions and their effect on the selected endpoints would help determine whether the scores remain comparable across docks.
Because Category 3 cases are not excluded prior to cell creation, all candidate docks for a given task are evaluated over the exact same set of valid initial object configurations. To provide quantitative transparency on how frequently each outcome occurs, we audited all candidate dock evaluations across our training set:
As these statistics show, Category 3 accounts for the majority of failed trials (67.69%), and including them in the denominator is essential for properly penalizing unsuitable docks.
We have also added visual examples of Category 2 and Category 3 failures at
Thank you for your thoughtful review and for suggesting concrete experiments to further test our pipeline. We have conducted additional closed-loop, ablation, correlation, and cross-policy experiments to address each of your points below.
Questionable co-evolution claim. The manipulation WAM remains frozen after post-training, while only the navigation VLM is further fine-tuned. ... The authors should better justify the term co-evolution.
To validate the closed-loop co-evolution between navigation and manipulation, we ran inference with the trained VLN policy in both simulation and real-world scenes. Using the actual stopping poses reached by the VLN policy as the initial base distribution, we re-collected manipulation training data and post-trained the WAM. We evaluated this closed-loop WAM on Nav-N-Manip in both simulation and the real world:
Post-training the WAM on rollouts from actual VLN endpoints adapts the manipulation policy to upstream navigation stopping characteristics, improving end-to-end success rates from 30% to 52% in simulation and from 17% to 30% in the real world.
| Training Setting | Sim SR@E2E (100 trials) | Real SR@E2E (30 trials) |
|---|---|---|
| Original WAM | 30% | 17% |
| Fine-tuned WAM | 52% | 30% |
Real-world · Overlap first: multiple attempts superimposed. Use the arrows to browse individual trials.
Insufficient validation of the three-stage pipeline. As shown in Table 3, RFT provides only a marginal improvement in end-to-end success rate (31.01% to 31.79%). A Co-Tuning + RLAIF baseline without RFT is needed to demonstrate the necessity of all three stages.
We added the Co-Tuning (MTC) + RLAIF baseline (skipping the RFT stage) and compared it against our original settings:
Skipping RFT (MTC + RLAIF) yields end-to-end success rate of 43.02%, lower than the full three-stage pipeline (52.13%). By filtering high-quality trajectories, RFT provides a stable behavioral prior that prevents performance degradation caused by overly large exploration spaces during direct RLAIF fine-tuning.
SR@3 | SR@1.5 | SR@1.0 | SR@0.6 | NE | SR@Manip | SR@E2E | |
MTC | 80.2 | 77.1 | 69.8 | 52.7 | 1.81 | 39.34 | 31.01 |
MTC + RLAIF | 84.5 | 80.0 | 78.5 | 64.2 | 1.46 | 51.16 | 43.02 |
MTC + RFT + RLAIF | 89.5 | 84.1 | 80.4 | 71.7 | 1.15 | 56.87 | 52.13 |
Potential mismatch between offline affordance labels and WAM capabilities. ... The authors should compare both at the same candidate base poses and report the correlation between offline affordance scores and actual WAM success rates.
Because the WAM is trained on expert trajectories provided by the Dexonomy-based oracle manipulation policy, their capabilities are inherently aligned. We evaluated both the Dexonomy oracle and the learned WAM across repeated trials at the same candidate navigation endpoints in simulation:
The Pearson correlation coefficient between the offline Dexonomy affordance scores and actual WAM grasp success rates is r = 0.9987 (Spearman rank correlation ρ = 1.00), showing that the offline labels match the actual capabilities of the learned policy.
| Candidate Endpoint | Oracle (Dexonomy) SR | Learned WAM SR |
|---|---|---|
| A | 100% (20/20) | 90% (18/20) |
| B | 55% (11/20) | 45% (9/20) |
| C | 0% (0/20) | 0% (0/20) |
| D | 0% (0/20) | 0% (0/20) |
Limited generalization across manipulation policies. ... Cross-WAM evaluation would strengthen the generalization claims.
In the same real-world shelf scene, we evaluated 3 independently trained WAMs—each targeting a different task and shelf level—directly from the stopping poses produced by our navigation policy:
The stopping poses produced by our navigation policy achieve consistent grasp success across all 3 independently trained WAMs, showing that the navigation model learns generalizable manipulation-aware spatial priors.
| Navigation Method | Real Task 1 SR@E2E (10 trials) | Real Task 2 SR@E2E (10 trials) | Real Task 3 SR@E2E (10 trials) |
|---|---|---|---|
| UniNaVid | 0% | 0% | 0% |
| Ours | 30% | 20% | 40% |
Real-world · Overlap first, followed by individual trials. Overlap superimposes multiple attempts.
Thank you for your time and helpful feedback on our method, figures, and experimental details. We have revised the manuscript and conducted additional experiments to address your comments below.
1. This method highly relies on simulator.
Following standard VLN training paradigms (Uni-NaVid [1], NavFoM [2], InternVLA-N1 [3]), training in simulation enables safe, large-scale exploration with strong foundational generalization. To ensure our method does not rely on a single simulator, we implemented three designs:
All real-world policies were trained strictly in simulation without overfitting to the real test environment, achieving high end-to-end success rates on physical hardware.
Both two finetuning process rely on resetting the environment to specific state and obtain different trajectories.
We apologize for the inaccurate phrasing in our original manuscript (specifically "We then reset the simulator to the same task initialization..." in Section 3.5, as well as the descriptions in Section 3.4 and Algorithm 1), which caused this confusion. In our actual implementation, neither RFT nor RLAIF requires resetting the environment to specific states to obtain different trajectories. Both stages rely purely on clean on-policy trajectories without state resets. Compared to SFT on static offline data, training and RL fine-tuning on reset-free on-policy data allow the policy to build an intrinsic understanding of the environment without state priors, addressing counterfactual inference and distribution shift more effectively. We have corrected these descriptions in Sections 3.4, 3.5, and Algorithm 1 of the revised manuscript.
2. The paper emphasize on bidirectional training. However, the manipulation policy is fixed after the first stage training. Only the navigation policy is improved with a frozen manipulation policy.
To complete the bidirectional loop, after VLN training we ran inference with the trained VLN policy in both simulation and real-world scenes, used the resulting navigation endpoints to re-collect WAM training data, and conducted a second round of post-training on the WAM:
Fine-tuning the WAM on rollouts from the updated VLN endpoints increases end-to-end success rates from 30% to 52% in simulation and from 17% to 30% in the real world, completing the bidirectional training loop.
| Training Setting | Sim SR@E2E (100 trials) | Real SR@E2E (30 trials) |
|---|---|---|
| Original WAM | 30% | 17% |
| Fine-tuned WAM | 52% | 30% |
Real-world · Overlap first: multiple attempts superimposed. Use the arrows to browse individual trials.
1. The presentation is bad. Figure 2 and figure 3 are purely AI-generated without any further polishment. Please at least make sure the letters in each word are equally spacing. The names for some modules are not consistent, such as Nav-N-Manip/NAV-N-MANIP.
We have redrawn and replaced Figure 2 and Figure 3 to fix the character spacing issues, and standardized all module names and notations across the main text in the revised manuscript.
2. The experiments section lacks of lots of details. The benchmark details, the training and evaluation setting are all very coarse.
We have added explicit pointers in Section 4.1 and the Reproducibility Statement, and expanded the appendix with full experimental details:
References
[1] Zhang et al., "Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks," RSS, 2025.
[2] Zhang et al., "Embodied Navigation Foundation Model," ICLR, 2026.
[3] InternVLA-N1 Team, "InternVLA-N1: An Open Dual-System Navigation Foundation Model with Learned Latent Plans," arXiv:2509.10838 / arXiv:2512.08186, 2025.
Results from inference in simulation.
Dexterous mobile manipulation is essential for robots to perform useful tasks in complex, unstructured environments. However, existing systems face two challenges. First, navigation remains too coarse for downstream manipulation, which often requires a precise base position and approach orientation rather than merely reaching a meter-scale goal region. Second, navigation and manipulation are commonly trained in isolation, lacking a feedback loop that enables the two capabilities to co-evolve. These limitations prevent current systems from reliably executing long-horizon dexterous mobile-manipulation tasks in real-world environments. To address them, we introduce Nav4Manip, a bidirectionally coupled co-evolution system for long-horizon dexterous mobile manipulation. It comprises two complementary architectures: Nav4Manip-N learns long-horizon navigation and refines its policy through manipulation feedback, while Nav4Manip-M trains a world-action model (WAM) for downstream dexterous manipulation. We train Nav4Manip-N in three stages: multi-task co-tuning learns navigation behavior and manipulation-aware affordance prediction; rejection sampling fine-tuning (RFT) rejects low-quality policy-generated paths and fine-tunes the model on retained trajectories; and reinforcement learning from AI feedback (RLAIF) uses counterfactual preferences to further improve navigation decisions. To support this framework, we establish the Nav-N-Manip benchmark and construct Nav4Manip-Pairs from benchmark rollouts, providing large-scale paired navigation–manipulation trajectories and affordance supervision. Extensive experiments on public vision-language navigation (VLN) benchmarks and Nav-N-Manip demonstrate state-of-the-art performance, while real-world evaluations show that Nav4Manip completes diverse dexterous mobile-manipulation tasks.
Real-world evaluations show that Nav4Manip completes diverse dexterous mobile-manipulation tasks.
Real-world demonstration
Real-world demonstration
Real-world demonstration
Navigation and dexterous manipulation in simulation (4× speed).
First-person navigation rollouts, including instruction following and object-goal navigation.
First-person manipulation rollouts in simulation.
Navigation rollouts on R2R.
Prompt: Walk until you reach the brown dresser with the vase on top of it, turn right, take a left and walk through the first doorway, stop immediatly after the doorway.
Prompt: Exit the laundry room and turn left. Walk towards the curio cabinet and turn left again. Enter the bedroom and walk past the dresser. Exit through the door to the left and wait near the sink.
Prompt: Go straight until you get to the frame on the wall of the teddy bear with the balloons saying "welcome" then turn right and go straight until you get to the room and wait in the entrance.
Navigation rollouts on RxR.
Prompt: We're facing some armchairs and a couch. Let's turn to our right and go straight. And go down the stairs. On this floor we should be able to see a white open door. Go through the open door. We are in a bathroom and we can stop here.
Prompt: We're going to start by looking at some paintings and a white sofa. We're going to turn out heads slightly to the right, and see a clock. Let's walk towards the clock. As we're now standing in front of the clock, let's turn our heads slightly to the right, and walk towards the small pathway between the kitchen and the living room. Now, as we're standing between the living room and the kitchen area, we're going to walk to the large cabinets that are in the corner of the room below some paintings. As we're standing in front of it, we're going to turn our heads to the right, all the way, and we're going to walk towards the small chair with some cushions that's in front of us. We should now be standing in the living room in front of the mat. We're going to turn to the right and walk towards the middle of the entire room: the kitchen, the living room, and the dining area. This will be our destination.
Prompt: You are facing a fireplace. You are going to turn around and you are going to hop over the left side of the couch going towards the picture on the wall. Once there you are going to take a step towards the corner of the island on your right. And then you are going to turn left and go thru the walkway. You are going to move forward down this hallway. You are going to pass a window on your right. And then you are going to turn into the last room on your left, you will see a washer and dryer. You are going to step onto the rug in there and then you are done.
Navigation rollouts on VLNVerse.
Prompt: Walk forward through the doorway, out of the bathroom area and past the washing machine on your left and the double vanity on your right. Continue straight through the narrow passage lined with white cabinets on both your left and right sides. Turn right at the end of the passage, entering a long hallway with the cabinets now on your left. Walk straight down the hallway, passing a large framed picture on your right. Continue straight, passing an ornate mirror on your right, and then enter the open area, keeping the white cabinet with glass doors on your right. Turn left to enter the living room, passing a white console on your left, and stop in front of the TV console with the large window and two chairs directly ahead of you.
Prompt: Walk forward into the hallway, leaving the bedroom area on your left. Walk straight down the hallway. Turn left into the doorway to enter the closet. Walk straight through the closet, with the dark shelves and cabinets on your right. At the end of the closet, turn left to enter the bathroom. Walk straight into the bathroom, passing the toilet and vanity on your right, and stop inside the shower stall in front of the window.
Prompt: Enter the living room through the doorway, keeping the sofa on your left and the TV on your right. Turn right into the adjacent hallway. Walk straight down the hallway with patterned wallpaper, then turn right at the end to enter the bedroom. Walk past the bed on your left and stop in front of the window.