paper / johncoogan / Jun 6
Current Vision-Language-Action (VLA) models fail on long-horizon mobile manipulation tasks due to compounding early errors and difficulty maintaining goal progress across spatially distributed subtasks — and finetuning on more data does not resolve this. MPVI (Motion Planner/VLA Interleaving) addresses this by integrating classical model-based motion planning with VLAs at inference time, using open-vocabulary object detection, frontier exploration, and VLM-based completion checking with proprioceptive triggers to manage module switching. Evaluated on the BEHAVIOR-1K benchmark, MPVI achieves a 113% improvement in task progress over a top end-to-end VLA baseline without any additional training.
vision-language-actionroboticsmotion-planningmobile-manipulationlong-horizon-tasksembodied-aivla-robustness
“MPVI achieves a 113% improvement in task progress over a top end-to-end VLA baseline on the BEHAVIOR-1K benchmark.”
paper / johncoogan / Jun 6
The paper introduces MA-AC-MPC, a hybrid framework that couples multi-agent actor-critic reinforcement learning with Model Predictive Control (MPC) to address the complementary weaknesses of each approach: MARL handles long-horizon cooperative policy learning from sparse rewards, while MPC enforces short-horizon dynamic feasibility and safety constraints. The combined algorithm is validated in two hardware scenarios — a pursuit-evasion task and a heterogeneous drone/rover cooperative landing task — where it significantly outperforms a comparable MARL-with-MLP baseline. The 100% vs. 60% hardware success rate gap between MA-AC-MPC and MA-AC-MLP on the landing task provides a concrete, quantified advantage for the hybrid architecture in real-world deployment.
multi-agent-reinforcement-learningmodel-predictive-controlcooperative-roboticsactor-criticpursuit-evasionautonomous-systemssafe-control
“MA-AC-MPC achieves a 100% success rate on cooperative drone-rover landing in hardware, compared to 60% for the MA-AC-MLP baseline.”
paper / johncoogan / May 13 / failed