GloVLA: Let Geometry Move and Local VLA Interact for Robust Object-Centric Manipulation in Unstructured Environments

Anonymous Authors

Abstract

Vision-language-action (VLA) models have shown promising generalization for language-conditioned robot manipulation, but deploying them in unstructured environments remains challenging. A single end-to-end VLA policy must simultaneously solve long-range transport of the end effector to task-relevant regions and short-horizon, contact-rich interaction upon arrival. This formulation is inefficient and brittle: small visual shifts, distractors, clutter, occlusions, or unfavorable initial gripper poses can push the policy outside the local state distribution in which it was trained. We introduce GloVLA, a hybrid framework that explicitly separates object-centric manipulation into two complementary regimes: a geometric transport controller moves the end-effector into interaction-centric handoff regions, and local VLA policies handle only the short-horizon interaction phases. GloVLA is model-agnostic and integrates with different VLA backbones with no additional demonstrations and no changes to the action space or success predicate. Experiments on standard LIBERO and LIBERO-Plus Object tasks, a newly introduced LIBERO-Challenge benchmark, and real-world unstructured settings show that GloVLA improves task success and substantially lowers VLA inference cost compared with full end-to-end VLA execution. On LIBERO-Challenge, full-trajectory GR00T N1.6 execution degrades to 20.9% average success while GloVLA retains 88.5%; on a physical UR10e, overall success improves from 35.6% to 90.0% while mean inference time is more than halved.

home
GloVLA on a real UR10e: a geometric transport controller (gray phases) moves the arm to object- and basket-centric handoff regions, then local VLA policies (orange phases) handle the contact-rich grasp and place. The badge tracks the current phase. Videos play at 3× speed.

Method

FullVLA baseline versus the proposed GloVLA: geometry for transport, VLA for contact
A full-trajectory VLA must stay visually stable over the entire episode, so failures occur before the robot ever reaches the contact phase. GloVLA factorizes the task: a geometric transport controller moves the end-effector into interaction-centric handoff regions, and a local VLA policy is invoked only inside the object-centric handoff region (dashed) where visual feedback and contact reasoning are genuinely needed.

For each LIBERO Object task, the original demonstrations are split into a grasp-sphere dataset around the target object and a place-sphere dataset around the basket, so the policy sees dense supervision near contact while the transport controller handles global transport. The controller emits the same 7D OSC-pose action format as the policy, and the episode is still judged by the official LIBERO success predicate. At rollout time execution is a four-mode switched state machine:

Home pose: arm at reset before transport
homeRobot at its reset pose; the target object is localized.
Transport controller drives the gripper toward the object (gray trace)
transport → objectController drives the gripper to the object handoff region.
Local grasp VLA closes on the cup (orange trace)
grasp VLAVLA closes the loop to grasp and lift the object.
Transport controller carries the object to the basket (gray trace)
transport → basketController carries the held object to the basket handoff region.
Local place VLA releases the object into the basket (orange trace)
place VLAVLA aligns and releases the object into the basket.
One real episode through the four-mode switched state machine. The gripper trace is colored by controller: gray segments are geometric transport, orange segments are the local VLA policies acting inside the handoff regions.

No extra data

Local policies train on the same demonstrations as the full-trajectory baseline — gains cannot come from extra supervision.

Same task definition

Action space, observations, and the official success predicate are unchanged — the task is not made easier.

Deterministic handoff

The controller–policy boundary is fixed object-centric offsets, not a learned component — it adds no new failure surface.

Backbone-agnostic

Any VLA that can be trained on the full task can be trained on the local phases instead.

Standard LIBERO Object Results

Even on the standard benchmark, where FullVLA is already strong, the factorization improves average success for every one of the four backbones — and reduces run-to-run variance (e.g. GR00T N1.6’s standard deviation drops from 6.0 to 1.9 points).

0 25 50 75 100 87.4 98.6 π₀ 97.8 99.6 π₀.₅ 88.8 98.4 GR00T N1.6 96.4 99.6 GR00T N1.7 FullVLA GloVLA (Ours)
Average closed-loop success rate (%) over the ten LIBERO Object tasks, 50 episodes per task.
Full per-task table (Table I)
LIBERO Object Task π0 π0.5 GR00T N1.6 GR00T N1.7
FullHybrid FullHybrid FullHybrid FullHybrid
Alphabet Soup94.0±12.0100.0±0.0100.0±0.0100.0±0.0100.0±0.0100.0±0.098.0±4.5100.0±0.0
BBQ Sauce82.0±4.590.0±0.0100.0±0.0100.0±0.060.0±15.890.0±8.994.0±5.598.0±4.5
Butter84.0±11.4100.0±0.096.0±5.5100.0±0.0100.0±0.0100.0±0.098.0±4.5100.0±0.0
Chocolate Pudding92.0±4.598.0±4.590.0±8.996.0±5.590.0±0.0100.0±0.094.0±5.598.0±4.5
Cream Cheese88.0±11.098.0±4.598.0±4.5100.0±0.060.0±14.1100.0±0.098.0±4.5100.0±0.0
Ketchup92.0±7.5100.0±0.0100.0±0.0100.0±0.0100.0±0.0100.0±0.0100.0±0.0100.0±0.0
Milk76.0±23.0100.0±0.0100.0±0.0100.0±0.096.0±5.5100.0±0.094.0±8.9100.0±0.0
Orange Juice74.0±15.2100.0±0.0100.0±0.0100.0±0.096.0±5.5100.0±0.0100.0±0.0100.0±0.0
Salad Dressing98.0±4.5100.0±0.0100.0±0.0100.0±0.094.0±12.098.0±4.5100.0±0.0100.0±0.0
Tomato Sauce94.0±5.5100.0±0.094.0±5.5100.0±0.092.0±7.596.0±5.588.0±13.0100.0±0.0
Average87.4±9.998.6±0.997.8±2.499.6±0.688.8±6.098.4±1.996.4±4.699.6±0.9

Robustness Challenge

A deterministic 50-scene LIBERO Object challenge set — five perturbation families (clutter, distraction, obstruction, visual shift, illumination) at three difficulty tiers, applied at evaluation time only. Official initial states and success predicates are unchanged, so the challenge isolates exactly one question: does a policy succeed through genuine interaction skill, or only by fitting an easy, unperturbed approach trajectory?

Ten example scenes from the LIBERO-Challenge benchmark, from single-perturbation easy settings to composed difficult settings
Example scenes from the 50-scene challenge set, rendered directly from the benchmark code. (a)–(e): easy single perturbations (clutter, distraction, illumination, obstruction, visual shift); (f), (g): medium settings with 2–3 simultaneous perturbations; (h)–(j): difficult settings with 4–5 combined perturbations.
20.9% → 88.5% all-scene success, FullVLA → GloVLA (+67.6 points)
+78.2 pp largest gap (distraction & medium compositions)
74.9% GloVLA retained on hard 4–5 perturbation scenes (FullVLA 1.5%)
0 25 50 75 100 31.4 8.3 1.5 93.8 86.5 74.9 Easy (30 scenes) Medium (10) Hard (10) FullVLA GloVLA (Ours)
GR00T N1.6 success rate (%) by difficulty tier on LIBERO-Challenge, 50 episodes per scene. FullVLA falls monotonically to 1.5%; GloVLA degrades gracefully, retaining 74.9% under 4–5 combined perturbations.
Clutter 23.7 94.5 Distraction 13.5 91.7 Obstruction 43.2 96.2 Visual shift 36.7 90.3 Illumination 39.7 96.2 Medium comp. 8.3 86.5 Hard comp. 1.5 74.9 FullVLA GloVLA (Ours)
GR00T N1.6 success rate (%) per perturbation family on LIBERO-Challenge (averaged over ten object tasks, 50 episodes per scene). The gain is broadly distributed across all five families and both composition tiers.
Per-task robustness table, all ten objects (Table III)
Object task Clutter Distraction Obstruction Visual shift Illumination
FullOurs FullOurs FullOurs FullOurs FullOurs
Alphabet soup26.3±20.999.7±0.812.0±14.4100.0±0.066.7±15.8100.0±0.054.3±19.6100.0±0.083.3±7.7100.0±0.0
BBQ sauce3.3±4.894.7±3.00.0±0.098.7±1.510.0±5.786.7±7.09.7±8.888.7±11.54.3±2.790.3±9.1
Butter16.3±15.594.0±5.85.0±5.657.3±22.933.3±17.091.7±12.737.3±21.180.3±20.936.3±8.595.3±4.5
Chocolate pudding24.0±17.898.0±1.810.7±15.194.0±7.435.0±14.498.3±2.737.3±22.788.0±9.940.0±8.494.7±6.8
Cream cheese14.3±7.390.0±4.720.0±9.496.0±3.324.7±4.796.7±1.014.3±12.586.0±19.729.0±6.396.3±1.5
Ketchup71.3±8.591.3±7.219.0±13.898.0±3.173.7±10.6100.0±0.028.7±27.981.0±30.928.0±7.3100.0±0.0
Milk15.3±13.7100.0±0.00.3±0.8100.0±0.028.7±4.3100.0±0.043.7±18.0100.0±0.032.7±14.6100.0±0.0
Orange juice11.7±14.2100.0±0.05.3±9.499.3±1.640.0±10.6100.0±0.027.0±27.899.7±0.831.3±9.9100.0±0.0
Salad dressing31.7±22.091.7±7.729.7±25.690.3±17.775.3±11.897.7±3.953.3±20.297.7±5.748.7±10.690.7±22.9
Tomato sauce23.0±16.585.7±6.132.7±24.683.3±13.844.3±23.090.7±6.461.7±18.481.3±10.963.3±7.994.3±5.4
Average23.7±14.194.5±3.713.5±11.991.7±7.143.2±11.896.2±3.436.7±19.790.3±11.039.7±8.496.2±5.0

Simulation Rollouts

Side-by-side LIBERO-Challenge rollouts on the chocolate-pudding task (third-person view). FullVLA on top, GloVLA below, from the same challenge scene. As perturbations compound, full-trajectory execution drifts off the approach and fails, while GloVLA’s geometric transport returns the arm to the local interaction distribution.

Clutter — easy
Distraction — easy
Obstruction — easy
Visual shift — easy
Illumination — easy
Medium composition 1 — medium
Medium composition 2 — medium
Medium composition 3 — medium
Hard composition 1 — hard
Hard composition 2 — hard

LIBERO-Plus Generalization

Beyond our own challenge set, GloVLA also leads on the independent LIBERO-Plus Object suite, which probes seven distribution-shift axes. GloVLA obtains the highest weighted average (88.2%) and leads on six of seven axes.

0 25 50 75 100 66.5 OpenVLA-OFT 72.7 π₀-FAST 77.1 OpenVLA-OFTm 76.6 GR00T N1.6 88.2 GloVLA
Weighted-average success (%) on the LIBERO-Plus Object suite, weighted by episodes per axis following the LIBERO-Plus protocol.
Per-axis LIBERO-Plus results (Table IV)
MethodAvg.Cam.RobotLang.LightBGNoiseLayout
OpenVLA-OFT66.538.925.499.073.797.672.371.8
π0-FAST72.772.027.671.571.095.293.184.5
OpenVLA-OFTm77.170.218.198.5100.091.994.177.4
GR00T N1.676.656.847.290.4100.099.676.381.6
GloVLA88.277.370.6100.0100.099.692.485.9

Data Efficiency

Because transport is delegated to geometry, the VLA can spend its limited data capacity on local interaction. Under matched source-demonstration budgets, GloVLA reaches near-perfect success with far fewer demos: at 10 demos it already averages 45.1% across unstructured conditions versus 9.1% for FullVLA, and by 50 demos it saturates at 99.9% (FullVLA 48.5%).

0 25 50 75 100 9.1 19.9 48.5 45.1 99.5 99.9 10 demos 30 demos 50 demos FullVLA GloVLA (Ours)
Average success (%) over five unstructured conditions versus demonstration budget (LIBERO-Challenge Task 1). GloVLA is dramatically more sample-efficient than full-trajectory training.

Real-Robot Rollouts

Real-robot tabletop trials mirror the simulated perturbation families: added objects, partial obstruction, illumination change, viewpoint shift, distractors, and composed medium / difficult scenes. All videos play at 3× speed.

Clutter — added objects around the target
Distraction — look-alike distractor objects
Obstruction — partial camera / target occlusion
Illumination shift
Visual shift — scene appearance change
Medium composition — two to three stressors
Difficult composition — four or more stressors combined

Real-Robot Results

On a physical UR10e, the factorization transfers cleanly: near-parity on the clean tabletop, and the largest gains exactly where long-range visual servoing is disrupted — illumination (0% → 80%), distraction (20% → 100%), and both composition tiers. Overall success rises from 35.6% to 90.0%, while mean inference time is more than halved.

35.6% → 90.0% overall real-robot success, FullVLA → GloVLA (+54.4 points)
60.1s → 28.2s mean inference time per episode (−53%)
Clean tabletop 90% 95% Clutter 30% 90% Obstruction 40% 100% Illumination 0% 80% Visual shift 40% 100% Distraction 20% 100% Medium comp. 10% 80% Hard comp. 0% 70% Overall 35.6% 90.0% FullVLA GloVLA (Ours)
Real-robot success rate on UR10e (clean: 20 trials; each perturbation: 10 trials). Overall success improves +54.4 points, from 35.6% to 90.0%.

BibTeX

@article{glovla2026,
  title   = {GloVLA: Let Geometry Move and Local VLA Interact for Robust
             Object-Centric Manipulation in Unstructured Environments},
  author  = {Anonymous Authors},
  journal = {Under review},
  year    = {2026}
}