Domain Randomization Strategies for Sim-to-Real Transfer

Domain randomization for sim-to-real transfer works best when its mechanism matches the source of failure. Visual mismatch, physics error, sensor noise, and missing scenarios are different problems. Effective transfer therefore begins with diagnosis, then uses targeted randomization, system identification, or real-data adaptation to address the constraint that limits deployment performance.

Why the sim-to-real gap is four distinct problems, not one

A policy performs well in simulation, then fails dramatically on real hardware. The failure points to a part of the simulated training experience that did not match deployment. A camera image can differ from a rendering even when the simulated dynamics are adequate. Conversely, a policy can perceive its environment correctly and still fail because contact behaves differently.

The visual gap comes from systematic differences in textures, lighting, and backgrounds. The physics gap comes from approximations of contact dynamics, friction, damping, and actuator behavior, with errors that compound during contact-rich tasks. The sensor gap concerns the noise distributions of IMUs, depth cameras, and proprioceptive encoders. The scenario gap concerns infrequent but consequential events absent from training. These causes require different classes of remedy because they change different parts of the problem the policy must solve.

How undifferentiated randomization fails — and what "matching strategy to gap" actually means

Treating these causes as one problem is a design error because variation in one dimension does not repair omissions in another. More varied textures cannot correct an inaccurate actuator model. A wider friction range also gives no guarantee that the policy will face an unusual task situation.

Breadth also comes at a cost to learning. Increased randomization can improve robustness while reducing optimality on a specific platform, and excessive variation slows convergence. Matching strategy to gap means identifying the binding constraint, then selecting a mechanism that addresses it directly. The aim is sufficient coverage of the uncertainty relevant to the task, without forcing the policy to accommodate unrelated variation.

Closing the visual gap: texture, lighting, and style-transfer strategies

Visual domain randomization changes textures, lighting, backgrounds, distractor objects, and camera viewpoints. It is the most mature gap-specific strategy and can support zero-shot transfer for perception-heavy tasks when applied deliberately. It targets mismatches between rendered appearance and real camera images. Visual diversity can reduce dependence on a narrow simulated appearance, but it does not correct errors in physical interaction.

Style-Identified CycleGAN extends this approach by translating simulated camera images into real-like modalities. The agent sees those image statistics in simulation, which allows visual zero-shot transfer without real-world agent interaction data. The GAN itself is trained on real images.

OpenAI Dactyl combined visual and physics randomization in MuJoCo to solve real-world block manipulation without accurate modeling or demonstrations. Its randomized policies achieved substantially more rotations than policies trained without randomization. The example joins two remedies rather than isolating one. Even within the visual component, synthetic diversity has a limit because randomized textures do not reproduce the contextual coherence of real environments. Varying appearances independently does not preserve the links between lighting and geometry.

Closing the physics gap: dynamics randomization and its adaptive variants

Static dynamics randomization is especially difficult to tune for contact-rich tasks. Contact dynamics are high-dimensional, and interactions between parameters can be irregular. A range that is too narrow leaves the policy brittle, while one that is too wide produces sub-optimal behavior. The difficult choice is the distribution itself. Adaptive methods handle that choice by changing the sampled dynamics as training proceeds, so the initial ranges can change.

BayRn uses Bayesian optimization over source distribution parameters and sparse real-world data to change the domain parameter distribution as the policy learns. This lets transfer performance guide the choice of ranges, but training the policy again and again can be costly. BayRnTune addresses that computational limitation by fine-tuning a previously learned policy each iteration. Reuse changes the cost of adaptation. The BayRnTune paper reports improvements ranging from modest gains up to 400% better rewards over vanilla domain randomization and Bayesian domain randomization across five environments, including BALLU walking and quadruped pushing.

DORAEMON progressively widens the dynamics sampling distribution through entropy maximization, subject to a minimum policy success-rate threshold. The policy’s success constrains how far diversity can expand. This removes the need for costly real-world data and manual tuning in its parameter-selection process.

Distribution width, however, is not the whole problem. GoFlow uses normalizing flows to adjust domain randomization adaptively, addressing settings where domain parameters have irregular dependencies. The GoFlow paper reports that it does better than fixed and other learning-based methods across a suite of environments, especially when those dependencies are present. This distinction is consequential for contact-rich dynamics. Expanding parameter ranges and representing their joint structure are different tasks, so a method that solves the first need not solve the second.

System identification presents a stronger objection to blanket dynamics randomization. PACE, or Precise Adaptation through Continuous Evolution, identifies a minimal set of physically interpretable parameters from simple encoder data. Its 2026 framework uses motor inertia, also called armature, viscous damping, Coulomb friction, joint bias, and global delay to achieve zero-shot locomotion without dynamics randomization. Measurement replaces breadth where the physical structure is identifiable. That objection holds only where the physical structure can be identified, because high-dimensional dexterous manipulation can lack known structure that can be interpreted, leaving adaptive randomization as the tool that can be used in practice.

Closing the sensor gap: matching noise distributions without time alignment

Sensor mismatch can remain after visual appearance and simulated dynamics have been addressed. Idealized Gaussian IMU noise and clean depth returns differ from sensor behavior under vibration, temperature drift, and contact shock. The discrepancy concerns what the policy measures as well as what the world looks like or how it moves. It needs separate treatment from those other gaps. Camera calibration, sensor latency, measurement delay, and communication jitter are randomization parameters that can otherwise remain fixed.

For proprioception, distribution matching frames the target as the noise statistics the policy encounters rather than exact time alignment between simulated and real trajectories. Latency and jitter still require deliberate attention. Check whether sensor latency and proprioceptive noise were randomized when broad visual and physics randomization still leaves deployment failures. An overlooked measurement mismatch may be the binding constraint.

Closing the scenario gap: curriculum strategies and structured diversity for rare events

Scenario coverage changes which situations the agent faces as well as the parameter values within them. A wide friction range alone does not make an object slide off a table mid-grasp. Parameter coverage and behavioral coverage are different. Closing this gap requires curriculum or adversarial structuring of the situations the agent encounters.

DeceptionNet learns adaptively challenging augmentations by adversarially maximizing uncertainty in the task network’s output. Continual domain randomization addresses a different part of exposure. Josifovski et al.’s 2024 approach introduces variability in stages and uses continual learning regularization to minimize catastrophic forgetting.

Diverse scenarios can also make joint optimization difficult. Distilled Domain Randomization separates exploration by training individual teacher policies per domain, then distilling them into a single deployable policy. This sidesteps high-variance optimization across incompatible scenarios rather than asking one training process to handle them all together. Domain randomization can generate only the events it was designed to include, so black swan events need real-world data augmentation or explicit adversarial scenario design.

When to supplement randomization with real data and system identification

A residual transfer gap does not always justify more simulation diversity. Small amounts of targeted real-world data or system identification can address it more efficiently. The combination follows from the diagnosis. Randomization supplies coverage, while real experience adapts the policy to the deployment context that broad coverage cannot describe precisely. Domain randomization with meta-learning therefore need not be treated as a fallback after a simulation-only approach fails.

PACE uses real encoder data to identify transfer-sensitive physical parameters, including motor inertia and joint friction, rather than compensating for their uncertainty with dynamics randomization. Where those parameters are identifiable, measurement can remove a source of uncertainty directly. The budget-allocation paper, How Should a Simulation-to-Reality Transfer Budget Be Spent?, supports this distinction in a controlled sim-to-sim pendulum setting. Its results favor measurement and training at estimated parameters once real data is available, but the study’s identifiable dynamics do not establish the same result for structural model mismatch.

Why not train only on real data once it becomes available? Real experience cannot safely cover rare scenarios or supply the volume needed for reinforcement learning on its own. Simulation retains its role in scale and coverage. Small real-data fine-tuning adds deployment accuracy without requiring either source to be sufficient alone. The practical choice is therefore which uncertainty to cover synthetically and which to resolve through measurement or adaptation.

Applying the gap-type framework: a diagnostic workflow for practitioners

The practical sequence begins with characterizing failure before expanding randomization. Run the simulation-trained policy on hardware and observe where it breaks. Visual confusion, contact instability, and sensor latency failures point to different things that hold the policy back. Use the four-gap decomposition to distinguish those causes rather than treating poor deployment performance as a request for more variation everywhere. Where visual and physics randomization are already broad, apply the sensor section’s diagnostic check to latency and proprioceptive noise.

For physics failures, check whether static ranges are enough or whether the way parameters interact calls for adaptation. High-dimensional contact dynamics and irregular dependencies favor the adaptive approaches discussed earlier, including GoFlow or BayRnTune, over simply widening a uniform distribution. Identifiable physical parameters instead support system identification. For visual failures, apply texture, lighting, or style-transfer strategies only when perception is in the loop. Purely proprioceptive control does not benefit from appearance randomization, which wastes compute and can slow convergence.

Next, check scenario coverage separately from parameter coverage. If rare contact events or environment configurations are absent, structure curriculum or adversarial exposure explicitly. Use real-to-sim initialization and fine-tuning with a small amount of real data to close the gap that remains. When real data is continuously available, Maddukuri et al.’s RSS 2025 work and Cheng et al.’s NeurIPS 2025 work on sim-and-real co-training point beyond a strictly sequential pipeline by combining simulated and real policy optimization.