Skip to content

Releases: strands-labs/robots

strands-robots v0.5.1

Choose a tag to compare

@cagataycali cagataycali released this 06 Aug 20:23
00cccb0

Release date: 2026-08-06 · Previous release: v0.5.0 (2026-08-03)

51 commits over three days. A correctness-only patch release: no new features,
no API additions. It closes a dependency-resolution gap that let v0.5.0 install
a lerobot that cannot serve its flagship bucket-streaming path, and continues
the numeric-input hardening pass across the training, tools, and simulation
surfaces.

strands-robots is the robot-control library for Strands Agents:
policy inference, teleoperation, calibration, recording, and simulation (MuJoCo,
Newton, and Isaac) behind one agent-callable tool surface.


Why upgrade

v0.5.0's lerobot floor is >=0.6.0, but bucket streaming needs >=0.6.1.
stream_dataset(repo_type="bucket") requires a StreamingLeRobotDataset that
accepts a repo_type argument — lerobot 0.6.0's constructor has none; 0.6.1 added
it. Because every lerobot-bearing extra floored at >=0.6.0, a lockfile or a
pre-existing lerobot 0.6.0 resolved cleanly but broken: the install succeeds,
then the runtime guard refuses the bucket read. A bare pip install picked 0.6.1
only by luck (newest candidate). v0.5.1 makes >=0.6.1 the enforced floor so 0.6.0
is unresolvable and the "must-have 0.6.1" requirement lives in the resolver, not
just in prose.

If you use stream_dataset(repo_type="bucket"), sync_to_bucket, or install from
a lockfile, upgrade to 0.5.1.


Highlights

  • lerobot floor raised to >=0.6.1,<0.7.0 across [lerobot], [lerobot-async],
    and [molmoact2] (#1930). 0.6.0 is now unresolvable; bucket streaming
    is guaranteed by the resolver rather than documented. The runtime guard is retained
    for pre-existing environments but now names the exact version and install command.
  • huggingface_hub floor raised to >=1.5.0 for bucket sync (#1957). hf buckets
    and hf sync first ship in hub 1.5.0; the gate, its upgrade message, and the [wbc]
    extra pin previously said >=1.0, so a caller on 1.0–1.4.x passed the gate and hit
    raw No such command 'buckets' CLI noise.
  • Isaac 6.0.x reset correctnessreset() now revives the articulation handles
    that world.reset() kills on Isaac 6.0.x (#1901, closes #1895), and the LIBERO eval
    pumps Kit on the main thread with the agent off-main so it cannot deadlock (#1899,
    closes #1896).
  • Continued numeric-input hardening — 29 commits extending v0.5.0's "refuse a value
    you cannot honor" pass to training hyperparameters, pose/serial/teleop tools, and the
    Isaac/Newton/MuJoCo action, gravity, joint-state, and timestep surfaces.

Dependency floors

Dependency v0.5.0 v0.5.1 Reason
lerobot (all extras) >=0.6.0,<0.7.0 >=0.6.1,<0.7.0 0.6.1 is the first release whose StreamingLeRobotDataset accepts repo_type (#1930)
huggingface_hub ([wbc]) >=1.0 >=1.5.0 hf buckets / hf sync CLI first ship in 1.5.0 (#1957)

Both floors are declared once as module constants
(BUCKET_STREAMING_MIN_LEROBOT, and the hub floor constant) so the gate, the
error-message remedy, the README guidance, and the extra pin cannot drift apart.


Fixed

Dependency resolution

  • #1930 floor lerobot at >=0.6.1 so bucket streaming is resolver-guaranteed.
  • #1957 the bucket-sync huggingface_hub floor is the release that ships the hf
    bucket CLI.

Simulation — Isaac

  • #1901 reset() revives the articulation handles world.reset() kills on 6.0.x
    (closes #1895).
  • #1899 pump Kit on main, run the agent off-main so the Isaac LIBERO eval cannot
    deadlock (closes #1896).
  • #1926 send_action applies the shared action-value domain.
  • #1928 set_joint_positions applies the shared joint-state domain.
  • #1968 cross-repo references name the repository they belong to.
  • #1977 add_object names an unusable keyword rather than dropping it.

Simulation — Newton / shared

  • #1927 every gravity surface applies the one shared gravity domain.
  • #1929 the recorded action schema is declared from robot_action_keys.
  • #1931 set_timestep applies the one shared timestep domain.
  • #1966 a viser dashboard port outside the TCP range is refused.

Training

  • #1935 a reward config that cannot obtain its pretrained asset reports the type and a remedy.
  • #1939 the run size a TrainSpec asks for is one shared positive-count domain.
  • #1943 the optimizer learning rate is one shared positive-finite domain.
  • #1947 the launch topology a TrainSpec asks for is one shared positive-count domain.
  • #1950 the reproducibility seed is one shared non-negative-count domain.
  • #1955 val_episodes is one shared positive-count domain for both writers of eval_split.
  • #1962 the LoRA adapter hyperparameters are one shared positive-count domain.

Tools

  • #1932 the teleop tool refuses a numeric option the lerobot CLI cannot honor.
  • #1933 pose_tool honors its steps / step_delay interpolation options.
  • #1934 serial_tool bounds a servo register write to the field it encodes into.
  • #1936 lerobot_train refuses a run size it cannot honor, before launching it.
  • #1938 pose_tool refuses a joint target the arm cannot honor.
  • #1940 robot_mesh bounds the two numeric options its command body cannot carry.

Inference / policies

  • #1948 a rollout seed is applied or refused, not silently dropped.
  • #1951 the VERA ports take the shared TCP-port domain.
  • #1953 a remote-inference port is validated on the domain its role actually has.
  • #1954 set_eval_seed requires a seed — None is refused, not half-applied.
  • #1973 a seed the client reseed cannot apply is refused, not half-applied.
  • #1985 a client timeout that names no wait budget is refused rather than blamed on the server.

Hardware / device / ROS

  • #1946 set_gripper reads an unlimited ctrlrange as unset, not as empty.
  • #1964 a Reachy Mini api_port outside the TCP range is refused.
  • #1976 a DDS domain id outside the RTPS port map is refused, not published process-wide.
  • #1979 a bridge loop period that cannot pace a thread is refused rather than run.

Datasets

  • #1969 streaming_dataset.open() refuses a numeric knob it cannot honor.
  • #1971 verify_dataset refuses an episode-count threshold outside its domain.
  • #1975 a per-camera LIBERO config add_camera cannot accept is refused, not dropped.

Performance

  • #1982 a recording that keeps no images does not force the render.

Docs & CI

  • #1956 / #1958 / #1945 notebook prerequisites name a release that exists; the
    lerobot_local install rows name a stack that can actually train; notebook 6
    orchestrates a heterogeneous fleet from one goal.
  • #1972 the torch stand-in skips with a reason instead of failing unactionably.
  • #1941 read the Dependabot config from .github/ instead of parsing it as a workflow.
  • #1963 CI refuses a closing keyword that only appears in a PR title.
  • #1983 the last-push-approval report can be pointed at the standing pull requests.
  • Process/agents docs: #1925, #1960, #1980, #1981.

Upgrade

pip install -U "strands-robots==0.5.1"
# with lerobot support:
pip install -U "strands-robots[lerobot]==0.5.1"

No API changes — this is a drop-in patch. If you were on lerobot 0.6.0, the
upgrade forces you onto 0.6.1 (required for bucket streaming to function).

strands-robots v0.5.0

Choose a tag to compare

@cagataycali cagataycali released this 04 Aug 01:32
ab14119

Release date: 2026-08-03 · Previous release: v0.4.1 (2026-07-01)

804 commits · ~146 merged pull requests across five weeks. This is the largest
release since the project began: a new NVIDIA Isaac Sim backend, agent-facing
analytic motion primitives, a remote-inference client/server split, terrain
locomotion curricula, and a deep hardening pass across every numeric input the
agent and mesh surfaces accept.

strands-robots is the robot-control library for Strands Agents:
policy inference, teleoperation, calibration, recording, and simulation (MuJoCo,
Newton, and now Isaac) behind one agent-callable tool surface.


Highlights

  • NVIDIA Isaac Sim backend - full IsaacSimulation with procedural scene loading,
    LeRobotDataset recording parity, a delta-EEF differential-IK controller for GR00T
    actuation, EEF-state grounding for LIBERO, and pip-isaacsim 6.0.x support.
  • Analytic motion primitives - move_to / set_gripper / rotate_wrist, backed
    by one shared mink IK bridge, give an agent the staging/transport/release vocabulary
    to position a robot around a learned policy's competence region.
  • stop_when= semantic early-return - a rollout terminates as soon as the world
    reaches a predicate state, with stopped_reason telemetry, turning a monolithic
    rollout into a retryable primitive.
  • Remote policy inference - a client/server split so an edge robot can drive a
    policy running on a remote GPU.
  • LeRobot 0.6 policy support - the [lerobot] / [molmoact2] extras now track
    lerobot>=0.6.0,<0.7.0, so MolmoAct2 resolves straight from PyPI (no from-source
    build) and the policy resolver covers every family lerobot 0.6.x ships.
  • Terrain locomotion curricula - create_world(terrain=..., difficulty=...) for
    stairs / slope / rough / pyramid heightfields, a full locomotion reward/predicate DSL,
    and shipped go2 / g1 / t1 benchmarks.
  • rosbridge transport - use_rosbridge + RosbridgeRobot drive ROS1 robots and
    any host with no ROS install (verified live against the NASA Curiosity Mars rover
    Gazebo sim).
  • A repository-wide input-hardening pass - every numeric knob on the simulation,
    hardware, mesh, teleop, and recording surfaces now refuses a value it cannot honor
    instead of silently clamping, coercing, or reporting a false success.

The changelog assembly moved to per-PR news fragments (changelog.d/) this
cycle, so overlapping PRs no longer conflict on the shared [Unreleased] anchor.


New backend: NVIDIA Isaac Sim

Isaac joins MuJoCo and Newton as a first-class simulation backend behind the same
tool surface.

  • IsaacSimulation - procedural scene building, robot/object/camera lifecycle,
    and physics stepping.
  • LeRobotDataset recording parity (IsaacRecordingMixin) - Isaac rollouts record
    the same dataset format as MuJoCo and Newton.
  • GR00T action actuation - an Isaac-side delta-EEF differential-IK controller
    converts GR00T's task-space {x, y, z, roll, pitch, yaw, gripper} deltas into joint
    targets (no robosuite dependency).
  • EEF state grounding - IsaacSimulation.get_body_state plus LiberoAdapter
    EEF-pose/gripper state sources so language-conditioned policies get the
    state.x/y/z/... they require; this also unblocks the predicate DSL on Isaac.
  • LIBERO init-state arm qpos - robosuite joint names are mapped onto the USD
    articulation so an episode starts from the LIBERO ready pose rather than the USD
    default.
  • pip-isaacsim 6.0.x support - synchronous physics-view invalidation, camera
    warm-up under a stopped timeline, matrix-parser tolerance, and documented install
    collateral (the coverage>=7.6 + OMNI_KIT_ACCEPT_EULA caveats).
  • URDF joint names - the importer's USD-mangled joint names are demangled back to
    the URDF's own names, so the same URDF reports one joint vocabulary on Isaac and MuJoCo.

Agent capabilities

Analytic motion primitives (move_to / set_gripper / rotate_wrist)

The Harness VLA staging vocabulary, backed by one shared mink DLS IK bridge
(strands_robots/simulation/ik.py, deduplicated from the cosmos3 and vera copies):

  • move_to(robot_name, position, orientation=None, ...) - position-only when
    orientation=None (the right contract for 5-DOF arms), DLS IK to the world-frame
    target then a servo-drive loop; returns the IK residual on an unreachable target
    rather than hanging or raising.
  • set_gripper(state="open"|"close") - ctrlrange-endpoint set-point with registry-driven
    open/close convention.
  • rotate_wrist(target_yaw) - wrist-yaw set-point while every other actuator holds.

All three refuse to run while a policy drives the same robot, take the engine lock per
control tick so renders and stop_policy interleave, and are not collision-aware
(a future curobo backend can drop in). Registry gripper metadata (gripper block per
robot) resolves which actuators the primitives command, so an arm actuator whose name
collides with a gripper hint is no longer silently excluded from IK.

The figure under pixel-to-world grounding below is the end-to-end demo: an agent
names a pixel, get_world_point turns it into a world coordinate, and move_to
puts the jaw there.

get_world_point(camera, pixels) - pixel-to-world grounding

Instead of reading privileged object poses, an agent picks pixels on a target's visible
surface and this call unprojects each through the pixel-aligned metric depth buffer
(p_world = T_world_cam · depth · K⁻¹ · [u, v, 1]), with built-in multi-pixel median
and per-pixel validity. Works identically on hardware with an RGB-D camera, so grounding
built on it transfers.

an agent names a pixel, get_world_point grounds it, move_to reaches it

Figure 1 - point at a pixel, the arm goes there (#1649 + #1654).
Left: the frame the agent saw, with the single pixel it named - no hand-written joint
targets. Middle: get_world_point unprojects that pixel and the result is compared
against the cube's privileged pose; the pixel resolves to the point on the visible
surface
, which is what "looking at a frame" actually means. Right: move_to
drives the jaw to that coordinate and reports its IK residual. Rendered headless
(MUJOCO_GL=egl) against the tagged source.

stop_when= semantic early-return

run_policy(stop_when={"predicate": "grasped", ...}) terminates the rollout as soon as
the world reaches the predicate state (checked every control step, on both the
synchronous and async-RTC paths), and reports stopped_reason
predicate | budget | cancelled | error plus steps_used. This turns a rollout into a
retryable primitive an agent can stage → invoke → inspect → re-invoke, and composes with
recording (the recorded episode's frame count equals steps_used).


Remote policy inference

A client/server split (strands_robots/inference/) so an edge robot can drive a policy
that runs on a remote GPU. lerobot_async gained a rename_map for remote
observation-key remapping and now forwards it to the PolicyServer, and a numeric
handshake (set_control_frequency / set_rtc_observed_delay) is validated at the call
rather than mid-rollout.


LeRobot 0.6 policies

The [lerobot] and [molmoact2] extras now require lerobot>=0.6.0,<0.7.0 (was
>=0.5.0,<0.6.0), and the 0.5.1-era torch/torchcodec overrides are dropped - lerobot
0.6 ships mature, platform-correct dependency pins of its own.

  • MolmoAct2 installs from PyPI. MolmoAct2Policy ships in lerobot 0.6 (lerobot
    PR #3604), so pip install 'strands-robots[molmoact2]' resolves it directly - the old
    "install lerobot from source" step, the transformers==5.3.0 pin, and the
    lerobot[smolvla]==0.5.1 guidance are all gone.
  • The policy resolver covers the full lerobot 0.6.x family - pi0, pi0.5,
    pi0_fast, SmolVLA, MolmoAct2, ACT, Diffusion, and the newer eo1 /
    vla_jepa - with a regression guard so a family a future lerobot release adds is
    picked up automatically. Both lerobot_async's server-supported set and the training
    expert_only set are sourced live from lerobot rather than a hard-coded list that
    drifts.
  • expert_only fine-tuning (freeze the VLM, train only the action expert) is now
    gated per policy: a run is allowed iff that policy's lerobot config actually declares a
    train_expert_only field, instead of against a fixed {pi0, pi05, pi0_fast, smolvla}
    tuple (PI0FASTConfig, for one, exposes no such field).
  • RTC seam documented honestly - run_policy(async_rtc=...) no longer claims
    SmolVLA/MolmoAct2 blend the chunk seam internally; the public lerobot/smolvla_base
    ships rtc_config=None and MolmoAct2 has none, so the library's own real-time-chunking
    path is what joins the seam.

Locomotion & terrain curricula

  • Terrain heightfields - create_world(terrain="stairs"|"slope"|"rough"|"pyramid", difficulty=...) as a curriculum knob a locomotion trainer ramps across resets, plus
    get_ground_height(x, y) to query the local surface, floating-base seating on the
    terrain surface, and terrain-relative reward measurement.
  • Reward/predicate DSL - base_velocity, base_height, base_orientation,
    base_lin_vel_z / base_ang_vel_xy, base_velocity_tracking reward terms; and
    base_beyond_x/y, base_yaw_beyond, base_below_z, base_tipped predicates.
  • Shipped benchmarks - go2_walk_forward, go2_strafe_left, go2_turn_left,
    g1_walk_forward, t1_walk_forward via `register_buil...
Read more

strands-robots v0.4.1

Choose a tag to compare

@cagataycali cagataycali released this 01 Jul 17:23
ed96a5d

strands-robots 0.4.1

GR00T Whole-Body-Control for the Unitree G1, PPO/FastSAC RL stack
with deterministic eval, LeKiwi in sim, and hard hot-path correctness fixes
(RTC sim/real parity, no silent open-loop rollouts, persistent-policy perf).

416 commits since v0.4.0. All clips rendered headless in MuJoCo (MUJOCO_GL=egl)
against the exact source being tagged.


Highlights at a glance

Feature What it does Demo
Whole-Body Control WBCPolicy wraps NVIDIA GR00T SONIC ONNX controllers for deploy-grade G1 locomotion - in-process ONNX (no torch, no sidecar). walk
WBC + steering Balance + walk ONNX sessions auto-selected by commanded speed; yaw while walking. turn
RL from scratch PpoTrainer, FastSacTrainer, VecSimEnv, BaseRLAlgo.evaluate(). FastSAC learns SO-100 reach from reward alone (−8.77 → −5.50). rollout
LeKiwi in sim Robot("lekiwi", mode="sim") now works - auto-downloads Ekumen-OS description (6-DOF arm on 3-omniwheel base). lekiwi

1. Whole-Body Control

Non-VLA provider wrapping GR00T Whole-Body-Control (SONIC / decoupled-WBC) ONNX
controllers. Faithful 86-dim observation stacked over 6 steps, two ONNX sessions
(balance + walk) auto-selected by commanded speed, upstream PD-to-torque law.

from strands_robots import Robot

g1 = Robot("unitree_g1", mode="sim", backend="mujoco")
g1.run_policy(
    policy="wbc",                 # GR00T SONIC whole-body controller
    command={"vx": 0.5},          # forward 0.5 m/s
    duration=6.0,
)
# forward walk: base +2.30 m, pelvis height 0.74 -> 0.75 m (upright, no fall)
vx omega result
Forward walk 0.5 m/s 0.0 +2.30 m, stays upright
Walk + yaw 0.3 m/s 0.4 rad/s steers while balanced

Weights: nepyope/GR00T-WholeBodyControl_g1 (NVIDIA Open Model License).

2. Reinforcement learning constructs

New strands_robots.training.rl stack. VecSimEnv runs N SimEnv as one
(N,D)-batched env over one reused thread pool. evaluate() is a deterministic
eval peer of train() - mean action, frozen normalizer, reports success_rate.

from strands_robots.training.rl import FastSacTrainer, VecSimEnv

env = VecSimEnv("so100", n_envs=8, task="reach")
trainer = FastSacTrainer(env)
trainer.train(iterations=80)          # CPU: mean return -8.77 -> -5.50
stats = trainer.evaluate(episodes=10) # deterministic: elbow 0.225 vs target 0.200

FastSAC learning curve

Also: SARM reward-model training + RA-BC production loop.

3. LeKiwi - now simulatable

from strands_robots import Robot

kiwi = Robot("lekiwi", mode="sim")   # was: "No model found"
# auto-downloads Apache-2.0 Ekumen-OS/lekiwi MuJoCo description
# 6-DOF SO-ARM on 3-omniwheel base (9 actuators), front/wrist cameras

4. Hot-path correctness (contract fixes - deploy-critical)

Fix Before After
RTC sim/real parity hardware loop ignored Real-Time Chunking contract loop honours RTC; inference_delay forwarded to denoiser; chunk-seam re-anchored for relative-action policies. Async-RTC latency masking now default
No silent open-loop state-key mismatch -> silent zero/open-loop rollout loud error; routing-degradation telemetry (positional_fallback_used, generic_state_keys_used) in run_policy/eval_policy JSON
Episode-count contract ambiguous run_policy episode count + verify_dataset_episodes; reset() during recording flushes buffered episode
PersistentPolicy reloaded heavy checkpoints per call resident worker + cache controls + policy_resident_rss_mb telemetry
Clean errors partial stub shadowed lerobot.policies; cached transient probe failures unknown policy types raise clean ImportError; probes no longer cache transient failures
get_mass_matrix broke on MuJoCo mj_fullM signature change works across signatures; numpy scalars retained in state/action vectors

5. Newton backend maturation

Scene-discovery + per-joint state parity with MuJoCo, domain randomization +
sensor-noise hooks, recording fix (camera frames captured even when policy skips images).

6. Security hardening

Output-path sandboxing across all three LLM-supplied filesystem sinks -
render(output_path=), run_policy(video=), start_cameras_recording -
centralized in simulation.safe_output: rejects .. traversal, backslash
separators, shell metacharacters, symlinked targets before any file opens.
Atomic writes; size caps.

7. Ergonomics / correctness

  • add_object rejects unknown kwargs with a structured error (was: silent grey default).
  • Zero-config robot discovery from robot_descriptions in the registry.
  • Simulation render output is ASCII-only; unified "no world" guard contract.

Media (higher-quality MP4 originals)

Clip GIF MP4
G1 WBC walk gif mp4
G1 WBC turn gif mp4
LeKiwi drive gif mp4
RL reach rollout gif mp4

Assets on branch release/0.4.1-visuals under release-assets/0.4.1/.

Full changelog: git log v0.4.0..v0.4.1 once tagged, or compare/v0.4.0...main.

strands-robots v0.4.0

Choose a tag to compare

@cagataycali cagataycali released this 17 Jun 07:51
5e3fe43

strands-robots v0.4.0

150+ commits since v0.3.8 (Feb 20). This is the release where strands-robots
stops being "policy inference glue for a single arm on your desk" and becomes a
platform you can simulate on, evaluate on, and deploy to a fleet with - without
swapping libraries at each stage. The through-line of this cycle was closing the gap
between "it runs a model" and "it runs a robot you'd trust in a room with people."

Python >=3.12 required (LeRobot >=0.5.0 floor).
pip install strands-robots · everything: pip install "strands-robots[all]"


The story of this release

Three forces drove almost every PR:

  1. The sim-to-real loop was broken in the middle. You could load a policy and you
    could talk to hardware, but there was no first-class simulator in between - so you
    couldn't develop a policy without a physical robot, and you couldn't reproduce a
    failure without the exact bench setup. We built a MuJoCo backend and a Robot()
    factory so the same code path runs in sim and on metal.

  2. "One robot" assumptions don't survive contact with a fleet. The moment more than
    one robot, or more than one operator, is involved, you need identity, presence,
    replay protection, an audit trail, and a human able to hit stop. We built the mesh
    and its AWS IoT transport around that reality, and spent a large fraction of the
    cycle hardening it rather than adding surface.

  3. "It produced an action array" is not "it succeeded." We added a real evaluation
    protocol (LIBERO) so policy quality is a number we can defend, not a vibe from
    watching a render.

Everything below ladders up to one of those three.


Simulation: so you can develop without a robot on the bench

Why: Iterating on a policy against physical hardware is slow, unsafe, and
unreproducible. A bug that only shows up at 50 Hz on a real servo is nearly impossible
to bisect. We needed a simulator that is byte-equivalent enough to trust and that
exposes the same agent-tool surface as the rest of the library.

  • MuJoCo backend as an AgentTool with 50+ actions (#85) on a foundation of
    models, the Policy/engine ABCs, a factory, a model registry, and asset
    auto-download (#84, #105/#106). The agent drives the world the same way it drives a
    real robot - world building, robot inject/eject, cameras, stepping, rollout, render.
  • Control-rate substepping (#353) and stepping physics for the full control
    period in eval
    (#429). Why it matters: position-servo policies were silently
    failing to track because we stepped the sim once per action instead of for the whole
    control interval - the policy looked broken when the integration was. This is the
    kind of bug that costs a week on hardware; we paid it down once, in sim.
  • Render fidelity fixes - blown-out white ground (#428), ground-plane z-fighting on
    attach (#360), conditional ground-strip + tendon scale + RNG parity (#400).
    Why it matters: renders are the policy's input in vision models. A blown-out
    frame isn't cosmetic - it's feeding the policy garbage and poisoning eval.
  • Naming/identity correctness: register sim robots under the user's name not the
    canonical one (#435), structured errors from resolvers (#417), optional robot_name
    across the state family (#412). Why: the sim has to address robots the way the user
    thinks about them, or multi-robot scenes become a guessing game.
  • SimEngine.describe() discovery surface (#407) - so an agent can ask "what can I do
    here?" instead of failing on an unknown action key. We also started surfacing valid
    actuator names on errors and propagating failures into run_policy status (#436),
    because a silent zero-action on failure is the single most dangerous default in robotics.

Robot() factory: one entry point

Why: Before this, choosing sim vs. real meant choosing a different code path, and
the path that touches a physical motor was just as easy to invoke by accident as the
safe one. That's backwards.

  • Robot() factory + top-level lazy imports (#86; hygiene follow-ups #145).
    Sim is the default; real hardware is an explicit opt-in (mode="real" /
    STRANDS_ROBOT_MODE=real). Lazy imports keep import strands_robots cheap so the
    factory doesn't drag the entire ML stack into a process that just wants to talk to a
    serial port.
  • Ergonomics for send_action, add_robot, render, get_robot (#431) - the
    paper cuts that make the difference between an API you demo and an API you live in.

Mesh + AWS IoT: because a fleet is a security problem, not a networking problem

Why: Connecting robots is easy. Connecting robots safely - where a stale command
can't be replayed, a compromised peer can't impersonate another, an operator can always
intervene, and every actuation is on an audit trail - is the actual job. We treated the
mesh as a trust boundary from day one, which is why the hardening PRs outnumber the
feature PRs here.

  • Core mesh - session, presence, RPC, streams, wiring, + AWS IoT transport (#101).
    Zenoh for the LAN, MQTT5/mTLS for the cloud, Device Shadow mirror, S3 camera offload,
    account-wide Fleet Provisioning.
  • The #195 hardening split landed as a deliberate sequence so each layer could be
    reviewed in isolation: PKI helpers + conftest (#220), payload validation / action
    allowlist (#223), Zenoh + ACL config with mTLS/downsampling/low-pass (#224),
    tamper-evident HMAC audit log with per-peer sequence + rotation (#221), cross-transport
    dedup + monotonic TTL + strict mode (#222), replay caches + override-resume + safety
    topic handlers (#225), and robot_mesh HITL via tool_context.interrupt + per-action
    rate limit (#227). Why split: security review fatigue is real; a 9-part series each
    reviewable in an afternoon catches more than one 4000-line PR nobody reads to the end.
  • Human-in-the-loop done right (#227, #411): a declined approval must not consume a
    rate-limit slot (or an operator's "no" could lock out a legitimate e-stop), and the
    operator's literal reply is never echoed back into the LLM context (that turns a human
    into a prompt-injection channel). Read-only actions are audited too (#411) - operators
    need "the agent read N frames at time T", not just actuation logs.
  • Replay/lockout safety pins: estop engages even when the per-issuer replay cache is
    full (#263/#339) - the cache bounds memory, never safety; resume-cache fairness mirror
    (#342); check-then-set estop replay lock (#273/#361); poison records on every degraded
    audit path so a stream gap is attributable, not silent (#410). Why this obsession:
    in this domain the failure mode isn't "wrong answer", it's "arm moves when it shouldn't"
    or "stop didn't take." Fail-loud, fail-safe, always.
  • AWS IoT provisioning hardening: CA pin + thing-name regex + scoped policy (#228),
    deny-by-default Fleet Provisioning hook (#333), operator-shadow + response publishes
    scoped to the device's own ThingName (#334, #336), atomic break-glass marker with
    explicit symlink reject (#388/#402). Why: fleet provisioning is the blast radius -
    a permissive default here means one bad cert owns the whole account.
  • Teleop integrity: validate input frames before apply (#332), route every teleop
    publish through the single Mesh.publish() chokepoint (#452). One door, guarded.
  • Sim is a first-class mesh peer: tell() dispatch maps to run_policy/start_policy
    (#304), sim joint state bridges to child peers (#422), sim cameras publish JPEG frames
    (#425). Why: if sim and real don't look identical on the mesh, your fleet tooling
    can't be tested in sim - which defeats the point of having a sim.

Policies: more brains, one socket

Why: The field is moving fast and no single policy wins everywhere - VLAs for
open-ended manipulation, classical planners for collision-aware motion. The Policy ABC
exists so adding a new brain doesn't fork the stack. This cycle we proved the abstraction
holds by hanging very different things off it.

  • NVIDIA Cosmos 3 omnimodal VLA (#317) with both a service backend (msgpack/websockets,
    GPU-isolated) and an in-process diffusers backend (#458) for when you'd rather not
    run a sidecar. We re-anchor IK on the achieved EE pose each step (#462) - why: over a
    long rollout, integrating the model's relative pose deltas drifts; anchoring on where the
    arm actually is bounds the tracking error instead of letting it compound.
  • MoveIt2Policy under [moveit2] (#305) and cuRobo migrated to the main API
    (#442) - collision-aware planners living under the same ABC as the VLAs, so an agent can
    choose "plan a safe path" vs. "imitate" without changing how it calls a policy.
  • GR00T N1.7 EA (#93) plus the unglamorous-but-essential wire-format fixes: service
    (B, T, ...) shape + float32 state (#149), container lifecycle (#152), command-builder
    flags (#150, #155). Why these are in a release at all: an off-by-one in the observation
    tensor shape doesn't error - it silently degrades inference. Pinning the wire format is
    what makes "GR00T support" a claim instead of a hope.
  • LeRobot local direct-HF inference with RTC (#56), device-resolution + postprocessor
    warnings (#430), and the LeRobot 0.5.2 recording pipeline overhaul - synchronized
    multi-robot, action-horizon batching, a camera-recorder race fix, full embodiment
    coverage (#366). Why: recording is how you get training data; a race in the recorder
    is silent data loss you only discover when your dataset trains a worse policy.

Evaluation: turning "looks good" into a number

Why: Without a benchmark, "the policy is better now" is unfalsifiable. We adopted a
benchmark-agnostic eval protocol (#129) and a LIB...

Read more

v0.3.8

Choose a tag to compare

@arronbailiss arronbailiss released this 23 Feb 17:50
🤖 Teach a hand once, and it dreams a thousand motions

The full robot lifecycle — collect, dream, train, deploy — unified behind natural language. From every idea, a motion. From every developer, a maker of machines.

Welcome to Strands Robots.