XLeRobot-Pro | Software Setup

Read This First

This page assumes no prior robotics experience. If you can copy a command into a terminal and read what comes back, you can finish it.

Do the hardware first

Nothing here works until the robot is built and wired. Both arms, the head, and the wheel base must be assembled, every motor must have its ID assigned, and both bus adapters must be plugged into the Jetson over USB. That is all on the Build Guide — finish it, then come back.

Which robot are you setting up?

The repository supports two shapes, and mixing them up is the single most common way to get stuck. Decide now which one you have:

Shape	What it is	Wiring
Full robot 2-bus	The complete platform: two arms, head, wheel base, 17 motors	`/dev/xle_arms` — left arm IDs 1–6, right arm 7–12 `/dev/xle_head` — head 1–2, wheels 3–5
Benchtop rig single-bus	One arm plus the head on a desk — how the vision and IK stack was developed and how its demos are written	One adapter: head IDs 1–2 and one arm at IDs 7–12 on the same chain

Steps 1–9 are identical for both. Steps 10–12 are where the two diverge, and each step says which shape it applies to.

Words you will see

Bus: One chain of motors sharing a single wire back to the computer. This robot has two.
Motor ID: A number, 1–253, stored inside each servo. It is how the computer addresses one motor out of a chain. Two motors with the same ID break the whole chain.
Calibration: A one-time recording of where each joint's centre and end stops are, so a commanded angle means the same thing on every robot.
udev symlink: A permanent nickname for a USB device. Without one, Linux renames your buses at random on every reboot.
venv: An isolated Python installation, so this project's packages cannot break anything else on the machine.

The Roadmap

Work top to bottom. Do not skip ahead — every step assumes the one above it passed.

Step	What you do	Time
Phase A — Prepare the computer
1	Flash JetPack 6.2.1 onto the Jetson	45–90 min
2	Get on Wi-Fi and switch to SSH	10 min
Phase B — Install the software
3	Create the Python environment	5 min
4	Install PyTorch from the Jetson index	15 min
5	Install XLeRobot-Pro and the vision stack	30–45 min
6	Verify the install — no hardware needed	5 min
Phase C — Connect the robot
7	Grant serial access and name the buses	10 min
8	Confirm every motor answers	10 min
9	Calibrate the robot, then check the calibration	20 min
Phase D — Make it move
10	First motion — the wheels	5 min
11	First motion — the arms	15 min
12	Autonomous cube grasp	20 min
13	Safety limits and Jetson tuning	10 min

ONE CHANGE AT A TIME. If a step fails, fix that step before moving on. Almost every hard-to-debug problem on this platform comes from a working subsystem being disturbed while chasing a broken one.

How to Read the Commands

Every command on this page is run on the Jetson, from the repository root, with the environment activated. Those three conditions look like this:

Every new terminal starts this way

$ source ~/.venvs/xlerobot-pro/bin/activate
$ cd ~/xlerobot-pro

You will know the environment is active because your prompt gains a (xlerobot-pro) prefix. If a command fails with ModuleNotFoundError, that prefix is almost always missing.

The repo root matters. Several scripts read and write capture files under ./outputs/, relative to wherever you launched them. Running from another directory puts the RealSense capture where the detector will not look for it.

1

Flash JetPack 6.2.1

Skip if your Jetson already runs JetPack 6.2.x · 45–90 minutes

THE VERSION IS PINNED, NOT PREFERENTIAL. The stack requires Ubuntu 22.04 / Python 3.10. JetPack 7.x ships Ubuntu 24.04 with a different Python and CUDA, so the pinned PyTorch wheel index does not resolve and the vision dependencies need reworking. Stay on the 6.2.x line even though 7.x is newer.

1a — Read the firmware version

Attach a monitor and USB keyboard, power on, and press Esc repeatedly at the NVIDIA splash. The UEFI banner shows the version. Note the carrier has DisplayPort, not HDMI — this is needed for first boot only.

Firmware	Meaning	What to do
36.x or newer	JetPack 6 generation	Single flash — skip to 1b
35.x or older	Factory firmware; cannot boot JetPack 6	Two-stage bridge required first

If you need the bridge: boot a JetPack 5.1.3 card, finish Ubuntu setup, connect to the internet, reboot once to let the initial firmware update run, then:

QSPI firmware bridge

$ sudo apt update && sudo apt install nvidia-l4t-jetson-orin-nano-qspi-updater

DO NOT INTERRUPT POWER while QSPI flashes after that reboot. This is the only genuinely risky moment in the whole build — a half-written firmware bricks the module.

1b — Write the card

Download jp62-r1-orin-nano-sd-card-image.zip (Jetson Linux 36.4.4) from NVIDIA's JetPack 6.2.1 page and write it with Balena Etcher. Etcher runs on macOS and Windows, so no Ubuntu host or SDK Manager is required for the devkit.

The microSD slot is on the module, not the carrier. It sits on the underside of the Orin module, tucked under its top edge — the opposite end from the I/O bank. Look into the gap between module and carrier with a flashlight; contacts face up. It is push-push: if it springs back out, it never seated.

1c — First boot

Complete the Ubuntu OEM wizard. When it asks for APP partition size, take the maximum — the default can leave a cramped root filesystem that is tedious to resize later. Join Wi-Fi during the wizard; step 2 depends on it.

Confirm the release

$ cat /etc/nv_tegra_release

Done when

That command prints R36, revision 4.4 or later, and the machine is on your Wi-Fi network.

If it fails

UEFI Interactive Shell v2.2 — Shell> means nothing bootable was found. Run map -r: if the only entries are MemoryMapped and Fv(...), no card is being seen — reseat it, or confirm firmware is 36.x.
Error: could not make network connection is not a network fault. UEFI exhausted its boot order and fell through to PXE. It disappears once a bootable card is present.
More boot-time signatures on the Troubleshooting page.

2

Switch to SSH

Work from your laptop instead of a monitor · 10 minutes

This is the single change that makes the rest of the setup pleasant, and it is not optional in practice. The Jetson has few USB ports, and the two bus adapters plus three cameras need five of them.

No hub required. A keyboard and monitor occupy ports you need for hardware. Working remotely leaves every port free — so you never have to put a motor adapter behind a hub, and you avoid the enumeration failures that come with it.
Every connection stays direct. Fewer things in the USB tree means fewer things that can drop out mid-run, and one less variable when something misbehaves.
You can work while the robot moves. No cable tethering you to the cart.

On the Jetson, once

$ sudo apt install -y openssh-server
$ hostname -I
# note the first address it prints

From your laptop

$ ssh <user>@<jetson-ip>

DISCONNECT THE MONITOR WHEN YOU ARE DONE. The robot is meant to drive around untethered — a DisplayPort cable running to a desk is a leash, and it will either stop the robot short or get yanked out of the carrier mid-run. Once SSH works, unplug the monitor and keyboard and leave them off.

Done when

You are logged into the Jetson from your laptop, and the monitor and keyboard are unplugged.

3

Create the Python Environment

System Python 3.10, isolated in a venv · 5 minutes

DO NOT INSTALL INTO A CONDA ENVIRONMENT. The Jetson PyTorch wheels are built against JetPack's system Python 3.10 and the system CUDA libraries. Inside conda they hit its bundled libstdc++ and OpenMP runtime, and either fail at import or — worse — leave torch.cuda.is_available() silently False. If a conda base auto-activates in your shell, run conda deactivate before building the venv.

Build the environment

$ sudo apt install -y git python3.10-venv python3.10-dev
$ /usr/bin/python3.10 -m venv ~/.venvs/xlerobot-pro
$ source ~/.venvs/xlerobot-pro/bin/activate
$ pip install --upgrade pip

python3.10-dev is not optional — quadprog, used by the IK solver, builds from source and needs the Python headers.

Clone the repository

$ git clone https://github.com/Minko82/xlerobot-pro.git ~/xlerobot-pro
$ cd ~/xlerobot-pro

Done when

Your prompt shows (xlerobot-pro).
python --version prints 3.10.x.
~/xlerobot-pro exists and contains pyproject.toml.

4

Install PyTorch

Jetson only — skip on a laptop · 15 minutes

Do this before installing the project. The generic PyPI torch wheel is CPU-only on aarch64, so NVIDIA's JetPack 6 build has to land first — otherwise pip pulls the wrong one and CUDA silently never works.

PyTorch from the Jetson index

$ pip install torch torchvision \
--index-url https://pypi.jetson-ai-lab.io/jp6/cu126

Check which wheel you got

$ python -c "import torch; print(torch.__version__, torch.cuda.is_available())"
# want: 2.10.0 True — bad: anything ending in +cpu, or False

WHY THE CPU WHEEL WINS IF YOU LET IT. PyPI publishes an aarch64 build tagged 2.10.0+cpu. Under PEP 440 a local-version segment sorts higher than plain 2.10.0, so if PyPI is reachable during the resolve — for instance via --extra-index-url — pip prefers the CPU build. Use --index-url exactly as written above. If you already got +cpu, reinstall by direct wheel URL from the index.

Note the index moved to pypi.jetson-ai-lab.io; the old .dev domain no longer resolves.

Done when

The check prints a version with no +cpu suffix, followed by True.

If it fails

False with a clean version number → torch is probably installed inside conda. Deactivate conda, rebuild the venv from step 3.
+cpu in the version → the resolve reached PyPI. Uninstall torch and reinstall with --index-url only.

5

Install XLeRobot-Pro

The robot stack, then the vision and IK stack · 30–45 minutes

Two installs. The first brings in the vendored LeRobot fork with all robot, camera, and teleop extras. The second adds perception and inverse kinematics.

1 — the robot stack

$ cd ~/xlerobot-pro
$ pip install -e ".[all]"
# equivalent to: make install

2 — the vision stack (Jetson)

$ grep -v '^open3d' requirements/requirements-vision.txt > /tmp/req-vision.txt
$ pip install -r /tmp/req-vision.txt --no-cache-dir

open3d is excluded deliberately on aarch64. No wheel exists for it there, so pip attempts a multi-hour source build. Nothing in the verification suite or the cube demo needs it — it affects only point_cloud.py and diagnostics/visualize_point_cloud.py. Everything else (pinocchio, pink, MuJoCo, RealSense) has aarch64 wheels, and quadprog builds from source in seconds given python3.10-dev.

On a laptop, install the file directly — pip install -r requirements/requirements-vision.txt — and keep open3d.

DISK RUNS TIGHT ON A 64 GB CARD. Expect roughly 50 GB used after a full install; pip cache purge reclaims 1–2 GB afterwards. Recorded LeRobot datasets live under ~/.cache/huggingface/lerobot/ — that directory also holds your calibrations, so never clear it wholesale.

Want an exactly reproducible machine? Use the pinned lockfiles instead of .[all]: requirements/requirements-ubuntu.txt (Ubuntu 24.04, x86_64) or requirements/requirements-macos.txt (macOS arm64). They match a known-good development machine, not the Jetson.

Done when

Both installs finish without errors and python -c "import lerobot, xlerobot_pro" prints nothing at all. Silence is success.

6

Verify the Install

No motors, no camera, no risk · 5 minutes

Prove the software before involving hardware. A failure here is far cheaper to diagnose on its own than one tangled up with a wiring fault.

Offline verification

$ make smoke
$ python tests/vision_offline_suite.py

make smoke compiles every Python file in the repo and runs the hardware-free test suite: model assets present, config paths resolve, calibration files parse, and the firmware limits are actually wired into every robot class and demo. The vision suite then exercises the whole perception → frame-transform → IK → FK chain in software.

Done when

make smoke reports 7 passed.
vision_offline_suite.py reports 49/49 passed, 0 failed.

A pass means every install decision above was correct. Stop here and fix anything that failed.

SEEING "6 passed, 1 skipped" INSTEAD? That is not a failure — the seventh test skips itself when the vision extras are absent, and says so in the skip reason. It means part 2 of step 5 did not take. Install requirements-vision.txt and re-run; you want all seven.

If it fails

ModuleNotFoundError: lerobot → the venv is not active, or step 5 did not finish. Re-run source ~/.venvs/xlerobot-pro/bin/activate then pip install -e ".[all]".
Errors mentioning pin, pink, or quadprog → the vision install is incomplete. Confirm python3.10-dev is installed and re-run part 2 of step 5.
libgomp / OpenMP errors → add echo 'export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1' >> ~/.bashrc and open a new shell.

7

Name the Buses

Serial permissions, then stable device names · 10 minutes

Plug both bus adapters directly into the Jetson — not through a hub — and power the motors. Then two things have to happen before any script can talk to them.

7a — Serial permissions

The buses are USB serial devices, so your user needs permission to open them.

Grant serial access

$ sudo usermod -aG dialout $USER
# log out and back in — the group does not apply to your current shell

On macOS no extra permission is needed; the buses appear as /dev/tty.usbmodem*.

7b — Give the buses permanent names

Both adapters use the same USB-serial chip, so Linux names them /dev/ttyACM0 and /dev/ttyACM1 in whatever order they happen to enumerate — and that order changes between boots and after any replug. Every entry point in this project opens /dev/xle_arms and /dev/xle_head instead. Those are stable nicknames keyed to each adapter's own serial number, and one script creates them for you.

Identify the buses and install the rules

# see what it found first
$ python diagnostics/detect_buses.py

# happy with the result? write the rules (asks for sudo)
$ python diagnostics/detect_buses.py --write

$ sudo udevadm control --reload-rules && sudo udevadm trigger
# now unplug and replug BOTH adapters
$ ls -l /dev/xle_arms /dev/xle_head

The script pings every ID on each port and works out which adapter is which by what answers: the arms bus replies on IDs 1–12, the head/base bus on 1–5. It then writes /etc/udev/rules.d/99-xlerobot.rules with each adapter's serial filled in.

Delete any older rules file first. If you have a 99-xlerobot.rules from a previous build with different serials, remove it. Stale rules are confusing and can define names that nothing uses.

PORTS ARE OVERRIDABLE with the XLEROBOT_ARMS_PORT and XLEROBOT_HEAD_PORT environment variables if you would rather point at raw device names. The symlinks are strongly preferred though — without them, a replug can silently swap which bus a script talks to, and the arms will move as if they were the head.

Done when

ls -l /dev/xle_arms /dev/xle_head lists both symlinks, each pointing at a ttyACM device.

If it fails

No /dev/ttyACM* devices at all → this is a USB problem, not a motor problem. Check dmesg for error -110 (timeout) or error -71 (protocol error) and unable to enumerate USB device. Try a new cable first, then a different port, and only then suspect the adapter. Never put a bus adapter behind a hub.
Devices exist but detect_buses.py finds no motors → the adapters have logic power but the servos do not. See step 8.
Both ports answer the same ID range → you have duplicate motor IDs across buses. Go back to Motor Configuration.

8

Confirm Every Motor Answers

Read-only — nothing moves · 10 minutes

Before anything is commanded to move, prove that all 17 motors are reachable and reporting sane values.

Check both buses

$ python diagnostics/check_bus.py --motors arm
# right arm, IDs 7-12, on /dev/xle_arms

$ python diagnostics/check_bus.py --motors head
# head, IDs 1-2, on /dev/xle_head

To sweep an entire bus for anything that answers — at any ID and any baud rate, which is what you want when a motor is missing:

Full bus scan

$ python -c "from lerobot.motors.feetech import FeetechMotorsBus; \
print(FeetechMotorsBus.scan_port('/dev/xle_arms'))"

Done when

A scan of /dev/xle_arms lists IDs 1–12 with no gaps.
A scan of /dev/xle_head lists IDs 1–5 with no gaps.
Both check_bus.py runs print positions rather than errors.

If it fails

Whole bus silent, but each motor works alone → duplicate IDs, essentially every time. Feetech is half-duplex: two motors on one ID transmit together and the collision corrupts every packet on the wire. A right arm still shipped as IDs 1–6 duplicates the entire left arm. Fix in Motor Configuration.
Scan returns fewer IDs than motors attached → the missing number names the culprit. Halve the chain and re-scan; about four rounds isolates the bad motor or cable.
Motors answer but report load pinned at 1024 + Torque_Limit with current 0 → the bus has logic power but no motor power. Check Present_Voltage: ~120–135 means 12 V (good), ~50 means the bus is on a 5 V rail. A 5 V USB bank will not drive these servos.
Permission denied: /dev/ttyACM0 → you added yourself to dialout but did not log out and back in.

9

Calibrate

Once per robot, then verified · 20 minutes

TORQUE IS RELEASED FOR THE WHOLE PROCEDURE. Both arms go limp and will fall from a raised pose. Support them by hand before you start, and clear everything from underneath them.

9a — Run it

lerobot-calibrate cannot see this robot. Its --robot.type does not offer xlerobot_new_wiring. The class is registered, but robots/utils.py imports robot types lazily inside its dispatch function, so the config module never loads and the registration decorator never runs. Drive the class directly with the script below instead.

Save this as calibrate_xlerobot.py in the repo root:

calibrate_xlerobot.py

from lerobot.robots.xlerobot.config_xlerobot import XLerobotNewWiringConfig
from lerobot.robots.xlerobot.xlerobot_new_wiring import XLerobotNewWiring

config = XLerobotNewWiringConfig(
    id="xlerobot",
    port1="/dev/xle_arms",
    port2="/dev/xle_head",
)
robot = XLerobotNewWiring(config)
robot.connect(calibrate=False)
try:
    robot.calibrate()
finally:
    robot.disconnect()

Run it

$ python calibrate_xlerobot.py

The run has four prompts, in this order:

Move both arms to the middle of their range and press Enter.
Sweep every arm joint through its full travel — all the way to each end stop, both arms.
Centre the head and press Enter.
The wheels are handled automatically as continuous-rotation motors.

The result is written to ~/.cache/huggingface/lerobot/calibration/robots/xlerobot_new_wiring/xlerobot.json. Reference calibrations from a known-good robot are versioned in the repo's calibration/ directory as a sanity check or fallback.

9b — Verify it. Do not skip this.

Calibration can complete successfully and still be wrong, and both failure modes are silent:

Range reads 0–4095 on a limited joint. The homing offset landed inside the travel range, so the raw count wrapped and min/max pinned to the encoder extremes. This is legitimate only for wrist roll and the wheels — a shoulder or elbow cannot travel 360°. Consequence: IK commands positions that do not exist and the arm drives into its hard stop.
Range far narrower than its mirror. That joint was not swept fully. Consequence: the arm is artificially limited and IK fails to reach valid targets.

The strongest check is left/right symmetry. Two independently-swept arms landing within a few percent of each other is not something you get by accident. Here is a known-good run:

Joint pair	Left span	Right span	Ratio
shoulder_pan	2715	2732	0.99
shoulder_lift	2418	2433	0.99
elbow_flex	2217	2197	0.99
wrist_flex	2358	2335	0.99
gripper	1526	1448	0.95

Done when

The calibration JSON exists in the cache directory above.
No limited joint shows a 0–4095 range.
Mirrored joints agree within a few percent.

If it fails

Bus drops mid-calibration, always at the same joint → the cable is being tugged at that joint's travel extreme. Range recording retries only once, so a single dropped packet aborts the run. Add a service loop at that joint. Do not raise the retry count to mask it — the same dropout will recur mid-grasp, under power.
An arm falls when the script ends → torque was cut rather than bled off. Support the arms; see the gotchas in the repo's diagnostics/README.md.

10

First Motion — the Wheels

Full robot only · 5 minutes

The base is the safest thing to move first: it is the only subsystem where a mistake cannot drop an arm on your hand. Start here.

PUT THE ROBOT SOMEWHERE IT CAN MOVE. The default run drives forward a few centimetres. Clear a metre of floor and keep a hand near the power switch.

Base drive check

# slow walk of a few centimetres — start here
$ python diagnostics/base_drive_check.py --vx 0.05 --seconds 2

# rotate in place instead
$ python diagnostics/base_drive_check.py --omega 15 --seconds 2

This touches the wheels only — the arms are never commanded. It answers three questions: do the wheels turn in the commanded direction, does wheel odometry read back the velocity that was commanded, and does the base stop reliably on every exit path. That last one is the one that matters; the wheels are stopped in a finally, on Ctrl-C, and on SIGTERM, and the stop is verified by reading the speeds back rather than assumed.

The script refuses anything outside its first-motion envelope rather than quietly clamping it — over 0.15 m/s, over 45 deg/s yaw, or longer than 10 s and it prints FATAL and exits without moving. So an over-ambitious command is a message, never a surprise.

Done when

The base moves in the direction you commanded.
Odometry agrees with the commanded velocity.
The base stops, and stays stopped, when the script exits.

If it fails

A wheel spins the wrong way → its motor ID does not match the position it is mounted in. IDs 3, 4, 5 map to left, back, right. Recheck against the wiring diagram.
A bus vanishes after the base drives → a cable was tugged loose. Check dmesg for error -110 or unable to enumerate, and strain-relieve the leads before any further driving.

11

First Motion — the Arms

Full robot and benchtop rig · 15 minutes

THE ARMS MOVE UNDER POWER. Clear the workspace, keep hands out of the swept volume, and keep a hand near the supply switch. Torque and acceleration are clamped by src/xlerobot_pro/firmware_limits.py — arms 450/40, wheels and neck 650/20 — but a clamped arm still hits hard enough to hurt.

Full robot — whole-platform teleoperation

On the finished 2-bus robot, the entry point that drives both arms, the head, and the base together is the Xbox teleop built on the XLerobotNewWiring class. It is the _new_wiring variant that matches this build — the plain 5_xlerobot_teleop_xbox.py is the old three-bus layout and will not connect.

Whole-robot teleop

# confirm the controller is seen first
$ python examples/pygame_test.py

$ python examples/5_xlerobot_teleop_xbox_new_wiring.py

Single-arm keyboard control

Start with joint control to confirm each motor moves in the direction you expect, then move to end-effector control. Use the copies under examples/xlerobot_examples/ — they prompt for the port at startup, so nothing needs editing.

Single-arm bring-up

# joint-by-joint, no IK
$ python examples/xlerobot_examples/0_so100_keyboard_joint_control.py

# end-effector control
$ python examples/xlerobot_examples/1_so100_keyboard_ee_control.py

# both ask for a port. Full robot: /dev/xle_arms
# Benchtop rig: /dev/ttyACM0 (or press Enter for that default)
# then answer "n" to "recalibrate?" — you already did that in step 9

THESE SCRIPTS DRIVE ONE ARM AT MOTOR IDs 1–6. On the full robot, that is the left arm. Pointing them at /dev/xle_arms moves the left arm and never the right — the right arm is at IDs 7–12, which needs a different motor map, not just a different port. For two-armed work, use the whole-robot teleop above. A full walkthrough of the end-effector script is on the First Reach tutorial.

AVOID THE COPIES DIRECTLY UNDER examples/. examples/0_so100_keyboard_joint_control.py and examples/2_dual_so100_keyboard_ee_control.py hardcode /dev/xle_right and /dev/xle_left — port names from the older three-bus layout that do not exist on this build. They fail to connect until you edit the port near the bottom of the file. Same filename, different behaviour; check the directory.

Cameras

Verify the camera streams

$ v4l2-ctl --list-devices
$ python examples/9_dual_wrist_camera.py
# RealSense head camera
$ rs-enumerate-devices

Done when

Every arm joint moves, and moves in the direction you expect.
The head tilts and pans.
All three camera streams open.

If it fails

Arm jerks or moves wildly → almost always a motor ID mismatch or a reversed servo horn. Recheck Motor Configuration.
Arm drops when a script ends → torque was cut rather than bled off. Never call disable_torque on a loaded arm; step Torque_Limit down over several seconds instead.
RealSense frames time out → replug the USB 3.0 cable and re-run rs-enumerate-devices.

12

Autonomous Cube Grasp

Benchtop rig — read the note before running on the full robot · 20 minutes

This is the payoff: a RealSense camera detects a coloured cube, the detection is transformed into the arm's base frame, the differential-IK solver plans to it, and the arm reaches, grasps, lifts, and releases. You already validated the whole chain in software back in step 6 — this runs it against real hardware.

THIS DEMO IS WRITTEN FOR THE SINGLE-BUS BENCHTOP RIG. grab_cube.py opens one port and expects head tilt/pan at IDs 1–2 and the arm at IDs 7–12 on that same chain. On the finished 2-bus robot, IDs 1–2 of /dev/xle_arms are the left arm's shoulder joints — so the demo would connect happily, read shoulder angles as head angles, and aim the grasp confidently at the wrong place.

Run it on a single-bus rig, or set XLEROBOT_BUS_PORT to a bus that genuinely matches that map. This is a known limitation of the demo, not of your build.

Autonomous cube grasp

$ cd ~/xlerobot-pro
# on a single-bus rig, point it at that adapter
$ export XLEROBOT_BUS_PORT=/dev/ttyACM0
$ python examples/vision/grab_cube.py

Expected sequence: calibration loads → head angles print → firmware limits applied → RealSense captures → red blob detected → camera coordinates transformed into the base frame → IK solves → the arm reaches, grasps, lifts, releases.

HEAD GEOMETRY FEEDS THE TRANSFORM. The demo reads live head tilt and pan to place the camera relative to the arm base. Getting the two swapped produces a transform that looks entirely plausible and sends the arm confidently to the wrong place. Confirm against your hardware that ID 1 moves up/down and ID 2 moves left/right before trusting a grasp.

No camera? Run the scripted motions

The scripted demos in examples/vision/scripted/ exercise the arm and the IK solver without any camera at all — useful for proving the motion half of the pipeline on its own.

Scripted motions (no camera)

$ python examples/vision/scripted/so101_simple.py
$ python examples/vision/scripted/so101_reach_forward.py
# also: so101_pen_pickup, so101_pen_drop, so101_grab_cube,
# so101_grab_apple, so101_hang

These hardcode /dev/ttyACM0 and the default SO-101 IDs 1–6. Edit the port at the top of the file to match your rig.

Debugging a grasp that misses

When the arm moves but lands in the wrong place, these separate perception error from transform error from mechanical error:

Tool	What it tells you
`diagnostics/check_vision_live.py`	Live comparison of camera detection against the transformed base coordinates
`diagnostics/diagnose_fk_vs_vision.py`	Place the gripper on the target by hand, then compare forward kinematics against what vision reports
`diagnostics/debug_transform_chain.py`	Dumps every intermediate matrix in the camera-to-base transform
`diagnostics/visualize_color_detect.py`	Overlays the detection on the last capture — is it even finding the cube?
`diagnostics/visualize_mujoco.py`	Replays the whole transform + IK pipeline in simulation

Done when

The arm reaches the cube, closes the gripper on it, lifts it, and releases. Accuracy within 1–2 cm is normal; better than that depends on how carefully the arm was assembled.

13

Safety Limits & Jetson Tuning

Read before changing anything · 10 minutes

Firmware limits live in exactly one file

System-wide maximum torque, acceleration, and speed are all in src/xlerobot_pro/firmware_limits.py. The defaults implement the paper's firmware saturation settings, sized so each power bus stays inside its fuse rating and a motor transient can never brown out the Jetson.

Bus	Torque limit	Acceleration	Fuse
Wheels / neck	650 (≈1.91 N·m)	20	10 A
Arms	450 (≈1.32 N·m)	40	5 A

They are applied automatically when a robot connects and by the vision demos. Edit that file to change platform maximums — never the demo scripts. Then confirm what the motors actually stored:

Verify the limits took

$ python diagnostics/verify_motor_limits.py

The arm limit of 450 is sized so that a dual-arm stall — every joint saturating at once — still fits inside the 5 A arm-bus fuse. Raising it changes that guarantee.

Jetson tuning

Goal	Command
Confirm GPU PyTorch is live	`python -c "import torch; print(torch.cuda.is_available())"` → `True`
Maximum inference performance	`sudo nvpmodel -m 0 && sudo jetson_clocks`
Fix `open3d` OpenMP errors	`export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1`

ON BATTERY, PREFER THE DEFAULT POWER MODE. Max power mode is for bench work with the compute rail on mains. Running it off the Tri-Bus battery pushes the compute rail outside its power envelope.

Where Everything Lives

Once the robot works, this is the map for going further.

Path	Contents
`src/lerobot/`	Vendored LeRobot fork with the XLeRobot-Pro robots, teleoperators, and power tooling
`src/xlerobot_pro/`	Vision + IK library, robot model, and the firmware safety limits
`examples/`	Tested bring-up, teleoperation, and policy examples
`examples/vision/`	Autonomous perception + IK demos
`examples/xlerobot_examples/`	Upstream references — unverified against this build, may need edits
`diagnostics/`	Bus, motor, and frame-transform debugging tools
`calibration/`	Reference motor calibrations from a known-good robot
`tests/`	Hardware-free checks (`make smoke`)

Going further

Record a dataset

Teleoperate while lerobot-record captures synchronised camera and joint data into a LeRobot dataset.

Train and run a policy

lerobot-train, then examples/policies/ for running SmolVLA and diffusion policies on the robot.

VR teleoperation

examples/teleop_hand_and_arm.py, plus the XLeVR/TeleVuer stack from the upstream XLeRobot VR docs.

Something is broken

The Troubleshooting page indexes failures by symptom, including the ones that present as something other than their cause.