Target
The cube position from the camera.
p* = (x, y, z)
Personal project / PyTorch / Webots / Aug to Sep 2026
UR5e neural inverse kinematics
A small network that gives the six joint angles for a target position.
I trained an MLP that solves IK for a UR5e with the tool pointing down. The loss runs the predicted angles through a PyTorch copy of the robot's forward kinematics, and the model drives a camera-guided pick-and-place loop in Webots.
01. Results
Error, held-out targets0.19 mm
Solve time, median0.53 ms
Against IKPy72x faster
Grasp point, net vs closed form0.33 mm
The error is on 2,021 test targets the network never saw. Time is on one thread, and IKPy gets the same pose problem. The 0.33 mm is the gap at the grasp point in Webots between the pose reached with the network and with the closed-form IK (one run; 4.8 mm mean over the scenario's 4 targets). The closed-form solver is still 2.3x faster.
A multi-head version trained without branch labels finds about half of the 8 IK branches, but with about twice the error, so I kept one branch.
02. How it works
Network
3 layers of 512, SiLU, 530k parameters.
q = f(p*)
Forward kinematics
The DH chain in PyTorch, so pose error has a gradient.
p(q), R(q)
Loss
Position and orientation error, plus a small data term that holds one branch.
|p - p*|² + ...
FK is only used in training. At run time it's one forward pass. The camera side is a color threshold plus a homography I fit on a calibration grid (1.1 mm median residual, on the calibration points themselves).
03. How I checked it
Solver benchmark
Slower than the closed form, much faster than IKPy
Same 200 pose targets for every solver, 200 warm-up calls on other targets, one thread. From the rest configuration IKPy fails on 58% of targets. The network had no failures inside its training box.
Checks
- The Jacobian matches finite differences to 3.2 × 10⁻¹⁰.
- The closed-form IK I inherited gave elbow-down solutions about 0.78 m off. I fixed it, and a smoke test round-trips all 8 branches.
- In Webots I send the arm to the same target with the closed form and with the network, and compare where it ends up.
What didn't work
- A frozen VGG16 (ImageNet weights) with a regression head trained on Webots camera images to predict the red cube's position: 49 mm error, against 8 mm for a color threshold (measured on an earlier camera setup). Its labels came from the threshold itself, so it could not beat it. I dropped it.
- The FK loss terms gave no significant gain on single-branch data (5 seeds, p = 0.26).
- On data that mixes branches, a plain supervised network averages them and lands 911 mm off.
04. Limitations
Outside the training box the error grows: 14.3 mm median on a z = 0.25 m slice, 7 to 11 mm above the tray. Past the reach limit it still returns six angles, up to 1.78 m off, with no warning, so it needs a reachability check in front. One tool orientation, one IK branch, simulation only.
05. Links
The Webots scene, the NumPy FK and the closed-form IK come from allan-almeida1/ur5-pick-and-place-webots (MIT). CREDITS.md lists what I inherited and what I wrote.