Four failures in a day, three of them mine

The robot lab is open again.

Thanks to everyone who turned up this morning and found it down. I spent the morning working through a stack of failures, and it is back up: robot.musserautomation.com.

Here is what the day before that looked like.

What a day of public traffic does to a machine

Yesterday, 196 sessions that weren't mine opened the lab from 66 locations. 96 of them took control of a real ABB arm.

Between them they surfaced four distinct failures in 24 hours. Three were my own doing.

That ratio is the honest headline. Putting a machine in front of strangers does not mostly find out what strangers do wrong. It finds out what you got wrong, faster than you ever would alone, because a few dozen people will exercise combinations you would never think to try.

A cable that tore, with nothing in the fault log

The failure that taught me the most happened overnight.

The gripper cable ripped out of its socket. That is the connector at the top of this post - look at the bent pins.

The striking part was the fault log: zero collision faults across the whole window. Pins like that usually mean somebody drove the arm into something. The log does not show that.

I am not certain what the specific trigger was. I would rather say so than hand you a tidy cause I cannot back. What I am sure of is that the wrist is involved. My own Home routine commanded joint 6 to absolute angles - zero, then ninety - before the arm settled near fifty-five degrees. Every single Home swept the wrist through roughly two hundred degrees, dragging the gripper cable around with it.

Whether that is the whole story or just the part I can see, I do not know yet. It is enough to act on.

I had already committed the fix. The robot had never loaded it.

That gap - between the code you have written and the code the machine is actually executing - is where this kind of thing lives. Anyone who has flashed a PLC, forgotten to activate a configuration, or shipped a recipe file that never got copied to the HMI has met it wearing a different costume. The repo was right. The robot was still wrong.

Who actually showed up

The spread is the part I did not expect.

Vienna, Aachen, Freiburg, Istanbul, Madrid, Paris, Chennai, Bengaluru, Portland, Las Vegas, Knoxville - and 52 more cities past the top fifteen. Those 196 sessions came from 95 distinct addresses, so a fair number of people came back for another go. Of the sessions I could put a device to, 47% were on a phone, which means a meaningful share of the people driving an industrial robot arm that day were doing it one-handed.

None of that is a vanity metric. It is the load case. A machine that has been driven from 96 sessions across 66 cities in a day has been tested in a way no amount of careful solo commissioning reproduces.

Still open

Not resolved, and I would rather say so than let a tidy ending imply otherwise:

• The camera calibration is still the degraded one from 2026-08-03 - a median reprojection error of 2.35 px against its predecessor's 0.83 px. Picks were landing 20 to 100 mm off, which is how balls ended up outside the pen. It needs redoing.
• Jog is the one public command that still winds the wrist.

Go break it

If you build machines for a living, I would like to know what you make of it. Drive it, then tell me what broke - there is a feedback box and it reaches me directly: robot.musserautomation.com.

It is a lab. Things break in public here, and that is the entire point.