The logging service behind the robot lab

I put a real industrial robot on the public internet this week. For the first stretch, my only evidence that anyone was using it was a vague sense that they were.

If you have not driven it yet, check it out first - no signup, no install: robot.musserautomation.com. The rest of this post is about how I know you were there.

Here is the last 24 hours: 85 sessions from 17 locations across three continents. 28 of them took control of the arm. The rest watched.

I only know that because of the least glamorous piece in the whole stack - the logging service running behind the lab.

What actually gets recorded

Every browser that opens the lab gets two things: a session row, and an event log.

The session row is the shape of the visit - when it connected, how long it held control, where it came from. The event log is the detail underneath it, one row per thing that happened, including every command that reached the robot. Both go from the hub out to a small SQLite database, so the record survives a hub restart and I can ask questions of it weeks later.

Location comes from a MaxMind GeoLite2 file sitting on disk next to the hub. An IP resolves to a country and a region, and that is all that is kept. No coordinates are ever stored. The pins on the map above are region centroids computed at draw time - they point at the middle of a region, not at a person. Free GeoIP is good enough to answer "roughly where are these people" and is not good enough for anything narrower, so I did not build anything narrower.

The rule that makes it safe to run

Instrumentation should not be able to break the thing it instruments.

That sounds obvious right up until the logging database has a bad afternoon and takes your machine down with it. So every write here is fire-and-forget. The hub hands the write off and moves on without waiting for it. If the database is unreachable, the failure is logged and the lab keeps running - I lose the record, never the session.

The whole store is optional. Pull the credentials out of the environment and the lab still works, minus its memory. That is deliberate: a visitor's session is the product, and the telemetry is a passenger. Passengers do not get to drive.

Anyone who has put a historian on a production line has met the other version of this - the data collection that has to be shot in the head at 2am because a full disk is stalling the PLC. Logging that can stop the line is worse than no logging, because you trusted it.

The part I did not expect to like

The hub emails me the first time someone shows up from an address it has not seen. One message per new IP, only for connections from outside my own region.

I built it as a debugging aid. It turned into the most motivating thing in the project. A robot arm sitting in a shop in Georgia is an experiment; the same arm being opened from Brazil, from Spain, from the Czech Republic is a service with users, and the difference between those two arrives in your inbox at odd hours.

What it changes

The dashboard on top of all this went in this morning - unique sessions, controllers versus viewers, the command mix, and the map. It reads back the same store the hub writes to, so there is no second source of truth to keep honest.

Now that I can see which parts of the lab people actually reach for, that is what decides what I build next. Before this, I would have guessed. I would probably have guessed wrong.

Which makes the most useful thing you can do pretty simple: go be a data point.

The lab is at robot.musserautomation.com. No signup, no install. Open it and you will show up as a pin on that map - and whichever button you reach for first is a small vote on what I build next.