A success rate is the first number anyone asks about an automation, and the easiest one to fudge without meaning to. Here's how I define ours, with the dashboard numbers both ways, so you can decide if I'm being fair.
The numbers
From our dashboard on 12 October 2026, the config-driven RPA platform at Edvoy had run 3,422 jobs across 71 configured portals, 66 of them active:
| Outcome | Jobs | What it means |
|---|---|---|
| Succeeded | 3,075 | The portal submission went through. |
| Failed | 175 | The platform couldn't finish the job. |
| Stopped by a user | 172 | An operator ended the job on purpose. |
Divide successes by every job and you get about 90%. Divide successes by successes plus failures and you get 94%. Same jobs, same dashboard, a four-point gap.
Why user-stopped jobs don't count
An operator stops a job for their own reasons. Maybe the record had a mistake, or they picked the wrong portal. Whatever it is, the platform didn't fail. It was told to stop, and it stopped.
If I counted those as failures, the success rate would move every time operators changed how they work, even if the platform didn't change at all. That makes it a bad number to manage the platform by. So the rate I report is:
success rate = successful jobs / (successful jobs + failed jobs)The important half of this rule is that user-stopped jobs don't count for the platform either. They're not in the top of the fraction or the bottom. They get their own line on the dashboard, and if that line grows, it's a question worth asking, just not a platform failure.
The trap: stopping is a loophole if you let it be
Excluding a category is only fair if nobody can push failures into it. If the bot could quietly "stop" itself when it got stuck, every hard failure would vanish from the denominator and the rate would look great.
So the platform tells the two apart. When a job ends because a person closed it, that's detected and recorded as a user stop. When it ends because the platform gave up, that's a failure, with a reason. The bot can't choose which bucket it lands in.
Give every failure a label
A success rate tells you how often things go wrong. It doesn't tell you what to do about it. For that, every failed job on our platform gets one of four labels:
- Config error: our mistake. Someone fixes the portal's config.
- Site changes: the portal moved something. The config needs an update.
- Connection issue: timeouts and slow pages. Usually fine on a retry.
- Blocked, attention required: the bot hit something it shouldn't guess at. A person looks.
The labels matter more than the rate. A 94% that's mostly connection blips is a healthy platform. A 94% that's mostly site changes on one portal is a config that needs attention this week.
Bot time is not wall-clock time
The other number people ask about is speed. By hand, one submission took about 30 minutes. The bot's own time averages about 5 minutes per job.
That 5 minutes is bot time only. A job's wall-clock time is longer, because it includes pauses where a person enters a one-time code or gets documents ready. Those pauses can be long, and they're not the bot's doing. If I quoted wall-clock time, I'd be blaming the bot for waiting on people. If I counted the pauses as savings, I'd be inflating the hours saved. So I quote bot time, and I say that's what it is.
A checklist for your own success rate
- Write down what counts as a success. "The bot finished" isn't it. "The target system confirmed it" is.
- Decide which outcomes aren't the platform's to own, and make sure the bot can't push failures into them.
- Report the excluded outcomes next to the rate, not instead of it.
- Label every failure with a reason that tells you who fixes it.
- Separate machine time from waiting time before you quote speed or savings.
None of this is clever. It just means that when someone asks "is 94% real?", you can show them the table above and let them check.
Figures are from the live dashboard of the platform described in the config-driven RPA case study, October 2026.