19 October 2026, 05:29 PM
Performance Testing Metrics Explained: Response Time, Throughput, TPS, and Error Rate
"We hit 400 TPS."
It's a proud sentence, and I've heard it in review meetings more times than I can count. It usually gets a nod. Then somebody quietly asks, "Four hundred what?" and the room discovers that three people have been picturing three different things. One heard requests per second. One heard completed purchases per second. One assumed it was database transactions, because that's what TPS meant at their last company.
All three were right, in their own world. None of them could compare the number with last quarter's, or with a vendor's brochure, or with the capacity plan.
That's the quiet problem with performance testing metrics. The words are familiar, the tools print them confidently, and the definitions underneath are looser than anyone admits. So this isn't a list of formulas. It's a tour of what each of the four common numbers really measures, what it hides, and how to read them together. Because on their own they mislead. Together, they tell a story.
Start With the Stopwatch Question
Take response time. Everyone knows it's "how long a request takes." But the stopwatch can start and stop in at least four places, and each gives a different number.
Protocol-level tools such as JMeter, k6 or Gatling give you the first three at best. They don't run a browser's rendering engine, so they can't tell you when a page looked ready. If a stakeholder cares about "how fast the page feels," a protocol-level response time is the wrong instrument.
Three other decisions change the number quietly. Does it include redirects? Does it include the sub-resources a page loads, or just the main document? And does your tool count think time? It shouldn't, and I've seen reports where it did.
Report a distribution, not a single figure. The median shows the typical experience, and the 95th and 99th percentiles show the unlucky ones. An average is a fine thing to mention in passing and a dangerous thing to put on a slide. I've written about the tail elsewhere, so I'll only add one rule here: always say which percentile, over which window, for which transaction.
Throughput: Requests, Bytes, or Work?
Throughput is the amount of something per unit of time, and the "something" is where the confusion starts.
Requests per second is the common one and the one most tools print. It's a good measure of how hard the server is being pushed.
Bytes per second matters for downloads, media and large API payloads. A system can have modest request rates and still saturate a network link.
Business operations per second (orders, payments, searches) is what the business cares about, and it needs a definition of what counts as one.
Here's the trap. A "request per second" is not a unit of customer value. If one checkout makes five requests, then 320 requests per second is about 64 checkouts per second. Quote the first number to an executive and they'll think you did 320 sales.
Also notice that throughput is an output of a test in most closed-loop setups. You choose the number of virtual users, and the system decides how much throughput it will deliver. If you want to fix throughput, you have to use an arrival-rate model. Mixing those two up is a classic source of "but we asked for 500 per second and only got 330" confusion.
TPS: The Most Overloaded Three Letters in Testing
TPS means transactions per second, and "transaction" has at least three common meanings.
- A database transaction. A commit, in the strict sense. This is the world of benchmark suites and DBAs.
- A single HTTP request. Many testers use it loosely as a synonym for request rate, which is how the term leaks into load tool reports.
- A business transaction. A complete user action: log in, place an order, make a payment. In JMeter you'd model it with a transaction controller.
None of these is wrong. Using them interchangeably is. My rule is blunt: in any report, write the unit out. "Checkouts per second" and "API requests per second" both carry information. "TPS" alone doesn't.
One more subtlety. TPS counts attempts unless you say otherwise. A system that returns errors quickly can post a high TPS while delivering very little. That's why I always show the successful rate next to the raw one.
Error Rate Is a Fraction, and Fractions Need Both Halves
"One percent errors" sounds precise. It isn't until you've answered two questions.
What counts as an error? HTTP 5xx, certainly. Timeouts, yes, because the user didn't get an answer. Connection failures, yes. A 4xx is trickier: a 401 on login with a wrong password is correct behavior, while a 429 from a rate limiter means you're being throttled and your test is now measuring the limiter. And then the worst category, the 200 that isn't a success: a response with a status of 200 and a body saying "something went wrong." Unless your test has assertions that look at content, those count as passes.
What's the denominator? Failed requests as a share of requests, failed transactions as a share of transactions, or affected users as a share of users all give different answers. A journey with ten steps and a 1 percent per-request failure rate fails roughly 1 minus 0.99 to the tenth power, about 9.6 percent of the time, if failures are independent. A tidy "1 percent" at the request level can hide one in ten customers failing to finish. That's an idealized calculation, but it shows why you should report at the level the customer experiences.
And remember the flattering effect of fast failures. Errors usually return quickly, so a system in trouble can show better response times while getting worse. Never read latency without the error rate next to it.
Reading Them Together
Now the useful part. Here's a made-up but realistic-shaped run, with load stepped up from 50 to 800 virtual users, one second of think time per user, and five requests per business transaction. Every number is illustrative, built to show the pattern, not taken from a real system.
From 50 to 300 users, throughput rises almost in proportion to users, and response time barely moves. The system has spare capacity. This is the healthy stretch.
Around 400 to 500, throughput flattens near 330 requests per second while response time starts to climb. That's the knee. The system is working as hard as it can, and extra users just wait longer. Errors are small but no longer zero.
Beyond 500, more load produces less output. Throughput falls, response time explodes, and errors accelerate. Queues, timeouts and retries are eating the capacity. Adding users here isn't a test of capacity anymore. It's a study of failure.
Notice also that these numbers obey Little's Law, which says users in the system equal throughput multiplied by the time each one spends per cycle, including think time. At 400 users: 320 × (0.25 + 1.0) = 400. When your own numbers don't satisfy that relationship, something's off, perhaps in your tool, your measurement window, or your assumptions. I treat it as a free sanity check.
A single figure from this table would have lied. "Peak throughput 333 per second" looks wonderful, and it's the same moment the error rate passed one percent and response time doubled. The usable capacity is nearer to 400 users, where the response time is still acceptable.
Computing the Numbers Honestly
Report generators are convenient, but I like to be able to recompute the headline numbers from raw results. Here's a query that does it from a table of raw samples, using a warm-up cut and a clear definition of an error. It runs on PostgreSQL and DuckDB. It assumes a table with a timestamp, an elapsed time in milliseconds, and a flag for errors that your loader sets as
status >= 500 OR timed_out OR assertion_failed.
SELECT
date_trunc('minute', ts) AS minute,
count(*) AS requests,
round(count(*) / 60.0, 1) AS requests_per_sec,
round(count(*) FILTER (WHERE NOT is_error) / 60.0, 1) AS successful_per_sec,
percentile_cont(0.95) WITHIN GROUP (ORDER BY elapsed_ms) AS p95_ms,
round(100.0 * count(*) FILTER (WHERE is_error) / count(*), 2) AS error_pct
FROM load_results
WHERE ts >= (SELECT min(ts) + interval '2 minutes' FROM load_results)
GROUP BY 1
ORDER BY 1;
Plot those per-minute values against the load level, and you'll get the same shape as the table above. If the shape looks different from what your tool's summary claimed, investigate before presenting anything.
Where These Metrics Mislead
A few limits are worth stating plainly.
Averaging across runs or across transactions blends the quick and the slow into something nobody experienced.
Comparing numbers between tools is risky. Tools differ in what they count as elapsed time, how they treat connection reuse and how they handle timeouts.
Targets without context are hollow. "500 TPS" means little without the transaction mix, the data volume, the environment size and the latency you demand at that rate.
Test environments that are smaller than production shift everything. Scaling results linearly to production is a hope, not a calculation, because caches, pools and shared services don't scale linearly.
A good number from a bad test is still a bad test. If the load generator was saturated, if the data was too uniform, or if the think time was missing, the metrics inherit the flaw.
Making It Work in a Large Organization
Three habits help at scale. First, write a metrics definition page before the first test: which clock, which percentiles, what counts as an error, what a transaction is, and which warm-up is excluded. Short, versioned, and agreed by QA, engineering and product. Second, align test metrics with production monitoring so the same words mean the same things in both places, and a test result can be compared with what the live system actually does. Third, tie them to explicit service level objectives, so a result is judged against a number someone signed up for, not against a mood.
If the System Behind the Numbers Is a Model
When an endpoint serves an LLM or an agent, the four metrics need translating. Response time splits into time to first token and total generation time, and users feel the first far more than the second when answers stream. Throughput is often best expressed as tokens per second, alongside requests, since two requests can differ in cost by an order of magnitude. Error rate should count provider throttling, timeouts, truncated or empty outputs, and guardrail refusals, which an HTTP status will never show. Quality isn't covered by any of the four, so you'll want separate evaluation checks. AI tools can help you summarize runs or spot odd patterns, but the pass or fail should rest on the objectives you defined.
Mistakes That Show Up Again and Again
Reporting TPS without naming the unit. Showing average response time as the headline. Calling the peak throughput the capacity. Leaving errors out of the same chart as latency. Counting only HTTP status codes as failures. Forgetting warm-up and ramp-up in the averages. And producing a report with no definitions, so that next quarter's numbers can't be compared with this quarter's.
A Closing Thought
Before your next test, spend fifteen minutes with the people who'll read the results and ask them each to write down what "TPS" means to them. Don't discuss it until everyone's finished. The differences you find will save you a week of confusion later, and they'll tell you more about your organization's performance culture than any dashboard.
I expect the vocabulary to slowly converge as more teams adopt the same observability standards in test and production, so that a test result and a live graph use identical definitions. Until then, the discipline is cheap: name the unit, name the clock, name the denominator. If you'd like help setting that up across your teams, PrimeQA Solutions works with engineering and product leaders to agree on Performance Testing Metrics that mean the same thing in the lab and in production.