Home/Glossary/Throughput

What is Throughput?

Throughput is the amount of data or number of operations successfully processed per unit of time. It is a fundamental measure of a system's capacity and performance, determining how many users your infrastructure can serve simultaneously.

Definition

Throughput measures the rate at which data is successfully delivered from one point to another, or the rate at which a system completes work. It is typically expressed in bits per second (bps) for network throughput or requests per second (RPS) for application throughput.

For example, if your web server handles 2,000 HTTP requests per second during peak traffic, your application throughput is 2,000 RPS. If your API endpoint transfers 50 MB of data per second to clients, your data throughput is 400 Mbps.

Throughput vs Bandwidth vs Latency

These three metrics are frequently confused but measure different aspects of network performance:

Metric	What It Measures	Analogy	Unit
Bandwidth	Maximum theoretical capacity	Width of a highway	Gbps, Mbps
Throughput	Actual data transferred	Traffic flow on the highway	Mbps, RPS
Latency	Time for a single trip	Speed limit on the highway	ms

The relationship: You can have high bandwidth but low throughput (a 10 Gbps link with heavy packet loss). You can have low latency but low throughput (a fast connection that can only handle one request at a time). Optimizing all three together is the goal.

How to Measure Throughput

Throughput measurement depends on what layer of the stack you are evaluating:

Network Throughput

Measured by transferring a known amount of data and dividing by the transfer time. Speed test tools perform this by downloading and uploading test files. AtomPing's speed test tool measures download throughput to your servers from multiple locations.

Throughput = Data Transferred / Transfer Time

Application Throughput (RPS)

Measured as requests per second (RPS) or transactions per second (TPS) that your application handles successfully. Use load testing tools to determine your maximum throughput under various conditions. Monitor actual throughput in production to detect degradation before users are impacted.

Database Throughput

Measured as queries per second (QPS) or transactions per second. Database throughput is often the bottleneck in web applications. Monitor read/write ratios, query execution times, and connection pool utilization to identify when your database becomes the limiting factor.

Factors Affecting Throughput

Many factors can limit throughput at different points in the stack:

Network Constraints

Available bandwidth, packet loss, network congestion, and the TCP congestion window all limit network throughput. High latency reduces TCP throughput because the protocol waits for acknowledgments before sending more data. The bandwidth-delay product determines the theoretical maximum throughput for a given path.

Server Resources

CPU saturation, memory pressure, disk I/O limits, and network interface capacity each impose ceilings on throughput. A CPU-bound application cannot serve more requests than its processing capacity allows, regardless of available bandwidth.

Application Architecture

Synchronous/blocking code, single-threaded bottlenecks, lack of caching, excessive database queries, and tight coupling between services all reduce throughput. Async architectures and efficient connection pooling can dramatically increase the number of concurrent requests a single server handles.

Protocol Overhead

HTTP headers, TLS encryption, TCP acknowledgments, and protocol framing all consume bandwidth without delivering application data. HTTP/2 reduces overhead through header compression and multiplexing. Protocol choice significantly affects achievable throughput.

How to Optimize Throughput

Improving throughput is about removing bottlenecks and increasing the capacity of your weakest link:

1. Horizontal Scaling

Add more servers behind a load balancer to distribute requests. This linearly increases throughput capacity. Ensure your application is stateless or uses shared state (database, cache) so any server can handle any request.

2. Caching

Every cached response is a request that does not hit your application or database. Implement caching at the CDN level (static assets), reverse proxy level (full page caching), application level (computed results), and database level (query result caching). Effective caching can increase effective throughput by orders of magnitude.

3. Async and Non-Blocking I/O

Synchronous, blocking architectures tie up a thread per request, limiting concurrency. Async I/O (event loops, coroutines) allows a single thread to handle thousands of concurrent connections, dramatically improving throughput for I/O-bound workloads.

4. Connection Optimization

Use HTTP/2 or HTTP/3 for multiplexed connections. Enable keep-alive to reuse connections. Implement connection pooling for databases and upstream services. Each new connection has overhead (TCP handshake, TLS negotiation) that reduces effective throughput.

5. Database Optimization

Database queries are often the throughput bottleneck. Add appropriate indexes, use read replicas for read-heavy workloads, implement query optimization, batch writes, and consider denormalization for frequently accessed data. Monitor slow query logs to identify the worst offenders.

Throughput Monitoring Best Practices

Effective throughput monitoring helps you detect capacity issues before they impact users:

Track throughput alongside latency: Throughput often drops when latency increases. A drop in RPS with rising response times signals a bottleneck forming.
Monitor error rates at high throughput: Your system may handle 5,000 RPS, but if 10% are errors, effective throughput is only 4,500 RPS. Track successful throughput separately.
Set capacity alerts: Alert when throughput approaches your known limits. If your peak capacity is 10,000 RPS, alert at 7,000 RPS to give time for scaling.
Baseline and trend: Establish normal throughput patterns (daily, weekly cycles) so you can detect anomalies. A sudden drop in throughput during peak hours requires immediate investigation.
Monitor from external locations: Internal throughput metrics may look healthy while users experience degraded performance due to network issues. Multi-region monitoring provides the user's perspective.

Pro tip: AtomPing's multi-region HTTP monitoring measures throughput from 10 European locations, giving you real-world performance data from the user's perspective. Combined with response time tracking, you get a complete picture of application performance.

Frequently Asked Questions

What is the difference between throughput and bandwidth?▼

Bandwidth is the maximum theoretical capacity of a network link (e.g., 1 Gbps). Throughput is the actual amount of data successfully transferred per unit time. Throughput is always less than or equal to bandwidth because of protocol overhead, congestion, packet loss, and other real-world factors. Think of bandwidth as the width of a highway and throughput as the actual traffic flow.

How do I measure throughput for my web application?▼

For web applications, measure throughput as requests per second (RPS) your server handles successfully. Use load testing tools (like k6, Locust, or Apache Bench) to send increasing traffic and measure how many requests complete successfully. Also track data throughput (MB/s) for file downloads and API responses. AtomPing monitors HTTP throughput as part of its response time checks.

What causes low throughput even with high bandwidth?▼

Common causes include: high latency (TCP window scaling limits throughput over long distances), packet loss (triggers retransmissions, halving TCP window), server resource bottlenecks (CPU, memory, disk I/O), connection limits, inefficient application code, and network congestion. A 1 Gbps link with 5% packet loss may only achieve a fraction of its capacity.

How does throughput relate to user experience?▼

Throughput directly affects how fast content loads. Low throughput means slower page loads, longer file downloads, and buffering during video playback. For APIs, low throughput means the server cannot handle enough concurrent requests, leading to queuing, timeouts, and errors for users during peak traffic.

What is goodput vs throughput?▼

Goodput is the application-level throughput: only the useful data delivered to the application, excluding protocol headers, retransmitted packets, and overhead. Throughput includes all data on the wire. If your link shows 100 Mbps throughput but 20% is retransmissions and overhead, your goodput is approximately 80 Mbps.

How can I increase my server's request throughput?▼

Key approaches: implement caching (reduces repeated work), optimize database queries (remove N+1 queries, add indexes), use async/non-blocking I/O (handle more concurrent connections), add horizontal scaling with load balancing, enable HTTP/2 multiplexing, and use connection pooling. Profile your application to identify the specific bottleneck before optimizing.

Does monitoring affect throughput?▼

Monitoring has minimal impact on throughput when done correctly. External monitoring (like AtomPing's HTTP checks) adds negligible load since checks run at intervals of seconds to minutes. Internal instrumentation (metrics collection, logging) should use asynchronous, non-blocking approaches to avoid impacting request processing.

Related Glossary Terms

→Latency →Load Balancing →Redundancy →Uptime Monitoring →Health Check

Monitor Throughput and Performance

AtomPing tracks HTTP response times and throughput from 10 European monitoring locations. Detect performance degradation before your users are affected. Free plan includes 50 monitors.

Start Monitoring Free