Cloud Infrastructure for Marketplaces: Traffic Spikes, Queues, Cache, CDN, and the Database

During a sale, a marketplace usually does not fail “because of a single server.” More often, the bottleneck appears at the architecture layer: the database runs out of connections, search cannot handle the filters, the payment API responds slowly, the cache is not warmed up, the queue accumulates tasks, and cloud account limits cannot be raised in a few minutes.

Autoscaling backend services is useful, but it does not guarantee resilience. If the application scales faster than the database, search, external APIs, and queues, the traffic spike is simply pushed further down the chain.

Before a sale, you need to check not only the number of servers, but the entire infrastructure stack:

• What the CDN serves and where the dynamic logic for the cart, order, and payment begins;
• Which data can be cached without risking price accuracy, inventory levels, and personalization;
• Which operations remain synchronous and which are moved to a queue;
• Whether the database, search, and external APIs can handle increased reads, writes, and latency;
• Whether limits, indexes, replicas, alerts, queue workers, and degradation scenarios are ready.

The main point: readiness for peak load is not about “adding more servers before launch,” but about a chain that has been validated in advance. The CDN reduces inbound traffic, the cache offloads reads, the queue smooths out background operations, while the database, search, external APIs, and monitoring define the platform’s real limits.

Peak Load Is a Test of the Entire Infrastructure

At the start of a sale, a marketplace rarely goes down because of a single specific server. More often, the system runs into a limit in a layer that was not considered critical in advance: the database exhausts its connections, search cannot handle the filters, the payment API starts responding slowly, the cache has not been warmed up, a queue has accumulated tasks, or the cloud account limits cannot be raised within a few minutes.

A marketplace is more complex than a conventional online store. It has more sellers, product listings, inventory records, pricing rules, integrations with warehouses, ERP/WMS systems, delivery services, payment providers, and notification services. Each user flow passes through several layers, and a weak link can wipe out the capacity headroom in adjacent components.

That is why, before a sale, the discussion should not be limited to “how many servers to add.” It is more important to understand how the entire chain behaves: the ingress layer, CDN, WAF, backend, cache, database, search, queues, external integrations, and monitoring.

Next, we will break down marketplace infrastructure by request paths: what happens in the user’s synchronous path, what is moved to the asynchronous flow, how to estimate peak load, and which layers cannot be scaled at the last minute.

Basic Marketplace Infrastructure Diagram

It is more useful to assess peak readiness not as a list of services, but as two flows: what the user expects immediately and what can be handled in the background. This helps separate the critical order path from auxiliary operations that should not block the buyer’s screen.

Synchronous path: what the user waits for immediately

User → CDN/WAF → load balancer or API gateway → backend services → cache → database / search / object storage → response to the user

This path includes the operations that determine whether the site feels like it is working to the user: a product page opens, a filter is applied, the cart is updated, an order is created, or a payment response is received. For example, when viewing a product page, images can be served through a CDN, some data can be retrieved from the cache, and up-to-date parameters can come from search or the database.

Order creation also remains partly in the synchronous path. The cart contents, price, and product availability must be checked, and the payment must be initiated, before responding to the user. Otherwise, the buyer will see a confirmation that the system cannot yet guarantee.

Asynchronous processing path: what can run in the background

Backend service event → queue or event bus → background task handlers → ERP/WMS / delivery / notifications / analytics / documents

This is where you should offload anything that does not have to be completed before responding to the user: sending email, SMS, and push notifications; generating documents; exchanging data with ERP/WMS; some delivery updates; analytics; and secondary statuses.

This processing path reduces pressure on the critical path. The user gets a clear result faster, while internal operations are performed at a controlled rate. However, the queue must not become a “black hole”: message age, errors, retries, and processing speed must be monitored.

A separate operational layer is also required: metrics, logs, tracing, alerts, and monitoring dashboards. Without it, the team may notice a problem too late—when the queue has already backed up, the database has exhausted its connections, and the payment API has started slowing down across the board.

For the customer, this architecture is a validation tool. Questions should cover not only the backend server group, but also the ingress layer, cache, database, search, queues, external integrations, and observability. A scalable application group alone will not save the platform if requests then run into slow search, an overloaded database, or a stalled queue.

What Happens to the Infrastructure During a Sale

When a sale starts, a marketplace does not receive one single wave of “high traffic,” but several different flows of operations.

The storefront primarily generates read traffic: the home page, categories, product pages, images, prices, inventory levels, and recommendations. Search and filters add heavy queries, sorting, and facets. The cart and checkout generate writes, validations, and transactions: the system must lock in the order contents, price, product availability, and payment initiation. After that, background events appear: notifications, documents, status updates, and synchronization with warehouses, sellers, and delivery services.

This is why the peak cannot be measured only by the number of users. Ten thousand visitors viewing a banner and ten thousand buyers simultaneously applying filters, adding items to the cart, and paying for orders create very different load profiles.

A condensed peak-load map looks like this:

During a sale, not one layer but several connected areas become overloaded. The CDN reduces inbound traffic, the cache offloads read traffic, and the queue smooths out background operations, but the database, search, external APIs, and monitoring define the actual limits of resilience.

After this map is in place, you can move on to capacity planning: instead of counting abstract “users on the site,” you need to calculate specific business scenarios—browsing, search, cart, checkout, payment, and background events.

How to assess throughput before a peak

Assessing throughput is not simply a matter of asking “how many users the site can handle.” For a marketplace, it is more useful to measure specific business scenarios rather than users in general: browsing the storefront, searching, opening a product page, adding an item to the cart, placing an order, making a payment, and background events after the order.

A minimal methodology looks like this:

• Split traffic into scenarios. For example: 60% of users browse the catalog, 25% use search, 10% add products to the cart, and 5% proceed to payment. It is better to derive the proportions from analytics rather than invent them before the test.
  
• Convert scenarios into requests per second. Opening a card is not a single request, but a chain of calls to the application, cache, search, DB, and object storage.
  
• Account for cache hits and misses. If 90% of cards are served from the cache, the database sees one load profile. If the cache is cold, the load changes sharply.
  
• Separate reads and writes. The storefront mostly reads, while the cart, order, and payment components write and require transactions.
  
• Check external constraints. The payment service, ERP/WMS, delivery, and notifications may have their own limits.
  
• Compare the arrival and processing rates in queues. If tasks arrive faster than the handlers pick them up, the queue will grow even if the interface is functioning normally.

Example with hypothetical numbers: at peak, 20,000 active users are expected per minute. If 10% of them add an item to the cart, that is about 2,000 cart operations per minute, or roughly 33 operations per second. If each operation generates 3 database requests and 2 cache requests, the data layer receives about 100 database requests per second from the cart alone.

If the cache hit rate drops, the load on the database can spike even if the user count has not changed. That is why load testing must model real sale scenarios rather than an abstract stream of requests. Otherwise, the test may show that the backend held up, but fail to surface database locks, heavy filter queries, a growing message age in the queue, or payment API latency.

After calculating the scenarios, it is worth moving to the first segment of the request path: which requests can be blocked or served before they reach the application backend.

CDN, WAF, and the ingress layer: what can be stopped before it reaches the application

A common mistake is to immediately calculate how many backend servers need to be added. The first step is to understand which requests should not reach the application, database, search, or order services at all.

A CDN is useful where content is public and predictable: product images, CSS/JS, media, guides, documents, some static sections, and public pages. For an international audience, this reduces latency and offloads the origin infrastructure: the user receives the file from the nearest node, and the backend does not have to respond to every identical request.

The boundary for CDN usage is where personalization and transactions begin. Cart contents, checkout, payments, personalized prices, up-to-date stock levels, and sensitive data must not be cached without strict rules. Otherwise, a user may see an incorrect price, outdated stock information, or even someone else’s data.

Before requests reach the application, you can usually stop or serve:

• Static assets and media;
• Some public pages;
• Malicious requests;
• Excessively frequent requests;
• Misbehaving clients and bots.

A WAF filters common attack patterns, suspicious parameters, and some bot traffic. Rate limits protect against scraping, brute-force attempts, aggressive integrations, and faulty clients that can generate load no less severe than real shoppers.

That is why, before a sale event, it is important to check not only “server capacity,” but also caching rules, limits, bot protection, timeouts, and API gateway behavior. However, even a strong ingress layer will not stop all dynamic traffic: product pages with current prices, carts, and orders will still reach the platform. The next layer is the backend and cache.

Backend and cache: how to reduce read load

After the CDN and WAF, dynamic traffic reaches the internal systems: a product card with the current price, the cart, inventory checks, checkout, and the payment flow. Backend services must be stateless locally: sessions, tokens, carts, and intermediate data must not be stored in the memory of a single instance. This allows instances to be added and removed without tying a customer to a specific server.

However, simply increasing the number of instances does not always help. If each new instance opens additional database connections, the peak may simply be shifted further down the chain. For this reason, state is moved to external storage, a cache, or a database, while data access is constrained through connection pools, timeouts, and graceful degradation rules.

Caching reduces read load when data is requested frequently and can tolerate controlled staleness:

For high-risk data, short record lifetimes, invalidation of stale values, protection against a cache stampede on the database during widespread cache misses, and fallback behavior if the cache fails are important.

Caching reduces the number of reads, but it does not eliminate transactional load, connections, locks, or slow queries. A poorly designed cache can make errors propagate faster: showing an outdated price, selling an out-of-stock item, or serving personal data to the wrong user. That is why both response speed and data correctness rules must be evaluated.

After the backend and cache, the next critical layer is the database. This is where orders, carts, prices, inventory, statuses, and transactions converge, and they cannot simply be spread across a cache or a queue.

Database: Transactions, Connections, and Locks

The database remains the core layer of a marketplace. It records orders, payment states, inventory levels, prices, carts, statuses, and the relationships between sellers, products, and buyers. The storefront can be sped up in part with caching and a CDN, but order creation still requires consistency: a purchase must not be confirmed at an incorrect price, and an item that is no longer available must not be sold.

During a sale, the database comes under pressure from several directions at once:

• Read traffic increases for product pages, inventory levels, prices, and carts;
• Write traffic increases when items are added to carts, orders are created, and statuses change;
• The number of connections from scaled backend services increases;
• Locks appear when inventory levels are updated and orders are committed;
• Slow queries become more visible because they are repeated frequently;
• Read replicas may start to lag behind the primary database.

Replicas help only where replication lag is acceptable. The catalog, reference data, and some analytical data can be read from replicas. However, the inventory check before placing an order, price recording, and transaction writes must not be blindly routed to a source that may be lagging.

Indexes also cannot be treated as a minor optimization. During peak load, a poor index turns a frequent query into a large-scale table scan. Queries involving carts, orders, inventory, sellers, statuses, and time ranges are especially risky. It is not enough to check that indexes exist; query execution plans must also be validated on data volumes close to production.

Connections are a separate risk. If the number of application instances triples and each instance maintains a large connection pool, the database may hit its connection limit before CPU or memory is exhausted. This is why bounded pools, connection proxies where appropriate, timeouts, and avoiding infinite retries are necessary.

The relationship between the database, cache, and queues must be explicit. Cache reduces read load, but it must not hide data correctness issues. Queues move background actions out of the critical path, but they must not turn a critical order transaction into an unmanaged deferred write. The minimum required persistence of the order and payment state is performed synchronously, while notifications, documents, integrations, and secondary updates move to the asynchronous path.

EU Cloud Infrastructure You Control

Run production workloads on dedicated resources across EU data centres. Transparent pricing, no hidden costs. Full control over compute, storage, and networking.

Explore Infrastructure

Predictable Virtual Machines for Production

Dedicated vCPUs, consistent performance, no noisy neighbours. Deploy and scale VMs in minutes. Full root access, fixed pricing.

View VM Plans

Up to €22,000 to Migrate Your Workloads

Cut OPEX when migrating to EU infrastructure. Get migration support, cost assessment, and infrastructure credits with no upfront commitment.

Apply for Bonus

Kubernetes with Full Control

Deploy production-ready clusters with control over node configuration, storage, and networking. Predictable costs and direct access to infrastructure.

Deploy Cluster

Search: the storefront’s separate capacity ceiling

Marketplace search is not just about reading product cards. It powers filters, sorting, full-text queries, facets, prices, brands, categories, availability, and regional conditions.

If these queries run against the primary transactional database, the storefront starts competing with orders and payments. That is why search is usually moved to a separate service or cluster, with its data updated from the main system through events, exports, or indexing processes.

Before a sale, you need to check:

• Whether the indexes are ready for popular filters;
• Whether sorting becomes expensive on large result sets;
• How quickly new prices and stock levels appear in search;
• How facets and filters behave during a simultaneous spike in requests;
• Whether there is a conflict between response speed and the freshness of commercial data.

Caching can help with popular categories and frequent queries, but it does not replace proper search configuration. You need to test the actual sale scenarios: promotion pages, discount filters, price sorting, regional availability, and fast click-throughs from email campaigns.

Once search can sustain the storefront, the next layer protects not read operations, but background operations after a user action. This is where queues come in.

Queues: How to Smooth Out Spikes and Protect Dependent Services

A queue is needed when an operation does not have to be performed while responding to the user. It accepts tasks and events, stores them until processing, and allows handlers to work at their own pace. For a marketplace, this is critical: not every action after an order should block the purchase confirmation screen.

What can be put in a queue

Queues are typically used for:

• Sending email, SMS, and push notifications;
• Generating documents;
• Synchronizing with ERP/WMS systems and sellers;
• Updating delivery statuses;
• Sending events to analytics;
• Reprocessing payment and logistics statuses;
• Secondary updates to the storefront and indexes, if they can tolerate a delay.

The main role of a queue is to smooth out spikes. If many orders are created within a minute, the platform does not have to send all notifications, documents, and external requests at the same time. It can record the critical state, place background tasks in a queue, and process them at a controlled rate. In this way, the queue protects the database, external APIs, and notification services from sudden surges.

What to monitor during peak load

A queue should not become a way to hide a problem. If tasks arrive faster than they are processed, both the queue size and message age increase. A user may already have received an order confirmation, while the warehouse system, delivery service, or notifications are lagging by tens of minutes.

During peak load, you should monitor more than just the queue size:

If the queue size is growing but message age remains low, the system may simply be riding out a short spike. However, an increase in the age of old messages means processing can no longer keep up with the load, and background processes are starting to lag behind the business.

When a queue falls behind, the next problem almost always shows up in retries. Handlers resend requests to external APIs, recreate documents, update statuses, or synchronize data. Without limits and idempotency, these retries may amplify the failure instead of helping the system recover.

Why Retries and Idempotency Matter

Retries must be limited. If an external API is temporarily unavailable, unbounded retries can make the incident worse. There must be pauses between attempts, a retry limit, and a separate error queue.

Another mandatory requirement is idempotency. Processing the same message more than once must not create two orders, two charges, two documents, or two conflicting delivery statuses. To achieve this, systems use unique operation identifiers, checks for actions that have already been completed, and retry-safe calls.

For the customer, the key question is this: the asynchronous layer must be a managed part of the system, not a place to dump tasks for “later.” It must be clear what can be sent to the queue, what remains synchronous, how handlers are scaled, how the DLQ works, and who deals with accumulated messages after a failure.

External APIs and Observability

A marketplace depends on external systems: payments, delivery, ERP/WMS, sellers, notification services, fraud prevention, and analytics. Even if its own cloud infrastructure scales properly, a third-party API may have rate limits, latency, a maintenance window, or unstable responses during a sale event.

How to Limit External Dependencies

Integrations require not only valid access keys, but also rules for handling delays and errors:

• Timeouts for each external call;
• Limits on the number of concurrent requests;
• Retries with a pause, rather than a continuous loop;
• A queue for operations that can be deferred;
• Fallback scenarios for non-critical features;
• Separate latency and error metrics for each provider.

Critical payment operations must be designed with particular care. Users should receive a clear status: payment accepted, declined, awaiting confirmation, or requiring additional verification. If the payment service responds slowly, the system must not hold the connection indefinitely and block resources. Some statuses may arrive asynchronously, but this must be accounted for in the order model.

What Needs to Be Visible During a Sale

Observability ties all layers into a manageable system. During a sale, it is not enough to know that “errors have increased.” You need to see where degradation began: the CDN, API gateway, application, cache, database, search, queue, handlers, or an external provider.

This requires metrics, logs, request tracing, alerts, and dashboards that show not only infrastructure but also business metrics:

• Orders created;
• Payment errors;
• Search latency;
• Cache miss rate;
• Number of database connections;
• Age of messages in the queue;
• Errors and latency in external APIs.

This is where operational maturity is tested. If the team cannot show in advance which metrics it will monitor in the first minutes of the sale and who makes the decision to degrade functionality, infrastructure capacity alone does not guarantee that orders will be preserved.

After that, the final practical question remains: which layers can be strengthened quickly before launch, and which must be prepared in advance.

What cannot be scaled at the last minute

Some compute resources can be added ahead of a traffic peak, but not every constraint can be expanded on short notice. Some components require upfront design, testing, limit approvals, and warm-up.

This table is useful as a criterion for procurement or for selecting an architectural solution. If a provider promises to "add capacity before launch," you need to clarify exactly which layers they can scale quickly and which must already be prepared.

Conclusion

A marketplace’s resilience during a sales event is determined not by the number of backend servers, but by the throughput of the entire chain. The CDN reduces inbound traffic, the cache offloads reads, the queue moves background operations out of the user request path, and the database, search, and external APIs define the system’s real limits.

The key question for a contractor or cloud team should not be “how many servers will we add,” but “which layers have already been tested for the peak scenario.” Before the sales event, limits, indexes, connection pools, caching rules, queue processing speed, failure behavior, and degradation metrics should all be clearly understood.

Readiness for peak load is not a one-time capacity increase, but a pre-validated architecture in which each layer has its own role, limits, and degradation plan.

FAQ

Why doesn’t backend autoscaling guarantee marketplace resilience?

Because the application scales faster than the database, search, external APIs, and queues. If you add more instances of the order service, they may open more database connections or call the payment API more frequently. As a result, the bottleneck shifts further down the chain rather than disappearing.

What data can be cached in a marketplace?

Typically, public and frequently read data is cached: product cards without the final personalized price, categories, reference data, static blocks, and some popular result sets and aggregates. Prices, stock levels, shopping carts, personalized discounts, and user data require caution. They need short TTLs, invalidation, and correctness checks before an order is placed.

What should remain synchronous during checkout?

Operations without which a correct result cannot be returned to the customer must be performed synchronously: validating the cart contents, prices, and product availability; creating the order; and initiating or confirming the payment. Notifications, documents, analytics, and some delivery and integration updates can be sent to a queue if the business allows a delay.

Which metrics are important during a sales event?

You need to monitor more than just CPU and memory utilization. Critical metrics include API latency, error rate, cache hit ratio, the number of database connections, locks, replica lag, search response time, queue length and age, external API errors, successful orders, and payment errors. These metrics must be visible across layers; otherwise, it is difficult to identify the cause of degradation in time.

Sources

1. Microsoft Azure Architecture Center — Queue-Based Load Leveling pattern

2. Microsoft Azure Architecture Center — Caching guidance

3. Cloudflare Docs — Rate limiting rules

4. Google Cloud Architecture Center — Patterns for scalable and resilient apps