Skip to content
Art2link ESB v2.02 LTS HomeDocumentationBlogContact
Patterns/Endpoints & consumption/Competing consumers

Competing consumers

When one consumer cannot keep up, run several against the same queue. They compete for messages, and the broker gives each message to exactly one of them — throughput scales, and no message is processed twice.

The pattern only works if something guarantees single delivery — each message goes to one consumer, never two. That guarantee does not come from the Art2link bus. On the bus, every port whose subscription matches a message receives its own copy; standing up several matching ports fans the work out, processing each message many times. That is publish-subscribe, the opposite of what you want here.

Competing consumers is therefore a broker-backed pattern. You land the work on a queue that arbitrates delivery, and let several consumers draw from it. In Art2link that queue is an Azure Service Bus queue, consumed through the Service Bus adapter with peek-lock: each message is leased to one consumer, completed only when its work succeeds, and released back for redelivery if that consumer fails. The broker, not Art2link, enforces exactly-one.

one Service Bus queue queued work peek-lock Consumer 1Consumer 2Consumer 3 each message → exactly one consumer add consumers to scale throughput
Two ways to scale

Parallelism comes at two levels, cheapest first. Reach for the second only once a single host is saturated.

Level 1 — one port, more concurrency queue Receive port Max Concurrent Messages = 4 one port leases several messages at once Level 2 — more hosts queue Host A · listenerHost B · listenerHost C · listener same listener, same queue, more instances

Level 1 — one port, more concurrency. Raise Max Concurrent Messages on the receive port above one. That single port leases and processes several messages in parallel from the queue; peek-lock still guarantees each message is held by exactly one handler. This is the whole pattern in one field, and it is where you should start.

Level 2 — more hosts. When a single host is saturated, run more runtime instances. Each runs the same listener against the same queue, and Service Bus still hands each message to exactly one of them. Single delivery holds because the queue arbitrates, not the host — so you scale out without touching the producer, the subscription, or the port definition.

The cost is ordering

Several consumers running in parallel finish in whatever order their work completes, so a strict input order is not preserved. Where that matters, you have three options. Enable Sessions on the queue: Service Bus pins all messages sharing a session id to one handler at a time, so each key stays ordered while different keys still run in parallel — the native answer, and usually the right one. Otherwise restore order downstream with a resequencer, or design the work to be order-independent so the question never arises. The convoy side of correlation is the same idea applied at the bus level. On the send side, the mirror image distributes deliveries — see load balancing.

One queue, not many ports. Competing consumers lives on a single broker queue that hands each message to one consumer. Duplicating subscriptions or ports does the opposite — it fans the same work out to all of them. Scale with concurrency and hosts, and reach for Sessions before you give up ordering.