$ snippet promise-concurrency-limit
Run promises with a concurrency limit
Process 500 URLs with at most 5 requests in flight — a pLimit / async pool in a dozen lines, keeping results in order.
/**
* Map over items with at most `limit` tasks running at once.
* Results keep the input order. Rejections reject the whole call (like Promise.all).
*/
async function mapLimit(items, limit, fn) {
const results = new Array(items.length);
let next = 0;
async function worker() {
while (next < items.length) {
const i = next++;
results[i] = await fn(items[i], i);
}
}
await Promise.all(Array.from({ length: Math.min(limit, items.length) }, worker));
return results;
}
/** Same, but never throws: returns { status, value | reason } per item. */
function mapLimitSettled(items, limit, fn) {
return mapLimit(items, limit, (item, i) =>
Promise.resolve().then(() => fn(item, i))
.then(value => ({ status: "fulfilled", value }), reason => ({ status: "rejected", reason })));
}How it works
Promise.all(items.map(fetch)) starts everything at once, which trips rate limits and exhausts sockets. The pool pattern starts limit workers that each pull the next index from a shared counter until the list is empty — no queue data structure needed. Because each worker writes to results[i], the output order matches the input regardless of completion order.
Gotchas
- If
fnthrows, remaining workers keep running until they notice; use the settled variant when partial failure is expected. - Choose the limit based on the target, not your machine: browsers cap ~6 connections per host, and APIs publish rate limits.
- For an infinite or streaming source of items, this pattern needs a generator instead of an array — it's the same idea with
for await.
Ad slot