How do you coordinate a cron job across multiple NestJS nodes?

Question

IT Interview Practice · Accepted Answer

First define what “only one node runs the cronjob” means. **Preventing simultaneous execution and preventing a repeated business effect are different requirements.** A shared scheduler coordinates execution; the operation itself must remain safe after a crash and retry.

Consider a daily invoice job deployed on three NestJS nodes. We want the scheduled work to run, without generating another invoice when a worker restarts.

## 1. Understand why all nodes start the task

Each application instance registers its own cron callback. At the scheduled time, each can start independently:

```text
00:00 -> Node A -> invoice task
      -> Node B -> invoice task
      -> Node C -> invoice task
```

A local isRunning flag only exists in one process. NestJS waitForCompletion can stop overlapping callbacks on that instance, but does not coordinate other nodes. Adding the same flag to all three nodes still leaves three separate decisions. [NestJS scheduling](https://docs.nestjs.com/techniques/task-scheduling).

We therefore need a shared place to decide which work exists and who may process it.

## 2. Choose the coordination model

| Situation | Option to evaluate | Responsibility that remains |
| --- | --- | --- |
| The platform already provides scheduled jobs | Run scheduling outside the web instances | Configure overlap, recovery and missed-run behavior. |
| BullMQ is already operated by the team | Shared Job Scheduler and dedicated queue | Configure shared concurrency and safe retries. |
| The existing task must stay on application nodes | Shared lock or lease | Handle ownership, expiry and stale workers correctly. |

For the invoice example, if Redis and BullMQ already exist, a shared scheduler with a stable scheduler ID is a practical candidate. Separate the schedule from workers so adding web nodes does not create independent schedules.

On a supported BullMQ version, set the dedicated queue's global concurrency to one. This limits active jobs across workers; local concurrency of one on each worker still allows several workers to run different jobs. [Global concurrency](https://docs.bullmq.io/guide/queues/global-concurrency).

Decide what to do if yesterday's work is still running: wait, skip or combine occurrences. Define the time zone and whether missed business dates should be caught up after downtime.

## 3. Protect the invoice, not just the job

Suppose a worker creates an invoice and crashes before recording job completion. Recovery may run the job again even though the invoice already exists.

```text
[Create invoice] -> [Crash before completion]
                              |
                            [Retry]
                              |
                  [Same business key: reuse result]
```

Use a stable business key, such as customer plus billing period. For local database work, a unique constraint and a transaction can connect invoice creation with its completed-run record. A retry then finds the existing result instead of creating another invoice.

For an external provider, use its idempotency key when supported and reconcile uncertain outcomes. A queue job ID or retained job record alone is not a permanent guarantee that a business action cannot repeat. [Idempotent jobs](https://docs.bullmq.io/patterns/idempotent-jobs).

## 4. Know what a lock can and cannot guarantee

A shared lock must be acquired atomically. Use an ownership token, an expiry and renewal; release only when the token still matches. Otherwise an old worker could release a newer worker's lock.

Expiry creates another case: a paused worker resumes after its lease expires while a new worker is already running. Critical writes may need fencing tokens, increasing ownership numbers that the destination checks to reject stale workers. Merely setting a long timeout does not eliminate this risk.

If coordination is unavailable, decide whether the task should wait rather than execute without ownership. This is often the safer choice for consequential work.

## 5. Verify the failure cases

Test multiple nodes, a crash after invoice creation, a pause longer than the lease, loss of Redis connectivity and a rolling deployment. Record the business date, worker, attempt and outcome so repeated attempts can be distinguished from repeated effects.

**Remember:** coordinate scheduling → limit shared execution → make the business action retry-safe → test recovery. “One active job” alone is not proof of exactly one business effect.

Situation	Option to evaluate	Responsibility that remains
The platform already provides scheduled jobs	Run scheduling outside the web instances	Configure overlap, recovery and missed-run behavior.
BullMQ is already operated by the team	Shared Job Scheduler and dedicated queue	Configure shared concurrency and safe retries.
The existing task must stay on application nodes	Shared lock or lease	Handle ownership, expiry and stale workers correctly.

1. Understand why all nodes start the task

2. Choose the coordination model

3. Protect the invoice, not just the job

4. Know what a lock can and cannot guarantee

5. Verify the failure cases

Related questions