Async Job Queues

Sending emails, generating PDFs, processing images — these do not belong in an HTTP request. A job queue solves this.

Celery (Python)¶

tasks.py¶

from celery import Celery app = Celery(‘tasks’, broker=’redis://localhost:6379’) @app.task def send_email(to, subject, body):

Slow operation — runs on a worker, not in the request¶

smtp.send(to, subject, body)

Invocation¶

send_email.delay(“[email protected]”, “Welcome”, “Hello!”) send_email.apply_async(args=[…], countdown=60) # In 60s

BullMQ (Node.js)¶

import { Queue, Worker } from ‘bullmq’; const queue = new Queue(‘emails’, { connection: redis }); await queue.add(‘welcome’, { to: ‘[email protected]’ }); const worker = new Worker(‘emails’, async (job) => { await sendEmail(job.data.to, ‘Welcome’); }, { connection: redis });

When to Use a Queue¶

• Operations > 500ms
• Email, SMS, push notifications
• Report and PDF generation
• Image/video processing
• Webhooks, integrations

Key Takeaway¶

Use a queue for anything that takes > 500ms. Celery for Python, BullMQ for Node.js. Redis as broker.