VERSION 1.0.0STATUS: JOB_QUEUE

BULK_CERTGEN

FastAPI and SQLAlchemy 2 on PostgreSQL 16, with WeasyPrint rendering, workers that scale horizontally via FOR UPDATE SKIP LOCKED, and 42 tests running against a real database in CI.

VIEW_SOURCE </>EXPLORE_DOCS ▤

QUEUE

SKIP_LOCKED

TESTS

42_PASSING

BATCH

1000_ROWS

WORKERS

HORIZONTAL

ABOUT_THE_PROJECT

Rendering 1000 PDFs takes over a minute, far too long for one HTTP request. The API validates the batch, stores every recipient as its own row and answers 202 straight away, then separate worker processes generate the certificates while the client polls progress and downloads a ZIP.

Postgres doubles as the queue, so there is no extra broker. Each certificate is rendered in its own transaction, so one bad row or rendering error never blocks the rest, and failed rows carry an error code that says whether a retry makes sense.

SYSTEM_ENVIRONMENT_MATRIX

// APIFASTAPI

// ORMSQLALCHEMY_2

// DBPOSTGRESQL_16

// PDFWEASYPRINT

// TESTSPYTEST

// INFRADOCKER_COMPOSE

Key_Challenges

[ ERR_01 ]
Several workers polling the same table could claim the same certificate; FOR UPDATE SKIP LOCKED lets each worker take a disjoint batch without blocking the others.
[ ERR_02 ]
A crashed worker leaves certificates stuck in PROCESSING; a lease timeout requeues them and gives up after three attempts.
[ ERR_03 ]
Client retries could create duplicate jobs; an Idempotency-Key with a UNIQUE constraint and a request-body hash returns the original job, even when two identical requests race.

Key_Learnings

[ LN_01 ]
Postgres is a perfectly good queue at this scale; reach for a broker when the numbers demand it.
[ LN_02 ]
Partial indexes keep hot queries fast as finished rows pile up in the same table.
[ LN_03 ]
Validate per row and store the failure reason instead of rejecting the whole batch.

//SYSTEM_COMPILE_COMPLETE

Have questions about this framework implementation or structural logic?

INITIALIZE_DISCUSSION →RETURN_TO_INDEX