An event-driven backend project

A backend that loses no events
and does no work twice.

A distributed issue tracker, built as microservices that talk to each other through events. Writes are idempotent, events stay durable even when something fails, and every piece scales on its own. Here are the architecture decisions and, above all, why they were made this way.

idempotency
event-driven
resilient
scalable
secure

The event travels from the issue being created all the way to the consumer, without getting lost.

What a senior job post asks for, solved

Requirement	How it is solved
idempotency	An Idempotency-Key header + Redis (SET NX EX) on writes; deduplication by event_id on consumption.
event-driven	Transactional outbox + relay + Redis Stream locally, or SQS/EventBridge on AWS, behind a replaceable port.
resilient	Outbox (no lost events), retries, FOR UPDATE SKIP LOCKED, a DLQ, health/readiness, and graceful shutdown.
scalable	API, relay, and consumer are independent processes that scale separately; consumer groups; ECS auto-scaling.
secure	Least-privilege IAM per service; CI/CD with OIDC (no static keys); managed secrets.

How an event flows

The style runs throughout: layers (controller → service → repository) with dependency injection and ports/adapters for anything that crosses the boundary (database, broker). The domain depends on interfaces, not technologies, so switching broker or database is writing a new adapter.

1
The client sends POST /issues with an Idempotency-Key.
2
The middleware reserves the key in Redis with SET NX EX; a retry does not create the issue twice.
3
The service opens a transaction with a Unit of Work.
4
In the same transaction it writes to issues and outbox: either both are saved, or neither.
5
The relay reads the unpublished events with FOR UPDATE SKIP LOCKED.
6
It publishes them to the Redis Stream through the EventPublisher port.
7
notification-service consumes them idempotently, with a DLQ and its own database.

Decisions and their reasoning

idempotency

Idempotency on writes

Problem: A client that retries a POST after a timeout must not create the issue twice.
Decision: A middleware using SET key value NX EX ttl in Redis; the first request wins and stores the response, duplicates get the already-computed response.
Why: Redis is coordination and deduplication, not the source of truth. Only 2xx responses are cached, and the lock has a short TTL so it does not stay locked if the process dies.

postgres

PostgreSQL without an ORM

Problem: Fine-grained control over queries and transactions is needed, without layers of indirection.
Decision: Direct access with pg and hand-written SQL, behind repositories with their interface.
Why: Transactions (the basis of the outbox), partial indexes, and FOR UPDATE SKIP LOCKED are used as-is; the cost of writing SQL by hand is acceptable.

migrations

Custom migrations, no ORM or library

Problem: The schema must be versioned and applied safely, and Docker's init step does not exist on RDS.
Decision: A custom runner over pg with a schema_migrations table, run as a step before the deploy.
Why: pg_advisory_lock means only one replica migrates at a time; checksums detect drift; one transaction per migration. Robust with several ECS tasks during a deploy.

outbox

Transactional outbox

Problem: Writing the issue and publishing the event separately can fail halfway: a lost event or a phantom event (dual-write).
Decision: The event is inserted into outbox within the same transaction as the issue, via a Unit of Work that shares the connection.
Why: The business fact and its event are committed together or not at all. It is the foundation of the system's reliability.

relay

Relay: a separate, always-on process

Problem: Something has to move events from the outbox to the broker without coupling that latency to the API.
Decision: A separate daemon reads the unpublished rows with FOR UPDATE SKIP LOCKED and publishes them; it sets published_at.
Why: Several instances can run in parallel without stepping on each other; if the relay goes down, events pile up and drain when it returns. Nothing is lost.

broker

Redis Streams, not Kafka or RabbitMQ

Problem: Reliable consumption is needed (groups, ACK, retries) without carrying the operational weight of a dedicated broker.
Decision: Redis Streams locally (a persistent log with consumer groups and XACK), behind the EventPublisher port.
Why: It was already in the stack and the volume does not justify Kafka/RabbitMQ. The broker is a replaceable detail: on AWS it is swapped for SQS/EventBridge without touching the domain.

consumer

Idempotent consumer with a DLQ

Problem: Delivery is at-least-once: an event may arrive duplicated, and a message that always fails must not block the group.
Decision: Deduplicate by event_id with ON CONFLICT DO NOTHING in the same transaction as the effect; reclaim pending messages with XAUTOCLAIM and send to a DLQ once the delivery threshold is exceeded.
Why: Duplicates do not repeat the effect; permanent errors go to the DLQ and transient ones are retried. The group never gets stuck.

ownership

Each service owns its data

Problem: Sharing tables between services creates a coupled distributed monolith.
Decision: notification-service has its own database; services communicate only through events.
Why: It lets each service evolve and deploy independently. It is what makes this real microservices.

From local to AWS

Every local piece has its managed equivalent. Designing with ports makes the jump mostly a matter of swapping adapters and configuration.

Locally	On AWS
redis	ElastiCache
postgres	RDS
redis stream	SQS / EventBridge (the port's production adapter)
processes	ECS Fargate: API, relay, and consumer as services that scale and fail separately
edge	API Gateway → internal ALB (throttling, authorizer, usage plans)
migrations	A pre-deploy task (same image, command migrate up)
ci/cd	GitHub Actions + OIDC (no static keys) → ECR → ECS; least-privilege IAM

Project status

issue-service — API, idempotency (Redis), PostgreSQL, and Unit of Work.
Custom migrations — advisory lock, checksums, one transaction per migration.
Transactional outbox + relay — publishing to Redis Streams behind the EventPublisher port.
notification-service — idempotent consumer + DLQ, its own database, EventConsumer port.
AWS — ECS/ECR, SQS/EventBridge adapters, least-privilege IAM, and CI/CD with OIDC.
Noted technical debt — stream trimming (XADD MAXLEN/XTRIM) and LISTEN/NOTIFY as a relay latency improvement.

A backend that loses no eventsand does no work twice.