Transaction Systems

Why transaction systems require a special approach¶

A transaction system is a different category of software than CRUD applications. When a payment goes halfway through — money leaves one account but doesn’t arrive at the other — you have a problem that costs customer trust and regulatory attention. We build systems where consistency is guaranteed at the architectural level, not at the “hopefully it works out” level.

ACID in the microservices world¶

In a monolith, ACID is simple — database transaction. In microservices architecture, where one business operation goes through 3-5 services, you need a different approach.

Saga Pattern: Distributed transaction as a sequence of local transactions. Each step has a compensating action — if step 3 fails, steps 1 and 2 are compensated (rollback). Two approaches:

• Orchestration — Central orchestrator controls the flow. Easier debugging, clear visualization.
• Choreography — Services react to events. Less coupling, but harder debugging.

We choose based on complexity and team structure. For critical financial operations, we prefer orchestration — explicit flow, better error handling.

Outbox Pattern: Guaranteed event delivery. Transaction writes data + event to the same database (atomically). Poller or CDC reads events and sends to Kafka. No lost events, no duplicates (idempotent consumers).

Payment system architecture¶

┌───────────────────────────────────────────────────────┐
│  GATEWAY LAYER                                         │
│  API Gateway → Auth → Rate Limiting → Routing         │
└──────────────┬────────────────────────────────────────┘
               │
               ▼
┌───────────────────────────────────────────────────────┐
│  ORCHESTRATION LAYER                                   │
│  Transaction Orchestrator (Saga)                      │
│  → Validation → Authorization → Settlement → Notify  │
└──────────────┬────────────────────────────────────────┘
               │
               ▼
┌───────────────────────────────────────────────────────┐
│  DOMAIN SERVICES                                       │
│  Account Service │ Payment Service │ Ledger Service   │
│  Fraud Detection │ Compliance │ Notification          │
└──────────────┬────────────────────────────────────────┘
               │
               ▼
┌───────────────────────────────────────────────────────┐
│  DATA LAYER                                            │
│  PostgreSQL (ACID) │ Redis (cache) │ Kafka (events)  │
│  Audit Log (append-only) │ Archive (cold storage)    │
└───────────────────────────────────────────────────────┘

High Availability & Disaster Recovery¶

For financial systems, availability is non-negotiable. Our HA/DR design:

• Active-Passive with automatic failover — Standby replica in another AZ. Automatic failure detection (health checks every 5s), failover under 30s. Synchronous replication for RPO = 0.
• Multi-AZ deployment — Services run in at least 2 availability zones. Load balancer automatically routes traffic to healthy instances.
• DR tests — Monthly automated failover drills. Quarterly full DR test with business stakeholders. Documented, measured, reported.
• Chaos Engineering — Random instance killing in production (controlled). Litmus Chaos or Chaos Monkey. Validation that failover actually works.

Idempotence and deduplication¶

In a distributed system, requests are retried — network timeout, retry logic, queue redelivery. Every endpoint must be idempotent:

• Idempotency key — Client sends unique key with each request. Duplicate request returns the same result without repeating the operation.
• Dedup at Kafka level — Exactly-once semantics with transactional producer.
• At-least-once delivery — Consumers are idempotent, duplicate processing is safe.

Audit Trail and Compliance¶

The regulator doesn’t need to hear “we trust it works.” They need data:

• Immutable audit log — Append-only, cryptographically signed. Who, what, when, from where, why.
• Event Sourcing for key domains — State is calculated from event history. Full state reconstruction to any point in time.
• Retention policies — Configurable per regulation (7 years for financial transactions, 10 years for AML).
• Export for audit — Structured reports, filterable, exportable. Audit ready in hours, not weeks.

Performance under load¶

A payment system must be fast even under load. Our optimizations:

• Connection pooling — PgBouncer for PostgreSQL, connection reuse.
• Prepared statements — Elimination of parse overhead for recurring queries.
• Indexes and partitioning — Transaction tables partitioned by month. Old data archiving without performance impact.
• Read replicas — Reporting and analytics go to read replica, not primary.
• Caching — Redis for hot data (account balances, rate limits). Cache invalidation via Kafka events.

Result: 5000+ transactions/second with consistent latency (P99 < 500ms) on standard infrastructure.

Souvisí s