Event-Driven Messaging: Kafka, RabbitMQ, CDC

Run RabbitMQ with its management interface using Docker Desktop.

Trace a message through an AMQP producer, exchange, binding, queue, and consumer.

Implement direct-exchange routing in a Spring Boot application.

Implement wildcard routing with a topic exchange in a Spring Boot application.

Select a fanout or headers exchange for a stated routing requirement.

Configure durable exchanges, durable queues, and persistent messages, then explain their guarantees and why publisher confirmation is still required.

Configure manual consumer acknowledgements and demonstrate redelivery of an unacknowledged message.

Tune prefetch and consumer concurrency for a work-queue workload.

Use publisher confirms and mandatory-message returns to detect unsuccessful publication.

Diagnose and stop an infinite RabbitMQ requeue and redelivery loop.

Build a bounded delayed-retry topology using TTL queues and dead-letter exchanges.

Route exhausted messages to a dead-letter queue and replay a corrected message safely.

Implement an idempotent RabbitMQ consumer backed by a persistent deduplication record.

Select classic queues, quorum queues, or RabbitMQ streams for a stated workload.

Use quorum and majority rules to predict queue availability and message safety during RabbitMQ node failures.

Use RabbitMQ management metrics to diagnose queue growth, unacknowledged messages, and consumer bottlenecks.

Run a local three-broker Kafka cluster in KRaft mode with Docker Desktop and inspect its brokers and topics.

Explain how partition leaders, followers, offsets, and in-sync replicas provide ordered storage and failover.

Create a replicated, multi-partition topic and inspect its records, offsets, leaders, and replicas with command-line tools.

Choose a record key that preserves the required business ordering scope.

Predict record distribution and ordering from a topic's keys and partition count.

Demonstrate how partitions are assigned across members of a consumer group.

Trace the lifecycle of a consumed record from fetch through offset commit.

Distinguish time-based retention from log compaction and select between them for a use case.

Configure producer acknowledgements and minimum in-sync replicas for a stated durability target, then predict write behavior during replica loss.

Configure retries and producer idempotence to prevent duplicates caused by producer retry attempts.

Tune batching, linger, and compression for a specified latency-throughput trade-off.

Compare automatic and manual offset commits by observing their behavior after consumer failure.

Configure consumer concurrency and poll settings without exceeding the available partitions.

Reduce disruptive consumer-group rebalances using an appropriate assignment and membership strategy.

Design bounded retries and dead-letter handling for transient failures and poison records while documenting the effect on ordering.

Implement a transactional consume-transform-produce flow and state the boundary of Kafka's exactly-once guarantee.

Estimate a Kafka partition count from throughput, consumer parallelism, and future-growth requirements.

Calculate topic storage from event size, event rate, replication factor, and retention period.

Assess the ordering and operational consequences of increasing a topic's partition count.

Interpret consumer lag, request latency, throughput, and under-replicated-partition metrics.

Define alert thresholds tied to message-delivery and consumer-lag service objectives.

Diagnose whether rising consumer lag is caused by production rate, processing time, rebalancing, or downstream failure.

Simulate a broker outage in the local cluster and verify leader failover and client recovery.

Design Kafka authentication, transport encryption, and least-privilege topic permissions for a stated deployment.

Apply consistent topic, exchange, routing-key, event-type, and version naming conventions.

Select JSON, Avro, or Protobuf for a stated interoperability and governance requirement.

Register a schema and configure a Spring Kafka producer and consumer to use it.

Distinguish backward, forward, and full schema compatibility using concrete producer-consumer versions.

Evolve an event schema additively without breaking existing consumers.

Verify a proposed schema change against a selected compatibility policy.

Handle unknown fields, missing optional fields, and malformed payloads in a consumer.

Define ownership and review rules for event-contract changes across teams.

Compare application dual writes, timestamp polling, database triggers, and log-based CDC for a given integration.

Explain how PostgreSQL WAL, logical decoding, publications, and replication slots support CDC.

Configure PostgreSQL logical-replication prerequisites for Debezium.

Deploy Kafka Connect with a Debezium PostgreSQL connector and capture a table change into Kafka.

Interpret Debezium before, after, operation, source, and timestamp fields for an emitted change event.

Distinguish initial snapshot processing from subsequent streaming and explain how Debezium hands off without losing committed changes.

Handle primary keys, updates, deletes, and tombstone records in a CDC consumer.

Explain how Debezium's stored offsets and PostgreSQL replication-slot LSNs support restart, including where duplicate delivery can occur.

Apply least-privilege capture scope by configuring a restricted Debezium database role and table or column filters for sensitive data.

Execute an incremental snapshot without replacing the ongoing change stream.

Use Debezium transaction metadata to reconstruct transaction boundaries and event order.

Assess the effect of a PostgreSQL schema change on the connector and downstream consumers.

Choose a safe re-snapshot or backfill strategy after WAL retention loss or replication-slot invalidation.

Write business data and an outbox record atomically in one Spring database transaction.

Route outbox rows into domain-event topics using Debezium's outbox transformation.

Build an idempotent consumer that materializes CDC events into an updatable read model.

Select unit, integration, contract, or end-to-end testing for a stated messaging risk.

Unit-test a Spring message handler for success, retryable failure, and non-retryable failure.

Integration-test a RabbitMQ producer and consumer using Testcontainers.

Integration-test Kafka publication, consumption, and offset behavior using Testcontainers.

Verify an end-to-end PostgreSQL-Debezium-Kafka CDC flow in Docker.

Inject duplicate delivery and component restarts, then verify recovery behavior.

Trace one business event across services by propagating event, correlation, and trace identifiers into structured logs and traces.

Configure RabbitMQ client authentication, TLS transport, and least-privilege vhost and resource permissions for a sample application.

Select choreography or orchestration for a multi-service business workflow.

Design saga steps and compensating actions for a partially completed transaction.

Design the ownership and rebuild strategy for a CQRS read projection.

Distinguish event sourcing from CDC, audit logging, and ordinary event publication.

Plan a replay or backfill that does not corrupt current consumer state.

Apply backpressure, throttling, or load shedding to protect a slow consumer dependency.

Choose RabbitMQ, Kafka, CDC, or a combination using workload-specific decision criteria.

Design a hybrid RabbitMQ-Kafka-CDC architecture and specify how each cross-system boundary handles loss, duplication, and replay.

Extract functional requirements, scale assumptions, failure expectations, and consistency needs from a messaging interview prompt.

Estimate event throughput, Kafka partitioning, retention storage, and RabbitMQ consumer concurrency for the capstone workload.

Produce context, container, event-flow, and failure-path diagrams for a hybrid event-driven system.

Defend the selection of RabbitMQ, Kafka, and CDC components using explicit trade-offs.

Create a failure-mode matrix mapping database, Debezium, Kafka, bridge, and RabbitMQ failures to detection and recovery actions.

Integrate the previously built outbox with Debezium and Kafka in the capstone, then verify rollback and restart recovery.

Implement a Kafka-to-RabbitMQ bridge that waits for publisher confirmation before committing Kafka offsets, then verify idempotent recovery from the remaining crash window.

Present and critique a technical-lead system-design answer covering reliability, scaling, observability, security, and operations.