Kafka поддерживает различные гарантии доставки — at-most-once, at-least-once и exactly-once — которые определяют, могут ли сообщения быть потеряны или продублированы. Понимание этих гарантий и способов их достижения важно для построения надежных систем.
Три семантики доставки
AT-MOST-ONCE → messages may be LOST but never duplicated:
→ commit offset BEFORE processing → if processing fails, the message is skipped (lost)
→ for: when occasional loss is acceptable and duplicates are not (rare)
AT-LEAST-ONCE → messages are never lost but may be DUPLICATED:
→ commit offset AFTER processing → if a crash occurs before commit, the message is
reprocessed (duplicate) → requires IDEMPOTENT processing
→ the common default; safe (no loss) but handle duplicates
EXACTLY-ONCE → each message is processed exactly once (no loss, no duplicates):
→ the strongest, but hardest; requires Kafka transactions + idempotent producers
