Menene Kafka's delivery semantics (at-least-once, exactly-once)?

Question

Accepted Answer

Kafka yana goyan bayan delivery guarantees daban-daban — **at-most-once**, **at-least-once**, da **exactly-once** — waɗanda suke ƙayyade ko messages za su iya ƙarewa ko maimaita. Fahimtar wannan da yadda za a sami su yana mahimmanci don gina tsarin da ke da tabbas.

## The three delivery semantics

```text
AT-MOST-ONCE → messages may be LOST but never duplicated:
  → commit offset BEFORE processing → if processing fails, the message is skipped (lost)
  → for: when occasional loss is acceptable and duplicates are not (rare)
AT-LEAST-ONCE → messages are never lost but may be DUPLICATED:
  → commit offset AFTER processing → if a crash occurs before commit, the message is
    reprocessed (duplicate) → requires IDEMPOTENT processing
  → the common default; safe (no loss) but handle duplicates
EXACTLY-ONCE → each message is processed exactly once (no loss, no duplicates):
  → the strongest, but hardest; requires Kafka transactions + idempotent producers
```

## Achieving the guarantees

```text
→ AT-LEAST-ONCE → process then commit; make consumers IDEMPOTENT (handle duplicates safely
  via dedup keys, idempotent operations) — practical and common
→ EXACTLY-ONCE → Kafka's transactional API + idempotent producer (enable.idempotence) →
  produce + commit offsets atomically (exactly-once within Kafka, e.g. Kafka Streams)
  ⚠️ exactly-once is complex and has overhead; often AT-LEAST-ONCE + idempotency is simpler
    and sufficient
```

## Practical guidance

```text
✓ Most systems use AT-LEAST-ONCE + idempotent processing (simple, reliable)
✓ Use EXACTLY-ONCE when duplicates are truly unacceptable AND idempotency is hard
✓ Producer acks (acks=all) → durability (don't lose on the produce side)
→ understand the trade-offs; design for the guarantee your use case needs
```

## Me yasa yana mahimmanci

Fahimtar Kafka's delivery semantics yana da daraja saboda **delivery guarantees sune suka ƙayyade tabbas** (ko messages za su iya ƙarewa ko maimaita), saboda haka fahimtar su yana mahimmanci don gina tsarin da ke da tabbas.

Zaban da aka zaɓa na delivery semantics yana shafar daidaituwa kai tsaye.

Fahimtar **three delivery semantics** — **at-most-once** (messages na iya ƙarewa amma ba maimaita ba, ta hanyar ƙaddamar da offset kafin aiwatarwa — don lokacin da ƙarewa kadan ya sabawa), **at-least-once** (messages ba su iya ƙarewa ba amma suna iya maimaita, ta hanyar ƙaddamar bayan aiwatarwa, yana buƙatar aiwatar da idempotent — na yau da kullun, babbar zaɓi mafi aminci), da **exactly-once** (kowane message aiwatacce sau daya, mafi ƙarfi amma mafi wahala, yana buƙatar jerin abubuwa da masu aikatarwa idempotent) — yana da mahimmanci don fahimtar alamun tabbas da za a miƙa.

Fahimtar **yadda za a sami guarantees** — at-least-once ta aiwatarwa sannan ƙaddamar da kuma sanya masu karba su zama **idempotent** (yin hanyar ɗin maimaita ta hanyar keys da aiwatarwa idempotent — na zahiri da na yau da kullun), da exactly-once ta hanyar Kafka's transactional API da masu aikatarwa idempotent (aikata da ƙaddamar offset atomically), tare da mahimmancin gani cewa exactly-once yana da hadari da cikin kolin **at-least-once da idempotency sau da yawa ya zama mafi sauƙi da ya isa** — ya nuna hanyar zahiri.

Fahimtar **practical guidance** — cewa yawancin tsarin suna amfani da at-least-once tare da aiwatar da idempotent (sauƙi da aminci), amfani da exactly-once ne kawai lokacin da maimaita jiya sai a saka sosai da idempotency yana da hadari, da amfani da producer acks don tsare-tsare — ya nuna ƙirar gida mai hankali.

Mahallin mahimmi shi ne cewa delivery semantics suna ƙulaƙula da ciniki (ƙarewa vs maimaita vs hadari), kuma zaɓi na yau da kullun, na zahiri shi ne at-least-once tare da idempotency, wanda yana da alaƙa da fihiriso mafi girma na idempotency a tsarin da ba a gina koli ba.

Tunda delivery guarantees sune suka ƙayyade ko messages za su iya ƙarewa ko maimaita (kai tsaye yana shafar tabbas) da fahimtar semantics, yadda za a sami su (musamman at-least-once tare da idempotency a matsayin zaɓi mafi zahiri na yau da kullun), da ciniki yana mahimmanci don gina tsarin da ke da tabbas, fahimtar Kafka's delivery semantics yana da daraja, zahiri-mahimmanci Kafka ilimi — tsakiya ga tabbas, mahimmanci don gina tsarin tare da guarantees daidai (yau da kullun at-least-once tare da idempotency), da nuna fahimtar buƙata don gina tsarin aminci na Kafka.