Anadai ake sarrafa kuskure da sakattuniyar saƙo a Kafka?

Question

Accepted Answer

Sarrafa **kuskure da sakattuniyar saƙo** yana da mahimmanci ga masu karɓa Kafka masu aminci — yanke shawara akan abin da ake buɓuga lokacin da sharadi na saƙo ya kuskuri (sake gwada, tsallake, ko tilasta zuwa **jeri mai sakattun saƙa**). Sarrafa kuskure da kyau yana hana asussu da masu karɓa da suka tsaya.

## Matsalar: gazawar sarrafawa

```text
When a consumer fails to PROCESS a message (bad data, downstream failure, bug):
  → BLOCKING retry forever → the consumer gets STUCK on a "poison" message (can't progress)
  → skipping silently → data LOSS (the message is lost)
  → crashing → consumer restarts, reprocesses, may get stuck again
→ need a deliberate error-handling strategy.
```

## Dabarun sarrafa kuskure

```text
✓ RETRY (with limits) → retry transient failures (with backoff); but LIMIT retries (don't
  retry forever on a permanent failure)
✓ DEAD LETTER QUEUE (DLQ) → after retries fail, send the message to a separate DLQ TOPIC →
  the consumer moves on (not stuck); the DLQ is inspected/reprocessed later
  → the standard way to handle messages that can't be processed (avoids blocking + loss)
✓ Distinguish TRANSIENT (retry) vs PERMANENT (DLQ/skip) failures
✓ IDEMPOTENT processing → safe retries/reprocessing (at-least-once → duplicates)
```

## Abubuwan da ake buɓuga a aikace-aikacen gida

```text
✓ Don't let one bad message BLOCK the partition (move it aside via DLQ → keep progressing)
✓ LOG/ALERT on errors and DLQ messages (visibility — don't lose track of failures)
✓ Manual offset commit → control exactly what's marked processed
✓ Monitor DLQ → failed messages need attention (a growing DLQ signals problems)
```

## Me ya sa haka ya zama mahaɗi

Ganewa abin da ake buɓuga sarrafa kuskure da sakattuniyar saƙo yana da mahimmanci saboda **sarrafa kuskure da kyau yana mahimmanci ga masu karɓa Kafka masu aminci**, saboda haka yana da mahimmanci ilimi aikace-aikace.

Lokacin da masu karɓa ba sa karɓar wani saƙo ba, sarrafaw da ba ta hankali ba tana haifar da matsaloli: gida mai tsai har abada yana sa masu karɓa **su tsaye akan "saƙo mai gida" ** (ba su iya gida ba), tsallake a hankali yana haifar da **asussu**, kuma katsewa yana haifar da sake sarrafaw da gida ake — saboda haka ana buɓugi dabarun sarrafa kuskure na gida.

Ganewa **dabarun** — **sake gwadawa tare da iyaka** (sake gwadawa gazawar gida tare da baje'wa amma iyakewa sake gwadawa don hana sake gwadawa har abada akan gazawar daɗi), **jeri mai sakattun saƙa (DLQ)** tsari (bayan sake gwadawa ya kasa, aika saƙo zuwa jeri DLQ daban wanda ma karɓa ya gida maimakon su tsaye, tare da jeri DLQ da ake duban da sake sarrafawa daga gida — hanya na gida don sarrafa saƙa da ba za a iya sarrafawa ba, hana tsai da asussu duka), bambanta juna gazawar gida (sake gwada) da ta daɗi (DLQ), da sarrafaw mai aminci (don sake gwadawa masu zaman kansa) — ita ce ilimi aikace-aikace mahalai don gina masu karɓa masu aminci.

**Tsarin DLQ** musamman ita ce hanya ta gida ga matsalar saƙo mai gida, da ita masu karɓa can cigaba naka da sarrafa sakattuniyar saƙo don sarrafawa baje'wa.

Ganewa **abubuwan da ake buɓuga a aikace-aikacen gida** — ba a ba saƙo gida gida iyaka jeri tare (tiyata DLQ don cigaba), rubuta alamomi akan kuskure da saƙo DLQ (ido), bukatar offset da jiya a gida don buɓugi, da sa ido jeri DLQ (tunda jeri DLQ mai girma yana nuni wa matsaloli) — yana nuni sarrafaw kuskure mai ƙarfi.

Sarrafa kuskure da kyau yana hana matsaloli na gaske na masu karɓa da suka tsaye (tsai) da asussu (tsallake a hankali) da sarrafaw da ba ta hankali ba tana haifar.

Tunda sarrafa kuskure da kyau (sake gwadawa tare da iyaka, jeri mai sakattun saƙa, bambanta nau'o'in gazawar) yana mahimmanci ga masu karɓa Kafka masu aminci — hana masu karɓa da suka tsaye da asussu — da tsarin DLQ ita ce hanya ta gida, da kuma ganewa waɗannan dabarun yana mahimmanci ilimi aikace-aikace, ganewa anadai ake sarrafa kuskure da sakattuniyar saƙo a Kafka yana da mahimmanci, ilimi aikace-aikace na gida — mahimmanci ga masu karɓa masu aminci, da tsakani jeri mai sakattun saƙa tsari don sarrafa saƙo da ba za a iya sarrafawa ba maimakon tsai da asussu, da mahimmanci don gina masu karɓa Kafka masu ƙarfi da suka sarrafa gazawar sarrafaw ta tsari.