Kafka 很强大,但充满了常见陷阱 — 对分区、顺序、消费者行为、传递语义和操作的误解和错误。理解这些有助于避免问题并正确使用 Kafka。
设计和分区陷阱
✗ Too FEW partitions → limits consumer parallelism (can't scale consumption beyond
partition count); too MANY → overhead, rebalancing pain → plan partition count carefully
✗ Wrong PARTITION KEY → uneven distribution (hot partitions) or wrong ordering scope →
choose a key for even distribution AND the ordering you need
✗ Expecting GLOBAL ordering → Kafka only orders WITHIN a partition (a common misunderstanding)
✗ Adding partitions changes key→partition mapping → breaks ordering for keys (be careful)
消费者和处理陷阱
✗ Ignoring CONSUMER LAG → consumers falling behind unnoticed → growing delay
✗ Not handling DUPLICATES → at-least-once means duplicates → need idempotent processing
(a very common mistake — assuming exactly-once without idempotency)
✗ Slow processing blocking the consumer → causes lag/rebalances; long processing → rebalance issues
✗ Frequent REBALANCES → from slow processing/timeouts → consumption pauses (tune, optimize)
✗ Improper offset committing → reprocessing or message loss
