p99 latency ya API ni sekunde 3 — unafanyaje debug na optimization?

Question

IT Interview Practice · Accepted Answer

p99 = **tail latency**: 1% ya maombi ya polepole zaidi huchukua sekunde 3. Hatua ya kwanza ni **kupima na kuchambua, si kukisia**. Linganisha p50 dhidi ya p99 — kama p50 ni haraka lakini p99 ni sekunde 3, njia si polepole kwa usawa; kitu fulani kinauma tu *chini ya load au katika kesi za pembeni* (pool saturation, GC pauses, N+1, cold cache, external call isiyopimwa muda).

**Mfano wa kufikiri:** p99 ni *foleni ndefu zaidi ya malipo*, si wastani wa kusubiri — kuharakisha wastani hakumfanyii chochote mteja aliyekwama nyuma.

## Mchakato (kutoka wapi hadi wapi)
1. **Pima na chambua** — RED metrics (Rate, Errors, Duration) + distributed tracing; soma p50/p95/p99. *Pengo* kati yao linasimulia hadithi.
2. **Bainisha hatua** — weka instrument kwa kila hatua kuona wapi maombi ya polepole yanatumia muda; usiboreshe kwa kubahatisha.

```js
// Pima muda wa kila hatua kuona wapi p99 inarundikana (usikisie)
async function handler(req, res) {
  const t0 = process.hrtime.bigint();
  const mark = (label) =>
    req.log.info({ label, ms: Number(process.hrtime.bigint() - t0) / 1e6 });
  const user = await db.user.find(req.id);   mark('db.user');
  const courses = await loadCourses(user);   mark('loadCourses'); // mshukiwa
  res.json(courses);                         mark('total');
}
```

3. **Tambua mshukiwa wa tail** — kwa kawaida *kusubiri*, si kukokotoa: connection-pool exhaustion, N+1 queries, external calls zisizopimwa muda, GC pauses, index inayokosekana, cold cache.

```js
// KABLA: N+1 — safari N za mfululizo, p99 inalipuka N inavyoongezeka
const courses = await db.course.findByUser(userId);
for (const c of courses) c.lessons = await db.lesson.byCourse(c.id);

// BAADA: kunja hadi query 1 -> tail inashuka (hakuna tena safari N za mfululizo)
const courses = await db.course.findByUser(userId);
const lessons = await db.lesson.byCourseIds(courses.map((c) => c.id)); // one IN(...)
const byCourse = Map.groupBy(lessons, (l) => l.courseId);
for (const c of courses) c.lessons = byCourse.get(c.id) ?? [];
```

Hapa p50 (N ndogo) inaonekana sawa, lakini chini ya load N inaongezeka na pool inajaa, kwa hivyo **p99 inalipuka**. Kwa external calls, weka daima **timeout + circuit breaker** ili dependency moja ya polepole isiweze kuburuta p99 ya endpoint nzima.

4. **Rekebisha kitu kimoja → pima p99 tena → rudia.** Kigezo kimoja kwa wakati, la sivyo hutajua kilichosaidia.
5. **Ihifadhi** — weka **SLO** (mfano p99 < 300ms) + alert ili isirudi nyuma.

## Mtazamo wa mahojiano
Mhojaji anachunguza mchakato wa *kimfumo*, si kubahatisha: je unatofautisha **tail na mean**, unapima kabla ya kubadilisha, na unaangalia **queues/locks (kusubiri) kabla ya CPU**. Jibu dhaifu: "ongeza caching / scale up servers." Jibu imara: pima → bainisha kwa tracing → rekebisha sababu mahususi ya tail → thibitisha kwa nambari.

## Kanuni
Boresha **tail (p99), si wastani**. "3s p99" nyingi hutoka kwa *kusubiri* (pool/queue/lock) badala ya *kukokotoa* — kwa hivyo angalia queues kabla ya CPU.

Mchakato (kutoka wapi hadi wapi)

Mtazamo wa mahojiano

Kanuni

Maswali yanayohusiana