Latency p99 sawijining API yaiku 3s — kepiye carane debug lan ngoptimalake?

Question

IT Interview Practice · Accepted Answer

p99 = **tail latency**: 1% request paling alon njupuk 3s. Langkah pisanan yaiku **ngukur lan mecah, dudu ngira**. Bandhingake p50 vs p99 — yen p50 cepet nanging p99 3s, jalure ora alon merata; ana sing nggigit mung *nalika beban dhuwur utawa ing kasus pinggir* (saturasi pool, jeda GC, N+1, cold cache, panggilan eksternal sing ora diwaktu).

**Model mental:** p99 iku *antrian kasir paling dawa*, dudu wektu ngenteni rata-rata — nyepetake rata-rata ora ana gunane kanggo pelanggan sing kepancal ing mburi.

## Prosese (saka ngendi menyang ngendi)
1. **Ukur & mecah** — metrik RED (Rate, Errors, Duration) + distributed tracing; waca p50/p95/p99. *Gap* ing antarane sing crita kabeh.
2. **Lokalisasi tahap** — instrumentasi saben langkah kanggo ndeleng ngendi request alon ngenggo wektu; aja ngoptimalake adhedhasar firasat.

```js
// Waktu saben tahap kanggo ndeleng ngendi p99 numpuk (aja ngira)
async function handler(req, res) {
  const t0 = process.hrtime.bigint();
  const mark = (label) =>
    req.log.info({ label, ms: Number(process.hrtime.bigint() - t0) / 1e6 });
  const user = await db.user.find(req.id);   mark('db.user');
  const courses = await loadCourses(user);   mark('loadCourses'); // tersangka
  res.json(courses);                         mark('total');
}
```

3. **Identifikasi tersangka tail** — biasane *ngenteni*, dudu ngitung: pool koneksi entek, query N+1, panggilan eksternal ora diwaktu, jeda GC, index ilang, cold cache.

```js
// SADURUNGE: N+1 — N round-trip urut, p99 mbledhos nalika N mundhak
const courses = await db.course.findByUser(userId);
for (const c of courses) c.lessons = await db.lesson.byCourse(c.id);

// SAWISE: ciutake dadi 1 query -> tail mudhun (ora ana N trip urut maneh)
const courses = await db.course.findByUser(userId);
const lessons = await db.lesson.byCourseIds(courses.map((c) => c.id)); // siji IN(...)
const byCourse = Map.groupBy(lessons, (l) => l.courseId);
for (const c of courses) c.lessons = byCourse.get(c.id) ?? [];
```

Ing kene p50 (N cilik) katon apik, nanging nalika beban dhuwur N mundhak lan pool jenuh, mula **p99 njeblug**. Kanggo panggilan eksternal, tansah setel **timeout + circuit breaker** supaya siji dependensi alon ora bisa narik p99 saka kabeh endpoint.

4. **Benerake siji perkara → ukur maneh p99 → baleni.** Siji variabel saben wektu, utawa kowe ora bakal ngerti apa sing mbantu.
5. **Kunci** — setel **SLO** (tuladha p99 < 300ms) + alert supaya ora mundur maneh.

## Sudut pandang wawancara
Pewawancara nyelidiki proses sing *metodis*, dudu tebakan begja: apa kowe mbedakake **tail saka mean**, ngukur sadurunge ngowahi, lan ndeleng **antrian/lock (ngenteni) sadurunge CPU**. Jawaban lemah: "tambah caching / scale up server." Jawaban kuwat: ukur → lokalisasi nganggo tracing → benerake panyebab tail spesifik → verifikasi nganggo angka.

## Prinsip
Optimalake **tail (p99), dudu rata-rata**. Umume "p99 3s" teka saka *ngenteni* (pool/antrian/lock) tinimbang *ngitung* — mula deleng antrian sadurunge CPU.

Prosese (saka ngendi menyang ngendi)

Sudut pandang wawancara

Prinsip

Pitakon sing gegandhengan