Backup na disaster recovery kwa database ya makumi ya TB

Question

IT Interview Practice · Accepted Answer

**Changanya periodic snapshots na continuous WAL archiving kwa point-in-time recovery, replicate offsite/cross-region, na thibitisha kwa restore drills za mara kwa mara dhidi ya targets za wazi za RPO/RTO.** Backup ambayo hujawahi kuirestore ni tumaini, si mpango.

## Usanifu

```text
Primary DB ──snapshot (daily)──▶ Object store (S3) ──cross-region replicate──▶ DR region
     │                               ▲
     └── WAL/binlog stream ──────────┘   (continuous, PITR to any second)
                                          │
                          restore drill ──┴──▶ verified restore ─▶ measure RTO, checksum RPO
```

## Jinsi inavyofanya kazi

- **Snapshots** ni nakala kamili ya volume/cluster (mfano daily EBS snapshot au `pg_basebackup`). Ni base unayorestore *kutoka* — lakini kurestore hadi usiku uliopita tu hupoteza siku ya writes.
- **WAL / binlog archiving** hustream kila mabadiliko tangu snapshot ya mwisho kwenda object store kwa kuendelea. Restore = snapshot + replay WAL hadi sekunde yoyote uliyochagua. Hii ni **PITR (point-in-time recovery)** — pona hadi 10:32:04, kabla tu ya `DELETE` mbaya.
- **Offsite / cross-region.** Backup katika region ile ile ya DB hufa pamoja na region. Replicate kwenda region ya pili (S3 cross-region replication) ili outage ya kikanda au ransomware isichukue zote mbili.
- **Restore drills.** Mara kwa mara restore kwenda scratch environment, pima muda ilichukua (**RTO**) na kiasi cha data ungepoteza (**RPO**), na checksum dhidi ya prod.

## RPO vs RTO

- **RPO (Recovery Point Objective)** = kiasi cha data unachoweza kumudu kupoteza. WAL archiving kila dakika 5 → RPO ≈ dakika 5. Synchronous replica → RPO ≈ 0.
- **RTO (Recovery Time Objective)** = jinsi unavyopaswa kurudi haraka. Kurestore 20TB kutoka S3 kunaweza kuchukua saa; warm standby replica hupunguza RTO hadi dakika.

## Mtazamo wa mahojiano

Mhojiwa huchunguza kama unajua **snapshots peke yake hazitoshi** (data loss kati yao), kama unaweza **kufafanua RPO/RTO kwa namba**, na kama **unajaribu restores**. Majibu bora sana husisitiza kuwa backups zisizojaribiwa hushindwa hasa wakati zinahitajika, na huchagua strategy kutoka RPO/RTO ya *biashara*, si kutoka urahisi. Mtego: kudhani read replica ni backup — replication hunakili `DROP TABLE` kwa uaminifu pia.

## Maelewano na hitilafu

- **Gharama vs RTO.** Warm cross-region standby hutoa dakika za RTO lakini huongeza gharama ya infra mara mbili; cold S3 restore ni nafuu lakini polepole. Nunua tier ambayo biashara inahitaji kweli.
- **Backup load.** Kusnapshot hot 20TB DB huongeza I/O; chukua snapshots kutoka replica.
- **Silent corruption.** WAL inaweza kuarchive page iliyoharibika; periodic full-restore + checksum huinasa.
- **Retention na compliance.** Weka history ya kutosha (mfano PITR ya siku 35 + monthly archives) kwa mahitaji ya kisheria; Glacier kwa long-term nafuu.
- **Wakati usifanye:** DB ndogo ni sawa na nightly `pg_dump` kwenda S3 — full WAL + cross-region standby ni kupita kiasi hadi data loss ipimwe kwa pesa halisi.

Usanifu

Jinsi inavyofanya kazi

RPO vs RTO

Mtazamo wa mahojiano

Maelewano na hitilafu

Maswali yanayohusiana