Yadda kuke gudanar da aiki na dama na kirki (Celery/ARQ)?

Question

Accepted Answer

Don **aiki na dama na kirki, mai ɗorewa** (wajey fasalin FastAPI mai sauƙi `BackgroundTasks`), kuna amfani da jeri na gaske: **Celery** (daidaita na ainihi) ko **ARQ** (sabon jeri na async). Waɗannan suna gudanar da aiki mai nauyi, da iya saba, da aiki da aka jira a cikin tsarin aiki daban-daban, wanda aka tallata da broker kamar Redis.

## Me yasa ba kawai BackgroundTasks?

```text
BackgroundTasks runs in the web process → NO persistence (lost on crash), NO retries,
NO scheduling, and heavy work ties up the worker. Fine only for quick fire-and-forget.
For durable/critical/heavy/scheduled jobs → use a real task queue (Celery / ARQ).
```

## Celery — ayyana da jera aiki

```python
# tasks.py
from celery import Celery
celery_app = Celery("tasks", broker="redis://localhost:6379")

@celery_app.task(bind=True, max_retries=3)
def process_video(self, video_id: int):
    try:
        do_heavy_processing(video_id)        # runs in a WORKER, not the web process
    except Exception as exc:
        raise self.retry(exc=exc, countdown=60)   # automatic retry on failure
```

```python
# in a FastAPI endpoint — QUEUE the task, return immediately
@app.post("/videos/{id}/process")
def process(id: int):
    process_video.delay(id)                  # enqueue → returns at once (doesn't block)
    return {"status": "processing started"}
# run workers separately: celery -A tasks worker
```

Tsibar haɗi tana jera aikin kuma tana amsa nan da nan; tsarin **aiki** daban-daban yana aiwatar da shi tare da dorewa da saba.

## ARQ — async-native (ya dace da FastAPI async model)

```python
# ARQ is built on asyncio — a natural fit for async FastAPI codebases
async def process_video(ctx, video_id: int):
    await do_async_processing(video_id)

# enqueue from an endpoint
await redis.enqueue_job("process_video", video_id)
```

ARQ shine mai sauƙi, zaɓi na farko-async — abin banaba lokacin da aƙwalwar ku tana async.

## Abin da jeri na aiki na gaske ke ba

```text
✓ Persistence — jobs survive crashes/restarts (stored in the broker)
✓ Retries with backoff — resilience against transient failures
✓ Scheduling — periodic/cron-like jobs (Celery Beat)
✓ Distribution & scaling — workers scale independently of the web servers
✓ Concurrency & rate limiting, task routing/priorities, result tracking
```

## Me yasa ya matufi

Gudanar da aiki na dama na kirki mai ƙarfin gida shine mahimmi don gina FastAPI ayyukan matakin sarrafa-koli wanda ke jiye aiki mai nauyi ko jirewa ta hakika — kuma fahimtar lokacin da kuma yadda ake amfani da jeri na aiki na gaske shine ilimin matakin babbar-tsammani.

Judgement na asali shine fahimtar **iyakokin `BackgroundTasks`** (babu dorewa, saba, ko jira — daidai kawai don sauri, aiki da ba mahimmi ba) da isa ga **Celery ko ARQ** lokacin da aiki ya kamata ya kasance mai ɗorewa, mai saba, da aka jira, mai nauyi, ko rarraba (sarrafa fayiloli/bidiyo, aika fayad masu yawa, juyar da bayani, samar da rahotanni).
\'Yan jeri waɗannan suna ba da ikon matakin sarrafa-koli na mahimmi: **dorewa** (aiki yana tsira da tumaɗi ta hanyar broker, saboda aiki ba ya ɓace ba), **saba ta atomatik** (karfi akan gida gazawar da ba jiya ba), **jira** (aiki mai lokaci-lokaci), da **leㄥotar sararin** na masu aiki daga satoshi na jari.

Sanin yadda ake ayyana da jera aiki (Celery's `.delay()`, ARQ's API na async-native), gidaje (broker + masu aiki), kuma musamman **yanke-yanke na lokacin amfani da jeri na aiki na gaske da farye BackgroundTasks**, shine mahimmi don tsara aiki mai aminci wanda ke nutsewa aiki mai nauyi ba tare da baɓa shi ba — buɗe-buɗe, buɗe na mahimmi wanda ke rarrabawa tsarin sarrafa-koli na kirki daga raunin da babu tsarawa.