Spike readiness: AI gets its own slice of threads, cheaper hot paths, classroom-friendly login

- /api/ai/ask: at most AI_MAX_CONCURRENT_PER_WORKER (12 of 32 threads) answers per worker;
  beyond that 503 assistant_busy + Retry-After before any quota is reserved. Without it a
  burst of questions (3-40 s each) took every thread and the map and quizzes queued behind.
  The slot is held until the stream is actually finished or closed.
- Answer submission 21 -> 14 queries, 15.4 -> 11.0 ms: one query for all choices, locked
  get_or_create for node progress and streak, and today's streak recorded once per day
  via the cache instead of a transaction per answer.
- /api/home 12.4 -> 8.6 ms: the user's completions are read by user_id instead of an IN list
  of ~600 lesson ids; the dict is returned directly (serializer stays the Swagger contract,
  HomeMatchesItsSerializerTests keeps them equal).
- otp_login per-IP 30 -> 300/hour: a class of 30 on one school Wi-Fi filled it; guessing is
  bounded per code and per phone.
- /api/health aiConcurrency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-11 20:55:24 +03:30
parent feaa5eb540
commit 2c708088e4
11 changed files with 279 additions and 19 deletions

View File

@@ -125,7 +125,7 @@ API docs: Swagger `/api/docs/`, Redoc `/api/redoc/`, admin `/admin/`.
**Structured logging**: `config.middleware.APIRequestLoggingMiddleware` (last in `MIDDLEWARE`, closest to the view) logs every `/api/` request as JSON (method/path/status/duration/user, no body/headers/secrets) via `config/log_formatters.py`.
**Postgres** is the server's DB since 2026-10-08 (own cluster `14/zareian`, port 5440, localhost only; DB `zareian_mathapp`, role `zareian_app`; selected by `DB_NAME`/`DB_USER`/`DB_PASSWORD`/`DB_HOST`/`DB_PORT` in `.env`; passwords in `~/mathapp-v2/.pg_password`, `.redis_password`). It is separate from the company's `main` cluster on 5432 — never touch that. Without `DB_NAME` the app falls back to SQLite (local dev, tests); both run the full suite green. Data was moved with `dumpdata`/`loaddata`; model-count comparison matched. Guards that relied on SQLite's `IMMEDIATE` now rely on real row locks; concurrency tests pass on Postgres. **Connections come from a per-worker pool** (`OPTIONS.pool`, `DB_POOL_MAX_SIZE` 8; `CONN_MAX_AGE` is gone and `DB_CONN_MAX_AGE` in `.env` stops startup): Postgres here has `max_connections=60`, and the old one-connection-per-thread setup (3 workers × 32 threads) turned 21% of requests into 500s at 100 concurrent users in a local load test against a 60-connection Postgres; with the pool 0 errors. A full pool gives 503 + `Retry-After` (`config/db.py`), not 500; `/api/health` → `dbPool` checks `workers × pool + 6 ≤ max_connections`. The AI tutor and the OTP SMS call hand their connection back before the network wait (`release_db_connection`). Worker count lives in `gunicorn.conf.py` (4; the systemd unit no longer passes `--workers`). Not load-tested on the server itself. `deploy.sh` backs up with `pg_dump -Fc` before `migrate`; the off-site live copy is in «بکاپ: کپیِ زنده» below. Operator guide: `~/Desktop/گزارشات Tadrisino/گزارش-۲۹/راهنمای-Postgres-و-Redis.html`. The pre-move `db.sqlite3` and migration dumps were deleted from the server; no SQLite file is left there.
**Postgres** is the server's DB since 2026-10-08 (own cluster `14/zareian`, port 5440, localhost only; DB `zareian_mathapp`, role `zareian_app`; selected by `DB_NAME`/`DB_USER`/`DB_PASSWORD`/`DB_HOST`/`DB_PORT` in `.env`; passwords in `~/mathapp-v2/.pg_password`, `.redis_password`). It is separate from the company's `main` cluster on 5432 — never touch that. Without `DB_NAME` the app falls back to SQLite (local dev, tests); both run the full suite green. Data was moved with `dumpdata`/`loaddata`; model-count comparison matched. Guards that relied on SQLite's `IMMEDIATE` now rely on real row locks; concurrency tests pass on Postgres. **Connections come from a per-worker pool** (`OPTIONS.pool`, `DB_POOL_MAX_SIZE` 8; `CONN_MAX_AGE` is gone and `DB_CONN_MAX_AGE` in `.env` stops startup): Postgres here has `max_connections=60`, and the old one-connection-per-thread setup (3 workers × 32 threads) turned 21% of requests into 500s at 100 concurrent users in a local load test against a 60-connection Postgres; with the pool 0 errors. A full pool gives 503 + `Retry-After` (`config/db.py`), not 500; `/api/health` → `dbPool` checks `workers × pool + 6 ≤ max_connections`. The AI tutor and the OTP SMS call hand their connection back before the network wait (`release_db_connection`). Worker count lives in `gunicorn.conf.py` (4; the systemd unit no longer passes `--workers`). **The AI tutor has its own slice of threads** (`AI_MAX_CONCURRENT_PER_WORKER`, 12 of 32, `ai_tutor/concurrency.py`): beyond that `/api/ai/ask` answers 503 `Retry-After: 10` before reserving quota, so a chat spike cannot starve the map and quizzes. The frontend spreads retries over 1–2× `Retry-After`. Not load-tested on the server itself. `deploy.sh` backs up with `pg_dump -Fc` before `migrate`; the off-site live copy is in «بکاپ: کپیِ زنده» below. Operator guide: `~/Desktop/گزارشات Tadrisino/گزارش-۲۹/راهنمای-Postgres-و-Redis.html`. The pre-move `db.sqlite3` and migration dumps were deleted from the server; no SQLite file is left there.
**SMS** goes through Kavenegar (`KAVENEGAR_API_KEY`/`KAVENEGAR_SENDER`); failures return explicit 503, not a fake success.