MathApp/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project

Django + DRF backend for Tadrisino, an educational app (Persian/Farsi). Codebase and API responses are in Persian; comments in source are Persian too.

## Commands

```bash
source venv/bin/activate
python manage.py runserver
python manage.py migrate
python manage.py test                 # all tests
python manage.py test accounts        # single app
python manage.py test accounts.tests.SomeTestCase.test_something  # single test
```

Local dev requires `DJANGO_DEBUG=1` (enables Swagger auth via `DEV_MASTER_OTP`/`devOtp` in responses). `DEBUG` defaults to `False`. Production also requires `DJANGO_SECRET_KEY`, `DJANGO_ALLOWED_HOSTS` (comma-separated), and `CACHE_URL` (Redis — required whenever `DEBUG=False` since rate limiting needs a shared cache; raises `ImproperlyConfigured` otherwise).

No pytest; uses Django's built-in test runner. No linter/formatter configured.

Docker (`docker compose up --build`) adds a local Loki+Grafana Alloy+Grafana log stack; not a production setup (still `runserver`, not gunicorn). See comments in `docker-compose.yml`, `docker/alloy/config.alloy`, and `LOG_DIR` in `config/settings.py`.

API docs: Swagger `/api/docs/`, Redoc `/api/redoc/`, admin `/admin/`.

## Architecture

**Apps** (each owns its own models, one responsibility):

| app | responsibility | key models |
|---|---|---|
| `accounts` | identity, sessions, OTP auth | `User`, `Profile`, `Session`, `OTPCode` |
| `content` | lessons/curriculum | `Grade`, `Unit`, `Node`, `Lesson`, `Question`, `Choice`, `NodeProgress` |
| `progress` | stats, leaderboard, streaks, gamification characters | `UserStats`, `LeaderboardEntry`, `StreakRecord`, `UserHearts`, `Character` |
| `ai_tutor` | AI assistant chat | `Conversation`, `Message`, `MessageFeedback` |
| `subscription` | plans | `Plan`, `Subscription` |
| `payment` | orders/transactions (Zarinpal gateway + Cafe Bazaar IAP) | `Order`, `Transaction` |
| `discount` | discount codes/offers | `DiscountCode`, `Redemption`, `Offer`, `UserOffer` |
| `notifications` | notifications | `Notification` |
| `affiliate` | referral program | `AffiliateAccount`, `Conversion` |
| `support` | tickets/bug reports | `BugReport`, `Ticket`, `TicketMessage` |
| `family` | parental control | `FamilyLink`, `ParentalSettings`, `WeeklyReportLog` |
| `statuslog` | generic cross-app status-change log | `StatusChangeLog` |
| `campaigns` | messaging campaign storage only, no send logic yet | `MessageCampaign` |
| `config` | project settings, root urls, middleware | — |

**Auth is custom, not Django's.** `accounts.User` is deliberately *not* `AUTH_USER_MODEL` — it's an independent business model. Auth is a stateless signed bearer token (`django.core.signing`, no JWT library) issued/verified in `accounts/authentication.py` (`TadrisinoTokenAuthentication`, header `Authorization: Bearer <token>`), backed by `accounts.Session`. Missing/invalid header → anonymous request, not a hard failure; each view's permission class decides if auth is required.

**Apps are decoupled by convention**: instead of real FKs across app boundaries, code uses loose refs (`user_id`, `object_id` + `label`) so any app can be split out later. Don't add cross-app FKs — follow the existing loose-reference pattern.

**Throttling**: DRF `ScopedRateThrottle` with per-endpoint `throttle_scope` strings, rates centralized in `REST_FRAMEWORK['DEFAULT_THROTTLE_RATES']` in `config/settings.py`. Guest users are throttled by IP (`NUM_PROXIES` setting controls how many XFF hops are trusted — must match real proxy count in prod or IP spoofing/throttle-bypass is possible). Adding a new sensitive endpoint means adding both a `throttle_scope` on the view and a matching rate entry in settings.

**`POST /api/ai/ask`** currently returns a stub; real model integration point is `generate_tutor_answer` in `ai_tutor/views.py`.

**Structured logging**: `config.middleware.APIRequestLoggingMiddleware` (last in `MIDDLEWARE`, closest to the view) logs every `/api/` request as JSON (method/path/status/duration/user, no body/headers/secrets) via `config/log_formatters.py`.

**SQLite** is current DB (`OPTIONS.timeout=20` to tolerate concurrent-write locks, e.g. campaigns tests); README notes Postgres migration is still pending.

**SMS** goes through Kavenegar (`KAVENEGAR_API_KEY`/`KAVENEGAR_SENDER`); failures return explicit 503, not a fake success.

**Gamification characters** (`progress.Character`, Duolingo-mascot-style): an admin-managed catalog (like `subscription.Plan` — created via Django admin or `seed_test_data`, not by users). Unlocking is currently just `UserStats.xp >= Character.unlock_xp` (no separate ownership table); `UserStats.selected_character_id` is the user's active pick, set via `POST /api/characters/select` after that check. First real content beyond the model/plumbing (art assets, more characters, purchase-with-coins, etc.) is expected to land incrementally later.

**Payment (Zarinpal)**: `create_order`/`CreateOrderView` never call Zarinpal directly — `paymentUrl` points at our own `GET /api/payment/orders/{id}/pay` (`PaymentRedirectView`), which calls Zarinpal's PaymentRequest API on first visit (storing `Order.gateway_authority`) and redirects to StartPay; this keeps order creation from depending on Zarinpal's uptime. Zarinpal redirects the user back to `GET /api/payment/zarinpal/callback` (`ZarinpalCallbackView`), which always re-verifies server-to-server via `payment/zarinpal.py::verify_payment` before marking an order paid — a forged `Status=OK` in the query string alone does nothing. Both `payment/zarinpal.py` and `payment/bazaar.py` fail closed (503 / invalid) when their credentials aren't configured, never a fake success.

## Where this is heading (the owner's intent)

The owner writes this backend; another developer writes the frontend; a third person
administers the server. None of them share a machine, and until recently the two halves
were built in parallel without a shared contract — which is why reconciling them cost
rework. The goal is not to merge the repos. **Two repos, one API contract** is the
intended shape, and the contract lives in this repo's OpenAPI schema (`/api/docs/`), not
in anyone's head.

What the owner is steering toward, in rough order:

1. **Both halves running against each other**, on real content, with nobody hand-editing
   files on the server. Most of the plumbing for this now exists.
2. **Nothing manual in a release.** Adding a dependency, a setting or a service must not
   add a step someone has to remember. When a change would, fold it into `deploy.sh` or
   make startup fail loudly and legibly instead.
3. **Content editable without a developer** — questions, curriculum, plan copy, all from
   the admin panel. A management command run over SSH is a last resort, not a workflow.
4. **Answer quality measurable**, not assumed: per-question reports, wrong-answer rates,
   prompt-version feedback. The senior's stated priority is proving the AI actually
   teaches, especially where students get stuck.
5. **Scale without drama** — thousands of concurrent users should be a non-event. Prefer
   the boring fix (an index, a cache, a cheap guard) over new infrastructure.

The owner is early in their career and learning the collaboration workflow as they go.
Explain the *why* alongside the *what*, flag when something is a product decision rather
than a coding one, and never assume a step was done just because it was described.

## Frontend ↔ backend (read before wiring the two together)

The app is two repos and two domains: the React frontend lives in
`Tadrisino/MathApp_frontend` (deployed at `math.tadrisino2.ir`, owned by another
developer) and this backend serves `api.math.tadrisino2.ir`. Our typed client for them
is `src/api/` inside their repo, delivered by pull request — we have read-only Gitea
access there, so changes go via a fork + PR, never a direct push.

**The one trap that keeps biting.** Their `App.tsx` calls relative paths
(`fetch('/api/auth/login')`). Those resolve against the *frontend* origin, where nginx
answers any unknown path with `index.html` — so the response is `200` with an HTML body
and the code dies on `res.json()` with `Unexpected token '<'`. Verify with
`curl https://math.tadrisino2.ir/api/grades`: HTTP 200, HTML. Two fixes exist, both
shipped in PR #3: a `vite` dev proxy (dev works unchanged) and an exported `apiUrl()`
for builds. **The better long-term fix is infrastructure, not code** — proxy `/api` on
the frontend domain to this backend, which makes everything same-origin and removes CORS
entirely. That needs the server admin, not us.

**Order matters when bringing the stack up:** backend first, then frontend. If the
frontend is tested while `api.math.tadrisino2.ir` is down, every screen fails and it
looks like the client is broken.

**Never hand-write the client from memory.** Three of fifteen calls were silently wrong
the first time. Dump the real contract with
`python manage.py spectacular --file schema.yaml --format openapi`, then drive the actual
client module against a running server (`runserver` on a spare port + `npx tsx`) before
sending anything. `content/tests_frontend_contract.py` is the backend-side guard.

**Content the frontend expects but nobody has filled in yet:** `Unit.blurb`,
`Plan.features`/`badge`, `Lesson.example`/`steps`. All optional, all editable from the
admin panel. `Node.icon`/`color` fall back to a deterministic palette so cards are never
colourless.

## Operations

`./deploy.sh` is the whole release: pull, install, migrate, collectstatic, restart, then
it polls `/api/health` and prints the likely cause if the service does not come back.
Settings boot with working defaults (hosts, CSRF, CORS, secret key, cache), so a stale or
partial `.env` on the server no longer breaks startup — env values are *merged with*
those defaults, not replaced by them.

`GET /api/health` is the outside view: database, unapplied migrations, cache, content
counts, security warnings, and recent traffic. It is how you check whether anyone has
actually connected without asking a human.

## Content: where it lives and how it gets to a server

`content/seed/` holds the whole curriculum and question bank as committed JSON —
`curriculum.json` (6 grades, 48 units, 230 nodes, 1,928 lessons) and `questions.json`
(1,441 multiple-choice questions). `deploy.sh` runs `seed_content` on every release, so a
fresh server fills itself with one command: **no Qdrant, no AI credit, no SSH session.**

`seed_content` is idempotent and never deletes. Questions bind to lessons **by title path**
(grade › unit › node › lesson), never by primary key — ids differ between any two
databases, and binding by id silently attaches questions to the wrong lesson.

`seed_content --export` writes the files back from the local DB. It refuses to overwrite a
non-empty `questions.json` from a DB that has no questions (that used to wipe the file
silently); use `--export --only curriculum` when you only mean the curriculum.

## Answer quality: the verification layer

Books are read by a **vision model from page images**, so extraction errors enter there —
a misread digit, a dropped decimal separator, a solution attached to the neighbouring
problem. These are invisible in retrieval and only surface when something trusts the text.

`rag_verify_solutions` re-solves every extracted problem independently and compares.
`LessonChunk.solution_verdict` is three-state — `agrees` / `disagrees` / `unjudgeable` —
and the verdict also rides in the Qdrant payload under a key `retrieval._payload_text`
does not read, so the tutor's context is unchanged. Current state: 1,112 agree, 123
disagree, 881 unjudgeable. **Question generation only ever reads `agrees` chunks.**

Two things that were learned the expensive way and are already encoded in the prompts:

- A boolean verdict is wrong. With two states every "I can't tell" becomes "wrong", which
  reported ~3× the real rate. Read samples before trusting any bulk run.
- `۳/۱۰۴` in these books is the decimal 3.104, not the fraction 3÷104.

`solution` itself is never rewritten — the extracted text stays as the record, and the
independent answer is stored separately in `checked_answer`.

## The figure gap (largest open data issue)

881 chunks (~42%) reference a figure or table that is not in the text, so they cannot be
answered or turned into questions. **The vision model saw the figure; it was simply never
asked to write it down.** `rag/services/pdf.py` now carries a self-contained rule telling
it to inline measurements and labels into the problem text — books ingested before that
rule predate it. Recovering them means re-ingesting those pages (~$2.7 of API credit).

## Quiz assembly

The app plays at **node** level (the island on the map) but questions hang off **lessons**,
and a lesson here is the finest grain — one problem type from the book, not a class
session. `GET /api/nodes/{id}/quiz` assembles a quiz from a node's lessons, at most one
question per lesson. Shuffle is seeded on `nodeId:userId`, so a refresh returns the same
quiz; `questionCount` is reported separately from `requested` so a thin node cannot look
like an error. 172 of 230 nodes currently hold a full four-question quiz.

## AI provider notes

`OPENAI_BASE_URL` points at GapGPT (an OpenAI proxy reachable from Iran) and it exposes
~127 models, including reasoning ones. Two failures look alike and must not be treated
alike — `is_account_limit()` in `rag_verify_solutions` is the shared guard:

- `429 initial_usage_limit_exceeded` — account cap, hours long. Retrying **extends** it.
- `403 insufficient user quota` — credit exhausted. Note the spaces: an earlier guard
  matched `insufficient_quota` with an underscore, missed it, and burned 770 doomed calls.

`/dashboard/billing/usage` returns cents and is the only reliable balance signal; the
`hard_limit_usd` in `/dashboard/billing/subscription` is a placeholder, not a balance.

**The key is currently out of credit.** Bulk work since then runs on Claude Code subagents
instead. Sonnet handles question authoring well (~88% yield); Haiku does not — it cannot
reliably pull the final answer out of a solution and produced ~40% broken output.

## Known gaps (per README pre-deploy checklist)

- Still on SQLite, not Postgres.
- Throttle rates need periodic review against real traffic.
- `DEV_MASTER_OTP` (`0000`) must stay gated behind `DEBUG` — never enabled in prod.
- No video content exists yet, though `Lesson.kind`/`video_url` can express it.
- 58 of 230 nodes still hold fewer than four questions; their verified chunks ran out.
- Kavenegar is unconfigured, so `POST /api/auth/otp/request` returns 503. With `DEBUG=0`
  there is no way to log into the API without creating an `OTPCode` row in the admin
  (`/admin/accounts/otpcode/add/`) — this is how to hand a tester a working login.

## The server: read this before running anything on it

`api.math.tadrisino2.ir` runs on a **shared company machine** (`administrator@78.38.35.249`,
hostname `haghzad`), not a box that belongs to this project. Our app is one tenant among
many. Also on it, all in production use by other people:

- **Gitea itself** (`gitea.service`) — the company's git server, including this repo's
  origin. Breaking it means nobody can push or pull anything, anywhere.
- Mail (`postfix`, `dovecot`), DNS (`named`), `mariadb`, `postgresql@14`, `rabbitmq`,
  `metabase`, `redis-server`, `nginx`.
- Other apps: `apirag`, `artinafront`, `metaverse`, `celerybeat`, and a **generic
  `gunicorn.service` that is not ours**.
- Other people's home directories: `aria`, `artina`, `behdad`, `zahra`, `git`, `gitea`.
  They are mode `0750` — unreadable as `administrator`, trivially readable under `sudo`.
  Do not read them. Nothing in this project needs them.

**Our account is `administrator`, and `sudo -l` reports `(ALL : ALL) ALL`.** It is also in
the `docker` and `lxd` groups, each of which is root by another route. There is no
technical guard rail between a careless command and the whole company's data. The only
guard rail is the operator.

Ours is exactly one directory: **`/home/administrator/mathapp-v2`**, served by
**`gunicorn-mathapp.service`** (unix socket, proxied by nginx). Note `~/mathapp` also
exists, is not a git repo, and is not what runs — do not touch it.

Rules that follow from this, and are not negotiable:

1. **Never run a destructive command outside `~/mathapp-v2`.** No `rm -rf`, no
   `git clean`, no `chown -R`, no dropping databases, anywhere else. When a command
   takes a path, write the path out in full rather than relying on the current directory.
2. **Restart only `gunicorn-mathapp`.** `systemctl restart gunicorn` is a different
   team's app. `deploy.sh` no longer guesses that name, and neither should you.
3. **Never delete the project directory or "re-clone fresh".** `db.sqlite3`, `.env` and
   `.django_secret_key` live inside it and are all gitignored. A fresh clone silently
   destroys the database, the config and every logged-in session. Use
   `git reset --hard origin/main` when the tree needs to go back to a known state.
4. **Never `git clean`** there for the same reason.
5. **Read `.env` only through Django.** `python -c` printing `settings.CACHES[...]` and
   booleans is fine; `cat .env` is not, and neither is pasting its values into a chat.
6. **Take a timestamped backup before editing anything under `/etc`,** and say where you
   put it.
7. **Disk is the standing risk.** It has been sitting at ~95% full (about 6 GB free on a
   113 GB volume). Check `df -h /` before anything that writes a lot — a full disk on
   this machine takes the mail server and Gitea down with us, not just our app.

`DEBUG` must never be `1` on this server: it turns on `DEV_MASTER_OTP` (`0000`), which
accepts any phone number as a valid login, and renders tracebacks with settings in them.
`DJANGO_DEBUG` belongs in `.env` only — not in the systemd unit, where it once sat as
`Environment="DJANGO_DEBUG=1"` and was saved only by `.env` happening to override it.

`DJANGO_SECRET_KEY` is what signs our bearer tokens. It must be a real random value that
exists nowhere in git. If the env var is simply absent and `DEBUG=0`, `config/settings.py`
generates a strong one and stores it in `.django_secret_key` (mode `0600`) — that is the
preferred setup, because there is then no key to leak. **Never let the server run on the
`django-insecure-…` fallback from `config/settings.py`: that string is committed here, so
anyone who can read this repo can forge a token for any user.**

## State as of 2026-09-17 (evening) — read this first

**The server is live and running current code.** `main` is at the latest commit, the
service `gunicorn-mathapp` was restarted against it, and `/api/health` reports
`grades: 6, lessons: 1928, questions: 1441`. `/api/nodes/1/quiz` returns a real
four-question quiz. 435 tests green.

What that took, and the lessons worth keeping:

1. **The server ran two-day-old code while every file on disk was current.** The pull and
   `seed_content` had happened; the service had simply never been restarted, so gunicorn
   still served the code it loaded on Sep 15. WhiteNoise builds its static-file index at
   startup too, which is why `collectstatic` had run and the admin panel still had no CSS.
   **A file on disk is not a deployed file.** Check
   `systemctl show <service> -p ActiveEnterTimestamp` before believing anything.
2. **`deploy.sh` reported success while this was true.** It health-checked a public URL
   that the *old* process answered with 200. It now compares the service start time
   before and after, and exits non-zero when the restart did not happen.
3. **Two files had been hand-edited on the server** (`config/urls.py`, `deploy.sh`), both
   only to add a trailing slash to the health route. That is why `git pull` kept
   conflicting. `api/health/` is now registered upstream alongside `api/health`, so the
   edits were reverted and are not needed again.

Still open, and deliberately not acted on:

- **`DJANGO_SECRET_KEY` on the server is the `django-insecure-…` fallback committed in
  `config/settings.py`.** Auth tokens are signed with it, so anyone who can read this repo
  can forge one. Fixing it means removing the key from `.env` *and* from the systemd unit
  and letting settings generate one. The owner decided this is the server admin's call,
  not ours. `manage.py check --deploy` flags it as `security.W009` on every release.
- `CACHE_URL` is unset, so rate limiting is per-worker (`LocMemCache`). Redis is already
  running on the box at `127.0.0.1:6379`; `db0` belongs to another app, `db1` is free.
- The frontend is still not wired to any API. `src/api/` was merged (PR #1) but is
  imported nowhere; `QuestionScreen` in `App.tsx` still renders four hardcoded
  multiplication questions, and **PR #3** (`zareian/api-client-2`) is still open upstream.
  Until someone connects the screens, none of the 1,441 questions are visible to a user.

The frontend's `Question` type carries a `type` field that nothing reads, and
`count_objects` in `Question.TYPE_CHOICES` is an empty shell — no rows, no serializer
branch, no rendering anywhere. Do not build content for it without asking first.