MathApp/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project

Django + DRF backend for Tadrisino, an educational app (Persian/Farsi). Codebase and API responses are in Persian; comments in source are Persian too.

## Commands

```bash
source venv/bin/activate
python manage.py runserver
python manage.py migrate
python manage.py test                 # all tests
python manage.py test accounts        # single app
python manage.py test accounts.tests.SomeTestCase.test_something  # single test
```

Local dev requires `DJANGO_DEBUG=1` (enables Swagger auth via `DEV_MASTER_OTP`/`devOtp` in responses). `DEBUG` defaults to `False`. Production also requires `DJANGO_SECRET_KEY`, `DJANGO_ALLOWED_HOSTS` (comma-separated), and `CACHE_URL` (Redis — required whenever `DEBUG=False` since rate limiting needs a shared cache; raises `ImproperlyConfigured` otherwise).

No pytest; uses Django's built-in test runner. No linter/formatter configured.

Docker (`docker compose up --build`) adds a local Loki+Grafana Alloy+Grafana log stack; not a production setup (still `runserver`, not gunicorn). See comments in `docker-compose.yml`, `docker/alloy/config.alloy`, and `LOG_DIR` in `config/settings.py`.

API docs: Swagger `/api/docs/`, Redoc `/api/redoc/`, admin `/admin/`.

## Architecture

**Apps** (each owns its own models, one responsibility):

| app | responsibility | key models |
|---|---|---|
| `accounts` | identity, sessions, OTP auth | `User`, `Profile`, `Session`, `OTPCode` |
| `content` | lessons/curriculum | `Grade`, `Unit`, `Node`, `Lesson`, `Question`, `Choice`, `NodeProgress` |
| `progress` | stats, leaderboard, streaks, gamification characters | `UserStats`, `LeaderboardEntry`, `StreakRecord`, `UserHearts`, `Character` |
| `ai_tutor` | AI assistant chat | `Conversation`, `Message`, `MessageFeedback` |
| `subscription` | plans | `Plan`, `Subscription` |
| `payment` | orders/transactions (Zarinpal gateway + Cafe Bazaar IAP) | `Order`, `Transaction` |
| `discount` | discount codes/offers | `DiscountCode`, `Redemption`, `Offer`, `UserOffer` |
| `notifications` | notifications | `Notification` |
| `affiliate` | referral program | `AffiliateAccount`, `Conversion` |
| `support` | tickets/bug reports | `BugReport`, `Ticket`, `TicketMessage` |
| `family` | parental control | `FamilyLink`, `ParentalSettings`, `WeeklyReportLog` |
| `statuslog` | generic cross-app status-change log | `StatusChangeLog` |
| `campaigns` | messaging campaign storage only, no send logic yet | `MessageCampaign` |
| `config` | project settings, root urls, middleware | — |

**Auth is custom, not Django's.** `accounts.User` is deliberately *not* `AUTH_USER_MODEL` — it's an independent business model. Auth is a stateless signed bearer token (`django.core.signing`, no JWT library) issued/verified in `accounts/authentication.py` (`TadrisinoTokenAuthentication`, header `Authorization: Bearer <token>`), backed by `accounts.Session`. Missing/invalid header → anonymous request, not a hard failure; each view's permission class decides if auth is required.

**Apps are decoupled by convention**: instead of real FKs across app boundaries, code uses loose refs (`user_id`, `object_id` + `label`) so any app can be split out later. Don't add cross-app FKs — follow the existing loose-reference pattern.

**Throttling**: DRF `ScopedRateThrottle` with per-endpoint `throttle_scope` strings, rates centralized in `REST_FRAMEWORK['DEFAULT_THROTTLE_RATES']` in `config/settings.py`. Guest users are throttled by IP (`NUM_PROXIES` setting controls how many XFF hops are trusted — must match real proxy count in prod or IP spoofing/throttle-bypass is possible). Adding a new sensitive endpoint means adding both a `throttle_scope` on the view and a matching rate entry in settings.

**`POST /api/ai/ask`** currently returns a stub; real model integration point is `generate_tutor_answer` in `ai_tutor/views.py`.

**Structured logging**: `config.middleware.APIRequestLoggingMiddleware` (last in `MIDDLEWARE`, closest to the view) logs every `/api/` request as JSON (method/path/status/duration/user, no body/headers/secrets) via `config/log_formatters.py`.

**SQLite** is current DB (`OPTIONS.timeout=20` to tolerate concurrent-write locks, e.g. campaigns tests); README notes Postgres migration is still pending.

**SMS** goes through Kavenegar (`KAVENEGAR_API_KEY`/`KAVENEGAR_SENDER`); failures return explicit 503, not a fake success.

**Gamification characters** (`progress.Character`, Duolingo-mascot-style): an admin-managed catalog (like `subscription.Plan` — created via Django admin or `seed_test_data`, not by users). Unlocking is currently just `UserStats.xp >= Character.unlock_xp` (no separate ownership table); `UserStats.selected_character_id` is the user's active pick, set via `POST /api/characters/select` after that check. First real content beyond the model/plumbing (art assets, more characters, purchase-with-coins, etc.) is expected to land incrementally later.

**Payment (Zarinpal)**: `create_order`/`CreateOrderView` never call Zarinpal directly — `paymentUrl` points at our own `GET /api/payment/orders/{id}/pay` (`PaymentRedirectView`), which calls Zarinpal's PaymentRequest API on first visit (storing `Order.gateway_authority`) and redirects to StartPay; this keeps order creation from depending on Zarinpal's uptime. Zarinpal redirects the user back to `GET /api/payment/zarinpal/callback` (`ZarinpalCallbackView`), which always re-verifies server-to-server via `payment/zarinpal.py::verify_payment` before marking an order paid — a forged `Status=OK` in the query string alone does nothing. Both `payment/zarinpal.py` and `payment/bazaar.py` fail closed (503 / invalid) when their credentials aren't configured, never a fake success.

## Where this is heading (the owner's intent)

The owner writes this backend; another developer writes the frontend; a third person
administers the server. None of them share a machine, and until recently the two halves
were built in parallel without a shared contract — which is why reconciling them cost
rework. The goal is not to merge the repos. **Two repos, one API contract** is the
intended shape, and the contract lives in this repo's OpenAPI schema (`/api/docs/`), not
in anyone's head.

What the owner is steering toward, in rough order:

1. **Both halves running against each other**, on real content, with nobody hand-editing
   files on the server. Most of the plumbing for this now exists.
2. **Nothing manual in a release.** Adding a dependency, a setting or a service must not
   add a step someone has to remember. When a change would, fold it into `deploy.sh` or
   make startup fail loudly and legibly instead.
3. **Content editable without a developer** — questions, curriculum, plan copy, all from
   the admin panel. A management command run over SSH is a last resort, not a workflow.
4. **Answer quality measurable**, not assumed: per-question reports, wrong-answer rates,
   prompt-version feedback. The senior's stated priority is proving the AI actually
   teaches, especially where students get stuck.
5. **Scale without drama** — thousands of concurrent users should be a non-event. Prefer
   the boring fix (an index, a cache, a cheap guard) over new infrastructure.

The owner is early in their career and learning the collaboration workflow as they go.
Explain the *why* alongside the *what*, flag when something is a product decision rather
than a coding one, and never assume a step was done just because it was described.

## Frontend ↔ backend (read before wiring the two together)

The app is two repos and two domains: the React frontend lives in
`Tadrisino/MathApp_frontend` (deployed at `math.tadrisino2.ir`, owned by another
developer) and this backend serves `api.math.tadrisino2.ir`. Our typed client for them
is `src/api/` inside their repo, delivered by pull request — we have read-only Gitea
access there, so changes go via a fork + PR, never a direct push.

**The one trap that keeps biting.** Their `App.tsx` calls relative paths
(`fetch('/api/auth/login')`). Those resolve against the *frontend* origin, where nginx
answers any unknown path with `index.html` — so the response is `200` with an HTML body
and the code dies on `res.json()` with `Unexpected token '<'`. Verify with
`curl https://math.tadrisino2.ir/api/grades`: HTTP 200, HTML. Two fixes exist, both
shipped in PR #3: a `vite` dev proxy (dev works unchanged) and an exported `apiUrl()`
for builds. **The better long-term fix is infrastructure, not code** — proxy `/api` on
the frontend domain to this backend, which makes everything same-origin and removes CORS
entirely. That needs the server admin, not us.

**Order matters when bringing the stack up:** backend first, then frontend. If the
frontend is tested while `api.math.tadrisino2.ir` is down, every screen fails and it
looks like the client is broken.

**Never hand-write the client from memory.** Three of fifteen calls were silently wrong
the first time. Dump the real contract with
`python manage.py spectacular --file schema.yaml --format openapi`, then drive the actual
client module against a running server (`runserver` on a spare port + `npx tsx`) before
sending anything. `content/tests_frontend_contract.py` is the backend-side guard.

**Content the frontend expects but nobody has filled in yet:** `Unit.blurb`,
`Plan.features`/`badge`, `Lesson.example`/`steps`. All optional, all editable from the
admin panel. `Node.icon`/`color` fall back to a deterministic palette so cards are never
colourless.

## Operations

`./deploy.sh` is the whole release: pull, install, migrate, collectstatic, restart, then
it polls `/api/health` and prints the likely cause if the service does not come back.
Settings boot with working defaults (hosts, CSRF, CORS, secret key, cache), so a stale or
partial `.env` on the server no longer breaks startup — env values are *merged with*
those defaults, not replaced by them.

`GET /api/health` is the outside view: database, unapplied migrations, cache, content
counts, security warnings, and recent traffic. It is how you check whether anyone has
actually connected without asking a human.

## Content: where it lives and how it gets to a server

`content/seed/` holds the whole curriculum and question bank as committed JSON —
`curriculum.json` (6 grades, 48 units, 230 nodes, 1,928 lessons) and `questions.json`
(1,441 multiple-choice questions). `deploy.sh` runs `seed_content` on every release, so a
fresh server fills itself with one command: **no Qdrant, no AI credit, no SSH session.**

`seed_content` is idempotent and never deletes. Questions bind to lessons **by title path**
(grade › unit › node › lesson), never by primary key — ids differ between any two
databases, and binding by id silently attaches questions to the wrong lesson.

`seed_content --export` writes the files back from the local DB. It refuses to overwrite a
non-empty `questions.json` from a DB that has no questions (that used to wipe the file
silently); use `--export --only curriculum` when you only mean the curriculum.

## Answer quality: the verification layer

Books are read by a **vision model from page images**, so extraction errors enter there —
a misread digit, a dropped decimal separator, a solution attached to the neighbouring
problem. These are invisible in retrieval and only surface when something trusts the text.

`rag_verify_solutions` re-solves every extracted problem independently and compares.
`LessonChunk.solution_verdict` is three-state — `agrees` / `disagrees` / `unjudgeable` —
and the verdict also rides in the Qdrant payload under a key `retrieval._payload_text`
does not read, so the tutor's context is unchanged. Current state: 1,112 agree, 123
disagree, 881 unjudgeable. **Question generation only ever reads `agrees` chunks.**

Two things that were learned the expensive way and are already encoded in the prompts:

- A boolean verdict is wrong. With two states every "I can't tell" becomes "wrong", which
  reported ~3× the real rate. Read samples before trusting any bulk run.
- `۳/۱۰۴` in these books is the decimal 3.104, not the fraction 3÷104.

`solution` itself is never rewritten — the extracted text stays as the record, and the
independent answer is stored separately in `checked_answer`.

## The figure gap (largest open data issue)

881 chunks (~42%) reference a figure or table that is not in the text, so they cannot be
answered or turned into questions. **The vision model saw the figure; it was simply never
asked to write it down.** `rag/services/pdf.py` now carries a self-contained rule telling
it to inline measurements and labels into the problem text — books ingested before that
rule predate it. Recovering them means re-ingesting those pages (~$2.7 of API credit).

## Quiz assembly

The app plays at **node** level (the island on the map) but questions hang off **lessons**,
and a lesson here is the finest grain — one problem type from the book, not a class
session. `GET /api/nodes/{id}/quiz` assembles a quiz from a node's lessons, at most one
question per lesson. Shuffle is seeded on `nodeId:userId`, so a refresh returns the same
quiz; `questionCount` is reported separately from `requested` so a thin node cannot look
like an error. 172 of 230 nodes currently hold a full four-question quiz.

## AI provider notes

`OPENAI_BASE_URL` points at GapGPT (an OpenAI proxy reachable from Iran) and it exposes
~127 models, including reasoning ones. Two failures look alike and must not be treated
alike — `is_account_limit()` in `rag_verify_solutions` is the shared guard:

- `429 initial_usage_limit_exceeded` — account cap, hours long. Retrying **extends** it.
- `403 insufficient user quota` — credit exhausted. Note the spaces: an earlier guard
  matched `insufficient_quota` with an underscore, missed it, and burned 770 doomed calls.

`/dashboard/billing/usage` returns cents and is the only reliable balance signal; the
`hard_limit_usd` in `/dashboard/billing/subscription` is a placeholder, not a balance.

**The key is currently out of credit.** Bulk work since then runs on Claude Code subagents
instead. Sonnet handles question authoring well (~88% yield); Haiku does not — it cannot
reliably pull the final answer out of a solution and produced ~40% broken output.

## Known gaps (per README pre-deploy checklist)

- Still on SQLite, not Postgres.
- Throttle rates need periodic review against real traffic.
- `DEV_MASTER_OTP` (`0000`) must stay gated behind `DEBUG` — never enabled in prod.
- No video content exists yet, though `Lesson.kind`/`video_url` can express it.
- 58 of 230 nodes still hold fewer than four questions; their verified chunks ran out.
- Kavenegar is unconfigured, so `POST /api/auth/otp/request` returns 503. With `DEBUG=0`
  there is no way to log into the API without creating an `OTPCode` row in the admin
  (`/admin/accounts/otpcode/add/`) — this is how to hand a tester a working login.

## The server: read this before running anything on it

`api.math.tadrisino2.ir` runs on a **shared company machine** (`administrator@78.38.35.249`,
hostname `haghzad`), not a box that belongs to this project. Our app is one tenant among
many. Also on it, all in production use by other people:

- **Gitea itself** (`gitea.service`) — the company's git server, including this repo's
  origin. Breaking it means nobody can push or pull anything, anywhere.
- Mail (`postfix`, `dovecot`), DNS (`named`), `mariadb`, `postgresql@14`, `rabbitmq`,
  `metabase`, `redis-server`, `nginx`.
- Other apps: `apirag`, `artinafront`, `metaverse`, `celerybeat`, and a **generic
  `gunicorn.service` that is not ours**.
- Other people's home directories: `aria`, `artina`, `behdad`, `zahra`, `git`, `gitea`.
  They are mode `0750` — unreadable as `administrator`, trivially readable under `sudo`.
  Do not read them. Nothing in this project needs them.

**Our account is `administrator`, and `sudo -l` reports `(ALL : ALL) ALL`.** It is also in
the `docker` and `lxd` groups, each of which is root by another route. There is no
technical guard rail between a careless command and the whole company's data. The only
guard rail is the operator.

Ours is exactly one directory: **`/home/administrator/mathapp-v2`**, served by
**`gunicorn-mathapp.service`** (unix socket, proxied by nginx). Note `~/mathapp` also
exists, is not a git repo, and is not what runs — do not touch it.

The same unit also answers to **`zareian-mathapp`** (a symlink alias in
`/etc/systemd/system/`, added 2026-09-18). Prefer that name in anything a human will type
or copy: `gunicorn` alone is a *different team's* service on this box, and `tadrisino` is
too generic for a machine where other people's projects may claim it later.

```bash
sudo systemctl restart zareian-mathapp     # ours, unambiguous
sudo systemctl restart gunicorn            # NOT ours — someone else's app
```

Rules that follow from this, and are not negotiable:

1. **Never run a destructive command outside `~/mathapp-v2`.** No `rm -rf`, no
   `git clean`, no `chown -R`, no dropping databases, anywhere else. When a command
   takes a path, write the path out in full rather than relying on the current directory.
2. **Restart only `gunicorn-mathapp`.** `systemctl restart gunicorn` is a different
   team's app. `deploy.sh` no longer guesses that name, and neither should you.
3. **Never delete the project directory or "re-clone fresh".** `db.sqlite3`, `.env` and
   `.django_secret_key` live inside it and are all gitignored. A fresh clone silently
   destroys the database, the config and every logged-in session. Use
   `git reset --hard origin/main` when the tree needs to go back to a known state.
4. **Never `git clean`** there for the same reason.
5. **Read `.env` only through Django.** `python -c` printing `settings.CACHES[...]` and
   booleans is fine; `cat .env` is not, and neither is pasting its values into a chat.
6. **Take a timestamped backup before editing anything under `/etc`,** and say where you
   put it.
7. **Disk is the standing risk.** It has been sitting at ~95% full (about 6 GB free on a
   113 GB volume). Check `df -h /` before anything that writes a lot — a full disk on
   this machine takes the mail server and Gitea down with us, not just our app.

### Handling the server password (every agent, every time)

Access is a password for the shared `administrator` account, not a key. Handle it exactly
like this — the steps exist for reasons, not ceremony:

```bash
printf '%s' '<password>' > "$SCRATCH/.pw"   # scratchpad, never the repo
chmod 600 "$SCRATCH/.pw"
sshpass -f "$SCRATCH/.pw" ssh administrator@78.38.35.249 '<command>'
ssh administrator@78.38.35.249 'sudo -S -p "" <command>' < "$SCRATCH/.pw"
shred -u "$SCRATCH/.pw"                      # immediately after, not "later"
```

- **Never put the password in the command line.** `ps` on this box shows every process's
  full argv to every logged-in user, and several other teams have accounts here.
- **Never write it into the repo, a memory file, a commit, or a note.** The scratchpad
  only, and only for as long as the commands take.
- **`shred -u` as soon as the work is done**, not at the end of the session. Recreate it
  if more work comes; that costs nothing.
- **Never echo it back into the conversation**, even partially, and never paste `.env`
  values either.
- Do not create SSH keys, edit `sshd_config`, or change the password unless the user asks
  for that specifically. It is a shared account; changing it locks the admin out.

The password reaches an agent only because the user supplies it in that session. If it is
not in the conversation, it is not available — say so and ask, rather than hunting for it
on disk. If the user wants to stop pasting it into chat, the macOS Keychain is the local
fix (`security add-generic-password -s tadrisino-server -a administrator -w`, then read it
back with `security find-generic-password -s tadrisino-server -w`), and an SSH key is the
real one.

`DEBUG` must never be `1` on this server: it turns on `DEV_MASTER_OTP` (`0000`), which
accepts any phone number as a valid login, and renders tracebacks with settings in them.
`DJANGO_DEBUG` belongs in `.env` only — not in the systemd unit, where it once sat as
`Environment="DJANGO_DEBUG=1"` and was saved only by `.env` happening to override it.

`DJANGO_SECRET_KEY` is what signs our bearer tokens. It must be a real random value that
exists nowhere in git. If the env var is simply absent and `DEBUG=0`, `config/settings.py`
generates a strong one and stores it in `.django_secret_key` (mode `0600`) — that is the
preferred setup, because there is then no key to leak. **Never let the server run on the
`django-insecure-…` fallback from `config/settings.py`: that string is committed here, so
anyone who can read this repo can forge a token for any user.**

## State as of 2026-09-17 (evening) — read this first

**The server is live and running current code.** `main` is at the latest commit, the
service `gunicorn-mathapp` was restarted against it, and `/api/health` reports
`grades: 6, lessons: 1928, questions: 1441`. `/api/nodes/1/quiz` returns a real
four-question quiz. 435 tests green.

What that took, and the lessons worth keeping:

1. **The server ran two-day-old code while every file on disk was current.** The pull and
   `seed_content` had happened; the service had simply never been restarted, so gunicorn
   still served the code it loaded on Sep 15. WhiteNoise builds its static-file index at
   startup too, which is why `collectstatic` had run and the admin panel still had no CSS.
   **A file on disk is not a deployed file.** Check
   `systemctl show <service> -p ActiveEnterTimestamp` before believing anything.
2. **`deploy.sh` reported success while this was true.** It health-checked a public URL
   that the *old* process answered with 200. It now compares the service start time
   before and after, and exits non-zero when the restart did not happen.
3. **Two files had been hand-edited on the server** (`config/urls.py`, `deploy.sh`), both
   only to add a trailing slash to the health route. That is why `git pull` kept
   conflicting. `api/health/` is now registered upstream alongside `api/health`, so the
   edits were reverted and are not needed again.

Still open, and deliberately not acted on:

- **`DJANGO_SECRET_KEY` on the server is the `django-insecure-…` fallback committed in
  `config/settings.py`.** Auth tokens are signed with it, so anyone who can read this repo
  can forge one. Fixing it means removing the key from `.env` *and* from the systemd unit
  and letting settings generate one. The owner decided this is the server admin's call,
  not ours. `manage.py check --deploy` flags it as `security.W009` on every release.
- `CACHE_URL` is unset, so rate limiting is per-worker (`LocMemCache`). Redis is already
  running on the box at `127.0.0.1:6379`; `db0` belongs to another app, `db1` is free.
- The frontend is still not wired to any API. `src/api/` was merged (PR #1) but is
  imported nowhere; `QuestionScreen` in `App.tsx` still renders four hardcoded
  multiplication questions, and **PR #3** (`zareian/api-client-2`) is still open upstream.
  Until someone connects the screens, none of the 1,441 questions are visible to a user.

The frontend's `Question` type carries a `type` field that nothing reads, and
`count_objects` in `Question.TYPE_CHOICES` is an empty shell — no rows, no serializer
branch, no rendering anywhere. Do not build content for it without asking first.