Commit Graph

82 Commits

Author SHA1 Message Date
feaa5eb540 Postgres connection pool, clean 503 under overload, time indexes, 4 workers
Postgres here allows 60 connections, but each of the 96 gunicorn threads held its
own persistent one: at 100 concurrent users (local Postgres with the same limit)
21% of requests were 500 "remaining connection slots are reserved". Now each worker
has a psycopg pool (8), so the same load gives 0 errors and 714 req/s (was 460).

- A full pool answers 503 + Retry-After (DRF handler and middleware), not 500.
- The AI tutor and the OTP SMS call return their connection before the network wait.
- statement_timeout 30 s, idle-in-transaction 60 s; bad pool settings and the old
  DB_CONN_MAX_AGE stop startup instead of failing every request.
- /api/health dbPool: per-worker pool stats and workers x pool + 6 vs max_connections.
- Indexes on the columns global counters filter by time (weekly leaderboard,
  global SMS cap, global AI cap); the leaderboard recomputes once per expiry.
- Worker count moves into gunicorn.conf.py (4 on a shared 8-core box).
- security doc: Redis item closed (server uses our own Redis since 2026-10-08).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:21:35 +03:30
4821df3914 CLAUDE.md: pointer to the next-APK notes and why app payments go through pay.html
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:54:12 +03:30
2e87a81cc5 CLAUDE.md: pre-move SQLite copy removed from the server; the old August backup script was never scheduled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:14:42 +03:30
a9cccc6550 Live off-site Postgres copy on 247 replaces the nightly pg_dump; backup retention in deploy.sh
- ops/pgbackup: SSH tunnel + pg_receivewal (slot zareian_247) + daily verified pg_basebackup (keep 7)
  and a restore check, all run on 78.38.35.247 as systemd units
- /api/health reports offsiteBackup (slot active/lag/lost) and warns; OFFSITE_WAL_SLOT setting
- removed statuslog backup_db/backup_verify and their health warnings
- deploy.sh keeps 3 pre-deploy dumps, 3 local patches, drops manual backups after 30 days
- docs: operations backup section, security item 14 closed, CLAUDE.md status

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:53:10 +03:30
19072ee534 Fourth companion renamed Tedi -> Omega (ω); slug stays tedi
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 01:42:44 +03:30
a39643b8a3 CLAUDE.md: paywall and subscriptions are live
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 01:20:51 +03:30
a5feee053a CLAUDE.md: Bazaar package name now matches the APK (com.tadrisino.math.app)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 14:12:28 +03:30
22c3196343 Close /api/health details and API docs to the public
Health: public gets only ok/error; details need X-Internal-Key. Docs/schema: admin session only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 13:05:19 +03:30
d6529956e1 Docs: Postgres + own Redis are the server's current state
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 12:43:48 +03:30
e00f6b80eb CLAUDE.md: lesson figures workflow, cost, review lessons, frontend cache and nginx headers
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 19:48:10 +03:30
d584d62904 Do not connect to the shared Redis: CACHE_URL stays empty, own Redis is an open decision
The owner decided (2026-10-07) that we do not connect to anything other teams run on the box, Redis
included. CACHE_URL was emptied on the server and our keys were removed from the shared Redis.

- settings: drop KEY_PREFIX / CACHE_KEY_PREFIX (it only existed because the Redis was shared); comments
  say CACHE_URL is intentionally empty and ResilientRedisCache is ready for our own Redis.
- config/cache.py docstring and tests updated accordingly (11 tests, still run against a local redis-server).
- CLAUDE.md: new server rule 8 (touch only our own files and service), status block, historical line
  that said "db1 is free" now says not to connect. README, operations, handbook, security doc (item 9
  reopened as an open decision) say the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:41:39 +03:30
c129f64180 Docs: Redis cache behaviour during an outage, what was tested live, CACHE_KEY_PREFIX
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:21:31 +03:30
7929f6bd85 Docs and requirements: signing key replaced and shared Redis cache are live; pin dependency versions
- CLAUDE.md: status block for 2026-10-07 (what changed on the server, how it was checked, what was not).
- docs/security-before-production.md: signing key, shared cache and unit DEBUG items moved to the guard table.
- requirements.txt: ranges replaced by the versions running on the server (pip freeze, 2026-10-07).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:02:51 +03:30
4c601087dc deploy.sh: health check through the gunicorn socket, optional DEPLOY_BUNDLE when Gitea is unreachable
From the server the public URL goes through ArvanCloud and returned 504/000 (2026-10-07), so the
final health step failed even for a healthy service. It now asks the gunicorn socket (path read from
the unit) with our Host header first. DEPLOY_BUNDLE=<file> fetches from a git bundle instead of Gitea,
only when set explicitly; without it an unreachable Gitea still stops the deploy.

Tested: probe_health against a throwaway unix-socket server and a dead URL (bash -n clean).
Not tested: the whole script end to end, until the next deploy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:57:50 +03:30
3e63bd0860 Docs: 7ecb337 is live; record the Gitea 504 from the server and the bundle deploy
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:48:20 +03:30
7ecb337263 Finishing a lesson or island needs recorded answers and an unlocked island; auto DB backup before migrate; throttles on 15 more routes
- content: POST nodes/{id}/quiz/complete and lessons/{id}/complete refuse a locked island
  (403 node_locked, same rule as /api/home) and a user who has not answered every question
  (409 answers_missing). Before, only a user with zero hearts and no answers was refused, so
  anyone else finished any island, locked or not, and finishing a locked island's lessons
  unlocked it. A node or lesson with no questions still passes when unlocked, with no XP.
  The client can no longer understate totalQuestions to buy stars.
- throttle_scope on 15 routes that had none (question bank without login, intro, hearts,
  notifications, tickets, plans, subscription status, orders, offers, affiliate, assistant
  history and feedback, logout).
- admin: OTPCode and Session can only be added/changed by a superuser.
- deploy.sh: integrity-checked DB backup before migrate (keeps the last 2); no migrate without it.
- docs and CLAUDE.md updated to say what is verified and what is not.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:30:24 +03:30
7e3854098f CLAUDE.md: guest login removed, state on the server
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-05 18:40:26 +03:30
663252a260 Remove the guest system entirely: no is_guest/device_id, no guest throttles, phone login is the only way in
accounts.0008 drops the columns and refuses to run while guest rows exist.
Docs describe the current state; tests that only proved guest behaviour are gone.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-05 18:20:10 +03:30
eb4e97753b CLAUDE.md: security review fixes are deployed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 05:09:50 +03:30
a973c41db2 Security review fixes: admin permissions, guest sessions, payment edge cases, assistant leaks
- Admin: refund/cancel/mark-resolved/Bazaar-link actions need change permission; question and
  curriculum import need add/delete; content browser needs view_question; guest device ids are
  shown to superusers only.
- Accounts: registering a guest's phone revokes the guest's earlier sessions; self-deletion keeps
  Redemption, UserOffer and the phone's OTP rows; the public demo guest cannot be deleted
  (DEMO_GUEST_DEVICE_IDS); display-name contact filter covers any separator and social handles.
- Payment: callback without Authority is 400; a second Bazaar purchase on a paid order is 409;
  orders snapshot credits/hearts/days at creation (payment.0011).
- Assistant: a broken diagram no longer makes a streamed answer free; a late off-topic marker
  refuses the answer (not stored, not cached); a disconnect saves the partial answer; only
  KNOWN_COLLECTIONS are searched.
- Infra: secret-key file is created atomically (workers no longer split keys); no default
  INTERNAL_SERVICE_KEY; Kavenegar key and Bazaar token kept out of error logs; Swagger/ReDoc
  pinned; schema throttled; docker compose bound to 127.0.0.1; non-ASCII digit ids are 400;
  ticket text capped at 5000.
- docs/security-before-production.md: remaining open items 24-31 and the guard tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 04:52:25 +03:30
645baf846b Signed payment links, hashed refresh tokens, login-only leaderboard, load test tools
- /api/payment/orders/{id}/pay requires the short-lived signature that paymentUrl carries;
  without it (or with another order's) it is the same 404 as an unknown order
- Session.refresh_token stores the SHA-256 of the token; migration accounts.0007 hashes the
  existing rows so tokens clients already hold keep working
- /api/leaderboard requires a token (guests have one); names of children were scrapable
- RAG mock only for `manage.py test`, not any command with an argument named "test"
- tools/loadtest: user factory, load generator, fake GapGPT+Qdrant, with method notes
- security-before-production.md lists only open items; closed ones become a guards table
  with the tests that hold them. Comments describe current behaviour, not history

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 21:12:28 +03:30
f441cfa96b CLAUDE.md: grades 4/5 and Bazaar purchase path are deployed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 03:08:29 +03:30
8cb326ba94 Grades 4 and 5, Bazaar purchase from the app, pack SKUs
- rag: MATH_4 / MATH_5 textbooks (TOC and page offsets read from the PDFs)
- content: grades 4 and 5 in the seed files: 7 units / 28 nodes each,
  204 + 194 questions (every node >= 4), all re-solved blind once
- subscription: Plan.bazaar_sku (admin); POST /api/subscription/purchase
  accepts paymentMethod=bazaar for packs, rejects unknown methods (was ignored)
- payment: idempotency key cannot switch provider; discount codes refused
  on Bazaar orders (Bazaar charges the fixed SKU price)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 20:01:44 +03:30
ab5e53730a Audit fixes 2026-10-03: seed keeps admin edits, plan cache, refresh limit, payment and tutor guards
- seed_content no longer undoes admin work: a SeedRecord ledger (content/seed_sync.py)
  stops deleted/renamed/re-worded content from being recreated and keeps admin-edited
  lesson text; new rows in the seed files still arrive (migration content.0010)
- plan list cache is invalidated on Plan save (receiver used a stale 'v1' key)
- token refresh per-IP cap 30/hour -> 600/hour (TOKEN_REFRESH_IP_RATE)
- a failed gateway result only changes a pending order (forged NOK no longer
  overwrites refunded/canceled)
- tutor gets the correct answer only after the student has answered
- support reply notification moved to a TicketMessage post_save signal
- global daily cap on paid AI questions (AI_GLOBAL_DAILY_MAX, default 3000)
- Bazaar verify uses only the order's SKU; bazaar orders require a SKU
- /api/health warns on the committed signing key; deploy.sh saves local commits;
  400 for bad ?unit= / ?period=; account delete keeps the referrer's Conversion
- earlier today: no stub answers outside DEBUG, ?grade= removed, logged fallbacks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 16:05:27 +03:30
7e027c167c CLAUDE.md: the no-fallbacks fixes are deployed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:03:16 +03:30
7307bfcadb Remove silent fallbacks found in the 2026-10-03 audit
- /api/ai/ask: the stub answer is reachable only under DEBUG or
  RAG_FORCE_MOCK. Without RAG keys on the server it used to return 200 with
  "this is a stub, set OPENAI_API_KEY" and spend the student's quota; now 503.
- /api/home: the `grade` alias is gone and sending it is a 400 instead of a
  silent fallback to the user's own grade. No client sent it.
- activate_subscription logs ERROR when the paid order's plan no longer exists
  (it silently became 30 days).
- resolve_grade_title logs instead of `except: pass`; curriculum_io no longer
  reads an `intro_text` key no file contains.
- CLAUDE.md: "No silent fallbacks" rules for future agents.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 10:38:42 +03:30
7e4b8b54a6 CLAUDE.md: the 2026-10-02 fixes are deployed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 23:20:27 +03:30
8a4753bd15 CLAUDE.md: what the 2026-10-02 review fixed and what is not deployed yet
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:43:50 +03:30
0f0b779a4e docs: plan with 700 toman per question; pack profit margins
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:19:06 +03:30
3bafd3d9c4 docs: measured model price ledger (docs/model-prices.md)
What GapGPT actually billed per model, with dates and evidence, so a
later price rise can be compared with what it was. gpt-4.1-mini and
gpt-4o-mini were re-checked from the 852 recorded calls in
docs/benchmarks/2026-09-26-tutor-model: every call equals the official
OpenAI per-token price or less (input cache), none more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 13:12:49 +03:30
4c7ccf10a0 docs: question packs, credit rules, real hearts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 12:14:45 +03:30
2edc56fc08 paid plans: 10 tutor questions a day at 179,000 / 1,790,000 toman
Five a day was too close to the free three. Pricing keeps the tutor under ~40% of net revenue
at typical use (a third of the cap); full numbers in the cost report and docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 10:11:31 +03:30
b3940e87f7 docs: age tone, public IDs, prices, account deletion and the test-account cleanup
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 06:50:33 +03:30
39247f6d53 senior's post-demo changes: parent report in the student's own account, Jalali birth date, invoices, math-only tutor, priced quotas
- remove the family app from the root (separate parent accounts, invite codes, parental
  settings); progress.0006 drops its tables, permissions and migration history, and
  Profile.parent_phone, the two family notification types and ?userId= on
  /api/progress/nodes go with it
- GET /api/user/weekly-report: last 7 days next to the 7 before (estimated minutes,
  questions, accuracy, islands, tutor questions) for the "parent report" page
- profile birthDate in Jalali (accounts/jalali.py), ages 5-99, birthYear derived from it
- GET /api/payment/invoices; orders now snapshot list price and plan title
- tutor answers only math: server pre-filter, scope rule in the prompt (tutor-v11),
  refusal marker replaced with fixed text; self-harm gets a help line instead
- streamed answers now record token usage (it was always 0)
- daily tutor quota: AI_DAILY_QUOTA_FREE (default 3) vs Plan.ai_daily_quota
- delete_users management command for wiping test accounts across every app
- docs/ai-cost-and-guardrails.md: measured cost per question and guardrail results

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 02:38:43 +03:30
5f4d7b1ed4 record the Bazaar connection: company account authorized, package com.tadrisino.app set
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:24:00 +03:30
957c9d0f1e record the 2026-09-29 final pre-presentation pass: what shipped, measured tutor latency, test writes to demo data
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:23:07 +03:30
8321dc22c3 record the 2026-09-29 deploy: diagrams live, backup name, test guests to remove
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 08:18:29 +03:30
50df7bd371 diagram kit: triangles and quadrilaterals from problem data, shape patterns, translate points, arithmetic/quote/duplicate-key repairs, clock guard (tutor-v10-shape-anchor)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 06:11:43 +03:30
a8cd681d65 an explicit 'draw it' request without topic words gets the whole diagram catalog (tutor-v8-asked-figure)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 03:34:12 +03:30
0917ab487f diagrams for every grade, sectioned prompt, repair of real model slips; fill lesson texts and thin islands
Diagrams: 14 kinds drawn by the app (plane with constructed points, graphs with
computed roots/extrema/tangents, number lines with intervals, solids, charts,
tables, Venn, mapping, trees, fractions, grids, groups, clock). The catalog is
split by topic and only the matching sections go next to a question (200-1073
tokens instead of 1854). parse() repairs slips seen on 96 real book problems:
Persian digits in JSON numbers, 2*pi, // comments, bare items, Venn with
Persian set names. Measured: 45/48 valid on the second round, 0/8 on plain
arithmetic; visual review of 83: 64 right, 3 weak, 16 wrong.

Content: 82 new questions so no island has fewer than four; fill_lesson_texts
writes Unit.blurb (58) and Lesson.example/steps (590) from verified book
problems, never the lesson's own quiz problem. seed_content only fills empty
fields, so admin edits survive a deploy. Tedi unlocks at 500 xp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 23:22:13 +03:30
33d8f43d74 draw a diagram in tutor answers: validated JSON from the model, rendered by the app; add the fourth companion tedi
The model ends a shape question with <diagram>{json}</diagram>; rag/services/diagram.py
splits it off before mathtext, validates it against a whitelist and stores it on
Message.diagram (ai_tutor.0006). It reaches the app in the SSE done event and in
conversation history. Prompt tutor-v6-diagrams; a hint next to the question (shape
words only) raised gpt-4.1-mini from 3/12 to 12/12 diagrams, 0/8 on arithmetic.

progress.0005 adds tedi (600 xp) to the companion catalog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-28 17:44:44 +03:30
74c3ee5e86 record the Zarinpal fix: shared tadrisino2.ir merchant installed, PaymentRequest now 100
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 22:25:13 +03:30
9a1ee36c0a record the new Bazaar client and the real cause of Zarinpal -14 (merchant registered for tadrisino.org)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 21:45:58 +03:30
301a6942da record that the login SMS now arrives: template SMSVerify on the server, status 10
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 21:45:31 +03:30
c1c2a270c2 record the new Bazaar authorize link for mathapi.tadrisino2.ir
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 02:11:28 +03:30
93334635f2 record the domain move and the live frontend: mathapi live, Zarinpal still -14, demo data intact
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 01:39:56 +03:30
1f38465d88 move the API's own address to mathapi.tadrisino2.ir
The API moved to mathapi.tadrisino2.ir on 2026-09-26 (nginx server_name
changed at 23:22) and api.math.tadrisino2.ir no longer resolves. The code
default for PUBLIC_BASE_URL and the docs' curl examples still named the dead
host, so every paymentUrl, the Zarinpal callback and the Bazaar redirect the
backend builds pointed nowhere. Historical status notes in CLAUDE.md keep the
old name on purpose. config/tests.py now reads the API host from
PUBLIC_BASE_URL instead of a literal. 624 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-27 01:06:48 +03:30
4b246e0c70 make the domain configuration, not code: two env vars move everything
A domain change is coming (the Zarinpal terminal rejects the current callback
domain). ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS, CORS defaults, the Bazaar
redirect URI and deploy.sh's public health URL each spelled the domain out.
They now derive from PUBLIC_BASE_URL and FRONTEND_URL, which keep today's
domain as their default; env values for the three lists are still added on
top, so the old domain can stay allowed while DNS moves.

ZARINPAL_CALLBACK_URL used os.environ.get(key, default), so an empty key in
.env produced an empty callback and every payment would fail closed. It now
uses the same `.strip() or` fallback as its neighbours; the new test caught it.

config/tests_domains.py loads settings in a subprocess with a new domain and
fails if any derived value still mentions the old one. docs/operations.md
gains the step-by-step domain move (DNS, .env, Zarinpal/Bazaar panels,
frontend config.js, verification). CLAUDE.md records that the frontend is
now ours and where the rebuild stands. 624 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 23:46:25 +03:30
2f8ed50b86 record the Bazaar re-authorization: personal-account token cleared, new link out, 7d205ca live
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 22:21:54 +03:30
ff212855a5 send the login code through Kavenegar's approved template when one is configured
The account's line cancels free-text SMS (status 13); the senior confirmed
the account is fine and only the template name was missing. With
KAVENEGAR_OTP_TEMPLATE set, OTP goes to verify/lookup with just the code as
token; without it the old sms/send path is unchanged. Both share one
provider call and response check. 620 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 22:02:01 +03:30