Commit Graph

118 Commits

Author SHA1 Message Date
4cd859b5fa 216 backups encrypted on 216: pgBackRest (AES-256) + age-encrypted payment CSVs, pushed write-only to 247 with rrsync; health watches pg_stat_archiver and OFFSITE_SYNC_STATUS_FILE; 216 scripts/configs in ops/server216; plaintext pg216 pipeline removed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 19:43:00 +03:30
dd98f16d12 Live backup of 216 on 247 (zareian-pg216-* units, Postgres 16 tools in our dir, SCRAM hash only on 216), backup scripts take PGB_ROOT/PGB_BIN/PGB_PORT with 249 defaults, anti-bot layers documented
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 19:22:56 +03:30
bd329f83ae Dedicated server 78.38.35.216: state and cut-over order in docs, CLAUDE.md entry, password snippet for two Password lines, archive-app-log.sh finds its project dir
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 19:06:04 +03:30
133233144e Seed: grade 12 questions (every lesson 4+ per lesson, answers computed in code), minus-sign and doubled-x fixes in grade 10 and 11
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 03:35:34 +03:30
a5a93ef6ba Seed: grade 11 questions (every lesson 4+ per lesson, answers computed in code)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 03:27:10 +03:30
5f963e2898 Seed: grade 10 questions (every lesson 4+ per lesson, answers computed in code)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 03:16:51 +03:30
7bd7c637b4 Seed: grade 9 questions (every lesson 4+ per lesson, answers computed in code)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 03:00:27 +03:30
ac96f93abc Seed: grade 8 questions (every lesson 4+ per lesson, answers computed in code)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 02:48:33 +03:30
9903e0ec91 Seed: grade 5 rebuilt (every lesson 4+ questions computed in code), wording fixes
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 02:30:33 +03:30
759bb0d575 Seed: grade 4 rebuilt (every lesson 4+ questions computed in code), slash decimals fixed in chapter 5
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 02:06:21 +03:30
f8a93a37c9 Seed: grade 7 rebuilt (every lesson 5+ questions computed in code), lost-exponent questions fixed
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 01:47:44 +03:30
9eae99340d Seed: grade 6 rebuilt (every lesson 5+ questions, answers computed in code), supplementary/complementary text fixed in grades 4/6/7, '/' decimals to '٫', wrong keys fixed
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-17 01:31:28 +03:30
f57deb9c1d Seed: grade 3 rebuilt (every lesson 5+ questions, wrong figures/lesson text fixed), Persian digits and plain fractions across all grades, apply_fixups gains lesson/question_set, ledger save chunked for SQLite
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-16 21:27:22 +03:30
5fbbd371fe Seed: questions for the 499 lessons that had none; every lesson now has questions (5,406/5,406)
998 four-choice questions (2 per lesson), written from each lesson's own text,
worked example and the book's extracted problem. Mostly grades 3, 6, 7 (books
imported before us), plus 4, 5 and a few in 8-12. Hands-on lessons (drawing,
measuring, folding) got text questions that test the same idea.
Checked: structure (4 distinct choices, no LaTeX), a blind re-solve of all 998
against the key (no mismatch), and content.tests_seed_content seeding the full
file. Not checked: an independent second solver.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:15:13 +03:30
e6e983f970 Figures must not give the answer away: blank titles on new figured questions, 7 trivial questions removed, 7 reworded; apply_fixups tool
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-15 02:21:02 +03:30
998b68e4bc Figured questions for grade-12 geometry (conics, systems, vectors: 50 new); docs: figured questions win in the island quiz
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-14 22:24:53 +03:30
a4e0c576bb Docs brought to the current state: CLAUDE.md rewritten, README formal, handbook and diagrams redrawn
- CLAUDE.md: 1,225 → ~430 lines, current state only (the dated status log stays in git history).
  Server rules moved to the top: what is ours on 249/247 and what is not, non-negotiable rules,
  announce-before-acting, password handling, and the full "deploy it" procedure with verification.
- README: formal entry page with current numbers (10 grades, 18,522 questions, 61 API operations,
  973 tests, PostgreSQL/Redis).
- Removed docs/frontend/ (Figma-era guide with guest login and its own api.ts; the frontend repo has
  the real client). Swagger's external-docs link now points at the frontend repo.
- architecture/operations/security/content-authoring/loadtest/textbooks: SQLite, guest login,
  demo accounts, question packs and the port-80 note replaced by what is true now.
- Technical handbook text updated throughout; all seven diagrams redrawn (PostgreSQL, phone-only
  login, subscriptions, current data model); PDF rebuilt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:45:51 +03:30
1c7c4b8eca API output without Persian diacritics or written ezafe ی (company decision); tutor answers too
plain_persian() strips kasra/fatha/damma/shadda/sukun/hamza-above and the written «ه‌ی» ezafe, keeping tanwin. Applied
at output, not in data: PlainPersianJSONRenderer runs one byte-level pass over every JSON response (~5 ms per MB
locally), and to_plain() cleans tutor answers including the stream. Rewriting titles in the DB/seed would have broken
question binding by title path and SeedRecord keys. Tests updated to expect the cleaned text; config/tests_plain_persian.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:57:17 +03:30
7b241c1aac CLAUDE.md: restore onto 249 was already done (owner)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:16:58 +03:30
e3c8962b7b Load test of 1000 players on the server: CPU is the ceiling (~135 req/s); player-simulation tools
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:48:00 +03:30
ba4ecff29e Server hardening: nginx origin locked to ArvanCloud + CSP, demo logins removed, docs
- ops/nginx/: the two nginx sites as they now run on 249. Only ArvanCloud
  edges reach the app (direct hits on the IP get 403, so Arvan's DDoS/WAF
  can't be skipped); the frontend gets CSP (own sources only, the one inline
  script by hash), X-Frame-Options, nosniff, Referrer-Policy,
  Permissions-Policy, HSTS; API body limit 100M -> 20M plus HSTS. Checked from
  outside and in Chrome (all tabs and an island quiz, logged in): no CSP errors.
- Demo accounts (users 11-15, 25) and TEST_LOGIN_PHONES removed from the server;
  the health warning for fixed-code logins now has a test and says how to
  test instead (an OTPCode made in the admin).
- SMS daily cap on the server is 2000 (launch of ~600 users).
- CLAUDE.md: never run tests with --parallel on macOS, always --noinput
  (both hang forever without an error).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:17:11 +03:30
78c3669119 Docs: app logs archived nightly and kept forever on the backup server
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 17:56:12 +03:30
f4831e93a7 Docs: payment event ledger and its CSV export on the backup server
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:47:24 +03:30
947f946909 Question packs removed at the root; all subscriptions 4 assistant questions a day; Bazaar is subscription-only
- subscription.0017/0018, payment.0013: pack plans deleted, QuestionCredit/HeartsPass models and Plan.kind /
  question_credits / hearts_days / Order.question_credits / hearts_days dropped. The only paid pack order was the
  company owner's test account (user 26).
- Assistant: no credit spending; quota is the daily cap only (free 2, every plan 4, gift subscription 4).
- Bazaar: verify_purchase is subscription-only (no inapp guess), start/purchase responses lose productType,
  verify refuses an order without a subscription. A paid order with no subscription now logs ERROR.
- Removed POST /api/payment/orders (internal; no caller). Its permission tests stay.
- Pack-based tests rewritten on subscriptions (refund, security, Bazaar token); pack-only test files deleted.
- docs: profit table for 4/day on both payment routes plus free users; handbook, operations, CLAUDE.md current.
949 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 20:33:38 +03:30
9c3adc3e60 Subscriptions only: Bazaar panel prices (598/898/1398k), assistant quota 5/3/3 from a cost calculation, packs off sale
- subscription.0016: prices equal to the Bazaar panel (senior), ai_daily_quota 5/3/3, card features; packs on_sale=False
  (one paid pack order, user 26, keeps its credits; pack paths stay for that order and for admin).
- Quota math in docs/ai-cost-and-guardrails.md: ~77% of a Bazaar sale reaches us; worst case (full quota every day)
  the assistant costs 23/27/35% of it.
- Assistant quota message points to the subscription, not packs.
982 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 19:57:16 +03:30
c0fa46be44 Payment providers as classes (Strategy + factory); 10342 = monthly; health logs a failed Bazaar check
- payment/providers.py: ZarinpalProvider / BazaarProvider with one interface (check_plan, sku_for, payment_url,
  purchase_extras, discount_blocker, reconcile_user); provider_for / provider_for_method pick one; unknown fails.
  create_order, purchase view, discount redeem and reconcile_user_orders use it instead of scattered ifs.
- subscription.0015: Bazaar SKU 10342 on the one-month plan (owner: "probably the monthly one").
- config/health: the Bazaar check no longer swallows its exception without a log.
982 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 18:21:45 +03:30
76b0ef976e Bazaar: plans cache cleared by migration 0014, enum name override, CLAUDE.md status
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 17:59:43 +03:30
2e0753587c Like/dislike + report under every question and lesson, lesson reports and vote columns in admin; practice islands refresh from file; audit fixes
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 12:56:47 +03:30
e662914efc No LaTeX/markdown anywhere students read (seed, import, DB migration 0021, tutor history, frontend guard); quiz of an island is fixed per user again
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 12:15:28 +03:30
dd10ff83f4 13,689 new quiz questions in 160 practice islands (grades 3-12), Question.figure, replay rotates the quiz, LaTeX removed from lesson/question text
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 11:58:18 +03:30
f9eea56895 CLAUDE.md: tester accounts
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 11:13:51 +03:30
33478179e2 CLAUDE.md: truth tables and island labels checked in the live app
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 10:57:01 +03:30
9c3596a132 CLAUDE.md: what is on the server after the subjects/figures release and what was checked
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 05:44:32 +03:30
8cb1ecb85e Subjects per grade (math, calculus, geometry, stats, discrete), 5 new textbooks, 202 new islands, long islands split, hand-written figures and lesson texts for the new books
- content.Subject + Unit.subject, /api/home?subject=, tutor grounded in the chosen book
- 5 new books ingested; ~1150 hand-written questions, islands named by hand
- node_split: islands over 7 lessons split into parts of <=6 (lesson ids and progress kept)
- Lesson.figures hand-authored and visually checked for 653 lessons of the new books;
  gen_lesson_figures got --subject and survives network errors
- seed files re-exported

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-12 05:35:18 +03:30
3d51b3b83a CLAUDE.md: capacity under load (measured vs estimated), open launch-day caps, how to load-test the shared server safely
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 22:27:54 +03:30
2c708088e4 Spike readiness: AI gets its own slice of threads, cheaper hot paths, classroom-friendly login
- /api/ai/ask: at most AI_MAX_CONCURRENT_PER_WORKER (12 of 32 threads) answers per worker;
  beyond that 503 assistant_busy + Retry-After before any quota is reserved. Without it a
  burst of questions (3-40 s each) took every thread and the map and quizzes queued behind.
  The slot is held until the stream is actually finished or closed.
- Answer submission 21 -> 14 queries, 15.4 -> 11.0 ms: one query for all choices, locked
  get_or_create for node progress and streak, and today's streak recorded once per day
  via the cache instead of a transaction per answer.
- /api/home 12.4 -> 8.6 ms: the user's completions are read by user_id instead of an IN list
  of ~600 lesson ids; the dict is returned directly (serializer stays the Swagger contract,
  HomeMatchesItsSerializerTests keeps them equal).
- otp_login per-IP 30 -> 300/hour: a class of 30 on one school Wi-Fi filled it; guessing is
  bounded per code and per phone.
- /api/health aiConcurrency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:55:24 +03:30
feaa5eb540 Postgres connection pool, clean 503 under overload, time indexes, 4 workers
Postgres here allows 60 connections, but each of the 96 gunicorn threads held its
own persistent one: at 100 concurrent users (local Postgres with the same limit)
21% of requests were 500 "remaining connection slots are reserved". Now each worker
has a psycopg pool (8), so the same load gives 0 errors and 714 req/s (was 460).

- A full pool answers 503 + Retry-After (DRF handler and middleware), not 500.
- The AI tutor and the OTP SMS call return their connection before the network wait.
- statement_timeout 30 s, idle-in-transaction 60 s; bad pool settings and the old
  DB_CONN_MAX_AGE stop startup instead of failing every request.
- /api/health dbPool: per-worker pool stats and workers x pool + 6 vs max_connections.
- Indexes on the columns global counters filter by time (weekly leaderboard,
  global SMS cap, global AI cap); the leaderboard recomputes once per expiry.
- Worker count moves into gunicorn.conf.py (4 on a shared 8-core box).
- security doc: Redis item closed (server uses our own Redis since 2026-10-08).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 20:21:35 +03:30
4821df3914 CLAUDE.md: pointer to the next-APK notes and why app payments go through pay.html
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 18:54:12 +03:30
2e87a81cc5 CLAUDE.md: pre-move SQLite copy removed from the server; the old August backup script was never scheduled
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 17:14:42 +03:30
a9cccc6550 Live off-site Postgres copy on 247 replaces the nightly pg_dump; backup retention in deploy.sh
- ops/pgbackup: SSH tunnel + pg_receivewal (slot zareian_247) + daily verified pg_basebackup (keep 7)
  and a restore check, all run on 78.38.35.247 as systemd units
- /api/health reports offsiteBackup (slot active/lag/lost) and warns; OFFSITE_WAL_SLOT setting
- removed statuslog backup_db/backup_verify and their health warnings
- deploy.sh keeps 3 pre-deploy dumps, 3 local patches, drops manual backups after 30 days
- docs: operations backup section, security item 14 closed, CLAUDE.md status

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 16:53:10 +03:30
19072ee534 Fourth companion renamed Tedi -> Omega (ω); slug stays tedi
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 01:42:44 +03:30
a39643b8a3 CLAUDE.md: paywall and subscriptions are live
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 01:20:51 +03:30
a5feee053a CLAUDE.md: Bazaar package name now matches the APK (com.tadrisino.math.app)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 14:12:28 +03:30
22c3196343 Close /api/health details and API docs to the public
Health: public gets only ok/error; details need X-Internal-Key. Docs/schema: admin session only.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 13:05:19 +03:30
d6529956e1 Docs: Postgres + own Redis are the server's current state
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-08 12:43:48 +03:30
e00f6b80eb CLAUDE.md: lesson figures workflow, cost, review lessons, frontend cache and nginx headers
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 19:48:10 +03:30
d584d62904 Do not connect to the shared Redis: CACHE_URL stays empty, own Redis is an open decision
The owner decided (2026-10-07) that we do not connect to anything other teams run on the box, Redis
included. CACHE_URL was emptied on the server and our keys were removed from the shared Redis.

- settings: drop KEY_PREFIX / CACHE_KEY_PREFIX (it only existed because the Redis was shared); comments
  say CACHE_URL is intentionally empty and ResilientRedisCache is ready for our own Redis.
- config/cache.py docstring and tests updated accordingly (11 tests, still run against a local redis-server).
- CLAUDE.md: new server rule 8 (touch only our own files and service), status block, historical line
  that said "db1 is free" now says not to connect. README, operations, handbook, security doc (item 9
  reopened as an open decision) say the same.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:41:39 +03:30
c129f64180 Docs: Redis cache behaviour during an outage, what was tested live, CACHE_KEY_PREFIX
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:21:31 +03:30
7929f6bd85 Docs and requirements: signing key replaced and shared Redis cache are live; pin dependency versions
- CLAUDE.md: status block for 2026-10-07 (what changed on the server, how it was checked, what was not).
- docs/security-before-production.md: signing key, shared cache and unit DEBUG items moved to the guard table.
- requirements.txt: ranges replaced by the versions running on the server (pip freeze, 2026-10-07).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 14:02:51 +03:30
4c601087dc deploy.sh: health check through the gunicorn socket, optional DEPLOY_BUNDLE when Gitea is unreachable
From the server the public URL goes through ArvanCloud and returned 504/000 (2026-10-07), so the
final health step failed even for a healthy service. It now asks the gunicorn socket (path read from
the unit) with our Host header first. DEPLOY_BUNDLE=<file> fetches from a git bundle instead of Gitea,
only when set explicitly; without it an unreachable Gitea still stops the deploy.

Tested: probe_health against a throwaway unix-socket server and a dead URL (bash -n clean).
Not tested: the whole script end to end, until the next deploy.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-07 13:57:50 +03:30