fix the campaigns concurrency test by fixing what it was catching
ExecuteCampaignRealConcurrencyTests had been written off as a known flake for weeks — about 45% red on a full-suite run. It was not timing noise. It was reporting a real defect, and the defect is in production code. execute_campaign looped with .iterator(), which keeps the read cursor on the user table open for the whole loop — a loop that writes a CampaignDelivery per user and, on the sms branch, makes a network call to Kavenegar in between. On SQLite that open cursor locks the table against any concurrent writer, and the lock is not released until the loop ends. So _retry_on_sqlite_lock could never recover from it no matter how long it waited: widening its window from 0.5s to 2s (also done here, since the linear backoff was too tight regardless) moved the failure rate not at all on its own. Both branches now materialise the ids before the first write. After the change: 20 consecutive runs of the test itself and 3 full-suite runs, all green. This is not only a test fix. While we are on SQLite the same pattern produces the same lock in production, where it surfaces as "database is locked" on unrelated requests that happen to land during a campaign run. Also in this pass ----------------- - CLAUDE.md carried three sections each claiming to be the current state, with three different question counts (1,441 / 1,783 / reality 2,463). There is now one current-state section and the older two are labelled as history, so whoever picks this up next is not reading stale numbers as fact. Three other figures that had drifted are corrected against the database: the verdict split, the figure-gap share (881/~42% is now 508/~14% after the five books were re-read), and the node coverage. - Dropped four genuinely unused imports. The remaining pyflakes hits are the signal registrations in apps.py, which are imported for effect. - .env.example suggested gpt-4o for vision; gpt-4.1-mini measured both cheaper and more accurate and is what the server actually runs. Audited all 2,463 questions across every grade: no question with the wrong number of options, none with zero or two correct answers, no empty or duplicated option, no orphan lesson or node, no duplicate question inside a lesson. 458 tests pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
58
CLAUDE.md
58
CLAUDE.md
@@ -191,8 +191,10 @@ problem. These are invisible in retrieval and only surface when something trusts
|
||||
`rag_verify_solutions` re-solves every extracted problem independently and compares.
|
||||
`LessonChunk.solution_verdict` is three-state — `agrees` / `disagrees` / `unjudgeable` —
|
||||
and the verdict also rides in the Qdrant payload under a key `retrieval._payload_text`
|
||||
does not read, so the tutor's context is unchanged. Current state: 1,112 agree, 123
|
||||
disagree, 881 unjudgeable. **Question generation only ever reads `agrees` chunks.**
|
||||
does not read, so the tutor's context is unchanged. Current state (2026-09-22, across
|
||||
all 3,517 chunks): 2,466 agree, 354 disagree, 508 unjudgeable, 189 not yet checked —
|
||||
those 189 are the other team's علوم/مطالعات collections, never run through the checker.
|
||||
**Question generation only ever reads `agrees` chunks.**
|
||||
|
||||
Two things that were learned the expensive way and are already encoded in the prompts:
|
||||
|
||||
@@ -205,11 +207,13 @@ independent answer is stored separately in `checked_answer`.
|
||||
|
||||
## The figure gap (largest open data issue)
|
||||
|
||||
881 chunks (~42%) reference a figure or table that is not in the text, so they cannot be
|
||||
508 chunks (~14%) reference a figure or table that is not in the text, so they cannot be
|
||||
answered or turned into questions. **The vision model saw the figure; it was simply never
|
||||
asked to write it down.** `rag/services/pdf.py` now carries a self-contained rule telling
|
||||
it to inline measurements and labels into the problem text — books ingested before that
|
||||
rule predate it. Recovering them means re-ingesting those pages (~$2.7 of API credit).
|
||||
it to inline measurements and labels into the problem text, and re-reading the five math
|
||||
books under that rule is what brought this down from 881 (~42%). What is left is
|
||||
concentrated in the three books that predate the rule (سوم/ششم/هفتم, the other team's);
|
||||
recovering them means re-ingesting those pages (~$2.7 of API credit).
|
||||
|
||||
## Quiz assembly
|
||||
|
||||
@@ -218,7 +222,8 @@ and a lesson here is the finest grain — one problem type from the book, not a
|
||||
session. `GET /api/nodes/{id}/quiz` assembles a quiz from a node's lessons, at most one
|
||||
question per lesson. Shuffle is seeded on `nodeId:userId`, so a refresh returns the same
|
||||
quiz; `questionCount` is reported separately from `requested` so a thin node cannot look
|
||||
like an error. 172 of 230 nodes currently hold a full four-question quiz.
|
||||
like an error. 224 of 263 nodes hold a full four-question quiz, and every node in
|
||||
grades 8–12 does; the 39 that fall short are all in grades 3/6/7.
|
||||
|
||||
## AI provider notes
|
||||
|
||||
@@ -348,7 +353,44 @@ preferred setup, because there is then no key to leak. **Never let the server ru
|
||||
`django-insecure-…` fallback from `config/settings.py`: that string is committed here, so
|
||||
anyone who can read this repo can forge a token for any user.**
|
||||
|
||||
## State as of 2026-09-17 (evening) — read this first
|
||||
## وضعیت فعلی (۲۰۲۶-۰۹-۲۲) — این را اول بخوان
|
||||
|
||||
سرور روی `api.math.tadrisino2.ir` زنده است و **کدِ فعلی را سرو میکند** (سرویس واقعاً
|
||||
ریاستارت شد، نه فقط فایلها روی دیسک). `/api/health` میدهد:
|
||||
`grades: 8, lessons: 2617, questions: 2463` و هیچ مهاجرتِ اعمالنشدهای ندارد.
|
||||
|
||||
- **هر نودِ پایههای هشتم تا دوازدهم آزمونِ کاملِ چهارسوالی دارد.** پایههای
|
||||
سوم/ششم/هفتم کتابِ تیمِ دیگرند: ۳۹ نود از ۱۴۵ کمتر از چهار سوال دارند، ولی هیچکدام
|
||||
خالی نیست، پس روی نقشه جزیرهی مرده وجود ندارد.
|
||||
- ۴۵۸ تست سبز، `check` و `check --deploy` و `makemigrations --check` تمیز، و
|
||||
`spectacular` بدونِ خطا و هشدار (قرارداد کامل است).
|
||||
- تستِ همزمانیِ campaigns که مدتها flaky بود **دیگر flaky نیست** — علتش یک باگِ
|
||||
واقعی بود، نه طبیعتِ تست؛ پایینِ همین فایل توضیح داده شده.
|
||||
|
||||
بخشهای تاریخیِ زیر بهترتیبِ زمان نگه داشته شدهاند چون درسهایشان هنوز معتبر است؛
|
||||
**اعدادِ محتواییشان کهنهاند و ملاک همین بخش است.**
|
||||
|
||||
---
|
||||
|
||||
### تستِ flakyِ campaigns یک باگِ واقعی بود (۲۰۲۶-۰۹-۲۲)
|
||||
|
||||
`ExecuteCampaignRealConcurrencyTests` ماهها بهعنوانِ «flakyِ شناختهشده» کنار گذاشته
|
||||
شده بود (~۴۵٪ قرمز روی اجرای کاملِ سوئیت). علتش تصادفیبودنِ زمانبندی نبود:
|
||||
|
||||
`execute_campaign` با `.iterator()` روی کاربرها حلقه میزد، یعنی **کرسرِ خواندن در
|
||||
تمامِ طولِ حلقه باز میماند** — حلقهای که وسطش `CampaignDelivery` مینویسد و در شاخهی
|
||||
sms یک فراخوانیِ شبکهای به کاوهنگار هم میکند. روی SQLite همین کرسرِ باز جدول را برای
|
||||
هر نویسندهی همزمان قفل میکند، و قفل تا پایانِ حلقه رها نمیشود — پس
|
||||
`_retry_on_sqlite_lock` هیچوقت نمیتوانست نجاتش دهد، هر چقدر هم صبر میکرد.
|
||||
|
||||
حالا هر دو شاخه پیش از اولین نوشتن، نتیجه را با `list(...values_list(...))` مادی
|
||||
میکنند. بعد از این تغییر: ۲۰ اجرای پیاپیِ خودِ تست و ۳ اجرای کاملِ سوئیت، همه سبز.
|
||||
|
||||
⚠️ **درسِ کلی: در این کدبیس `.iterator()` را دورِ حلقهای که مینویسد یا I/O بیرونی
|
||||
میزند نگذارید.** تا وقتی روی SQLite هستیم، این الگو در پروداکشن هم همان قفل را
|
||||
میسازد — آنجا بهشکلِ «database is locked» برای درخواستهای بیربطِ دیگر.
|
||||
|
||||
## تاریخچه — وضعیت ۲۰۲۶-۰۹-۱۷ (اعداد کهنه)
|
||||
|
||||
**The server is live and running current code.** `main` is at the latest commit, the
|
||||
service `gunicorn-mathapp` was restarted against it, and `/api/health` reports
|
||||
@@ -389,7 +431,7 @@ The frontend's `Question` type carries a `type` field that nothing reads, and
|
||||
`count_objects` in `Question.TYPE_CHOICES` is an empty shell — no rows, no serializer
|
||||
branch, no rendering anywhere. Do not build content for it without asking first.
|
||||
|
||||
## وضعیت ۱۴۰۵/۰۶/۲۹ (۲۰۲۶-۰۹-۲۰) — این را اول بخوان
|
||||
## تاریخچه — وضعیت ۱۴۰۵/۰۶/۲۹ (۲۰۲۶-۰۹-۲۰) (اعداد کهنه)
|
||||
|
||||
سرور روی `3026df0` است، کد و محتوا هر دو منتشر شدند، سرویس واقعاً ریاستارت شد، و
|
||||
`/api/health` عدد `questions: 1783` میدهد. `/api/nodes/1/quiz` چهار سوالِ واقعی برمیگرداند.
|
||||
|
||||
Reference in New Issue
Block a user